What Is Claude 3.5 Sonnet? How It Works, Use Cases, and Artifacts

Claude 3.5 Sonnet: First Impressions

I was intrigued by the benchmark results, so I went straight to Claude.ai to test the new Claude 3.5 Sonnet model.

I first asked Claude 3.5 Sonnet to create a graph using the data in the table below:

I reckon I didn’t put much thinking effort into the prompt I used—Generate a graph for this table—and yet the model generated this interactive grouped bar chart:

Claude 3.5 Sonnet implemented the chart as a React component using the Recharts library. We can see and copy the code by switching to the Code tab from the top right corner:

One thing that bothered me about the generated bar chart is that the color legend at the bottom overlaps with the x-axis labels. Besides that, there’s just too much information to consume for one single graph. I thought it might be a cool idea to have a separate infographic for each benchmark, so I prompted Claude 3.5 Sonnet with this:

For the code benchmark, generate an infographic where you add a very short explanation about what the benchmark is and what it measures and a little comparison graph on the side to show how each LLM scores on that benchmark

I have to say, the result was pretty impressive, considering how bad my prompt was:

The infographic is well organized, with a good choice of colors and font properties. I already see so many applications here—with detailed prompting, we could tailor the design to our needs and add brand-specific customizations, for example.

If you haven’t yet tried Claude 3.5 Sonnet yourself, you might be wondering what’s that UI component that shows both the graph and the code. Enter Artifacts.

Artifacts: Claude AI’s New Feature

There’s no point in me explaining what Artifacts are when the official video does it so much better. The reason I encourage you to watch this is because they show how to create a side-scrolling game in HTML 5 in one minute:

Artifacts feel so dynamic!

At first glance, the Artifacts feature might seem like just another window or tab in the interface. However, Anthropic isn’t merely adding another way to display information—they are providing a new way to interact with their AI.

Artifacts move beyond traditional text-based interactions by introducing a dynamic, interactive layer. This allows us to visualize and manipulate the AI’s outputs directly, rather than just reading them.

This shift is a notable development. It suggests a move towards a more collaborative relationship with AI, where we can actively engage with and refine the AI’s output.

It will be interesting to see how this feature evolves and how it impacts the way we utilize AI in the future.

And what excites me is Antrophic’s promise to design it for team collaboration:

How to activate and use Artifacts

To access Claude Artifacts, you'll need to enable it in the settings. First, click on your account initials, and then click Feature Preview.

Then, activate the Artifacts by switching the toggle to On.

Once activated, you can trigger the feature by asking Claude to generate content that can be displayed in an Artifact, such as code, data visualizations, or even mockups.

When Claude creates an Artifact, it appears in a separate panel within the interface, divided into Code and Preview tabs. The Code tab displays the raw output generated by Claude, while the Preview tab provides a visual or interactive representation of that output.

How to Access Claude 3.5 Sonnet

The most straightforward way to access Claude 3.5 Sonnet is to go to Claude.ai and have fun in their friendly graphical user interface:

Claude 3.5 Sonnet is free to use on Claude.ai, but I do need to mention that I reached usage limits very quickly (after about ten prompts).

If you want to use Claude 3.5 Sonnet regularly, there’s no doubt that you’ll need to buy a Pro subscription.

Claude 3.5 Sonnet is also available via the Anthropic API, Google Cloud’s Vertex AI, and Amazon Bedrock.

How Claude 3.5 Sonnet Works and Performs

While Anthropic hasn't released the full technical details, we know that Claude 3.5 Sonnet is built upon their previous model, Claude 3 Opus. It's a large language model, meaning it's trained on a massive dataset of text and code, allowing it to understand and generate human-like language.

What sets Claude 3.5 Sonnet apart is its enhanced performance. It has been fine-tuned and optimized, resulting in improvements in various areas.

Graduate-level reasoning has long been considered a hallmark of human intelligence, a frontier where AI has traditionally struggled to keep pace.  While there is still much work to be done, Claude Sonnet 3.5 demonstrates significant progress.

Regarding math, Claude 3.5 Sonnet lightly trails behind GPT-4o, which leads with a score of 76.6%. But Claude 3.5 Sonnet's score of 71.1% is still notably high. This performance suggests that Claude 3.5 Sonnet has a robust grasp of mathematical concepts, formulas, and logical reasoning, making it a promising tool for tasks that require mathematical expertise.

The BIG-Bench-Hard benchmark focuses on evaluating AI models' ability to tackle multifaceted problems, demonstrate advanced reasoning, and apply knowledge across various domains. Claude 3.5 Sonnet significantly outperforms other models in this benchmark, achieving a score of 93.1%. This indicates that Claude 3.5 Sonnet possesses a high level of proficiency in handling complex tasks, reasoning effectively across different subject areas, and applying its knowledge flexibly to novel situations.

When it comes to visual capabilities, Claude 3.5 Sonnet particularly excels in visual math reasoning, scoring 67.7% on the MathVista test, a significant lead over the other models. Claude 3.5 Sonnet also performs well in visual question answering (MMMU) and document visual Q&A (ANLS score), showcasing its ability to understand and process visual information effectively.

While its performance on science diagrams and chart Q&A is slightly lower than GPT-4o and Gemini 1.5 Pro, it still maintains a high accuracy rate above 90%. Overall, the results highlight Claude 3.5 Sonnet's strong capabilities in understanding both visual and textual data, making it a promising tool for various applications that require multimodal comprehension.

Claude 3.5 Sonnet Use Cases

Claude 3.5 Sonnet is Anthropic’s strongest vision model, and it’s no surprise it does so well on tasks that require visual reasoning—like interpreting and creating graphs. However, it’s also marketed as a great coding or writing partner, so let’s take a closer look.

Visual presentations

If you have data you need to present visually, Claude 3.5 Sonnet seems to be able to really work magic.

In Anthropic’s example, a biology professor prompts the model with two graphs and asks Claude to extract relevant information to create a Javascript-based presentation that uses reveal-js, a popular framework for creating HTML presentations. Let’s see if the professor got the slides she wanted:

Coding partner

Let’s say you’re a software engineer who tries to write tests and fix bugs in a code that crops images into circles (so users can have nice profile images). Can Claude 3.5 Sonnet help with that? Let’s see:

Writing partner

Anthropic also showcases Claude 3.5 Sonnet as a writing partner, but their example couldn’t be further from a real-world use case—novels are generally not about “sarcasm-laden crabs that live on bustling reefs.” The genogram Claude creates is awesome, though, and once again showcases its amazing visual abilities.

Price Comparison: Sonnet vs. GPT-4o vs. Gemini Pro 1.5

Let’s see how Claude 3.5 Sonnet's prices compare to those of OpenAI and Google. Of course, this is a good opportunity to ask Claude 3.5 Sonnet to create a graph:

For personal use, all three platforms charge about $20/month.

Conclusion

For the past few years, we’ve all seen so much news and many breakthroughs in the AI space that it’s become harder to get excited about new releases—especially with so many cherry-picked examples in demonstration videos.

However, I am genuinely excited about Claude 3.5 Sonnet's visual capabilities and Artifacts feature. I’ll personally try it for work and keep an eye on its development.