Evals for Agents with Arize

As AI agents become more autonomous, testing and debugging their behavior becomes both more important—and more challenging. Traditional metrics often fall short when agents reason, plan, and act across multiple steps. To build reliable agentic systems, teams need strong observability and evaluation practices baked in from day one.

In this code-along webinar, Laurie Voss, Head of Developer Relations at Arize, will show you how to automatically test and debug AI agents using the Arize AI engineering platform. You’ll learn core principles of agent evaluation, then build and instrument a simple agent to track performance, behavior, and failure modes. By the end of the session, you’ll have a practical framework for monitoring agents in development and beyond.

Set-Up Instructions

The notebook requires 3 secrets, which you can enter as colab secrets, or just paste directly into cell 2:

• os.environ["ANTHROPIC_API_KEY"]
• os.environ["PHOENIX_API_KEY"]
• os.environ["PHOENIX_COLLECTOR_ENDPOINT"]

For Anthropic, you can get a free API key from https://platform.claude.com/settings/keys

Next you'll need access to Phoenix Cloud! This is free to use.(You could also run Phoenix locally on your laptop but this is easier!)

• Go to https://app.phoenix.arize.com and sign inIf you see a screen with a "launch space" button, launch your space.
• You should then choose "Settings" in the bottom left. You'll see ascreen as shown.
• For the API key, click the "+ System Key" button where the big redarrow is pointing.
• For the collector endpoint, you want your hostname, shown by thesmaller red arrow.