LLM Application Evaluation with LangSmith

You will learn to design comprehensive AI application evaluation systems that measure performance across accuracy, cost, and latency dimensions using evaluation datasets and multiple evaluator types — from algorithmic matching to LLM-as-judge approaches — enabling you to establish success criteria upfront and measure progress toward release-ready applications.

You will learn to implement evaluation systems in practice using LangSmith for dataset creation, evaluator definition, and experiment execution — building algorithmic evaluators for objective comparisons, LLM-as-judge evaluators for subjective assessments, and multi-metric evaluators for comprehensive quality analysis.