After doing these courses, I feel confident creating professional visualizations and dashboards
Shoebur RahmanProcurement Specialist, Giz Bangladesh
Course Description
A description of the course.
Prerequisites
Curriculum
Course outline
1
LLM Application Evaluation
Evaluation Fundamentals
You will learn to design comprehensive AI application evaluation systems that measure performance across accuracy, cost, and latency dimensions using evaluation datasets and multiple evaluator types — from algorithmic matching to LLM-as-judge approaches — enabling you to establish success criteria upfront and measure progress toward release-ready applications.
Evaluation Implementation
You will learn to implement evaluation systems in practice using LangSmith for dataset creation, evaluator definition, and experiment execution — building algorithmic evaluators for objective comparisons, LLM-as-judge evaluators for subjective assessments, and multi-metric evaluators for comprehensive quality analysis.
Conversation Evaluation
You will learn to evaluate conversational AI applications using online evaluation with criteria-based assessment — implementing turn-level and full-conversation evaluation patterns through LLM-as-judge evaluators — enabling you to systematically measure chatbot quality across coherence, task completeness, and efficiency.
LLM Application Evaluation with LangSmith
Course
Complete
