Machine Learning Artificial Intelligence Data Scientist

Evaluating LLM Responses

In this session, we cover the different evaluations that are useful for reducing hallucination and improving retrieval quality of LLMs.

Josh Reini

2023년 11월 29일

무료 계정 만들기

실습 문제하고 트랙하세요.

Google로 계속하기계속 진행하시면 다음 사항에 동의하는 것으로 간주됩니다: 이용약관 및 개인정보 처리방침 그리고 귀하의 데이터가 미국에 저장된다는 점에 동의하는 것으로 간주됩니다.

함께 코드 따라하기

코드 따라하기 ↗

슬라이드 보기 ↗

LLMs should be considered hallucinatory until proven otherwise! A lot of us have turned to augmenting LLMs with a knowledge store (such as Zilliz) to solve this problem. But this RAG setup can still face issues with hallucination. In particular - this can be caused from retrieving irrelevant context, not enough context, and more.

TruLens is built to solve this problem. TruLens sits as the evaluation layer for the LLM stack, allowing you to shorten the feedback loop and iterate on your LLM app faster. We'll also talk about the different metrics you can use for evaluation and why you should consider LLM-based evals when building your app.

Key Takeaways:

Learn about common failure modes for LLM apps
Learn the different evaluations that are useful for reducing hallucination, improving retrieval quality & more.
Learn about how to evaluate LLM apps with TruLens

Additional Resources

TruLens Documentation

TruLens GitHub

Find the prompts used for LLM-based feedback functions in TruLens' open-source github repository here.

[SKILL TRACK] AI Fundamentals

[COURSE] Working with the OpenAI API

[TUTORIAL] How to Build LLM Applications with LangChain