Machine Learning Artificial Intelligence Data Scientist

Fine-Tuning Your Own Llama 2 Model

In this session, we take a step-by-step approach to fine-tune a Llama 2 model on a custom dataset.

Maxime Labonne

2023년 11월 17일

무료 계정 만들기

실습 문제하고 트랙하세요.

Google로 계속하기계속 진행하시면 다음 사항에 동의하는 것으로 간주됩니다: 이용약관 및 개인정보 처리방침 그리고 귀하의 데이터가 미국에 저장된다는 점에 동의하는 것으로 간주됩니다.

데이터셋 보기 ↗

The advent of large language models has taken the AI world by storm. Outside of proprietary foundation models like GPT-4, open-source models are playing a pivotal role in driving the AI revolution forward, democratizing access for anyone looking to leverage these models in production. One of the biggest challenges in generating high-quality output from open-source models rests in fine-tuning, where we improve their outputs based on a series of instructions.

In this session, we take a step-by-step approach to fine-tune a Llama 2 model on a custom dataset. First, we build our own dataset using techniques to remove duplicates and analyze the number of tokens. Then, we fine-tune the Llama 2 model using state-of-the art techniques from the Axolotl library. Finally, we see how to run our fine-tuned model and evaluate its performance.

Key Takeaways: