본문으로 바로가기

강의

Reinforcement Learning from Human Feedback (RLHF)

고급기술 수준

업데이트됨 2024. 10.

인간적 가치를 충실히 반영하는 GenAI 모델을 설계하고, 최신 LLM을 직접 다루며 실무 역량을 키우십시오.

무료로 강의 시작

PythonArtificial Intelligence

4시간

13 동영상

38 연습 문제

2,900 XP

3,663

성취 증명서

수천 개 기업의 학습자들이 사랑하는

팀을 교육하시나요?

비즈니스용으로 체험해 보세요

강의 설명

이 강의에서는 Generative AI의 효율성과 인간 전문성의 통찰을 결합한 Reinforcement Learning from Human Feedback를 다룹니다. GenAI 모델이 인간의 가치와 선호를 충실히 반영하도록 만드는 방법을 배우고, LLM을 직접 다루며 실습해 보세요. 또한 보상 모델의 복잡성을 이해하고, LLM을 기반으로 실제 환경에 잘 학습하고 적응하는 AI를 구축하는 방법을 익힙니다.

선수 조건

Deep Reinforcement Learning in Python

1

Foundational Concepts

This chapter introduces the basics of Reinforcement Learning with Human Feedback (RLHF), a technique that uses human input to help AI models learn more effectively. Get started with RLHF by understanding how it differs from traditional reinforcement learning and why human feedback can enhance AI performance in various domains.

Introduction to RLHF

Text generation with RLHF

Classifying generated text for RLHF

RL vs. RLHF

Exploring pre-trained LLMs

Tokenize a text dataset

Fine-tuning for review classification

Preparing data for RLHF

Preparing the preference dataset

Extracting prompts

2

Gathering Human Feedback

Discover how to set up systems for gathering human feedback in this Chapter. Learn best practices for collecting high-quality data, from pairwise comparisons to uncertainty sampling, and explore strategies for enhancing your data collection.

Methods for high-quality feedback gathering

Understanding comparison and rating in RLHF

Comparing slogans for a gym campaign

Measuring feedback quality and relevance

Low confidence

K-means for feedback clustering

Active learning

Implementing an active learning pipeline

Active learning loop

3

Tuning Models with Human Feedback

In this Chapter, you'll get into the core of Reinforcement Learning from Human Feedback training. This includes exploring fine-tuning with PPO, techniques to train efficiently, and handling potential divergences from your metrics' objectives.

Reward models explored

Initializing the reward

Setting up the reward trainer

Training with PPO

Initialize the PPO trainer

PPO fine-tuning

Efficient fine-tuning in RLHF

Prepare for 8-bit Training

Train with LoRA

4

Model Evaluation

Explore key techniques for assessing and improving model performance in this last Chapter of Reinforcement Learning from Human Feedback (RLHF): from fine-tuning metrics to incorporating diverse feedback sources, you'll be provided with a comprehensive toolkit to refine your models effectively.

Model metrics and adjustments

Mitigating negative KL divergence

Checking the reward model

Incorporating diverse feedback sources

Majority voting on multiple data sources

Unreliable data source identification

Evaluating RLHF models

Interpreting curves

Evaluating RLHF with metrics

Wrapping up your RLHF journey

Reinforcement Learning from Human Feedback (RLHF)

강의
완료

수료증 획득

LinkedIn 프로필, 이력서 또는 CV에 이 인증서를 추가하세요
소셜 미디어와 성과 평가에서 공유하세요지금 등록

19백만 명 이상의 학습자와 함께 Reinforcement Learning from Human Feedback (RLHF)을(를) 시작하세요!

DataCamp for Mobile을 통해 데이터 분석 능력을 향상시키세요.

모바일 강좌와 매일 5분 코딩 챌린지를 통해 이동 중에도 학습 효과를 높이세요.