Skip to main content

Course

Extreme Gradient Boosting with XGBoost

IntermediateSkill Level

4.8+

Updated 03/2026

Learn the fundamentals of gradient boosting and build state-of-the-art machine learning models using XGBoost to solve classification and regression problems.

Start Course for Free

PythonMachine Learning4 hr16 videos49 Exercises3,750 XP60,118Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Training 2 or more people?

Try DataCamp for Business

Course Description

Do you know the basics of supervised learning and want to use state-of-the-art models on real-world datasets? Gradient boosting is currently one of the most popular techniques for efficient modeling of tabular datasets of all sizes. XGboost is a very fast, scalable implementation of gradient boosting, with models using XGBoost regularly winning online data science competitions and being used at scale across different industries. In this course, you'll learn how to use this powerful library alongside pandas and scikit-learn to build and tune supervised learning models. You'll work with real-world datasets to solve classification and regression problems.

Prerequisites

Supervised Learning with scikit-learn

1

Classification with XGBoost

This chapter will introduce you to the fundamental idea behind XGBoost—boosted learners. Once you understand how XGBoost works, you'll apply it to solve a common classification problem found in industry: predicting whether a customer will stop being a customer at some point in the future.

Welcome to the course!

Which of these is a classification problem?

Which of these is a binary classification problem?

Introducing XGBoost

XGBoost: Fit/Predict

What is a decision tree?

Decision trees

What is Boosting?

Measuring accuracy

Measuring AUC

When should I use XGBoost?

Using XGBoost

2

Regression with XGBoost

After a brief review of supervised regression, you'll apply XGBoost to the regression task of predicting house prices in Ames, Iowa. You'll learn about the two kinds of base learners that XGboost can use as its weak learners, and review how to evaluate the quality of your regression models.

Regression review

Which of these is a regression problem?

Objective (loss) functions and base learners

Decision trees as base learners

Linear base learners

Evaluating model quality

Regularization and base learners in XGBoost

Using regularization in XGBoost

Visualizing individual XGBoost trees

Visualizing feature importances: What features are most important in my dataset

3

Fine-tuning your XGBoost model

This chapter will teach you how to make your XGBoost models as performant as possible. You'll learn about the variety of parameters that can be adjusted to alter the behavior of XGBoost and how to tune them efficiently so that you can supercharge the performance of your models.

Why tune your model?

When is tuning your model a bad idea?

Tuning the number of boosting rounds

Automated boosting round selection using early_stopping

Overview of XGBoost's hyperparameters

Tuning max_depth

Tuning colsample_bytree

Review of grid search and random search

Grid search with XGBoost

Random search with XGBoost

Limits of grid search and random search

When should you use grid search and random search?

4

Using XGBoost in pipelines

Take your XGBoost skills to the next level by incorporating your models into two end-to-end machine learning pipelines. You'll learn how to tune the most important XGBoost hyperparameters efficiently within a pipeline, and get an introduction to some more advanced preprocessing techniques.

Review of pipelines using sklearn

Exploratory data analysis

Encoding categorical columns I: LabelEncoder

Encoding categorical columns II: OneHotEncoder

Encoding categorical columns III: DictVectorizer

Preprocessing within a pipeline

Incorporating XGBoost into pipelines

Cross-validating your XGBoost model

Kidney disease case study I: Categorical Imputer

Kidney disease case study II: Feature Union

Kidney disease case study III: Full pipeline

Tuning XGBoost hyperparameters

Bringing it all together

Final Thoughts

Extreme Gradient Boosting with XGBoost

Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance reviewEnroll Now

Don’t just take our word for it

*4.8

from 235 reviews

86%

13%

1%

0%

0%

Sort by

Mudathir

2 days ago

Tezendra

2 days ago

Daniel

4 days ago

Jerry

5 days ago

Gyula

7 days ago

Omar Joaquin

last week

Mudathir

Daniel

Jerry

FAQs

What makes XGBoost different from other machine learning methods?

XGBoost is a fast, scalable implementation of gradient boosting that excels on tabular data. It regularly wins data science competitions and is widely used across industries for its performance.

What Python libraries are used alongside XGBoost in this course?

You will use XGBoost together with pandas for data manipulation and scikit-learn for building pipelines, preprocessing data, and evaluating model performance.

What types of prediction problems will I solve?

You will apply XGBoost to classification tasks like predicting customer churn and regression tasks like predicting house prices using the Ames, Iowa housing dataset.

Does the course cover hyperparameter tuning for XGBoost?

Yes. Chapter 3 teaches you the variety of XGBoost parameters and efficient tuning strategies to optimize model performance without exhaustive manual experimentation.

Will I learn to use XGBoost within scikit-learn pipelines?

Yes. The final chapter teaches you to incorporate XGBoost into end-to-end pipelines with preprocessing steps and hyperparameter tuning integrated into a single workflow.

Join over 19 million learners and start Extreme Gradient Boosting with XGBoost today!

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Grow your data skills with DataCamp for Mobile

Make progress on the go with our mobile courses and daily 5-minute coding challenges.