Interactive Course

Extreme Gradient Boosting with XGBoost

Learn the fundamentals of gradient boosting and build state-of-the-art machine learning models using XGBoost to solve classification and regression problems.

Start Course For Free

4 hours
16 Videos
49 Exercises
12,662 Participants
3,750 XP

Create Your Free Account

By continuing you accept the Terms of Use and Privacy Policy, that your data will be stored outside of the EU, and that you are 16 years or older.

Loved by learners at thousands of top companies:

Course Description

Do you know the basics of supervised learning and want to learn to use state-of-the-art models on real-world datasets? Gradient boosting is currently one of the most popular techniques for efficient modeling of tabular datasets of all sizes. XGboost is a very fast, scalable implementation of gradient boosting that has taken data science by storm, with models using XGBoost regularly winning many online data science competitions and used at scale across different industries. In this course, you'll learn how to use this powerful library alongside pandas and scikit-learn to build and tune supervised learning models. You'll work with real-world datasets to solve classification as well as regression problems.

This chapter will introduce you to the fundamental idea behind XGBoost - boosted learners. Once you understand how XGBoost works, you'll apply it to solve a common classification problem found in industry, namely, predicting whether a customer will stop being a customer at some point in the future.

This chapter will teach you how to make your XGBoost models as performant as possible. You'll learn about the variety of parameters that can be adjusted to alter the behavior of XGBoost and how to tune them efficiently so that you can supercharge the performance of your models!

After a brief review of supervised regression, you'll apply XGBoost to the regression task of predicting house prices in Ames, Iowa. Along the way, you'll learn about the two kinds of base learners that XGboost can use as its weak learners, and review how to evaluate the quality of your regression models.

Here, you'll take your XGBoost skills to the next level by incorporating your models into two end-to-end machine learning pipelines. You'll learn how to tune the most important XGBoost hyperparameters efficiently within a pipeline, as well as be introduced to some more advanced preprocessing techniques, all the while applying everything you've learned in the first three chapters. Enjoy!

1
Classification with XGBoost
Free

This chapter will introduce you to the fundamental idea behind XGBoost - boosted learners. Once you understand how XGBoost works, you'll apply it to solve a common classification problem found in industry, namely, predicting whether a customer will stop being a customer at some point in the future.
View Chapter Details Play Chapter Now
Regression with XGBoost

After a brief review of supervised regression, you'll apply XGBoost to the regression task of predicting house prices in Ames, Iowa. Along the way, you'll learn about the two kinds of base learners that XGboost can use as its weak learners, and review how to evaluate the quality of your regression models.
View Chapter Details Play Chapter Now
Fine-tuning your XGBoost model

This chapter will teach you how to make your XGBoost models as performant as possible. You'll learn about the variety of parameters that can be adjusted to alter the behavior of XGBoost and how to tune them efficiently so that you can supercharge the performance of your models!
View Chapter Details Play Chapter Now
Using XGBoost in pipelines

Here, you'll take your XGBoost skills to the next level by incorporating your models into two end-to-end machine learning pipelines. You'll learn how to tune the most important XGBoost hyperparameters efficiently within a pipeline, as well as be introduced to some more advanced preprocessing techniques, all the while applying everything you've learned in the first three chapters. Enjoy!
View Chapter Details Play Chapter Now

What do other learners have to say?

“I've used other sites, but DataCamp's been the one that I've stuck with.”

Devon Edwards Joseph

Lloyd's Banking Group

“DataCamp is the top resource I recommend for learning data science.”

Louis Maiden

Harvard Business School

“DataCamp is by far my favorite website to learn from.”

Ronald Bowers

Decision Science Analytics @ USAA

Sergey Fogelson

VP of Analytics and Measurement Sciences, Viacom

Sergey began his career as an academic at Dartmouth College, where he researched the neural bases of visual category learning and obtained his Ph.D. in Cognitive Neuroscience. After leaving academia, Sergey got into the rapidly growing startup scene in the NYC metro area, where he has worked as a data scientist in digital advertising, cybersecurity, finance, and media. He is heavily involved in the NYC-area teaching community and has taught courses at various bootcamps, and has been a volunteer teacher in computer science through TEALSK12. When Sergey is not working or teaching, he is probably hiking. (He thru-hiked the Appalachian trail before graduate school).

If you type... We will search for...

Press enter to see all search results

Courses

Tracks

Instructors

Extreme Gradient Boosting with XGBoost

Create Your Free Account

Loved by learners at thousands of top companies:

Course Description

Classification with XGBoost

Welcome to the course!

Which of these is a classification problem?

Which of these is a binary classification problem?

Introducing XGBoost

XGBoost: Fit/Predict

What is a decision tree?

Decision trees

What is Boosting?

Measuring accuracy

Measuring AUC

When should I use XGBoost?

Using XGBoost

Fine-tuning your XGBoost model

Why tune your model?

When is tuning your model a bad idea?

Tuning the number of boosting rounds

Automated boosting round selection using early_stopping

Overview of XGBoost's hyperparameters

Tuning eta

Tuning max_depth

Tuning colsample_bytree

Review of Grid Search and Random Search

Grid Search with XGBoost

Random Search with XGBoost

Limits of Grid Search and Random Search

When Should you Use Grid Search and Random Search?

Regression with XGBoost

Regression review

Which of these is a regression problem?

Objective (loss) functions and base learners

Decision trees as base learners

Linear base learners

Evaluating model quality

Regularization and base learners in XGBoost

Using regularization in XGBoost

Visualizing individual XGBoost trees

Visualizing feature importances: What features are most important in my dataset

Using XGBoost in pipelines

Review of pipelines using sklearn

Exploratory data analysis

Encoding categorical columns I: LabelEncoder

Encoding categorical columns II: OneHotEncoder

Encoding categorical columns III: DictVectorizer

Preprocessing within a pipeline

Incorporating XGBoost into pipelines

Cross-validating your XGBoost model

Kidney disease case study I: Categorical Imputer

Kidney disease case study II: Feature Union

Kidney disease case study III: Full pipeline

Tuning XGBoost hyperparameters

Bringing it all together

Final Thoughts