Skip to main content

Course

Machine Learning with Tree-Based Models in Python

IntermediateSkill Level

4.8+

Updated 12/2025

In this course, you'll learn how to use tree-based models and ensembles for regression and classification using scikit-learn.

Start Course for Free

PythonMachine Learning5 hr15 videos57 Exercises4,650 XP110K+Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Training 2 or more people?

Try DataCamp for Business

Course Description

Decision trees are supervised learning models used for problems involving classification and regression. Tree models present a high flexibility that comes at a price: on one hand, trees are able to capture complex non-linear relationships; on the other hand, they are prone to memorizing the noise present in a dataset. By aggregating the predictions of trees that are trained differently, ensemble methods take advantage of the flexibility of trees while reducing their tendency to memorize noise. Ensemble methods are used across a variety of fields and have a proven track record of winning many machine learning competitions. In this course, you'll learn how to use Python to train decision trees and tree-based models with the user-friendly scikit-learn machine learning library. You'll understand the advantages and shortcomings of trees and demonstrate how ensembling can alleviate these shortcomings, all while practicing on real-world datasets. Finally, you'll also understand how to tune the most influential hyperparameters in order to get the most out of your models.

Prerequisites

Supervised Learning with scikit-learn

1

Classification and Regression Trees

Classification and Regression Trees (CART) are a set of supervised learning models used for problems involving classification and regression. In this chapter, you'll be introduced to the CART algorithm.

Decision tree for classification

Train your first classification tree

Evaluate the classification tree

Logistic regression vs classification tree

Classification tree Learning

Growing a classification tree

Using entropy as a criterion

Entropy vs Gini index

Decision tree for regression

Train your first regression tree

Evaluate the regression tree

Linear regression vs regression tree

2

The Bias-Variance Tradeoff

The bias-variance tradeoff is one of the fundamental concepts in supervised machine learning. In this chapter, you'll understand how to diagnose the problems of overfitting and underfitting. You'll also be introduced to the concept of ensembling where the predictions of several models are aggregated to produce predictions that are more robust.

Generalization Error

Complexity, bias and variance

Overfitting and underfitting

Diagnose bias and variance problems

Instantiate the model

Evaluate the 10-fold CV error

Evaluate the training error

High bias or high variance?

Ensemble Learning

Define the ensemble

Evaluate individual classifiers

Better performance with a Voting Classifier

3

Bagging and Random Forests

Bagging is an ensemble method involving training the same algorithm many times using different subsets sampled from the training data. In this chapter, you'll understand how bagging can be used to create a tree ensemble. You'll also learn how the random forests algorithm can lead to further ensemble diversity through randomization at the level of each split in the trees forming the ensemble.

Define the bagging classifier

Evaluate Bagging performance

Out of Bag Evaluation

Prepare the ground

OOB Score vs Test Set Score

Random Forests (RF)

Train an RF regressor

Evaluate the RF regressor

Visualizing features importances

4

Boosting

Boosting refers to an ensemble method in which several models are trained sequentially with each model learning from the errors of its predecessors. In this chapter, you'll be introduced to the two boosting methods of AdaBoost and Gradient Boosting.

Define the AdaBoost classifier

Train the AdaBoost classifier

Evaluate the AdaBoost classifier

Gradient Boosting (GB)

Define the GB regressor

Train the GB regressor

Evaluate the GB regressor

Stochastic Gradient Boosting (SGB)

Regression with SGB

Train the SGB regressor

Evaluate the SGB regressor

5

Model Tuning

The hyperparameters of a machine learning model are parameters that are not learned from data. They should be set prior to fitting the model to the training set. In this chapter, you'll learn how to tune the hyperparameters of a tree-based model using grid search cross validation.

Tuning a CART's Hyperparameters

Tree hyperparameters

Set the tree's hyperparameter grid

Search for the optimal tree

Evaluate the optimal tree

Tuning a RF's Hyperparameters

Random forests hyperparameters

Set the hyperparameter grid of RF

Search for the optimal forest

Evaluate the optimal forest

Congratulations!

Machine Learning with Tree-Based Models in Python

Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance reviewEnroll Now

Don’t just take our word for it

*4.8

from 685 reviews

86%

13%

1%

0%

0%

Sort by

Eddie

15 hours ago

Maha

2 days ago

Dzsenifer

3 days ago

GÖZDE

3 days ago

Tezendra

5 days ago

Jerry

7 days ago

Eddie

Maha

Dzsenifer

FAQs

Is this course suitable for beginners?

Yes, this course is suitable for beginners! It provides a thorough introduction to decision trees and tree-based models through Python and the user-friendly scikit-learn machine learning library.

Will I receive a certificate at the end of the course?

Yes, you will receive a certificate of completion at the end of the course.

Does the course cover the concept of the bias-variance trade-off?

Yes, the course covers the concept of the bias-variance trade-off. You will gain insight into how to diagnose the problems of overfitting and underfitting, and understand how ensembling can alleviate these shortcomings.

Does the course provide an understanding of how to tune hyperparameters?

Yes, the course provides a thorough understanding of how to tune the hyperparameters of a tree-based model using grid search cross-validation.

Who will benefit from this course?

This course is suitable for those interested in data science and machine learning who are looking to gain a better understanding of decision trees and tree-based models. It would provide an excellent foundation for those starting out in data science to get a better understanding of the fundamentals of machine learning and model tuning.

Join over 19 million learners and start Machine Learning with Tree-Based Models in Python today!

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Grow your data skills with DataCamp for Mobile

Make progress on the go with our mobile courses and daily 5-minute coding challenges.