Skip to main content
HomeSpark

Course

Foundations of PySpark

IntermediateSkill Level
4.7+
585 reviews
Updated 03/2025
Learn to implement distributed data management and machine learning in Spark using the PySpark package.
Start Course for Free
SparkData Engineering4 hr45 Exercises3,850 XP150K+Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Group

Training 2 or more people?

Try DataCamp for Business

Course Description

In this course, you'll learn how to use Spark from Python! Spark is a tool for doing parallel computation with large datasets and it integrates well with Python. PySpark is the Python package that makes the magic happen. You'll use this package to work with data about flights from Portland and Seattle. You'll learn to wrangle this data and build a whole machine learning pipeline to predict whether or not flights will be delayed. Get ready to put some Spark in your Python code and dive into the world of high-performance machine learning!

Prerequisites

Introduction to Python
1

Getting to know PySpark

In this chapter, you'll learn how Spark manages data and how can you read and write tables from Python.
Start Chapter
2

Manipulating data

3

Getting started with machine learning pipelines

4

Model tuning and selection

Foundations of PySpark
Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance reviewEnroll Now

Don’t just take our word for it

*4.7
from 585 reviews
78%
18%
3%
0%
0%
  • Rafał
    11 hours ago

  • Gabrielle
    2 days ago

  • Osama
    3 days ago

  • Dung
    5 days ago

    okok

  • Monson
    last week

    NA

  • Natalia
    last week

Rafał

Gabrielle

"okok"

Dung

FAQs

Is this course suitable for beginners?

Yes! This course is ideal for those with little or no prior exposure to Spark and PySpark. You will learn all the basics you need to start using PySpark for data analysis.

Will I receive a certificate at the end of the course?

Yes, upon completing this course, you will receive a certificate from DataCamp.

Who will benefit from this course?

Data Scientists, Data Engineers, and DevOps Engineers who want to use Spark and PySpark for data analysis and machine learning models will benefit from this course.

What topics are covered in this course?

In this course you will learn about data wrangling, machine learning pipelines, model tuning and selection, and other related topics using PySpark.

Is knowledge of Python necessary for this course?

Prior experience with Python is beneficial but not absolutely necessary. If you are comfortable with Python basics, you will be able to get the most out of this course.

What kind of useful applications can I create with the skills I learn in this course?

You will gain the skills necessary to create applications that can read and write data in Spark, perform data wrangling, and build and tune machine learning pipelines.

    Join over 19 million learners and start Foundations of PySpark today!

    Create Your Free Account

    or

    By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

    Grow your data skills with DataCamp for Mobile

    Make progress on the go with our mobile courses and daily 5-minute coding challenges.