Skip to main content
HomeSparkFeature Engineering with PySpark

Feature Engineering with PySpark

Learn the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering.

Start Course for Free
4 hours16 videos60 exercises14,662 learnersTrophyStatement of Accomplishment

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.
Group

Training 2 or more people?

Try DataCamp for Business

Loved by learners at thousands of companies


Course Description

The real world is messy and your job is to make sense of it. Toy datasets like MTCars and Iris are the result of careful curation and cleaning, even so the data needs to be transformed for it to be useful for powerful machine learning algorithms to extract meaning, forecast, classify or cluster. This course will cover the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering. With size of datasets now becoming ever larger, let's use PySpark to cut this Big Data problem down to size!
For Business

GroupTraining 2 or more people?

Get your team access to the full DataCamp library, with centralized reporting, assignments, projects and more
Try DataCamp for BusinessFor a bespoke solution book a demo.

In the following Tracks

Big Data with PySpark

Go To Track
  1. 1

    Exploratory Data Analysis

    Free

    Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!

    Play Chapter Now
    Where to Begin
    50 xp
    Where to begin?
    50 xp
    Check Version
    100 xp
    Load in the data
    100 xp
    Defining A Problem
    50 xp
    What are we predicting?
    100 xp
    Verifying Data Load
    100 xp
    Verifying DataTypes
    100 xp
    Visually Inspecting Data / EDA
    50 xp
    Using Corr()
    100 xp
    Using Visualizations: distplot
    100 xp
    Using Visualizations: lmplot
    100 xp
  2. 3

    Feature Engineering

    In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.

    Play Chapter Now
For Business

GroupTraining 2 or more people?

Get your team access to the full DataCamp library, with centralized reporting, assignments, projects and more

In the following Tracks

Big Data with PySpark

Go To Track

datasets

2017 St Paul MN Real Estate Dataset

collaborators

Collaborator's avatar
Adrián Soto
Collaborator's avatar
Nick Solomon
John Hogue HeadshotJohn Hogue

Lead Data Scientist, General Mills

See More

What do other learners have to say?

Join over 14 million learners and start Feature Engineering with PySpark today!

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.