课程

Feature Engineering with PySpark

高级技能水平

更新时间 2026年1月

Learn the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering.

免费开始课程

SparkData Manipulation4 小时16 视频60 练习5,000 经验值17,615成就声明

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

深受数千家公司学习者的喜爱

培训2人或更多？

试用DataCamp for Business

课程描述

The real world is messy and your job is to make sense of it. Toy datasets like MTCars and Iris are the result of careful curation and cleaning, even so the data needs to be transformed for it to be useful for powerful machine learning algorithms to extract meaning, forecast, classify or cluster. This course will cover the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering. With size of datasets now becoming ever larger, let's use PySpark to cut this Big Data problem down to size!

先决条件

Supervised Learning with scikit-learn Introduction to PySpark

1

Exploratory Data Analysis

Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!

Where to Begin

50 经验值

Where to begin?

50 经验值

Check Version

100 经验值

Load in the data

100 经验值

Defining A Problem

50 经验值

What are we predicting?

100 经验值

Verifying Data Load

100 经验值

Verifying DataTypes

100 经验值

Visually Inspecting Data / EDA

50 经验值

Using Corr()

100 经验值

Using Visualizations: distplot

100 经验值

Using Visualizations: lmplot

100 经验值

2

Wrangling with Spark Functions

Real data is rarely clean and ready for analysis. In this chapter learn to remove unneeded information, handle missing values and add additional data to your analysis.

Dropping data

50 经验值

Dropping a list of columns

100 经验值

Using text filters to remove records

100 经验值

Filtering numeric fields conditionally

100 经验值

Adjusting Data

50 经验值

Custom Percentage Scaling

100 经验值

Scaling your scalers

100 经验值

Correcting Right Skew Data

100 经验值

Working with Missing Data

50 经验值

Visualizing Missing Data

100 经验值

Imputing Missing Data

100 经验值

Calculate Missing Percents

100 经验值

Getting More Data

50 经验值

A Dangerous Join

100 经验值

Spark SQL Join

100 经验值

Checking for Bad Joins

100 经验值

3

Feature Engineering

In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.

Feature Generation

50 经验值

Differences

100 经验值

100 经验值

Deeper Features

100 经验值

Time Features

50 经验值

Time Components

100 经验值

Joining On Time Components

100 经验值

100 经验值

Extracting Features

50 经验值

Extracting Text to New Features

100 经验值

Splitting & Exploding

100 经验值

Pivot & Join

100 经验值

Binarizing, Bucketing & Encoding

50 经验值

Binarizing Day of Week

100 经验值

100 经验值

One Hot Encoding

100 经验值

4

Building a Model

In this chapter we'll learn how to choose which type of model we want. Then we will learn how to apply our data to the model and evaluate it. Lastly, we'll learn how to interpret the results and save the model for later!

Choosing the Algorithm

50 经验值

Which MLlib Module?

50 经验值

Creating Time Splits

100 经验值

Adjusting Time Features

100 经验值

Feature Engineering Assumptions for RFR

50 经验值

Feature Engineering For Random Forests

50 经验值

Dropping Columns with Low Observations

100 经验值

Naively Handling Missing and Categorical Values

100 经验值

Building a Model

50 经验值

Building a Regression Model

100 经验值

Evaluating & Comparing Algorithms

100 经验值

Understanding Metrics

50 经验值

Interpreting, Saving & Loading

50 经验值

Interpreting Results

100 经验值

Saving & Loading Models

100 经验值

Final Thoughts

50 经验值

Feature Engineering with PySpark

课程完成

获得成就证明

将此证书添加到你的 LinkedIn 档案、简历或履历中
在社交媒体和绩效评估中分享立即注册

加入超过19百万学习者，今天就开始Feature Engineering with PySpark！

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

通过 DataCamp for Mobile 提升您的数据技能

随时随地通过我们的移动课程和每日 5 分钟编程挑战提升技能。