After doing these courses, I feel confident creating professional visualizations and dashboards
Shoebur RahmanProcurement Specialist, Giz Bangladesh
Course Description
The real world is messy and your job is to make sense of it. Toy datasets like MTCars and Iris are the result of careful curation and cleaning, even so the data needs to be transformed for it to be useful for powerful machine learning algorithms to extract meaning, forecast, classify or cluster. This course will cover the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering. With size of datasets now becoming ever larger, let's use PySpark to cut this Big Data problem down to size!
Prerequisites
Curriculum
Course outline
1
Exploratory Data Analysis
Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!
- Where to Begin50 XP
- Where to begin?50 XP
- Check Version100 XP
- Load in the data100 XP
- Defining A Problem50 XP
- What are we predicting?100 XP
- Verifying Data Load100 XP
- Verifying DataTypes100 XP
- Visually Inspecting Data / EDA50 XP
- Using Corr()100 XP
- Using Visualizations: distplot100 XP
- Using Visualizations: lmplot100 XP
2
Wrangling with Spark Functions
Real data is rarely clean and ready for analysis. In this chapter learn to remove unneeded information, handle missing values and add additional data to your analysis.
3
Feature Engineering
In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.
4
Building a Model
In this chapter we'll learn how to choose which type of model we want. Then we will learn how to apply our data to the model and evaluate it. Lastly, we'll learn how to interpret the results and save the model for later!
Feature Engineering with PySpark
Course
Complete
