Skip to main content

ETL in Python

Leverage your Python and SQL knowledge to create a pipeline ingesting, transforming and loading data into a database.

Start Course for Free
4 Hours16 Videos48 Exercises
3850 XP

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA. You confirm you are at least 16 years old (13 if you are an authorized Classrooms user).

Loved by learners at thousands of companies

Course Description

Want to grow your data engineering skills and more efficiently process big data? Well, it’s time to develop your ETL skills. In this course, you’ll learn the foundations of creating pipelines to efficiently extract, transform, and load data into the systems your company commonly uses. You’ll get hands-on experience by helping a fictional private equity firm process the sales data they need to make informed business decisions when buying real estate. Jump in, learn how to create ETL pipelines, and develop one of the most in-demand engineering skills needed in the market.

  1. 1

    Explore the data and requirements

    Free

    In this first chapter, you’ll be introduced to your role as a data engineer in a private equity fund. You'll be exposed to the whole ETL pipeline before deep-diving into its first phase: the extraction process.

    Play Chapter Now
    Introduction to ETL in Python
    50 xp
    The ETL process
    100 xp
    Downloading a ZIP file
    100 xp
    Exploring a ZIP file
    100 xp
    Ask the right questions
    50 xp
    Reading from a CSV file
    100 xp
    Writing to CSV
    100 xp
    Extracting
    50 xp
    Downloading the new dataset file from web
    100 xp
    Project folder structure
    50 xp
    Extract 'em all!
    100 xp
  2. 2

    Create the ETL foundations

    In this chapter you're going to create some important foundations for our ETL pipeline. In particular, along with data transformation, you'll start setting up the components needed to communicate with the database.

    Play Chapter Now
    Let's talk with the database
    50 xp
    SQLAlchemy core components
    50 xp
    Engines and sessions
    100 xp
    Database tables
    50 xp
    Table class definition
    100 xp
    Columns definition
    100 xp
    Data cleaning
    50 xp
    Lower string and date format
    100 xp
    Price and description
    100 xp
    Put transform operations together
    50 xp
    Setup base script
    100 xp
    Create tables
    100 xp
    Transform 'em all!
    100 xp
  3. 3

    From raw to clean data

    This chapter is all about moving transformed data to a clean table, from which insights can be built. You'll explore how to create a unique key to perform insert and delete statements on SQLAlchemy. At the end of this chapter you'll complete the load process, the last step of the ETL pipeline.

    Play Chapter Now
    Unique key definition and clean table
    50 xp
    Date data type definition
    100 xp
    Unique key definition
    100 xp
    Insert and delete operations
    50 xp
    Querying
    100 xp
    Insert and delete
    100 xp
    Put load operations together
    50 xp
    Insert operation
    100 xp
    Delete operation
    100 xp
    Load 'em all!
    100 xp
  4. 4

    From clean data to meaningful insights

    This chapter will show you how the data the ETL pipeline processes every month is used to build insights, readable by the fund’s shareholders. You'll explore key SQL components to build more complex queries and create these insights. You'll also explore libraries that will translate raw SQL queries into more readable Excel files.

    Play Chapter Now
    Operators
    50 xp
    Sales for Dublin and Cork
    100 xp
    First month 2021 sales
    100 xp
    Sqlalchemy func
    50 xp
    Aggregate functions
    50 xp
    Average, max and min functions
    100 xp
    Create the insights
    50 xp
    Creating the insights view
    100 xp
    How many counties?
    50 xp
    Working with Excel files
    50 xp
    Create a simple Excel file
    100 xp
    Add a table into Excel file
    100 xp
    Export monthly insights
    100 xp
    Wrap-up
    50 xp

Datasets

Property price register 2021

Collaborators

Hadrien Lacroix

Prerequisites

Data Engineering for EveryoneIntroduction to Data EngineeringPython Data Science Toolbox (Part 2)
Stefano Francavilla Headshot

Stefano Francavilla

Stefano is the CEO and co-founder of Geowox.

Stefano is the CEO and co-founder of Geowox, a company using AI and big data to value residential properties. In a previous life, he studied Computer Science at the polytechnic university of Milan while founding a software development company. He then worked as a product engineer at Intercom, advised portfolio startups at Growing Capital, a seed investment firm.
See More

What do other learners have to say?

I've used other sites—Coursera, Udacity, things like that—but DataCamp's been the one that I've stuck with.

Devon Edwards Joseph
Lloyds Banking Group

DataCamp is the top resource I recommend for learning data science.

Louis Maiden
Harvard Business School

DataCamp is by far my favorite website to learn from.

Ronald Bowers
Decision Science Analytics, USAA