ETL in Python
Leverage your Python and SQL knowledge to create an ETL pipeline to ingest, transform, and load data into a database.
Start Course for Free4 Hours16 Videos48 Exercises
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.Loved by learners at thousands of companies
Course Description
Build Your ETL Skills
Developing your ETL skills will improve your data engineering processes and means that you can work with data more efficiently. This course covers the foundations of creating pipelines to efficiently extract, transform, and load data into your company’s systems. You’ll get hands-on experience by helping a fictional private equity firm process sales data to make data-driven decisions when buying real estate.Learn to Set up ETL Pipelines
The course opens with an explanation of the ETL process and a deep-dive into data extraction. You’ll then move on to reviewing the ETL pipeline and the tools and techniques you need to transform data. Once the data is in your desired format, you can move it to a clean table and eventually move on to the last stage of the pipeline; loading your data ready to be used.You’ll finish the course by looking at how the ETL pipeline is used to build useful insight for the fictional company’s shareholders. You’ll look at more complex queries such as aggregation, averages, and max/min functions, before moving on to ways that you can translate raw SQL queries into readable Excel files.
Practice with Popular ETL Tools and Techniques
Throughout this course, you’ll be introduced to ETL tools and techniques that will simplify your workflow and create better structures for your data. These tools include SQLAlchemy, which can help you to perform insert and delete statements on your data, as well as offering aggregation functionality.- 1
Explore the data and requirements
FreeIn this first chapter, you’ll be introduced to your role as a data engineer in a private equity fund. You'll be exposed to the whole ETL pipeline before deep-diving into its first phase: the extraction process.
- 2
Create the ETL foundations
In this chapter you're going to create some important foundations for our ETL pipeline. In particular, along with data transformation, you'll start setting up the components needed to communicate with the database.
Let's talk with the database50 xpSQLAlchemy core components50 xpEngines and sessions100 xpDatabase tables50 xpTable class definition100 xpColumns definition100 xpData cleaning50 xpLower string and date format100 xpPrice and description100 xpPut transform operations together50 xpSetup base script100 xpCreate tables100 xpTransform 'em all!100 xp - 3
From raw to clean data
This chapter is all about moving transformed data to a clean table, from which insights can be built. You'll explore how to create a unique key to perform insert and delete statements on SQLAlchemy. At the end of this chapter you'll complete the load process, the last step of the ETL pipeline.
- 4
From clean data to meaningful insights
This chapter will show you how the data the ETL pipeline processes every month is used to build insights, readable by the fund’s shareholders. You'll explore key SQL components to build more complex queries and create these insights. You'll also explore libraries that will translate raw SQL queries into more readable Excel files.
Operators50 xpSales for Dublin and Cork100 xpFirst month 2021 sales100 xpSqlalchemy func50 xpAggregate functions50 xpAverage, max and min functions100 xpCreate the insights50 xpCreating the insights view100 xpHow many counties?50 xpWorking with Excel files50 xpCreate a simple Excel file100 xpAdd a table into Excel file100 xpExport monthly insights100 xpWrap-up50 xp
Datasets
Property price register 2021Collaborators
Stefano Francavilla
See MoreStefano is the CEO and co-founder of Geowox.
Stefano is the CEO and co-founder of Geowox, a company using AI and big data to value residential properties. In a previous life, he studied Computer Science at the polytechnic university of Milan while founding a software development company. He then worked as a product engineer at Intercom, advised portfolio startups at Growing Capital, a seed investment firm.
Don’t just take our word for it
*4.3
64%
18%
9%
9%
0%
Sort by
- Paniz F.14 days
Dear DataCamp, I recently completed the ETL with Python course and wanted to provide some feedback. Overall, I found the course provided a good introduction to ETL processes and practical examples to reinforce the concepts. The flow of material was easy to follow, likely because I had some prior experience with ETL pipelines before starting the course. A few areas that could potentially be improved: Adding more challenging hands-on exercises and projects would take the learning to the next level. For example, a capstone project focused on building an end-to-end ETL pipeline with Python would allow learners to apply all the skills covered. Including more real-world datasets and use cases beyond the simple retail dataset used throughout the course would make the material more applicable. Modeling the ETL process for a complex dataset would better mimic on-the-job tasks. Providing opportunities to work with different ETL tools beyond Pandas/Numpy would round out the curriculum. Even a brief overview of other Python libraries/packages used for ETL would help learners understand the broader ecosystem. Overall though, I found the course content well-structured and easy to grasp as a beginner. The combination of videos, hands-on exercises, and quizzes kept me engaged throughout. I enjoyed learning through DataCamp and hope this feedback will help improve future iterations of the course. Please let me know if I can provide any other details on my experience.
- Elías M.4 months
Pretty Good, aprticularly the inclusion of a more complex environment other than a plain editor of text that most courses offer. Either webinars or courses on how to set up a local environment for data science, how to connect to server and databases in other environment should really take the platform to a next level
- Amit K.6 months
Useful hands-on exercises that are getting built step by step throughout the course.
- Smit S.6 months
Excellent job done by Datacamp and Stefano in explaining the ETL concept from base and interactive code-along hands-on experience was exceptional. I would highly recommend to take this course to someone who has no idea about what is ETL and how data pipeline is created and executed.
- Nakul S.9 months
Excellent content and practical project oriented exercises. Learnt a lot of SQLAlchemy along the way! I especially liked that the instructor led us through a practical ETL project from scratch, so it also gives one a peek into his project folder structure and ETL script management best practices. I hope DataCamp adds projects (guided and unguided) which use python ETL, as I believe it is an essential skill for any data engineer/scientist/analyst. I am also looking forward to seeing web scraping based projects on the DataCamp platform which involve use of libraries like Beautiful Soup , Scrapy and so on.
FAQs
Join over 12 million learners and start ETL in Python today!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.