Introduction to Airflow in Python

Learn how to to implement and schedule data engineering workflows.
Start Course for Free
4 Hours16 Videos55 Exercises11,977 Learners
4050 XP

Create Your Free Account

GoogleLinkedInFacebook
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA. You confirm you are at least 16 years old (13 if you are an authorized Classrooms user).

Loved by learners at thousands of companies


Course Description

Delivering data on a schedule can be a manual process. You write scripts, add complex cron tasks, and try various ways to meet an ever-changing set of requirements—and it’s even trickier to manage everything when working with teammates. Airflow can remove this headache by adding scheduling, error handling, and reporting to your workflows. In this course, you’ll master the basics of Airflow and learn how to implement complex data engineering pipelines in production. You'll also learn how to use Directed Acyclic Graphs (DAGs), automate data engineering workflows, and implement data engineering tasks in an easy and repeatable fashion—helping you to maintain your sanity.

  1. 1

    Intro to Airflow

    Free
    In this chapter, you’ll gain a complete introduction to the components of Apache Airflow and learn how and why you should use them.
    Play Chapter Now
  2. 2

    Implementing Airflow DAGs

    What’s up DAG? Now it’s time to learn the basics of implementing Airflow DAGs. Through hands-on activities, you’ll learn how to set up and deploy operators, tasks, and scheduling.
    Play Chapter Now
  3. 3

    Maintaining and monitoring Airflow workflows

    In this chapter, you’ll learn how to save yourself time using Airflow components such as sensors and executors while monitoring and troubleshooting Airflow workflows.
    Play Chapter Now
  4. 4

    Building production pipelines in Airflow

    Put it all together. In this final chapter, you’ll apply everything you've learned to build a production-quality workflow in Airflow.
    Play Chapter Now
In the following tracks
Data Engineer
Collaborators
Hadrien LacroixLis Sulmont
Mike Metzger Headshot

Mike Metzger

Data Engineer Consultant @ Flexible Creations
Mike is a consultant focusing on data engineering and analysis using SQL, Python, and Apache Spark among other technologies. He has a 20+ year history of working with various technologies in the data, networking, and security space.
See More

What do other learners have to say?

I've used other sites—Coursera, Udacity, things like that—but DataCamp's been the one that I've stuck with.

Devon Edwards Joseph
Lloyds Banking Group

DataCamp is the top resource I recommend for learning data science.

Louis Maiden
Harvard Business School

DataCamp is by far my favorite website to learn from.

Ronald Bowers
Decision Science Analytics, USAA