Build your ultimate AI agent
Course Description
Now Updated to Apache Airflow 3.1.6 - Delivering data on a schedule can be a manual process. You write scripts, add complex cron tasks, and try various ways to meet an ever-changing set of requirements—and it's even trickier to manage everything when working with teammates. Apache Airflow can remove this headache by adding scheduling, error handling, and reporting to your workflows. In this course, you'll master the basics of Apache Airflow and learn how to implement complex data engineering pipelines in production. You'll also learn how to use Directed Acyclic Graphs (Dags), automate data engineering workflows, and implement data engineering tasks in an easy and repeatable fashion—helping you to maintain your sanity.
Prerequisites
Curriculum
Course outline
1
Intro to Airflow
In this chapter, you’ll gain a complete introduction to the components of Apache Airflow and learn how and why you should use them.
- Introduction to Apache Airflow50 XP
- Dag counter50 XP
- Running your first Dag50 XP
- Airflow Dags50 XP
- Creating a simple Dag100 XP
- Working with Dags and the Airflow shell50 XP
- Examining Airflow commands50 XP
- Testing a task in Airflow50 XP
- Airflow web interface50 XP
- Navigating the Airflow UI50 XP
- Examining Dags with the Airflow UI50 XP
- Dag errors50 XP
2
Building Dags in Airflow
What’s up Dag? Now it’s time to learn the basics of implementing Airflow Dags. Through hands-on activities, you’ll learn how to set up and deploy operators, tasks, and scheduling.
3
Maintaining and monitoring Airflow workflows
In this chapter, you’ll learn how to save yourself time using Airflow components such as sensors and executors while monitoring and troubleshooting Airflow workflows.
4
Controlling Dag Logic
Put it all together. In this final chapter, you’ll apply everything you've learned to build a production-quality workflow in Airflow.
Introduction to Apache Airflow in Python
Course
Complete
