首页 Data Engineering

课程

Introduction to Apache Airflow in Python

高级技能水平

更新时间 2025年6月

Learn how to implement and schedule data engineering workflows.

免费开始课程

AirflowData Engineering4 小时16 视频55 练习4,050 经验值61,143成就声明

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

深受数千家公司学习者的喜爱

培训2人或更多？

试用DataCamp for Business

课程描述

Now Updated to Apache Airflow 2.7 - Delivering data on a schedule can be a manual process. You write scripts, add complex cron tasks, and try various ways to meet an ever-changing set of requirements—and it's even trickier to manage everything when working with teammates. Apache Airflow can remove this headache by adding scheduling, error handling, and reporting to your workflows. In this course, you'll master the basics of Apache Airflow and learn how to implement complex data engineering pipelines in production. You'll also learn how to use Directed Acyclic Graphs (DAGs), automate data engineering workflows, and implement data engineering tasks in an easy and repeatable fashion—helping you to maintain your sanity.

先决条件

Intermediate Python Introduction to Shell

1

Intro to Airflow

In this chapter, you’ll gain a complete introduction to the components of Apache Airflow and learn how and why you should use them.

Introduction to Apache Airflow

50 经验值

Testing a task in Airflow

50 经验值

Examining Airflow commands

50 经验值

Airflow DAGs

50 经验值

Defining a simple DAG

100 经验值

Working with DAGs and the Airflow shell

50 经验值

Troubleshooting DAG creation

100 经验值

Airflow web interface

50 经验值

Starting the Airflow webserver

50 经验值

Navigating the Airflow UI

50 经验值

Examining DAGs with the Airflow UI

50 经验值

2

Implementing Airflow DAGs

What’s up DAG? Now it’s time to learn the basics of implementing Airflow DAGs. Through hands-on activities, you’ll learn how to set up and deploy operators, tasks, and scheduling.

Airflow operators

50 经验值

Defining a BashOperator task

100 经验值

Multiple BashOperators

100 经验值

Airflow tasks

50 经验值

Define order of BashOperators

100 经验值

Determining the order of tasks

100 经验值

Troubleshooting DAG dependencies

50 经验值

Additional operators

50 经验值

Using the PythonOperator

100 经验值

More PythonOperators

100 经验值

EmailOperator and dependencies

100 经验值

Airflow scheduling

50 经验值

Schedule a DAG via Python

100 经验值

Deciphering Airflow schedules

100 经验值

Troubleshooting DAG runs

50 经验值

3

Maintaining and monitoring Airflow workflows

In this chapter, you’ll learn how to save yourself time using Airflow components such as sensors and executors while monitoring and troubleshooting Airflow workflows.

Airflow sensors

50 经验值

Sensors vs operators

100 经验值

Sensory deprivation

50 经验值

Airflow executors

50 经验值

Determining the executor

50 经验值

Executor implications

100 经验值

Debugging and troubleshooting in Airflow

50 经验值

DAGs in the bag

50 经验值

Missing DAG

100 经验值

SLAs and reporting in Airflow

50 经验值

Defining an SLA

100 经验值

Defining a task SLA

100 经验值

Generate and email a report

100 经验值

Adding status emails

100 经验值

4

Building production pipelines in Airflow

Put it all together. In this final chapter, you’ll apply everything you've learned to build a production-quality workflow in Airflow.

Working with templates

50 经验值

Creating a templated BashOperator

100 经验值

Templates with multiple arguments

100 经验值

More templates

50 经验值

Using lists with templates

100 经验值

Understanding parameter options

50 经验值

Sending templated emails

100 经验值

50 经验值

Define a BranchPythonOperator

100 经验值

Branch troubleshooting

50 经验值

Creating a production pipeline

50 经验值

Creating a production pipeline #1

100 经验值

Creating a production pipeline #2

100 经验值

Adding the final changes to your pipeline

100 经验值

Congratulations!

50 经验值

Introduction to Apache Airflow in Python

课程完成

获得成就证明

将此证书添加到你的 LinkedIn 档案、简历或履历中
在社交媒体和绩效评估中分享立即注册

加入超过19百万学习者，今天就开始Introduction to Apache Airflow in Python！

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

通过 DataCamp for Mobile 提升您的数据技能

随时随地通过我们的移动课程和每日 5 分钟编程挑战提升技能。