课程

使用 Python 的 ETL 和 ELT

中级技能水平

更新时间 2026年1月

学习使用提取、转换和加载原则构建高效、性能优异且可靠的数据管道。

免费开始课程

PythonData Engineering

4小时

14 视频

53 道练习

4,450 XP

37,235

成就证明

深受数千家公司学习者的喜爱

需要团队培训？

企业版试用

课程描述

通过数据管道赋能分析

数据管道是每个强大数据平台的基础。构建这些管道是数据工程师的一项核心技能，他们为准备迈向数据驱动未来的企业提供了巨大的价值。这门入门课程将帮助你磨练构建高效、性能优异且可靠的数据管道所需的技能。

构建和维护 ETL 解决方案

在本课程中，您将深入了解构建数据管道的完整流程。您将提升技能，利用 Python 库，如 pandas 和 json，从结构化和非结构化来源中提取数据，然后再将其转换并持久化，以供后续使用。在此过程中，你将掌握架构图、单元测试和监控等实用工具与技术，这些都将帮助你的数据管道脱颖而出。随着学习的深入，你将通过动手练习来检验新掌握的技能。

为数据工作流加速赋能

完成本课程后，你将能够设计、开发并使用数据管道，在工作、新职业或个人项目中大幅提升你的数据工作流程。

先决条件

Data Warehousing Concepts Streamlined Data Ingestion with pandas

1

Introduction to Data Pipelines

Get ready to discover how data is collected, processed, and moved using data pipelines. You will explore the qualities of the best data pipelines, and prepare to design and build your own.

Introduction to ETL and ELT Pipelines

Running an ETL Pipeline

ELT in Action

ETL and ELT Pipelines

Building ETL and ELT Pipelines

Building an ETL Pipeline

The "T" in ELT

Extracting, Transforming, and Loading Student Scores Data

2

Building ETL Pipelines

Dive into leveraging pandas to extract, transform, and load data as you build your first data pipelines. Learn how to make your ETL logic reusable, and apply logging and exception handling to your pipelines.

Extracting data from structured sources

Extracting data from parquet files

Pulling data from SQL databases

Building functions to extract data

Transforming data with pandas

Filtering pandas DataFrames

Transforming sales data with pandas

Validating data transformations

Persisting data with pandas

Loading sales data to a CSV file

Customizing a CSV file

Persisting data to files

Monitoring a data pipeline

Logging within a data pipeline

Handling exceptions when loading data

Monitoring and alerting within a data pipeline

3

Advanced ETL Techniques

Supercharge your workflow with advanced data pipelining techniques, such as working with non-tabular data and persisting DataFrames to SQL databases. Discover tooling to tackle advanced transformations with pandas, and uncover best-practices for working with complex data.

Extracting non-tabular data

Ingesting JSON data with pandas

Reading JSON data into memory

Transforming non-tabular data

Iterating over dictionaries

Parsing data from dictionaries

Transforming JSON data

Transforming and cleaning DataFrames

Advanced data transformation with pandas

Filling missing values with pandas

Grouping data with pandas

Applying advanced transformations to DataFrames

Loading data to a SQL database with pandas

Loading data to a Postgres database

Validating data loaded to a Postgres Database

4

Deploying and Maintaining a Data Pipeline

In this final chapter, you’ll create frameworks to validate and test data pipelines before shipping them into production. After you’ve tested your pipeline, you’ll explore techniques to run your data pipeline end-to-end, all while allowing for visibility into pipeline performance.

Manually testing a data pipeline

Testing data pipelines

Validating a data pipeline at "checkpoints"

Testing a data pipeline end-to-end

Unit-testing a data pipeline

Validating a data pipeline with assert

Writing unit tests with pytest

Creating fixtures with pytest

Unit testing a data pipeline with fixtures

Running a data pipeline in production

Orchestration and ETL tools

Data pipeline architecture patterns

Running a data pipeline end-to-end

Congratulations!

使用 Python 的 ETL 和 ELT

课程完成

获得成就证明

将此证书添加到您的 LinkedIn 档案、简历或履历中
在社交媒体和绩效评估中分享立即注册

加入超过19百万学习者，今天就开始使用 Python 的 ETL 和 ELT！

通过 DataCamp for Mobile 提升您的数据技能

随时随地通过我们的移动课程和每日 5 分钟编程挑战提升技能。