专业数据工程师 在 Python 中

了解现代数据架构的关键组件，从摄取和服务到治理与编排。

Unix 命令行帮助用户以新方式组合现有程序、自动化重复任务，并在集群和云上运行程序。

掌握虚拟机、容器、Docker 和 Kubernetes 的基础知识。 了解差异，快速上手！

本课程介绍 dbt，用于数据建模、转换、测试和构建文档。

掌握面向对象编程（OOP）的基础概念，构建自定义类和对象！

掌握 NoSQL，提升数据工作流效率。 学习 Snowflake 处理大数据，使用 Postgres JSON 处理文档数据，并用 Redis 处理键值数据。

在这门 DevOps 入门课程中，你将掌握 DevOps 基础，并学习提升生产力的关键概念、工具和技术。

掌握 Python 测试： 掌握方法、创建检查，并使用 pytest 和 unittest 确保代码无错误。

Sharpen your debugging skills to enhance sales data accuracy.

了解 Docker 入门，掌握它在数据专业人士工具箱中的重要性。 了解 Docker 容器、镜像等内容。

精通 PySpark，轻松处理大数据——学习处理、查询和优化海量数据集，释放强大分析能力！

This chapter introduces the exciting world of Big Data, as well as the various concepts and different frameworks for processing Big Data. You will understand why Apache Spark is considered the best framework for BigData.

The main abstraction Spark provides is a resilient distributed dataset (RDD), which is the fundamental and backbone data type of this engine. This chapter introduces RDDs and shows how RDDs can be created and executed using RDD Transformations and Actions.

In this chapter, you'll learn about Spark SQL which is a Spark module for structured data processing. It provides a programming abstraction called DataFrames and can also act as a distributed SQL query engine. This chapter shows how Spark SQL allows you to use DataFrames in Python.

Step into a data engineer's shoes and master data cleaning with PySpark on an e-commerce orders dataset!

In this chapter, we learn how to download data files from web servers via the command line. In the process, we also learn about documentation manuals, option flags, and multi-file processing.

In the last chapter, we bridge the connection between command line and other data science languages and learn how they can work together. Using Python as a case study, we learn to execute Python on the command line, to install dependencies using the package manager pip, and to build an entire model pipeline using the command line.

了解批处理与流式处理的区别、流式系统扩展及实际应用。

精通 Apache Kafka！从核心概念到高级架构，学习创建、管理和排查 Kafka，解决真实世界的数据流挑战！

在本课程中，你将学习 Kubernetes 基础，并使用 Manifests 和 kubectl 指令部署和编排容器。

Understand how data engineering can impact your business.