Skip to main content

Introduction to Data Pipelines

This introductory course will help you hone the skills to build effective, performant, and reliable data pipelines.

Start Course for Free
4 Hours15 Videos57 Exercises
2,444 Learners

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies


Course Description

Empowering Analytics with Data Pipelines Data pipelines are at the foundation of all analytics projects. You’ve probably heard the age-old adage, that 90% of data science is cleaning and transforming data. This introductory course will help you hone the skills to build effective, performant, and reliable data pipelines. Data pipelines are at the foundation of every strong data platform. Building these pipelines is an essential skill for data engineers, who provide incredible value to a business ready to step into a data-driven future. This introductory course will help you hone the skills to build effective, performant, and reliable data pipelines. Building and Maintaining ETL Solutions Throughout this course, you’ll dive into the complete process of building a data pipeline. You’ll grow skills leveraging Python libraries such as `pandas` and `json` to extract data from structured and unstructured sources before it’s transformed and persisted for downstream use. Along the way, you’ll grow confidence tools and techniques such as architecture diagrams, unit-tests, and monitoring that will help to set your data pipelines out from the rest. As you progress, you’ll put your new-found skills to the test with hands-on exercises. Supercharge Data Workflows After completing this course, you’ll be ready to design, develop and use data pipelines to supercharge your data workflow in your job, new career, or personal project.
  1. 1

    Introduction to Data Pipelines

    Free

    Get ready to discover how data is collected, processed, and moved using data pipelines. You will explore the qualities of the best data pipelines, and prepare to design and build your own.

    Play Chapter Now
    Introducing data pipelines
    50 xp
    What is a data pipeline?
    50 xp
    Components of a data pipeline
    100 xp
    Producers and consumers of data pipelines
    100 xp
    Designing data pipelines
    50 xp
    Architecture diagrams for data pipelines
    50 xp
    Reading architecture diagrams
    50 xp
    Data pipeline design process
    100 xp
    Qualities of great data pipelines
    50 xp
    Building quality data pipelines
    50 xp
    Persisting data throughout a pipeline
    50 xp
    Qualities of sound data pipelines
    100 xp

Datasets

scores.csvschools_modified.csvamazon_sales_cleaned_sql.csvtax_rate_cleaned.csv

Collaborators

Collaborator's avatar
George Boorman
Collaborator's avatar
Arne Warnke
Collaborator's avatar
Anastasia Dvoryanchikova
Collaborator's avatar
Katerina Zahradova
Jake Roach HeadshotJake Roach

Data Engineer

Hi all! I'm Jake, a Data Engineer and DataCamp Instructor. I use Python and Airflow to extract, transform, and load data into a state-of-the-art data platform powered by Astronomer, AWS, MongoDB and Postgres. I'm born and raised in Buffalo, NY, so I'm used to seeing a Snowflake or two. When I'm not working with data, you can find me out at the golf course playing a quick nine holes before dark!
See More

What do other learners have to say?

Join over 12 million learners and start Introduction to Data Pipelines today!

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.