Vai al contenuto principale

Corso

Scaling and Optimizing Data Pipelines with Polars

IntermedioLivello di competenza

Aggiornato 06/2026

Learn to optimize, scale, and test Polars data pipelines for production-ready performance.

Inizia Il Corso Gratis

PythonData Manipulation

4 h

15 video

56 Esercizi

4,800 XP

Attestato di conseguimento

Preferito dagli studenti di migliaia di aziende

Formare un team?

Prova per il Business

Descrizione del corso

Take your Polars skills to production scale. Learn to read query plans and unlock the optimizer's full potential, work efficiently with Parquet, CSV, and database sources, and exploit advanced dtypes like List, Struct, Categorical, and Enum. You'll also stream large queries to disk, process data in batches, and build testable pipelines with built-in assertions. By the end, you'll be equipped to build high-performing data workflows that handle datasets of any size.

Prerequisiti

Data Transformation with Polars

1

Query Optimization Deep Dive

Learn how to keep queries lazy for maximum optimization, read and interpret query plans, and unlock fast paths with profiling and sorted data.

Effective query execution

Previewing the data lazily

Summarizing digital checkouts

Collecting two reports together

Optimization and query plans

Reading the naive plan

Reading the optimized plan

What query optimization actually does

Using optimization to full effect

Lazy pivot to wide format

Profiling a lazy query

Fast-path filter on sorted data

Inizia Il Capitolo

2

Efficient Data Input and Output

Learn how to read and write Parquet files, parse messy CSVs, scan multifile and hive-partitioned datasets, and query databases from Polars.

Working with Parquet files

Inspecting a Parquet schema

Summarizing Parquet data

Writing a Parquet snapshot

Parsing CSVs

Parsing a messy CSV

Overriding the inferred schema

Ignoring parse errors

Working with Multifile Datasets

Scanning multiple files

Inserting missing columns

Scanning a hive-partitioned dataset

Working with Databases

Reading from SQLite with a URI

Appending to a SQLite history table

Querying DuckDB into Polars

Inizia Il Capitolo

3

Advanced Dtypes for Optimal Analysis

This chapter covers working with List and Struct columns, encoding repeated strings as Categorical and Enum dtypes, and reducing memory use through numeric downcasting.

Handling array data

Extracting a primary genre

Counting genres per movie

Counting genre popularity

Working with nested columns

Unnesting production_country

Renaming country fields

Creating an Object dtype

Using Categorical and Enum dtypes

Creating a Categorical column

Using a categorical expression

Creating an Enum column

Reducing memory pressure

Inspecting numeric range

Downcasting numeric columns

Estimating DataFrame size

Inizia Il Capitolo

4

Working with Polars at Scale

Learn how to use the streaming and GPU engines, sink large query results directly to disk with partitioning, and test pipelines with Polars' built-in assertions.

Working with the streaming and GPU engines

Targeting the streaming engine

Processing query results in batches

Sinking large queries to disk

Sinking a cleaned extract to Parquet

Multiplexing Parquet sinks

Using a custom batch sink

Testing Polars Pipelines

Comparing DataFrames with equals

Testing schemas

Testing full DataFrames

Congratulations!

Inizia Il Capitolo

Scaling and Optimizing Data Pipelines with Polars

Corso
completato

Ottieni Attestato di conseguimento

Aggiungi questa certificazione al tuo profilo LinkedIn, al curriculum o al CV
Condividila sui social e nella valutazione delle tue performanceIscriviti Ora

Unisciti a oltre 19 milioni di studenti e inizia Scaling and Optimizing Data Pipelines with Polars oggi!

Aumenta le tue competenze sui dati con l'app di DataCamp

Avanza ovunque ti trovi con i nostri corsi per dispositivi mobili e le nostre sfide di programmazione quotidiane da 5 minuti.