Accéder au contenu principal

Cours

Scaling and Optimizing Data Pipelines with Polars

IntermédiaireNiveau de compétence

Actualisé 06/2026

Learn to optimize, scale, and test Polars data pipelines for production-ready performance.

Commencer le cours gratuitement

PythonData Manipulation

4 h

15 vidéos

56 Exercices

4,800 XP

Certificat de formation

Apprécié par des utilisateurs provenant de milliers d'entreprises

Former une équipe ?

Essayez pour les entreprises

Description du cours

Take your Polars skills to production scale. Learn to read query plans and unlock the optimizer's full potential, work efficiently with Parquet, CSV, and database sources, and exploit advanced dtypes like List, Struct, Categorical, and Enum. You'll also stream large queries to disk, process data in batches, and build testable pipelines with built-in assertions. By the end, you'll be equipped to build high-performing data workflows that handle datasets of any size.

Prérequis

Data Transformation with Polars

1

Query Optimization Deep Dive

Learn how to keep queries lazy for maximum optimization, read and interpret query plans, and unlock fast paths with profiling and sorted data.

Effective query execution

Previewing the data lazily

Summarizing digital checkouts

Collecting two reports together

Optimization and query plans

Reading the naive plan

Reading the optimized plan

What query optimization actually does

Using optimization to full effect

Lazy pivot to wide format

Profiling a lazy query

Fast-path filter on sorted data

Commencer le chapitre

2

Efficient Data Input and Output

Learn how to read and write Parquet files, parse messy CSVs, scan multifile and hive-partitioned datasets, and query databases from Polars.

Working with Parquet files

Inspecting a Parquet schema

Summarizing Parquet data

Writing a Parquet snapshot

Parsing CSVs

Parsing a messy CSV

Overriding the inferred schema

Ignoring parse errors

Working with Multifile Datasets

Scanning multiple files

Inserting missing columns

Scanning a hive-partitioned dataset

Working with Databases

Reading from SQLite with a URI

Appending to a SQLite history table

Querying DuckDB into Polars

Commencer le chapitre

3

Advanced Dtypes for Optimal Analysis

This chapter covers working with List and Struct columns, encoding repeated strings as Categorical and Enum dtypes, and reducing memory use through numeric downcasting.

Handling array data

Extracting a primary genre

Counting genres per movie

Counting genre popularity

Working with nested columns

Unnesting production_country

Renaming country fields

Creating an Object dtype

Using Categorical and Enum dtypes

Creating a Categorical column

Using a categorical expression

Creating an Enum column

Reducing memory pressure

Inspecting numeric range

Downcasting numeric columns

Estimating DataFrame size

Commencer le chapitre

4

Working with Polars at Scale

Learn how to use the streaming and GPU engines, sink large query results directly to disk with partitioning, and test pipelines with Polars' built-in assertions.

Working with the streaming and GPU engines

Targeting the streaming engine

Processing query results in batches

Sinking large queries to disk

Sinking a cleaned extract to Parquet

Multiplexing Parquet sinks

Using a custom batch sink

Testing Polars Pipelines

Comparing DataFrames with equals

Testing schemas

Testing full DataFrames

Congratulations!

Commencer le chapitre

Scaling and Optimizing Data Pipelines with Polars

Cours
terminé

Obtenez un certificat de réussite

Ajoutez cette certification à votre profil LinkedIn, à votre CV ou à votre portfolio
Partagez-la sur les réseaux sociaux et dans votre évaluation de performanceS'inscrire maintenant

Rejoignez plus de 19 millions d'utilisateurs et commencez Scaling and Optimizing Data Pipelines with Polars dès aujourd'hui !

Apprenez où que vous soyez avec l'application DataCamp

Progressez où que vous soyez grâce à nos cours conçus pour mobile et à nos défis quotidiens de 5 minutes.