Pular para o conteúdo principal

Gratuito Curso

Scaling and Optimizing Data Pipelines with Polars

IntermediárioNível de habilidade

Atualizado 06/2026

Learn to optimize, scale, and test Polars data pipelines for production-ready performance.

Iniciar Curso Gratuito

Incluído gratuitamente

PythonData Manipulation

4 h

15 vídeos

56 Exercícios

4,800 XP

Certificado de conclusão

Preferido por alunos de milhares de empresas

Training a Team?

Try for Business

Descrição do curso

Take your Polars skills to production scale. Learn to read query plans and unlock the optimizer's full potential, work efficiently with Parquet, CSV, and database sources, and exploit advanced dtypes like List, Struct, Categorical, and Enum. You'll also stream large queries to disk, process data in batches, and build testable pipelines with built-in assertions. By the end, you'll be equipped to build high-performing data workflows that handle datasets of any size.

Pré-requisitos

Data Transformation with Polars

1

Query Optimization Deep Dive

Learn how to keep queries lazy for maximum optimization, read and interpret query plans, and unlock fast paths with profiling and sorted data.

Effective query execution

Previewing the data lazily

Summarizing digital checkouts

Collecting two reports together

Optimization and query plans

Reading the naive plan

Reading the optimized plan

What query optimization actually does

Using optimization to full effect

Lazy pivot to wide format

Profiling a lazy query

Fast-path filter on sorted data

Iniciar Capítulo

2

Efficient Data Input and Output

Learn how to read and write Parquet files, parse messy CSVs, scan multifile and hive-partitioned datasets, and query databases from Polars.

Working with Parquet files

Inspecting a Parquet schema

Summarizing Parquet data

Writing a Parquet snapshot

Parsing CSVs

Parsing a messy CSV

Overriding the inferred schema

Ignoring parse errors

Working with Multifile Datasets

Scanning multiple files

Inserting missing columns

Scanning a hive-partitioned dataset

Working with Databases

Reading from SQLite with a URI

Appending to a SQLite history table

Querying DuckDB into Polars

Iniciar Capítulo

3

Advanced Dtypes for Optimal Analysis

This chapter covers working with List and Struct columns, encoding repeated strings as Categorical and Enum dtypes, and reducing memory use through numeric downcasting.

Handling array data

Extracting a primary genre

Counting genres per movie

Counting genre popularity

Working with nested columns

Unnesting production_country

Renaming country fields

Creating an Object dtype

Using Categorical and Enum dtypes

Creating a Categorical column

Using a categorical expression

Creating an Enum column

Reducing memory pressure

Inspecting numeric range

Downcasting numeric columns

Estimating DataFrame size

Iniciar Capítulo

4

Working with Polars at Scale

Learn how to use the streaming and GPU engines, sink large query results directly to disk with partitioning, and test pipelines with Polars' built-in assertions.

Working with the streaming and GPU engines

Targeting the streaming engine

Processing query results in batches

Sinking large queries to disk

Sinking a cleaned extract to Parquet

Multiplexing Parquet sinks

Using a custom batch sink

Testing Polars Pipelines

Comparing DataFrames with equals

Testing schemas

Testing full DataFrames

Congratulations!

Iniciar Capítulo

Scaling and Optimizing Data Pipelines with Polars

Curso
concluído

Obtenha um certificado de conclusão

Adicione esta credencial ao seu perfil do LinkedIn, currículo ou CV
Compartilhe nas redes sociais e em sua avaliação de desempenhoInscreva-se Agora

Faça como mais de 19 milhões de alunos e comece Scaling and Optimizing Data Pipelines with Polars hoje mesmo!

Desenvolva suas habilidades em dados com o app do DataCamp

Continue progredindo em qualquer lugar com nossos cursos para celular e desafios diários de programação de 5 minutos.