After doing these courses, I feel confident creating professional visualizations and dashboards
Shoebur RahmanProcurement Specialist, Giz Bangladesh
Course Description
This course introduces the data engineering role on Google Cloud. You'll learn about data sources, sinks, formats, and storage solutions, then explore replication, migration, and ETL/ELT architectures using BigQuery, Dataform, Dataproc, and Cloud Composer. The course includes hands-on labs with Datastream, BigLake, and Serverless Spark.
Prerequisites
There are no prerequisites for this course
Curriculum
Course outline
1
Course Introduction
This section welcomes you to the Introduction to Data Engineering on Google Cloud course, and provides an overview of the course structure and goals.
2
Data Engineering Tasks and Components
This module provides an introduction to the role of a data engineer. It covers key concepts such as data sources and sinks, data formats, storage options on Google Cloud, metadata management, and the use of Analytics Hub for data sharing within and outside an organization.
3
Data Replication and Migration
This module provides an overview of data replication and migration on Google Cloud. It covers the basic architecture, the 'gcloud' command-line tool, Storage Transfer Service, Transfer Appliance, and Datastream, along with their functionalities and use cases.
4
The Extract and Load Data Pipeline Pattern
This module focuses on data extraction and loading processes on Google Cloud, particularly with BigQuery. It covers the basic extraction and loading architecture, the bq command-line tool, BigQuery Data Transfer Service, and BigLake as an alternative to traditional extract-load patterns.
5
The Extract, Load, and Transform Data Pipeline Pattern
This module provides an overview of ELT (extract, load, transform) processes on Google Cloud. It covers the basic ELT architecture, a common ELT pipeline example, BigQuery's capabilities for scripting and scheduling SQL, and the functionality and use cases of Dataform.
6
The Extract, Transform, and Load Data Pipeline Pattern
This module provides an overview of ETL (extract, transform, load) processes on Google Cloud. It covers the basic ETL architecture, GUI tools, batch and streaming data processing options (Dataproc, Dataproc Serverless), and the role of Bigtable in data pipelines.
7
Automation Techniques
This module focuses on automation patterns and options for pipelines on Google Cloud. It covers various tools and services like Cloud Scheduler, Workflows, Cloud Composer, Cloud Run functions, and Eventarc, along with their functionalities and use cases for automation.
8
Course Summary
In this final section, we review what was presented in this course and discuss the next steps to continue your cloud learning journey.
R
Introduction to Data Engineering on Google Cloud
Course
Complete
