After doing these courses, I feel confident creating professional visualizations and dashboards
Shoebur RahmanProcurement Specialist, Giz Bangladesh
Course Description
What powers the systems that we use every day without realizing, like fraud detection to keep our transactions secure or traffic signals that keep traffic flowing smoothly? The answer is streaming data (data that is continuously generated by different sources) and serverless technologies—like Amazon Kinesis and AWS Lambda.
In this course, you’ll learn how to leverage these powerful technologies by helping a fictional data engineer named Cody. Your goal is to help her to collect real-time streaming data from city-owned vehicles, analyze the data, and send relevant alerts like speed warnings to drivers. Using Amazon Kinesis and Firehose, you’ll learn how to ingest data from millions of sources before using Kinesis Analytics to analyze data as it moves through the stream. You’ll also spin up serverless functions in AWS Lambda that will conditionally trigger actions based on the data received. By the end of this training you’ll know how to create live ElasticSearch dashboards with AWS QuickSight and CloudWatch—and hopefully helped Cody complete her ambitious project.
In this course, you’ll learn how to leverage these powerful technologies by helping a fictional data engineer named Cody. Your goal is to help her to collect real-time streaming data from city-owned vehicles, analyze the data, and send relevant alerts like speed warnings to drivers. Using Amazon Kinesis and Firehose, you’ll learn how to ingest data from millions of sources before using Kinesis Analytics to analyze data as it moves through the stream. You’ll also spin up serverless functions in AWS Lambda that will conditionally trigger actions based on the data received. By the end of this training you’ll know how to create live ElasticSearch dashboards with AWS QuickSight and CloudWatch—and hopefully helped Cody complete her ambitious project.
Prerequisites
Curriculum
Course outline
1
Streaming in the cloud
In this first chapter, you will learn about the differences between batch and stream processing, create your first stream, manage its permissions, write to it and read from it.
- What is streaming and why does it matter?50 XP
- Batch vs stream100 XP
- Producers and destinations100 XP
- Managing Firehose delivery streams100 XP
- Getting ready for the first stream50 XP
- Creating roles50 XP
- IAM users vs. roles100 XP
- Permissions practice100 XP
- S3 bucket creation100 XP
- Working with the Firehose delivery stream50 XP
- Create your first Firehose stream100 XP
- Writing to a Firehose stream100 XP
- Reading Firehose data100 XP
2
Going serverless
The next step in your streaming data journey is learning how to use transformational lambda functions to go serverless. Through hands-on exercises, you will add lambda layers and trigger lambda functions on specific conditions.
3
Analyzing streaming data
You're now ready to encode and decode streaming data and analyze data directly in the stream. You will even use multiple streams to get daily vehicle top speeds.
4
Monitoring and visualizing streaming data
In this final chapter, you will discover how to monitor your stream's performance using logs, metrics, alarms and dashboards. You will use Elasticsearch and build your own Kibana dashboard.
Streaming Data with AWS Kinesis and Lambda
Course
Complete
