Skip to main content

Streaming Data with AWS Kinesis and Lambda

Learn how to work with streaming data using serverless technologies on AWS.

Start Course for Free
4 Hours22 Videos56 Exercises2,594 Learners
4500 XP

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA. You confirm you are at least 16 years old (13 if you are an authorized Classrooms user).

Loved by learners at thousands of companies


Course Description

What powers the systems that we use every day without realizing, like fraud detection to keep our transactions secure or traffic signals that keep traffic flowing smoothly? The answer is streaming data (data that is continuously generated by different sources) and serverless technologies—like Amazon Kinesis and AWS Lambda.

In this course, you’ll learn how to leverage these powerful technologies by helping a fictional data engineer named Cody. Your goal is to help her to collect real-time streaming data from city-owned vehicles, analyze the data, and send relevant alerts like speed warnings to drivers. Using Amazon Kinesis and Firehose, you’ll learn how to ingest data from millions of sources before using Kinesis Analytics to analyze data as it moves through the stream. You’ll also spin up serverless functions in AWS Lambda that will conditionally trigger actions based on the data received. By the end of this training you’ll know how to create live ElasticSearch dashboards with AWS QuickSight and CloudWatch—and hopefully helped Cody complete her ambitious project.

  1. 1

    Streaming in the cloud

    Free

    In this first chapter, you will learn about the differences between batch and stream processing, create your first stream, manage its permissions, write to it and read from it.

    Play Chapter Now
    What is streaming and why does it matter?
    50 xp
    Batch vs stream
    100 xp
    Producers and destinations
    100 xp
    Managing Firehose delivery streams
    100 xp
    Getting ready for the first stream
    50 xp
    Creating roles
    50 xp
    IAM users vs. roles
    100 xp
    Permissions practice
    100 xp
    S3 bucket creation
    100 xp
    Working with the Firehose delivery stream
    50 xp
    Create your first Firehose stream
    100 xp
    Writing to a Firehose stream
    100 xp
    Reading Firehose data
    100 xp

Collaborators

Hadrien LacroixJen BrickerLis Sulmont

Prerequisites

Introduction to AWS Boto in PythonIntroduction to ShellStreaming Concepts
Maksim Pecherskiy Headshot

Maksim Pecherskiy

Data Engineer

Maksim is a Data Engineer at the World Bank, and formerly the Chief Data Officer at the City of San Diego. He also works with tech companies to help them effectively connect technology to government. he came to the US from Ukraine when he was 10 years old, lived in 5 countries and speaks 3 languages.
See More

What do other learners have to say?

I've used other sites—Coursera, Udacity, things like that—but DataCamp's been the one that I've stuck with.

Devon Edwards Joseph
Lloyds Banking Group

DataCamp is the top resource I recommend for learning data science.

Louis Maiden
Harvard Business School

DataCamp is by far my favorite website to learn from.

Ronald Bowers
Decision Science Analytics, USAA