Skip to main content

Streaming Concepts

Learn about the difference between batching and streaming, scaling streaming systems, and real-world applications.

Start Course for Free
2 Hours16 Videos47 Exercises3750 XP

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Course Description

Streaming is a huge aspect of the data world right now and is being used by nearly every industry from manufacturing to healthcare. Would you like to learn more about the general concepts behind data pipelines and how the processes work?

This course provides a general introduction to streaming concepts including batching, queuing, and stream processing along with where they fit into data processing frameworks. It covers real-world examples of how streaming is implemented in production. It is designed as a general introduction to these concepts and does not require an extensive background in data processing.
  1. 1

    Methods for Processing Data

    Free

    In this chapter, you’ll be given an introduction to basic concepts in batch data processing, revolving around implementation, scaling, and potential challenges during use.

    Play Chapter Now
    Intro to batch processing
    50 xp
    Batch characteristics
    100 xp
    Batch ordering
    100 xp
    Scaling batch processing
    50 xp
    On the scale
    100 xp
    Horizontally opposed
    50 xp
    Batch issues
    50 xp
    Batch problems
    50 xp
    Batch scenarios
    100 xp
  2. 2

    Intro to Streaming

    Here, you’ll learn about the basics of streaming, event-based data processing, and the advantages over traditional batch-based data processing.

    Play Chapter Now
    Intro to event-based computing
    50 xp
    In the event of...
    100 xp
    Welcome to the event!
    100 xp
    Queuing
    50 xp
    Queue characteristics
    100 xp
    To queue or not to queue
    100 xp
    Single system data streaming
    50 xp
    Log stream order
    100 xp
    Log options
    100 xp
    Batching vs. streaming
    50 xp
    Batch, queue, or stream?
    100 xp
    Log stream processor
    100 xp
  3. 3

    Streaming Systems

    This chapter introduces the concepts behind modern streaming systems, how to scale them, and some of the issues found in using streaming processes.

    Play Chapter Now
    Intro to real-time streaming
    50 xp
    Real-time?
    100 xp
    Is it real this time?
    100 xp
    Vertically scaling streaming systems
    50 xp
    Scaling reasons
    50 xp
    To vertically scale...?
    100 xp
    Horizontally scaling streaming systems
    50 xp
    Upscaled out
    100 xp
    SLA guarantees
    100 xp
    Streaming roadblocks
    50 xp
    Streaming attributes
    100 xp
    Issue types
    100 xp
    Streaming challenges
    100 xp
  4. 4

    Real-World Use Cases

    In this final chapter, you’ll learn about common streaming systems and consider the implementation of streaming data processing in different use cases.

    Play Chapter Now
    Popular streaming systems
    50 xp
    Streaming truths
    100 xp
    Crossing the streams...
    100 xp
    Real-world use case: streaming music service
    50 xp
    Message components
    100 xp
    Answer me this...
    100 xp
    Real-world use case: sensor data
    50 xp
    Great order of the SLAs
    100 xp
    Sensor scaling considerations
    100 xp
    Real-world use case: vaccination clinic
    50 xp
    Vaccination clinic - classify areas
    100 xp
    A new problem...
    100 xp
    Congratulations!
    50 xp

Collaborators

Hadrien Lacroix
Kelsey McNeillie

Prerequisites

Understanding Data Engineering
Mike Metzger Headshot

Mike Metzger

Data Engineer Consultant @ Flexible Creations

Mike is a consultant focusing on data engineering and analysis using SQL, Python, and Apache Spark among other technologies. He has a 20+ year history of working with various technologies in the data, networking, and security space.
See More

What do other learners have to say?

    Join over 11 million learners and start Streaming Concepts today!

    Create Your Free Account

    GoogleLinkedInFacebook

    or

    By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.