Skip to main content

Home Data Engineering

Course

Streaming Concepts

BasicSkill Level

4.7+

Updated 01/2025

Learn about the difference between batching and streaming, scaling streaming systems, and real-world applications.

Start Course for Free

TheoryData Engineering2 hr16 videos47 Exercises3,750 XP6,643Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Training 2 or more people?

Try DataCamp for Business

Course Description

Streaming is a huge aspect of the data world right now and is being used by nearly every industry from manufacturing to healthcare. Would you like to learn more about the general concepts behind data pipelines and how the processes work?This course provides a general introduction to streaming concepts including batching, queuing, and stream processing along with where they fit into data processing frameworks. It covers real-world examples of how streaming is implemented in production. It is designed as a general introduction to these concepts and does not require an extensive background in data processing.

Prerequisites

Understanding Data Engineering

1

Methods for Processing Data

In this chapter, you’ll be given an introduction to basic concepts in batch data processing, revolving around implementation, scaling, and potential challenges during use.

Intro to batch processing

Batch characteristics

Batch ordering

Scaling batch processing

On the scale

Horizontally opposed

Batch issues

Batch problems

Batch scenarios

2

Intro to Streaming

Here, you’ll learn about the basics of streaming, event-based data processing, and the advantages over traditional batch-based data processing.

Intro to event-based computing

In the event of...

Welcome to the event!

Queue characteristics

To queue or not to queue

Single system data streaming

Log stream order

Log options

Batching vs. streaming

Batch, queue, or stream?

Log stream processor

3

Streaming Systems

This chapter introduces the concepts behind modern streaming systems, how to scale them, and some of the issues found in using streaming processes.

Intro to real-time streaming

Is it real this time?

Vertically scaling streaming systems

Scaling reasons

To vertically scale...?

Horizontally scaling streaming systems

Upscaled out

SLA guarantees

Streaming roadblocks

Streaming attributes

Issue types

Streaming challenges

4

Real-World Use Cases

In this final chapter, you’ll learn about common streaming systems and consider the implementation of streaming data processing in different use cases.

Popular streaming systems

Streaming truths

Crossing the streams...

Real-world use case: streaming music service

Message components

Answer me this...

Real-world use case: sensor data

Great order of the SLAs

Sensor scaling considerations

Real-world use case: vaccination clinic

Vaccination clinic - classify areas

A new problem...

Congratulations!

Streaming Concepts

Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance reviewEnroll Now

Don’t just take our word for it

*4.7

from 444 reviews

80%

19%

1%

0%

0%

Sort by

Francis

16 hours ago

Junior

yesterday

Juan David

yesterday

Juan Pedro

yesterday

Aayush

last week

Challenging yet fun to complete.

Justin

last week

Francis

Junior

Juan David

FAQs

Is any programming experience required for this streaming concepts course?

No. This is a theory-based course that introduces streaming concepts without requiring coding. A basic understanding of data engineering is the only prerequisite.

What is the difference between batch processing and stream processing covered here?

The course explains how batch processing handles data in large chunks at intervals, while stream processing handles data continuously as events occur, and when each approach fits best.

Does the course cover real-world streaming use cases?

Yes. The final chapter presents real-world examples of streaming in production across industries like manufacturing and healthcare, showing how these concepts apply in practice.

How long does this course take to finish?

It is a two-hour course with 47 exercises across four chapters. Most learners complete it in about one and a half to two hours.

What streaming system topics are discussed beyond the basics?

You will learn about scaling streaming systems, handling common issues in stream processing, and understanding how queuing and event-based architectures fit into data pipelines.

Join over 19 million learners and start Streaming Concepts today!

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Grow your data skills with DataCamp for Mobile

Make progress on the go with our mobile courses and daily 5-minute coding challenges.