Track
AWS Cloud Practitioner (CLF-C02)
0 min
Mastering AWS can be challenging, especially when it comes to understanding its two core services: Amazon Simple Queue Service (SQS) and Amazon Simple Notification Service (SNS).
Although both of these services are integral to cloud solutions, they are often misunderstood and misused.
In this article, I’ll break down the roles of SQS and SNS, identify their similarities and differences, and demonstrate how these services can be used together to build high-performing cloud architectures.
SQS vs SNS: Short Answer
If you’re in a rush, here are the key differences:
- SQS is a poll-based message queue service for decoupling services, supporting A2A communication and message retention.
- SNS is a push-based notification service for real-time messaging, supporting both A2A and A2P communication.
Continue reading for a more detailed explanation, including use cases, architecture diagrams, and best practices!
What is Amazon SQS?
Amazon Simple Queue Service (SQS) is a fully managed message queueing service that facilitates communication between software components. SQS uses queues to decouple microservices, enabling you to create asynchronous workflows in your applications.
SQS involves three core components:
- Producers: These are applications that send messages to the queue.
- Queues: A queue stores messages until they are processed. The retention period for these messages can be set anywhere from 60 seconds to 14 days.
- Consumers: These are services that receive the messages from the queue.
Amazon SQS architecture diagram, including main components. Image by Author.
SQS offers two types of message queues:
- Standard queues offer unlimited throughput, at-least-once delivery (i.e., a message will always be delivered), and best-effort-ordering.
- FIFO queues offer high throughput, exactly-once processing, and first-in-first-out delivery (i.e., messages are received in the exact order they are sent).
For more information regarding these queues, please visit the AWS documentation.
Amazon SQS is designed with message persistence to safely store all messages until they are processed, ensuring no message loss. SQS first allows several retries to process a given message. If a message cannot be processed after exhausting all tries, you can store unprocessed messages in a dead-letter queue, which can then be reviewed and reprocessed.
By decoupling applications, Amazon SQS boasts a number of benefits. It accommodates high message throughput, ensuring high performance even with greater traffic loads. Moreover, it provides independent scaling, ensuring that each service gets sufficient resources based on its individual needs. It also enhances security and durability by encrypting messages and storing them in multiple servers.
To learn more about this queueing system, consider checking out our SQS comprehensive tutorial.
Become a Data Engineer
Become a data engineer through advanced Python learning
What is Amazon SNS?
Amazon Simple Notification Service (SNS) is a fully managed messaging service that supports application-to-application (A2A) and application-to-person (A2P) communication. It is designed for real-time notifications and delivers messages to applications and end users in a scalable and efficient manner.
Similar to SQS, Amazon SNS supports message persistence by letting users create dead-letter queues to store messages that cannot be processed.
Amazon SNS uses a publish/subscribe (pub/sub) messaging model and comprises 3 parts:
- Publisher: A publisher is a system that creates and sends messages to an SNS topic.
- Topic: A topic is an access point that enables message delivery to multiple endpoints.
- Subscriber: A subscriber, or consumer, is an endpoint that receives a message from the topic.
SNS messaging model architecture diagram. Image source: AWS
SNS is highly performant and can send messages to multiple subscribers simultaneously. For A2A communication, SNS can deliver messages to AWS Lambda, Amazon Kinesis Firehose, and even Amazon SQS. For A2P communication, SNS can send messages directly to users via text, email, or push notifications.
If you’d like to master AWS and be able to integrate many of its services to create optimal solutions, consider taking the AWS Cloud Technology and Services course.
Features, including alerting systems, fan-out architectures, and real-time notifications, contribute to Amazon SNS's high robustness and reliability. Due to its high message throughput and capacity for A2A and A2P communication, SNS is often an integral component in event-driven architectures.
Core Differences Between SQS and SNS
Choosing between SQS and SNS requires a solid understanding of the differences in these messaging services. This section outlines the main criteria distinguishing the two.
Push vs. poll-based
Amazon SQS uses a polling system where consumers periodically poll the queue to check for new messages.
Amazon SNS, a push-based system, automatically pushes messages to subscribers as soon as they are published on a topic.
Recipient types
Amazon SQS solely supports A2A communication, enabling messages to be sent to other AWS services, such as AWS EC2, AWS Lambda, AWS RDS, and AWS Redshift.
Amazon SNS supports A2A and A2P communication, sending messages to AWS services (e.g., AWS S3, AWS EC2) and user contacts (e.g., SMS and email) as subscribers.
Message retention
Amazon SQS can store messages for as long as 14 days, allowing consumers enough time to process them. On the other hand, Amazon SNS does not retain any messages, delivering messages to subscribers as soon as they are published.
Message delivery model
Amazon SQS uses a one-to-one model, where only one consumer processes each message. Even if multiple consumers poll for messages at the same time, only one consumer will get a given message. While the given consumer processes a message, SQS utilizes the visibility timeout feature to make that message invisible to other consumers.
Amazon SNS supports a fan-out pattern, where the same message on a topic is sent to every subscriber simultaneously. These subscribers process the received message independently.
Batching
Amazon SQS allows users to batch messages, which entails sending messages to consumers in groups. The batch size can be configured in AWS.
Amazon SNS does not support batching, only allowing one message to be processed at a time.
The following table summarizes these key differences:
|
Criteria |
Amazon SQS |
Amazon SNS |
|
Push versus poll-based |
Poll-based: Consumers poll the queue for new messages |
Push-based: Automatically pushes messages to subscribers |
|
Recipient types |
Supports A2A communication |
Supports A2A and A2P communication |
|
Message retention |
Stores messages for up to 14 days |
It does not store messages |
|
Message delivery model |
Supports one-to-one messaging |
Supports fan-out (one-to-many) messaging |
|
Batching |
Supports batching of messages |
It does not support batching of messages |
When to Use SQS
SQS is the preferred option for a cloud architecture when the message delivery system needs:
- A polling system - consumers can choose when to poll messages.
- Asynchronous processing - services can process messages independently.
- Message retention - messages can be stored for up to 14 days.
- Guaranteed message delivery - no messages in the queue are lost.
- Batching - services process messages in groups.
Consider the following real-life example: An e-commerce platform wants to process customers’ orders, which entails storing the data pertaining to the order in an RDS database.
Architecture diagram of a system using Amazon SQS to process data—image by Author.
In this workflow, customers submit orders, which enter the SQS queue. The lambda function (i.e., the consumer) polls the queue, receives the messages, and then writes the data to the RDS database.
There are several benefits to using SQS for this use case. Since SQS decouples the process, customers can submit orders even when the database is not operational. Since SQS offers at-least-once delivery, all orders will be processed without risk of message loss. Finally, SQS supports batching, which allows messages to be processed in groups, thereby improving efficiency.
Amazon SNS would not work here as it does not retain messages and would instead immediately push messages to the lambda function regardless of the database's status. So, if the database becomes unavailable, messages can be lost. Also, since SNS does not support batching, it would process messages one at a time, making for a less optimal solution.
When to Use SNS
SNS is the preferred option for a cloud architecture when the message delivery system needs:
- Real-time notifications - messages are immediately pushed to subscribers at low latency.
- Fan-out distribution - multiple subscribers can receive the same message simultaneously.
- Multiple notification subscribers - messages can be sent to applications (e.g., AWS Lambda) and end users (e.g., email).
Consider a scenario where a developer monitors the CPU usage of their EC2 instance. To ensure that the usage remains low, they want to monitor the CPU utilization metric on Cloudwatch and receive a notification whenever it exceeds a given threshold.
Amazon SNS would greatly aid the developer in creating a solution since it pushes notifications in real time and supports both A2A and A2P processes. Using its fan-out pattern, SNS can immediately send notifications to lambda functions that can work on alleviating the issue and to the developer via email to inform them about the triggered alarm.
Architecture diagram of a system using Amazon SNS for monitoring—image by Author.
Amazon SQS would be a less effective tool for this job due to its message retention. If developers wish to respond to threshold breaches, they would need to receive real-time alerts, which would not be possible if the messages were stored in queues. Furthermore, since SQS uses a polling system, messages are only accessed when polled by consumers, which increases the latency.
Using SQS and SNS Together: Best of Both Worlds
Amazon SQS and Amazon SNS have distinct features and traits, but AWS users can get the best out of both services by combining them to build an efficient, scalable, and fault-tolerant messaging system.
Integrating the two services is easy since SQS queues can be subscribers to SNS topics. So, SNS can send real-time notifications to SQS queues, after which SQS will decouple all processes while ensuring message delivery.
Consider the following example: An AWS developer is creating a document submission system that extracts content from documents submitted by end users. This system must perform two tasks for any submitted document:
- Process the document using a text-processing application.
- Send a confirmation email to the end user.
The developer can combine SQS and SNS to create an efficient solution:
Architecture diagram of a system using SNS in conjunction with SQS—image by Author.
In this workflow, a message is sent to the SNS topic when a user submits a document, which immediately pushes the message to the lambda function and the SQS queue. The lambda function works on sending the confirmation email, while the SQS queue will handle the messages, which will be polled by the EC2 instance that hosts the text processing application.
Conclusion
Amazon SQS and SNS are core services within the AWS ecosystem, each offering distinct advantages. SNS offers real-time notifications, while SQS offers reliable message processing.
The ideal messaging solution for a problem can incorporate SNS, SQS, or both. By understanding the ins and outs of both services, AWS developers can build a messaging solution that caters to their unique business needs, ensuring efficiency, scalability, and resilience.
If you’d like to take your AWS knowledge to the next level, consider taking these two advanced courses:
Or even prepare for the AWS Cloud Practitioner certification with our skill track:
Get certified in your dream Data Engineer role
Our certification programs help you stand out and prove your skills are job-ready to potential employers.
Can I integrate Amazon SQS and SNS with other AWS services?
How do I ensure message delivery reliability with SQS and SNS?
Can I combine SQS and SNS in a single architecture?
What are the pricing models for SQS and SNS?
How can I optimize performance when using SQS and SNS?
Topics
