What is Pub/Sub Messaging?

Publish-subscribe messaging, or pub/sub messaging, is an asynchronous communication model between software services. In modern software architectures, applications are decoupled into smaller, independent building blocks called services. Pub/sub messaging provides instant event notifications for these distributed systems. It supports scalable and reliable communication between independent software modules.

Pub/sub messaging systems allow publishers to send messages related to specific topics without direct communication with subscribers. Publishers send messages through an intermediary message broker, typically a message bus or topic. Subscribers receive messages only from topics that they subscribed to. Here are some key components of the pub/sub system.

Messages

A message is communication data sent from the sender to the receiver. Message data types can be anything from strings to complex objects representing text, video, sensor data, audio, or other digital content.

Topics

Every message has a topic associated with it. The topic acts like an intermediary channel between senders and receivers. It maintains a list of receivers who are interested in messages about that topic.

Subscribers

A subscriber is the message recipient. Subscribers have to register (or subscribe) to topics of interest. Every subscriber operates independently from the others. Typically, they can perform other operations while processing messages simultaneously.

Publishers

The publisher is the component that sends messages. It creates messages about a topic and sends them to all subscribers of that topic. This interaction between the publisher and subscribers is a one-to-many relationship. The publisher doesn’t need to know who is using the information it is broadcasting, and the subscribers don’t need to know where the message comes from.

Applications developed with a publish-subscribe (pub/sub) pattern have separate application and communication logic. The messaging infrastructure decouples message delivery between publishers and subscribers and broadcasts to different subscribers asynchronously.

The pub/sub communication system has the following key features.

Delivery guarantees

Pub/sub messaging system is commonly built around the at-least-once delivery model. In this model, messages that publishers send will be retained for a pre-determined period. In applications that prioritize strict ordering, such as finance, engineers can architect a first-in-first-out model and exactly-once delivery to avoid duplicate message processing.

Push delivery

Pub/sub messaging immediately pushes asynchronous event notifications when messages are published to the message topic. Subscribers are notified when a message is available.

Multiple delivery protocols

Topics connect to multiple types of endpoints, such as message queues, serverless functions, HTTP servers, and email addresses.

Read about message queueing with AWS »

Fanout

This scenario happens when a message is sent to a topic and then replicated and pushed to multiple endpoints. Fanout provides asynchronous event notifications, which in turn allow for parallel processing.

Filtering

The filtering feature empowers the subscriber to create a message filtering policy. So, they’ll only get the notifications they’re interested in, as opposed to receiving every single message posted to the topic.

Bi-directional messaging

In some cases, publishers can also be subscribers. You can create a bidirectional message stream that works consistently without setting up external channels.

The publish-subscribe (pub/sub) model enables event-driven architecture, which is required in several modern applications. You can use events to trigger and communicate between decoupled services. An event is a state change, or an update, like an item being placed in a shopping cart.

Pub/sub messaging provides significant advantages to developers who build applications that rely on real-time events. Here are some of the advantages.

Eliminate polling

Message topics allow instantaneous, push-based delivery, eliminating the need for message consumers to periodically check or poll for new information and updates. This promotes faster response time and reduces the delivery latency that can be particularly problematic in systems where delays cannot be tolerated.

Dynamic targeting

The pub/sub pattern makes the discovery of services easier, more natural, and less error-prone. Instead of maintaining a roster of peers so an application can send messages, a publisher will simply post messages to a topic. Then, any interested party will subscribe its endpoint to the topic and start receiving these messages. Multiple subscribers can change, upgrade, or disappear, and the system adjusts dynamically.

Decouple and scale independently

Pub/sub makes the software more flexible. Publishers and subscribers are decoupled and work independently from each other, which allows you to develop and scale them independently. You can decide to handle orders one way this month and then another the following month. Adding or changing functionality won’t send ripple effects across the system, because you can integrate additional pub/sub components without rearchitecting the core logic.

Simplify communication

Code for communications and integration is some of the hardest code to write. The publish-subscribe model reduces complexity by removing all the point-to-point connections with a single connection to a message topic. The topic will manage subscriptions to decide what messages should be delivered to which endpoints. Fewer callbacks result in looser coupling, plus code that is easier to maintain and extend.

Durability

Pub/sub messaging services often provide very high durability, and at-least-once delivery, by storing copies of the same message on multiple servers.

Security

Message topics authenticate applications that try to publish content and allow you to use encrypted endpoints to secure messages in transit over the network. Many models support server-side encryption at rest.

Here are some popular use cases of the publish-subscribe (pub/sub) messaging system below.

Perform parallel asynchronous processing

Events published to a message topic can trigger multiple workers to perform necessary but unrelated tasks simultaneously.

Deliver application and system alerts

Deliver critical updates and asynchronous event notifications to affected application components and your users.

Manage asynchronous workflows

Relay events among applications, move data between data stores, refresh distributed caches, or update records in business systems.

Balance workloads

Batch up tasks for bulk processing, or break up a larger task into many smaller ones and use message queuing to divide the work among multiple workers.

Log to multiple systems

Record events to analyze later, capture logs for operations and security, or archive to meet backup or compliance requirements.

Use fanout for replication

Replicate data between production and development environments, or develop and test with live data.

Coordinate serverless applications

Fanout asynchronous event notifications to distributed functions and message queues to coordinate the components of your serverless application.

Stream IoT Data

The pub/sub pattern is a useful way for Internet of Things (IoT) devices to interact. Devices can easily stream data to backend systems or each other.

A message queue is another form of asynchronous communication used in serverless and microservices architectures. Messages are stored in the queue until they are processed and deleted. Message ordering may also cause bottlenecks in the system.

In contrast, the publish-subscribe (pub/sub) pattern allows for more flexibility. Several interested subscribers can receive messages simultaneously and asynchronously.

Publishers don't need to know who the subscribers are.

Read about serverless technologies on AWS »

Amazon Web Services (AWS) has two services you can use for different publish-subscribe (pub/sub) applications.

Pub/sub APIs

AWS AppSync gives application developers the ability to access data from multiple databases, micro-services, and AI models with a single GraphQL API request. Get started quickly by instantly creating GraphQL APIs from your Amazon DynamoDB and Aurora databases. Combine multiple GraphQL APIs into a federated super-graph as you grow.

AppSync Events allows you to easily publish and subscribe to real-time data updates and events, like live sports scores and stats, group chat messages, price and inventory level changes, or location and schedule updates, without having to deploy and manage WebSockets infrastructure.

Pub/sub messaging

Amazon Simple Notification Service (SNS) is a fully managed Pub/Sub service for application-to-application and application-to-person messaging. Simplify your architecture and reduce costs with message filtering, batching, ordering, and deduplication.

Get started with pub/sub on AWS by creating a free AWS account today.