Grokking Messaging Patterns: Queues, Pub/Sub, and Event Streams

This blog explores three fundamental messaging patterns—publish-subscribe (pub-sub), message queues, and event streams—explaining what they are, how they differ, and when to use each.

Imagine clicking “Place Order” on a shopping app.

Instantly, multiple things start happening—payment processing, inventory updating, sending you a confirmation email—all handled by different services.

If each service waited on another via direct API calls, one slow service could delay or crash the whole experience.

What if the payment service is down or the email server is swamped?

Does your entire app freeze?

This is where asynchronous communication comes to the rescue.

Messaging systems let one part of your application send a message and move on without waiting for others to respond.

In simple terms, messaging decouples the sender from the receiver: one service hands off a message to a message broker and doesn’t stall – the broker safely holds the message until the receiving service is ready to process it.

If the receiver is slow or temporarily offline, the message patiently waits in line instead of breaking the whole system.

This decoupling is a game-changer for building resilient, scalable applications.

However, not all messaging systems work the same way.

In fact, there are three common messaging patterns you’ll encounter in system design, each with unique behavior:

• Message Queues: one producer, one consumer – each message goes to a single recipient (point-to-point communication).

• Publish-Subscribe (Pub-Sub): one producer, many consumers – messages are broadcast to all subscribers who are interested.

• Event Streams: a continuous, durable log of events – multiple consumers can read the full history of messages at their own pace.

Each pattern shines in different scenarios and comes with trade-offs in delivery guarantees, ordering, and complexity.

Let’s explore each and see how they compare.

Message Queues: One-to-One Workhorses

A message queue is the simplest messaging pattern – think of it like a line at a coffee shop.

Orders (messages) form a queue, and a barista (consumer) takes each order one by one.

If more baristas are hired, they all pull from the same queue to serve customers faster.

In tech terms, one service (producer) sends messages into the queue, and one consumer service receives each message to process it, ensuring each task is handled once and only once.

Once a message is consumed and processed, it’s removed from the queue.

This pattern is fantastic for work distribution and load leveling.

For example, suppose an e-commerce site needs to generate PDFs of invoices. The order service can post “generate invoice” jobs into a queue, and multiple invoice-worker services will each grab jobs from the queue to process in parallel.

This way, if there’s a spike in orders, the work is automatically load-balanced among workers.

Queues excel at smoothing out traffic bursts and preventing any single worker from getting overwhelmed.

They also typically provide strong delivery guarantees – the system makes sure that a message isn’t lost and eventually gets processed even if workers crash or go offline temporarily.

Technologies:

Many systems implement queues – for instance, RabbitMQ and Apache ActiveMQ are popular message brokers for queues, and cloud services like Amazon SQS provide a fully-managed queue service.

(For a deeper dive into message queues and their role in system design, see the course Grokking System Design Fundamentals.)

Publish-Subscribe (Pub-Sub): One-to-Many Broadcasting

The publish-subscribe pattern, often called pub-sub, works more like a TV broadcast or a YouTube live stream.

Here, a producer (publisher) sends each message out not to a specific receiver but to a topic or channel that multiple listeners have subscribed to.

Every subscriber interested in that topic gets their own copy of the message.

In other words, pub-sub is a one-to-many messaging model.

The publisher doesn’t need to know who is listening; it just publishes events, and the messaging system (broker) delivers those events to all current subscribers.

This pattern shines when you want to broadcast information to multiple services at once without hard-coding who they are.

For example, imagine an online store where a new order event needs to notify several systems: the inventory service should decrement stock, the shipping service should schedule delivery, and the analytics service should log the sale.

Using pub-sub, the order service can publish an “Order Placed” event to the Orders topic, and all three subscriber services will instantly receive it and act accordingly.

The publisher simply says “Hey, an order was placed!” and any service that cares about orders is kept in the loop, all asynchronously.

The big benefit of pub-sub is decoupling and flexibility.

New subscribers can join or leave without affecting the publisher or requiring code changes in the publishing service. It’s great for building scalable, event-driven architectures where components react to events in real time.

However, pub-sub usually offers weaker delivery guarantees compared to queues – if no subscriber is listening at the moment an event is published, that event might be lost (unless you use a system that can buffer or store events).

Also, each message is typically ephemeral in pure pub-sub; once delivered to current subscribers, it’s not stored for future subscribers.

Technologies:

Pub-sub messaging is implemented by systems like Redis Pub/Sub (for simple low-latency messaging), Google Cloud Pub/Sub, or AWS SNS (Simple Notification Service).

Even Apache Kafka can act as a pub-sub broker, delivering messages to multiple consumers, though Kafka is more famously used for event streaming (coming up next).