Instead of calling each other directly, services often drop messages and move on. It's more resilient — but it raises real questions. What if the receiver is down? What if a message gets processed twice and a customer is charged double? These 5 patterns are the answers.
The 5 patterns
| # | Pattern | One-line |
|---|---|---|
| 1 | Message Queue | A buffer between sender and receiver |
| 2 | Publish-Subscribe | One message, many independent receivers |
| 3 | Competing Consumers | Many workers share one queue to scale |
| 4 | Dead-Letter Queue | Where failing messages go to be inspected |
| 5 | Idempotent Consumer | Processing twice = same result as once |
1. Message Queue
A buffer that holds messages between sender and receiver. One message → one consumer.
Order svc → [ ▪▪▪ QUEUE ▪▪▪ ] → Payment svc
(holds messages safely)
| ✅ Good | ❌ Bad |
|---|---|
| Sender & receiver decoupled | Adds a component to manage |
| Receiver can be down — the message waits | Not instant (it's async) |
| Absorbs traffic spikes | Ordering can be tricky |
Swiggy: "charge this order" is dropped on a queue. If Payment is briefly down, the message waits — nothing is lost.
Azure: Service Bus Queue.
Key idea: each message is consumed once, by one worker.
2. Publish-Subscribe (Pub-Sub)
One message published → delivered to MANY subscribers, each independently.
The difference from a queue: a queue delivers to one. Pub-sub delivers a copy to everyone who subscribed.
"Order placed" → [ TOPIC ] →→ Payment (own copy)
→→ Restaurant (own copy)
→→ Analytics (own copy)
| ✅ Good | ❌ Bad |
|---|---|
| Add new subscribers without touching the sender | Harder to trace who got what |
| Fully decoupled | Eventual consistency |
| One event, many reactions | — |
Swiggy: "order placed" → payment, restaurant, and analytics each get their own copy and react independently.
Azure: Service Bus Topic (with subscriptions), or Event Grid.
Queue vs Topic — lock this in:
| Queue | Topic |
|---|---|
| One message → one consumer | One message → many subscribers |
| "Assign this task" | "Announce this happened" |
3. Competing Consumers
Multiple workers pull from the SAME queue to process faster. They compete for messages.
┌→ Worker 1
[ QUEUE ] ───┼→ Worker 2 (whoever's free grabs the next message)
└→ Worker 3
| ✅ Good | ❌ Bad |
|---|---|
| Scale by adding workers | Message order isn't guaranteed |
| Auto load-balances | Workers must be stateless |
| Drains faster at peak | — |
Swiggy: at dinner peak, 50,000 "charge order" messages pile up. Run 100 payment workers off one queue → it drains fast. Each message still goes to exactly one worker (no double-processing).
This is how you scale a queue consumer.
4. Dead-Letter Queue (DLQ)
A separate queue for messages that keep failing — so one bad message can't block everything.
The problem it solves is a poison message: one malformed message that fails, retries, fails, retries… forever, jamming the queue.
[ QUEUE ] → try process → fail → retry → fail (3x) → move to [ DLQ ]
↑ inspect later; queue keeps flowing
| ✅ Good | ❌ Bad |
|---|---|
| One bad message can't block the line | Needs monitoring |
| Failed messages saved for inspection | Manual cleanup |
Swiggy: a corrupt order message fails 3 times → it's shoved into the DLQ → the main queue keeps flowing for everyone else. An engineer inspects the DLQ later.
Azure: Service Bus has a DLQ built-in on every queue and subscription.
⚠️ The trap: teams set up a DLQ and never watch it. Messages die silently. Always alert on DLQ depth.
5. Idempotent Consumer
Processing the same message twice produces the SAME result as processing it once.
Why you need it: queues can deliver a message twice (a network hiccup, a retry). Without protection, "charge ₹450" runs twice → the customer is charged ₹900. 💥
Message "charge order #123" arrives twice:
❌ Not idempotent: charge ₹450 + charge ₹450 = ₹900 (disaster)
✅ Idempotent: "already processed #123?" → skip the 2nd → ₹450 ✅
How it works: track a unique message/operation ID; if you've seen it, skip it.
| ✅ Good | ❌ Bad |
|---|---|
| Safe against duplicate delivery | Must store processed IDs |
| Retries become harmless | Adds a check on every message |
Swiggy: every payment carries an idempotency key. A duplicate "charge" message is detected and skipped. Charged once.
This is non-negotiable in money flows — "at-least-once delivery" means duplicates will happen, so consumers must be idempotent.
How it all connects
Message Queue → the basic buffer (1 → 1)
Publish-Subscribe → the broadcast version (1 → many)
Competing Consumers → scale the consumer side (many workers, 1 queue)
Dead-Letter Queue → catch the failures
Idempotent Consumer → survive duplicate deliveries
The whole thing in one line
A queue assigns one task; a topic announces to all. Then you scale it (competing consumers), protect it (dead-letter queue), and make it safe to retry (idempotent consumer). Get those right and messaging becomes the most reliable way for services to talk.
