Message Queues

Message queues solve the synchronous coupling problem. Mental model: think of a queue as a buffer between fast producers and slow consumers. Event-driven architecture is the natural extension: instead of services calling each other, they react to events. Start with the basic producer-consumer, build up to event sourcing and saga patterns. Why Async? The Producer-Consumer Pattern Imagine you walk into a coffee shop and order a latte. In a synchronous world, you'd stand frozen at the counter, unable to move, speak, or think until the barista hands you the cup. In the asynchronous world — the real one — you get a number, step aside, and go check your phone. The barista works at their own pace. You work at yours. Neither blocks the other. This is exactly the problem message queues solve in software systems. The Synchronous Problem In a typical web request, your API server calls downstream services synchronously: If the Email Service is slow (or down), your user waits. If it crashes, your request fails. If you get 10,000 sign-ups simultaneously, your Email Service gets 10,000 simultaneous calls and collapses. The Async Solution: Message Queues With a message queue, the API server drops a message into the queue and immediately responds to the user. The Email Service picks it up whenever it's ready. Core Vocabulary Producer: The component that creates and publishes messages (your API server) Consumer: The component that reads and processes messages (your Email Service) Broker: The intermediary that stores and routes messages (SQS, Kafka, RabbitMQ) Queue: A buffer that holds messages until a consumer processes them Message: A payload — typically JSON — describing an event or command Three Powers of Async Decoupling Resilience: If the Email Service goes down for an hour, messages accumulate safely in the queue. When it comes back up, it processes the backlog. No emails are lost. Without a queue, those sign-ups would have gotten errors. Load leveling: Black Friday sends 50,000 orders per minute. Your Order Processing Service can only handle 5,000 per minute. With a queue, orders pile up safely and are processed at a sustainable rate. Without it, the service crashes. Loose coupling: The API server doesn't know (or care) that an Email Service exists. You can add an SMS service, a fraud detection service, or an analytics pipeline without touching the producer at all. Real-World Example: Amazon When you place an order on Amazon, the checkout service doesn't synchronously call inventory, payments, fraud detection, shipping, and notifications. It publishes a single event to a queue. Dozens of downstream services consume that event independently. This is why Amazon can process millions of orders per day without the checkout page timing out. When NOT to Use Async Async isn't always the answer. Use synchronous calls when: You need the result immediately to continue (e.g., checking if a username is available) Strong consistency is required (e.g.