How message queues let services talk without waiting.

A message queue is a broker that holds messages between producers and consumers, decoupling them in time and space. Producers can keep producing while consumers are slow; consumers can come and go without producers caring. The four major products (RabbitMQ, Apache Kafka, AWS SQS, NATS) make different choices about ordering, persistence, and delivery semantics.

A message queue inserts a buffer between two services. The producer doesn't need the consumer to be up, and the consumer doesn't need to keep pace with the producer. The broker holds the difference.

That buffer provides four properties: temporal decoupling (the consumer can be down), throughput smoothing (spikes get absorbed), retries (the broker holds a message until it is acked), and topology decoupling (services don't need each other's addresses, which is service discovery's job). The cost is one more system to operate, with its own failure modes.

Most queue patterns are one of three shapes. Brokers call them queues, topics or channels, but the semantics are these three.

At-most-once: the producer fires and forgets, or the broker delivers without retries. A message can be lost but is never delivered twice. This is similar to UDP, and suits telemetry but not payments.

At-least-once: the broker holds until the consumer acks. Crashes mid-processing get redelivered. The consumer must be idempotent: same message twice should be a no-op the second time, often using a distributed ID as the dedup key. This is the default for most production systems.

Exactly-once: at-least-once plus a deduplication mechanism. Kafka does this with idempotent producers + transactional commits; SQS FIFO with content-hash dedup; databases with primary-key constraints. End-to-end exactly-once across multiple systems is very hard, and "at-least-once plus an idempotent consumer" is usually the practical choice.

The consumer reads a message and the broker stores it as in-flight. The consumer then either acks (delete it), nacks (return to the queue immediately), or times out (return after visibility timeout). The visibility timeout is the longest the consumer may take before the broker assumes it died and redelivers.

Set the timeout a little longer than your slowest reasonable handler. If it is too short you get duplicates, and if it is too long a crashed consumer leaves its work invisible for minutes. Long-poll, periodic renewLock calls (Service Bus), or SQS's ChangeMessageVisibility let you extend the timeout during processing.

A message that fails repeatedly is a poison message. Without a DLQ, it cycles forever and starves the rest of the queue. With one, after N retries the broker moves it to a separate queue for inspection. The main queue keeps moving and the operator can investigate later.

A DLQ that only ever grows usually means the consumer has a bug or the producer is sending malformed data. Alert on DLQ growth rate as well as on the main queue's depth.

Most production systems run at-least-once delivery, which pushes the hard work onto the consumer. Take a worker that captures payments from a payment.authorized queue. The broker promises every message arrives at least once, so the worker will eventually see the same authorization twice, for example after a crash, after a visibility timeout fires during a slow card-network call, or after a producer retry. Handle the duplicate first.

The dedup key is a producer-assigned ID that names the operation. Here that is the authorization ID, not the broker's delivery tag, which changes on every redelivery. The consumer inserts it into a processed_messages table in the same transaction as the capture itself. Checking first and processing afterwards without a shared transaction reintroduces the race, because a crash between the check and the write lets the duplicate through. With the transactional insert, a redelivered message hits a primary-key conflict, and the worker treats that as "already done" and acks.

Now consider a poison message. One authorization arrives with a malformed amount field, the handler throws, and the nack requeues it so it comes straight back. On a plain queue it wastes a worker's time on every retry. On a key-partitioned queue it is worse, because per-key ordering means the consumer cannot skip it and every message behind it for that key waits. A stalled partition looks like a consumer outage for one key.

The fix is a retry cap plus a dead-letter queue. After, say, five attempts, move the message aside with its attempt count and last exception attached, ack the original, and let the partition drain. Then monitor the DLQ itself. Alert when its depth stays above zero for more than a few minutes, and use the growth rate to tell one bad message from a bad deploy. A DLQ nobody alerts on is just a place where data gets lost.

This guide, like most, mixes two kinds of system that share vocabulary but not mechanics. AMQP-style brokers (RabbitMQ, SQS, Service Bus) track per-message state: ack deletes, nack redelivers, the DLQ is a broker feature, and a poison message can be moved aside without touching its neighbours. Kafka-style logs have none of that. The consumer holds an offset, "ack" means committing it, the broker never redelivers one message in isolation, and a DLQ is something you build yourself by producing the failure to a dead-letter topic and committing past it. The ack and visibility sections above describe the first family, and the stalled partition is what a poison message looks like in the second.

A single-partition, single-consumer queue preserves order. Add parallelism and order is lost, because multiple workers process in parallel and the broker hands out messages by availability rather than enqueue time.

Most systems handle this by partitioning by key. SQS FIFO uses a "message group ID"; Kafka uses partition keys; RabbitMQ uses consistent-hash exchanges. The same key goes to the same partition and stays ordered, while different keys may interleave. This preserves per-entity order while keeping overall parallelism.

Each popular broker is a different design point, so choose by the trade-offs you want.

Kafka is technically a partitioned commit log rather than a queue, though it is often used as one, and it has its own page. Messages are not deleted on consumption, consumers track offsets, and replay is straightforward. If you are weighing it against a broker, the Kafka vs RabbitMQ comparison compares them side by side.

Queue lag: consumer falls behind producer. Either scale consumers (with autoscaling tied to depth), or accept that processing is asynchronous. Graph queue depth, where a flat line is healthy and steady growth means work is being deferred.

Hot partition: one key dominates traffic and pins a single consumer. Re-partition by a more uniform key, or split the hot key across shards.

Stuck consumers: a consumer holds a message past the visibility timeout without crashing, so another consumer receives a redelivery. Fix the slow path, or extend the visibility timeout explicitly on long jobs.

Apache Kafka

~1 million msgs/sec per broker on commodity hardware (LinkedIn original benchmark, 2014). Modern Confluent Cloud claims 10× that with KRaft and tiered storage. Latency: ~5-10 ms p99 within a region. Best fit: high-throughput event streams, replayable logs.

RabbitMQ

~50k msgs/sec per node with persistence and ack; ~1 million msgs/sec without persistence. Latency: ~1-5 ms p99 in-region. Best fit: complex routing (exchanges, bindings), per-message acks, low-latency RPC patterns.

AWS SQS

Standard queue: effectively unlimited throughput, at-least-once, no ordering. FIFO queue: ~3,000 msgs/sec per queue with batching, exactly-once processing with dedup. Latency: ~10-30 ms p99 (HTTP-based, not gRPC). Best fit: serverless workflows where you want a fully managed service.

NATS / NATS JetStream

Core NATS: ~10 million msgs/sec per node (in-memory, fire-and-forget). JetStream (persistent): ~1 million msgs/sec. Sub-millisecond latency in-region. Best fit: latency-critical IoT, microservice messaging, real-time signalling.

Apache Pulsar

Throughput similar to Kafka (~1M msgs/sec), with a different storage model: separate broker and BookKeeper layers, geo-replication built in, multi-tenancy first-class. Best fit: large multi-tenant deployments, geo-replicated streaming.

Redis Streams

~100k msgs/sec per Redis instance, with Redis-grade reliability. Best fit: small-scale streaming inside an existing Redis stack; not a Kafka replacement.

Raw throughput rarely decides the choice, since almost any of these handles the load most teams have. Choose by operational model: managed (SQS) vs self-hosted; replay (Kafka, Pulsar) vs ack-and-discard (RabbitMQ); strict ordering (Kafka per partition, FIFO SQS) vs free-form (Standard SQS, RabbitMQ default).

LinkedIn: Kafka, the original case. Kafka was born at LinkedIn (2010-2011) to handle their activity-stream and operational metrics. By 2024 they run ~7 trillion messages per day across thousands of clusters. The original published throughput was about 50 MB/sec per broker on commodity hardware. Their engineering blog is the main reference.

Stripe: RabbitMQ for delivery, SQS for batch. Stripe routes API webhooks through RabbitMQ (per-customer ordering, configurable retry policy with exponential backoff, dead-letter topology). Background batch jobs (invoicing, reports) run through SQS for the simpler at-least-once semantics. Stripe's engineering posts describe both pipelines.

Slack: Kafka for fan-out, in-memory for hot paths. Slack's "Flannel" connection multiplexer reads from Kafka topics that fan out workspace events. ~10 million concurrent WebSocket connections per region; the Kafka topic-per-workspace model lets the connection layer scale independently of the event-bus throughput. Slack has described the architecture in conference talks.

In all three cases the queue separates the producer's request rate from the consumer's. The operational property you need matters more than the broker: replay (Kafka), strict per-key order (Kafka partitions, FIFO SQS), simple at-least-once (SQS Standard) or routing flexibility (RabbitMQ).