Queues, backpressure and dead letters: the heuristic we use

Somewhere between a principle and a habit sits the useful kind of rule: right most of the time, and wrong in ways you'll notice straight away. This is one of those.

A queue doesn't reduce work, it moves when the work happens. That's genuinely useful, and it's not the same as capacity. If the arrival rate exceeds what you can process, a queue converts an obvious failure into a growing, invisible one.

A QUEUE IS A BUFFER, NOT CAPACITY

Queues absorb bursts. They don't absorb a sustained mismatch between arrival and service rate. That just makes a number go up until something else breaks. The failure is worse than the one you avoided, because it presents as latency rather than errors and takes much longer to notice.

So the queue needs a depth alarm tied to what the business can tolerate, not to a round number. "Older than the SLA" is a better alarm than "more than ten thousand", and it's the one that tells you something real.

DEAD LETTERS NEED AN OWNER

Every queue eventually has messages it can't process. Sending them to a dead-letter queue is correct and insufficient. A DLQ with no owner and no dashboard is a place where work goes to be forgotten, which is worse than dropping it, because everyone believes it's safe.

The rule we use: a DLQ is either drained by a person on a schedule, or it doesn't exist. And whatever's in it has to be replayable without hand-editing, or nobody will ever drain it.

Checked against clinical operations: at about 9,000 referrals a month it holds comfortably. It'd start to wobble an order of magnitude higher, where the fixed costs it ignores stop being small.

ORDERING IS EXPENSIVE

Global ordering costs you parallelism, and most systems don't need it. What they need is ordering within a key: everything about one shipment in sequence, while unrelated shipments proceed independently. That's much cheaper and usually what people meant.

Get this wrong in the strict direction and you have a single-threaded system with a queue in front of it. Get it wrong in the loose direction and you process a cancellation before the thing it cancels.

The rule would have caught a referral sat unrouted for eleven days because its owner had left and didn't, because nobody applied it to the integration written under deadline. That's the usual story. The heuristic is fine, the coverage isn't.

WHERE IT GOES WRONG

• A dead-letter queue with no owner, quietly holding three weeks of unprocessed work.
• Global ordering where per-key ordering would do, turning the whole pipeline single-threaded.
• No ordering at all, so a cancellation arrives before the thing it cancels.
• A depth alarm set to a round number instead of to the SLA the business actually promised.

A queue buys you time, not throughput. Alarm on age, and give the DLQ an owner.

WHEN THE RULE FAILS

Treat it as a prompt to think, not a substitute for thinking. Its job is to stop the same conversation happening a fourth time, not to end it.