What on-call pain as an architecture metric actually costs

The question is never whether something is worth doing in the abstract. It's whether it's worth doing at your volume, with your failure rate, valuing your team's time properly. So let's do the sum.

How often people get woken up is a measurement of the system, not of the rota. Treat it as feedback on the design and it becomes the most honest quality signal you have, because unlike most metrics nobody can argue with it.

WHERE THE PAGES CLUSTER

Count pages by component over a quarter and the distribution won't be flat. Two or three things will account for most of them, and they'll be the components everyone already avoids. That list is your architecture backlog, ordered by evidence rather than by opinion.

It's also the most persuasive artefact we know for getting remedial work funded. "This module woke someone up eleven times last quarter" survives a prioritisation meeting in a way that "this module is badly designed" does not.

THE TWO KINDS OF ALERT

Some pages mean a human decision is required now. The rest mean something the system should have handled itself, or something nobody needed to know until Tuesday. Sorting a quarter's pages into those two piles is a couple of hours and reliably uncomfortable.

Anything in the second pile is either automation you haven't written or an alert that should be a report. Both are fixable, and both are usually cheaper than the ongoing cost of the interruption.

At around 20,000 samples a month with a two percent exception rate, lab managers were absorbing about nine hours of manual reconciliation a week. That's the number the build had to beat, and it's a lower bar than anyone in the room expected.

IT DECIDES WHO YOU CAN HIRE

Bad on-call is a retention problem before it's an engineering one, and it compounds: the people who leave are the ones with options, which are the ones who could have fixed it. That's worth naming explicitly when a client asks why we want to spend a sprint on reliability.

The inverse holds too. A rota that's genuinely quiet is a strong signal about a codebase, and it's one of the few things about engineering culture that's hard to fake in an interview.

The interesting term wasn't engineering time. It was the cost of a batch was reported before its confirmatory run had finished landing once in the wrong quarter, which the client could size to the pound and we couldn't size at all.

WHERE IT GOES WRONG

• Losing the engineers with options, who were the ones who could have fixed it.
• Rebalancing the rota instead of fixing the two components generating most of the pages.
• Alerts that require no immediate decision, firing at 3am for six months.
• No page-per-component numbers, so remedial work competes with features on vibes alone.

Count pages by component. That list is your architecture backlog.

WHERE THE MODEL BREAKS

Do the numbers before the meeting, not during it. A decision that survives arithmetic tends to survive the next reorg too, because the reasoning outlives the people who made it.