What designing the degraded mode first actually costs

The question is never whether something is worth doing in the abstract. It's whether it's worth doing at your volume, with your failure rate, valuing your team's time properly. So let's do the sum.

Every dependency will be unavailable at some point. What your system does during that window is a design decision whether or not anyone makes it deliberately. Making it deliberately is most of the work.

THE DEFAULT IS USUALLY THE WORST OPTION

Left alone, an unavailable dependency produces a stack trace in front of a user and a lost piece of work. That's rarely the best available behaviour. Often you could accept the input, tell the truth about what's pending, and reconcile later, but only if someone decided that in advance.

The decision has to be per-dependency, because they aren't equivalent. Losing the address validator should degrade to accepting the address unvalidated. Losing the payment authoriser should stop the transaction. Same outage shape, opposite correct answers.

TELL THE TRUTH ON THE SCREEN

Degraded modes fail on honesty more than on engineering. A system that silently drops the enrichment step and shows an incomplete record as though it were complete has turned a visible outage into a data quality problem, which is much harder to find later.

So the degraded state needs to be visible in the record, not just in the logs. "Pending verification" is a real state that belongs in the model. It's also the thing that lets someone clean up afterwards, because they can find the affected rows.

At close to 80,000 listings a month with a two percent exception rate, trust and safety reviewers were absorbing about nine hours of manual reconciliation a week. That's the number the build had to beat, and it's a lower bar than anyone in the room expected.

PRACTISE IT

An untested fallback is a guess. Turn the dependency off in a staging environment that has real traffic shape and watch what actually happens. It's routinely not what the code suggests, usually because of a timeout somewhere nobody configured.

The failures that surprise us most are almost always timeout-related. A dependency that's down is easy. A dependency that's slow holds every connection until the whole system is unavailable for reasons that look unrelated.

The interesting term wasn't engineering time. It was the cost of a suspended seller kept transacting through a second account landing once in the wrong quarter, which the client could size to the pound and we couldn't size at all.

WHERE IT GOES WRONG

• No timeout on a dependency, so slow is worse than down.
• The same failure policy for the address validator and the payment authoriser.
• Silently skipping enrichment, so incomplete records look complete and can't be found later.
• A fallback path that's never been exercised and doesn't work when it's needed.

Decide what happens when each dependency is gone. Then make the degraded state visible.

WHERE THE MODEL BREAKS

Do the numbers before the meeting, not during it. A decision that survives arithmetic tends to survive the next reorg too, because the reasoning outlives the people who made it.