Quality gates at the point of ingestion: a cost model for clinical operations

The question is never whether something is worth doing in the abstract. It's whether it's worth doing at your volume, with your failure rate, valuing your team's time properly. So let's do the sum.

Bad data gets more expensive the further it travels. Rejecting it at the boundary is unpopular and cheap. Cleaning it up in the warehouse is popular and never finishes.

THE COST CURVE

A malformed record caught at ingestion costs one message to one sender. The same record found in a report costs an investigation, a correction, and a conversation about whether any other numbers are wrong. Same record, two orders of magnitude difference.

Downstream cleaning also can't ever be complete, because you're inferring intent from a corrupted artefact. Upstream you can just ask. That asymmetry is the whole argument, and it's worth making early because the political pressure runs the other way.

REJECT WITHOUT LOSING IT

Rejecting isn't discarding. The record goes somewhere durable and inspectable, with the reason attached and a way to resubmit once it's fixed. A gate that drops things is worse than no gate, because it destroys evidence.

Tell the sender in terms they can act on. "Constraint violation on column 7" produces an email to your team. "Row 412: closing date is before opening date" produces a corrected file.

At about 9,000 referrals a month with a two percent exception rate, care coordinators were absorbing about nine hours of manual reconciliation a week. That's the number the build had to beat, and it's a lower bar than anyone in the room expected.

WHERE STRICTNESS BACKFIRES

Being strict about things that don't matter is how gates get switched off. Reject on what makes the record unusable: missing identifiers, impossible dates, referential breaks. Tolerate cosmetic variation, and normalise it quietly rather than making it someone's problem.

Watch the rejection rate as a signal about the sender rather than the data. A rate that jumps usually means something changed at their end, and asking is faster than diagnosing.

The interesting term wasn't engineering time. It was the cost of a referral sat unrouted for eleven days because its owner had left landing once in the wrong quarter, which the client could size to the pound and we couldn't size at all.

WHERE IT GOES WRONG

• Rejection messages written in database vocabulary, so the sender can't act on them.
• Strictness about cosmetic formatting, giving everyone a reason to disable the gate.
• Cleaning up in the warehouse, inferring intent from an artefact that's already corrupted.
• A gate that drops rejected records, destroying the evidence needed to fix them.

Reject at the boundary, keep what you rejected, and say why in the sender's language.

WHERE THE MODEL BREAKS

Do the numbers before the meeting, not during it. A decision that survives arithmetic tends to survive the next reorg too, because the reasoning outlives the people who made it.