POST-MORTEMPRODUCT

How modelling the domain before the schema broke an energy metering system

Modelling the domain before the schema, learned the expensive way on an energy metering system that stayed up while being wrong.

FILED
READ
AUTHOR
REF

The alert fired at a bad hour, as they do. The system wasn't down. That was the problem. It was up and confidently serving wrong answers, which is a lot worse than an outage because nobody comes looking.

A schema is a set of answers. If you write it before you've settled the questions, you've encoded guesses into the hardest part of the system to change. Spend the extra week on the model and the schema takes an afternoon.

NOUNS ARE DECISIONS

Every entity you name is a claim that this thing exists independently, has a life of its own, and can be pointed at. Most arguments that look like technical arguments are really disagreements about whether something is an entity or an attribute, and they get much shorter once you say that out loud.

The test we use is lifecycle. Does it change on its own schedule? Does anybody care about its history? Does someone own it? Three yeses and it's an entity. Three noes and it's a column, and making it a table will cost you joins forever for no return.

GET THE VERBS TOO

Modelling usually stops at the nouns, which is half a model. The verbs are where the domain actually lives: the transitions, who's allowed to make them, what has to be true first. Write them down as a list of allowed moves before you write a single migration.

Do that and something useful falls out for free: the states nobody mentioned. Every domain has two or three legitimate states that don't appear in any documentation because everyone handles them by hand. Those are the ones that generate support tickets for the next five years.

The trigger was boring. A timezone change silently duplicated an hour of consumption. The system had no way to represent that, so it picked one, and settlement analysts spent the next day proving it wrong with exported CSVs.

// Transitions first. The schema is downstream of this.
type Transition = {
  from: State
  to: State
  actor: Role
  requires: Guard[]
}

const TRANSITIONS: Transition[] = [
  { from: "draft",     to: "submitted", actor: "coordinator", requires: ["complete"] },
  { from: "submitted", to: "accepted",  actor: "reviewer",    requires: ["in_scope"] },
  { from: "submitted", to: "returned",  actor: "reviewer",    requires: ["reason_given"] },
  // The state nobody mentioned in the workshop, and the reason we do this first:
  { from: "accepted",  to: "withdrawn", actor: "coordinator", requires: ["before_cutoff"] },
]

THE WEEK THIS COSTS

It looks like a week of not building. In practice it's a week of finding out that two departments use the same word for different things, which you were going to find out anyway, just later, in production, after the migration.

We've never regretted the week. We have repeatedly regretted skipping it, and the regret always arrives in the same shape: a nullable column added in a hurry with a name like status_2.

By the time anyone looked, several million reads a day had gone through the affected path. Only a slice of it was wrong, and we couldn't tell which slice without a full replay. The replay was the one thing we'd never tested.

WHERE IT GOES WRONG

  • Modelling the nouns and leaving the transitions to whoever writes the controller.
  • Discovering in month four that two teams mean different things by the same word.
  • Adding a nullable status column in a hurry, then never being able to remove it.
  • Generating the schema from a UI mock, which bakes this month's screen layout into next decade's data.

Settle the nouns and the allowed moves before you write a migration.

WHAT CHANGED AFTERWARDS

Two action items survived: the two that removed a decision. Everything on the list that asked someone to be more careful was quietly dead within a quarter, which is roughly what we expected when we wrote it.

RELATED
SAME GROUND, DIFFERENT ANGLE
ALL TRANSMISSIONS