99.9% Is Not a Business Case

99.9% accuracy sounds finished. It is the number that ends the meeting.

It should not be, because an error rate on its own tells you almost nothing.

Run the system a hundred times a day and that one in a thousand arrives three times a month. That is the arithmetic people do.

Here is the arithmetic they skip. The wins and the losses are not the same size.

Say the system saves a second per case. A thousand cases is about seventeen minutes saved. Now one of those thousand is wrong. Someone has to notice it, work out what happened, and repair what it touched downstream.

If that takes seventeen minutes, you worked for nothing.

A rate of one in a thousand only means one in a thousand if the one costs what the thousand saved

It usually does not. Errors cost more than successes save, because a success is silent and an error has to be found, understood, undone, and explained to whoever it reached.

And usually you cannot tell which one it was. If finding the bad output means checking all thousand, then checking is the new job, and the saving is gone before the error has cost you anything at all.

Then there is which case fails. That is not a random draw. The one that breaks is the unusual one, and unusual tends to mean consequential.

The thousand you got right were the easy thousand.

Before you buy the accuracy number, price the error. What does one bad output cost to catch, to fix, and to explain. Multiply it by three times a month.

That is the number the business case needed. It is almost never the one on the slide.