A customer disputed the discount applied to an order in March. It took us three weeks to establish that we could not answer him. The record showed the rate as it stands today, and the last person to save it was a nightly integration account, in August.
Not what the rate had been in March. Not who changed it. Not on whose authority, or whether it had been changed once or four times.
Our core systems store the present. That is what they are built to do, and for the work they were bought for it is entirely sufficient. History exists in our estate in three accidental places, and each one is an accident. One application keeps a change log, retained for thirty days because that was the default. The data warehouse takes a nightly snapshot, which is why we can sometimes answer a question to the day and never to the hour. And backups hold everything, at the cost of a restored environment, a week of effort and a specialist, which is a price nobody will pay to settle a dispute about a discount.
The same gap cost us four times in eighteen months. A regulator asked when a consent flag had been set. A supplier disagreed with us about a contracted price and had better records than we did. A payroll correction could be demonstrated as wrong and not as when it became wrong. And an internal investigation ended without a conclusion because the evidence available was a current value and a last-modified date.
This was never a technology limitation. Every one of those platforms supports change tracking. It was switched off because nobody specified it, and nobody specified it because history costs storage and requires somebody to decide a retention period.
We stopped trying to keep everything. About forty fields across six systems were agreed as consequential, meaning money, entitlements, consent, prices and identity, and change capture was enabled for those alone, recording the old value, the new value, the actor and the source, kept for seven years beside the record itself. The warehouse tracks the same set. New software is now asked to demonstrate it during evaluation, and two products could not.
A system that holds only the present can tell you what is true. It can never tell you what you did.
– Serguey Shinder
