Our nightly export stopped carrying customer names on a Wednesday. We had not deployed anything for nine days, their status page was green, and their support desk confirmed there had been no incident. Their API had moved us onto a newer version of itself, on a schedule they had published eighteen months earlier.
They version by date header. Send a date and you get that version's behaviour for as long as they support it. Send nothing and you get whatever is current. Our client sent nothing, because the integration was written in an afternoon against their quickstart, and their quickstart does not send the header either.
The change itself was sensible and well documented. A customer object that used to be returned inline on each order is now returned as an identifier string, to cut payload size, with an explicit parameter to expand it. Our mapper read the name field from inside that object. Given a string where it expected an object, the deserialiser produced a null instead of raising, because the field was declared optional in our model, which it had been for a legitimate reason nobody now remembers.
So sixty two thousand rows went into the CRM with an empty name column, overwriting values that had been correct, and the only reason anybody noticed within a day is that a salesperson opened an account and saw a blank.
The version is pinned now, in one constant, and the client checks it against a version endpoint at startup and refuses to run if the two disagree. Every partner we integrate with has an entry in a register recording how they version, what we are pinned to, what happens when you send nothing, and the date their support for our pin ends. Writing that register was the useful part: two of our six partners have no versioning mechanism at all, which is a thing worth knowing before it matters rather than afterwards. The mapper is strict as well, so an unexpected type is a failure rather than a null.
Sending no version is still a choice. It chooses whatever the other company thinks is best that morning, and it renews itself every day, and it is the one decision in the integration that appears nowhere in your own code.
– Sergey Shinder
