Ask how to reserve stock across two services and keep it correct, and the textbook answer arrives fast: you need a distributed transaction coordinator. Two-phase commit, or a try/confirm/cancel (TCC) protocol run by something like Seata, sitting between the services and making sure both sides agree.
I placed an order that splits across two vendors, reserves stock in a separate inventory service, and survives two buyers racing for the last unit of the same SKU — with no coordinator anywhere in the request path. Not "eventually add one." None, by design. I want to show you why that holds up, because the reasoning generalises past inventory.
The invariant is smaller than the transaction
Here is the constraint that actually has to hold: stock must never go negative. That's it. Not "the order and the reservation must be created atomically" — just that one row, available, cannot drop below zero no matter how many requests hit it at once.
That is single-row arithmetic, and Postgres already has an answer for single-row arithmetic under concurrency: a conditional update.
UPDATE stock_item SET available = available - ?, reserved = reserved + ?
WHERE sku_id = ? AND available >= ?
If the row's available is high enough, the update commits and the hold exists. If it isn't, zero rows are affected, StockService.reserve throws, and order.create() never persists anything. There is no tentative state, no half-reserved row waiting for a second message to confirm it. The WHERE clause is the entire oversell defence, and it's a property of one row in one service's own database — nothing about it requires a second service to agree on anything, in the same instant or otherwise.
The habit that leads people to a coordinator is conflating "this spans two services" with "this needs cross-service atomicity." Placing an order spans two services. The part that must never be wrong — the stock count — does not. Once you separate those two questions, most of the case for a coordinator goes away on its own.
Try/confirm/cancel survive, the coordinator doesn't
The reservation lifecycle really is shaped like TCC. I didn't argue that away — I kept the shape and cut the coordinator out of the middle of it:
| TCC step | What actually runs | Where the correctness lives |
|---|---|---|
| try |
order.create() calls inventory's reserve endpoint synchronously, one HTTP call, no saga framework |
The conditional UPDATE above, inside inventory's own local transaction |
| confirm |
order writes order-paid to its outbox in the same transaction as the PAID state change; inventory consumes it later and deducts the hold |
Kafka's at-least-once delivery + an idempotent consumer |
| cancel | Same shape, on order-cancelled — buyer cancel or the timeout sweep |
Same idempotent consumer, same code path |
try is synchronous because the caller is still on the line and the answer is needed right now: either the hold exists or the order was never worth creating. confirm and cancel are asynchronous because they're genuinely eventual — payment takes real time, might never happen, and nothing about "eventually tell inventory to convert or release the hold" needs to happen at the same instant as anything else. A coordinator's whole value is enforcing that two updates happen in the same instant. Nothing here asks for that.
What crosses the service boundary isn't a joint decision, it's a sequence: reserve now, then confirm or cancel later, exactly once. Outbox-plus-Kafka already delivers that sequence. Adding a coordinator on top wouldn't remove the "later, eventually" step — it would add a second system whose only job is remembering where in the sequence everyone currently is, duplicating the job Kafka and an idempotent consumer already do.
If the reservation succeeds but the order fails to persist right after — a duplicate Idempotency-Key loses a race, or the insert throws for an unrelated reason — that's the one compensation that runs synchronously: order.create()'s catch block calls inventory's release endpoint before returning the error to the caller, because the failure is already known and there's no reason to hand it to an async pipeline for something the request thread can just fix immediately.
The unglamorous parts that make it safe
None of the above works without three pieces of plumbing that will never make it onto a conference slide.
Idempotency, four layers deep. Kafka promises at-least-once, which means the consumer will eventually see the same message twice, or — worse — see order-cancelled arrive after it already applied order-paid for the same order because something upstream misbehaved. Four checks catch that, cheapest first:
-
processed_message (message_id, consumer_group)— a redelivered message is recognized before any business logic runs. -
stock_ledger'sUNIQUE (order_id, sku_id, type)— a secondRESERVE/RELEASE/DEDUCTrow for the same order and SKU can't insert, so even a message that slipped past layer 1 (a crash between marking it processed and committing) is caught at the data level. - The conditional arithmetic itself, unconditionally guarded against going negative.
-
Terminal-movement mutual exclusion — nothing above stops
releasefrom firing on an order that was alreadydeducted.releaseanddeducteach check the ledger for the opposite terminal movement first and refuse if it's already there. This is the layer that turns "our own producer emitted the wrong event" into a loud no-op instead of a silent double-adjustment ofavailable. It also happens to be the layer I added a fix round after the first version shipped — the original code idempotency-checked duplicate deliveries but never checked whether the opposite terminal state had already landed by a different route. It's the kind of gap that four layers of "handle duplicates" doesn't catch, because a bypassed state machine isn't a duplicate.
Timeouts found by scanning, not by waiting. Kafka has no delayed-delivery primitive, so "cancel this order if nobody pays within 30 minutes" can't be a message with a fuse sitting in a queue. Instead a scheduled job selects expired PENDING orders with SELECT ... FOR UPDATE SKIP LOCKED against a partial index on status = 'PENDING' — the locking means running the scanner on more than one instance is safe, and the partial index means it never touches rows that were never candidates. The cancel and its outbox write commit together, so there's no window where the sweep decided to cancel but never told anyone.
Reconciliation as a feature, not an afterthought. The four idempotency layers guarantee duplicates and bypasses are caught. They don't guarantee anyone finds out. A scheduled job diffs stock_ledger movements against live order state and files a row for a human to look at on any mismatch. Without it, "eventually consistent" is a hope, not a property — a system can be silently a little bit wrong forever if nothing is watching for it.
One more small, unglamorous fact worth naming: the Idempotency-Key on POST /orders is scoped to the buyer, not global. The first version wasn't — a header collision between two different buyers would have returned buyer A's order to buyer B. UNIQUE (buyer_id, idempotency_key) instead of UNIQUE (idempotency_key) closed it. Small column, real information leak if it's missing.
When you actually need a coordinator
I don't think "you never need one" is true, and saying so out loud is the point of writing this down as an ADR rather than a blog opinion. A coordinator earns its cost when one business operation requires two or more services to update local state in the same instant, where neither side has a meaningful "happens now, the other happens eventually" split — the canonical case is a ledger transfer that must debit one account and credit another with no window where the money exists in neither or in both.
Reservation doesn't have that shape. try fully completes before the order exists at all; confirm/cancel have a real business reason to be delayed — waiting for a human to pay. The other trigger for a coordinator is a compensation that can't be expressed in business terms — if "give the stock back" weren't a meaningful operation, Saga-style compensation would have no move to make. Every compensation here — release a hold, cancel an order, retry a payment — is an ordinary write to ordinary data. None of that applies.
Numbers
Two buyers race for the last unit of a SKU with available = 1, fired from two threads released by the same latch so they genuinely overlap rather than queue politely:
Tests run: 1 -- ConcurrencyTest
exactly 1 winner (200), exactly 1 loser (INSUFFICIENT_STOCK)
No lock taken in application code, no coordinator consulted — Postgres's own row-level locking on the conditional UPDATE decides the winner.
Against the running stack (verify-m2.sh, not a unit test): a buyer places one order across two vendors and it becomes one Order with two SubOrders, each line carrying its price and name snapshot. A second buyer tries to over-order a scarce SKU (one in stock) and gets a fast 409 with the stock count untouched. A paid order's hold converts to a deduction the moment the event is consumed — same traceId visible on both sides of the Kafka hop. An unpaid order, given a 15-second timeout for the demo, gets cancelled by the sweep and its hold comes back. Four sentences, four real outcomes, no coordinator watching any of them.
This is part of a series on building a multi-vendor commerce platform. The open-source half, stallora-cloud-starter, carries the outbox library this reservation flow is built on. Next up: three production incidents that the outbox pattern's four-sentence blog-post version doesn't warn you about.
