There is a class of failure in long-running AI agents that does not show up in unit tests, does not trigger an exception, and does not get caught by a retry decorator. It happens at the boundary between your agent's internal state and the external world, and when it happens, your agent will confidently do the wrong thing twice.
Here is what it looks like in practice.
An agent is working through a multi-step task: pull customer record, calculate refund amount, issue refund via payment API, update the ticket, notify the customer. The payment API call succeeds. Money leaves the account. Then the process crashes before the agent writes anything to its state store.
On retry, the agent has no memory of the payment. So it pays again.
This is not a hypothetical. It is a predictable consequence of how most people build agentic systems today.
The Gap Between Action and Record
The core problem is that most agent implementations treat state as a summary of what the agent intends to do next, not as an immutable record of what has already happened. When a crash occurs, the agent reloads its last known state and resumes from there. If the last known state says "about to send payment," the agent sends payment.
The action and the record of that action are two separate writes happening in two separate systems. A process crash between them is all it takes to create a duplicate side effect.
You can try to patch this with idempotency keys, and for some external APIs that works. But idempotency only helps if the downstream system supports it and if you are generating keys deterministically enough to reproduce them on retry. That is a lot of ifs stacked on top of each other.
State Transitions as Events, Not Mutations
Event-driven architecture approaches this differently. Instead of mutating a state record in place, every meaningful thing that happens gets written as an immutable event to an append-only log first. The state of the agent is derived from replaying those events, not stored directly.
This changes the failure story completely. Before the agent calls the payment API, it emits a payment_initiated event. After the API responds successfully, it emits payment_confirmed with the transaction ID. If the process crashes between those two events, a resuming agent can inspect the log, see that payment_initiated exists but payment_confirmed does not, and decide what to do: check the API for the transaction status, wait, alert a human. It does not blindly reissue the payment.
def handle_payment_step(agent_context, event_log):
last_event = event_log.latest_for(agent_context.task_id)
if last_event.type == "payment_confirmed":
# Already done, move to next step
return proceed_to_notification(agent_context)
if last_event.type == "payment_initiated":
# Crash happened mid-flight, check status before retrying
status = payment_api.check_status(last_event.transaction_ref)
if status == "success":
event_log.append("payment_confirmed", transaction_ref=last_event.transaction_ref)
return proceed_to_notification(agent_context)
# Handle pending or failed status explicitly
# Safe to initiate
ref = payment_api.charge(agent_context.customer_id, agent_context.amount)
event_log.append("payment_initiated", transaction_ref=ref)
The agent resumes from a known checkpoint with evidence, not assumptions. That distinction matters a lot when the side effects are irreversible.
Human-in-the-Loop Flows Stop Being a Hack
There is a second problem that event-driven architecture quietly solves: human approval steps.
Most agentic systems that need human approval implement it as a polling loop. The agent checks a database field every N seconds, or sets a long timeout and prays, or wraps the whole thing in something complicated that still feels fragile. It works until the agent process restarts mid-wait, at which point you are back to undefined behavior.
With an event log, this becomes straightforward. The agent emits an awaiting_approval event and stops. It is not sleeping, not polling, not holding a connection open. It is just done with this execution cycle. When a human approves the action in whatever interface you build, that system appends an approval_granted event to the log. A consumer picks that event up and resumes the agent from exactly where it paused.
[task_started]
[data_fetched]
[awaiting_approval] <-- agent stops here
...
[approval_granted] <-- triggered by human action
[payment_initiated]
[payment_confirmed]
[notification_sent]
[task_complete]
The audit trail is a side effect you get for free. So is the ability to replay or inspect any task's history without adding separate logging infrastructure.
What This Actually Requires
To build agents this way, you need a few things in place. An append-only event store that your agents treat as the source of truth. Deterministic event naming so replaying a log produces the same state every time. And agents that check the event log before acting, not just at startup but before any step that touches the external world.
None of that is exotic. Event sourcing has been a mature pattern in distributed systems for years. What is new is applying it to the specific failure modes that appear when an LLM-based agent is making real API calls on behalf of real users.
The agents are getting more capable quickly. The infrastructure they run on mostly has not caught up. Treating every external action as an event that gets recorded before and after execution is one of the more concrete ways to close that gap.
