I open-sourced the other half of agent observability: reading beliefs, not just traces

Agents fail in ways traces cannot explain. A trace records what an agent did: the tool it called, the latency, the response. It does not record why. The why lives in a belief that existed for perhaps half a second - "I assume the config is at /etc/app/config.yaml" - and by the time you open the trace UI, that belief is gone.

This is the gap I kept hitting while debugging agent pipelines. So I built the other half of observability: read the agent's beliefs straight from its own streamed output, while they are still happening.

Beliefs as first-class events

Axion is a middleware that sits in front of any OpenAI-compatible endpoint. It tees the model's streamed output (SSE passthrough, zero added latency) and extracts belief events: assumptions, intentions, and plans. Each event carries a confidence score, a timestamp, and a session attachment.

{
"type": "intention",
"belief": "the user wants a retry button on the error page",
"confidence": 0.8,
"span": { "start": 1234, "end": 1290 }
}

Extraction is pattern-based: deterministic, free, auditable. A dot only ends a clause when followed by whitespace or end of input, so URLs and file paths survive extraction intact. The patterns themselves are open source, so extending the vocabulary is a PR, not a research project.

The layer-on-top design

Axion deliberately does not replace Langfuse, Arize, or Braintrust. It emits belief batches over a webhook (axion.belief_batch.v1), signed with HMAC, which you can forward into your existing spans as structured metadata. Your trace pipeline stays; it gains a dimension it could not have: the reasoning that produced the behavior.

The suite

VisReplay: full session recording (thoughts, tool calls, errors) into versioned files, replayed frame by frame
VisCompile: deterministic behavioral diffs between runs, with a --fail-on-regression exit code for CI gates
VisBoard: a shared workspace where agents and humans collaborate - scoped agent tokens, versioned notes with ETag semantics, live SSE events

Verification before shipping

Every repo was verified with live tests, not just unit suites: SSE belief streams with redaction against a mock upstream, byte-deterministic snapshots across repeated runs, full board CRUD and agent auth semantics against a real Postgres.

Honest gaps

Belief extraction is pattern-based, not semantic. It catches stated beliefs, not implicit ones.
Self-hosted only. There is no hosted multi-tenant lane yet.
VisBoard automation currently ships webhook/notify actions.