From Prompts to Infrastructure: Building Trustworthy, Scalable AI Agents in the Age of A2A and Agent Sandboxes

Originally published on tamiz.pro.

The naive image of an AI agent is a chatbot that gets smarter each turn. The production reality is far more demanding: agents that coordinate across services, respect security boundaries, follow standardized communication contracts, and survive unbounded request volume. The shift from prompts to infrastructure isn't philosophical — it's a hard engineering transition that separates demos from deployed systems.

Why Prompt-Level Design Fails at Scale

Early AI applications leaned on prompt chaining: sequence of LLM calls linked by a human-readable narrative. This works until you need five agents handling 10,000 concurrent requests, each calling multiple downstream tools, with latency budgets measured in seconds and audit trails required for compliance.

Prompt-level systems face three fatal scaling problems:

1. Non-deterministic control flow — LLM outputs are probabilistic, so routing logic built into prompts breaks under edge cases.
2. No isolation boundary — A compromised or malicious prompt can leak secrets, overwrite state, or exfiltrate tool outputs.
3. Opaque ownership — When one monolithic prompt orchestrates everything, debugging requires re-reading dozens of lines of instruction text instead of inspecting structured state transitions.

The alternative is treating agents as infrastructure — services with explicit contracts, bounded execution contexts, and standardized inter-agent communication.

A2A: The Agent-to-Agent Protocol Layer

A2A (Agent-to-Agent) refers to the emerging class of protocols that let agents communicate with each other without a central orchestrator making every decision. Think RPC for agents: structured, typed, versioned, and observable.

What A2A Solves

Without A2A, agent ecosystems look like this:

• Agent A calls a REST endpoint on Agent B, hardcoding the request format.
• Agent B expects Agent A's context in a specific JSON structure.
• When either agent changes its schema, both break.
• Debugging requires sniffing network traffic because there's no shared contract.

With A2A, agents speak a common protocol. Each agent publishes a capability contract — a machine-readable description of what inputs it accepts, what outputs it produces, and what side effects it may have. Other agents discover and invoke capabilities through a typed interface, not by guessing JSON shapes.

The Core Abstractions

A well-designed A2A system rests on three primitives:

Cardinality-bounded messages. Every inter-agent message has a defined lifecycle: sent, acknowledged, completed, or failed. Unlike fire-and-forget HTTP, A2A messages carry sequence numbers and correlation IDs so agents can reconstruct conversation history.

Typed capability descriptors. Instead of POST /agent/process, a capability is declared as:

{
  "type": "tool",
  "name": "database_query",
  "version": "1.2.0",
  "input_schema": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "required": ["query", "connection_id"],
    "properties": {
      "query": { "type": "string", "maxLength": 4096 },
      "connection_id": { "type": "string", "pattern": "^conn-[a-f0-9]+$" }
    }
  },
  "output_schema": {
    "type": "object",
    "properties": {
      "rows": { "type": "array", "maxItems": 1000 },
      "truncated": { "type": "boolean" }
    }
  },
  "error_schemas": [
    { "code": "QUERy_TOO_LARGE", "message": "Query exceeds 4096 characters" },
    { "code": "CONNECTION_UNAVAILABLE", "message": "Connection pool exhausted" }
  ]
}

This schema isn't decorative. It enables compile-time validation at the call site, runtime enforcement at the capability boundary, and automatic test generation for every contract change.

Trust-bound invocation context. Every A2A call carries an invocation context that answers: who is calling? what tools may they access? what is the budget? A2A protocols embed this as a signed token or policy bundle rather than a header you trust because you built the client.

Why A2A Matters for Agent Sandboxes

A2A and sandboxes are complementary. A2A defines the contract — what can be asked and what must be returned. Sandboxes define the execution boundary — where and how the agent runs its logic. Together they create a system where agents can cooperate without trusting each other's internals.

Agent Sandboxes: Execution Isolation as a First-Class Concept

An agent sandbox is a controlled environment where agent code runs with explicit resource limits, network restrictions, and output filtering. The sandbox is not a luxury — it is the mechanism that makes multi-agent systems safe.

The Threat Model

Consider what goes wrong when agents run without sandboxes:

• An LLM generates a tool call that extracts all rows from a production database because the prompt didn't include a row limit.
• A downstream agent receives a prompt injection in a user's message and redirects API keys to an external endpoint.
• A recursive agent loop consumes all available tokens and billing quota before any guardrail detects it.
• A compromised agent writes to shared state that other agents read, poisoning the entire pipeline.

Each of these scenarios is addressable at the infrastructure layer rather than hoping the next prompt improvement catches it.

Sandbox Layers

A production-grade agent sandbox implements three layers:

Compute sandbox. Isolated process execution with CPU, memory, and timeout budgets. Tools run as subprocesses or WebAssembly modules, not arbitrary Python inside the agent process. Tools that exceed their allocation are killed and reported as errors rather than hanging the orchestrator.

Network sandbox. Agents only reach networks they are authorized for. A read-only agent cannot call POST https://webhook.example.com/keys. Egress is enforced by the host, not by the agent's code. Ingress is filtered against allowlists for inbound tool responses.

Data sandbox. Secrets, credentials, and PII live outside the agent's execution context. Tools receive tokens, not full credentials. Output streams are scanned for sensitive patterns before leaving the sandbox boundary.

Implementing the Compute Sandbox in Practice

Here is a minimal but production-oriented sandbox wrapper around a tool invocation:

import asyncio
import json
import time
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import Any, Optional

@dataclass
class SandboxConfig:
    max_memory_mb: int = 256
    timeout_seconds: float = 30.0
    allowed_networks: list[str] | None = None
    max_output_bytes: int = 65536
    secret_prefixes: list[str] | None = None

class SandboxError(Exception):
    pass

class SandboxTooMuchMemory(SandboxError):
    pass

class SandboxTimeoutError(SandboxError):
    pass

class SandboxNetworkBlocked(SandboxError):
    pass

@asynccontextmanager
async def run_tool_in_sandbox(
    tool_code: str,
    arguments: dict[str, Any],
    config: SandboxConfig,
):
    start = time.monotonic()

    # 1. Encode the call as a self-contained script payload
    payload = json.dumps({
        "args": arguments,
        "limits": {
            "memory_mb": config.max_memory_mb,
            "timeout_s": config.timeout_seconds,
            "max_output_bytes": config.max_output_bytes,
        },
        "allowed_networks": config.allowed_networks,
        "secret_prefixes": config.secret_prefixes or [],
    }).encode()

    # 2. Spawn a sandboxed subprocess. In practice this would use
    # gVisor, Firecracker, or WASI — here we show the contract.
    proc = await asyncio.create_subprocess_exec(
        "sandbox-runner",
        "--tool-code", "-",
        "--payload", "-",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )

    try:
        stdout, stderr = await asyncio.wait_for(
            proc.communicate(input=payload),
            timeout=config.timeout_seconds,
        )
    except asyncio.TimeoutError:
        proc.kill()
        raise SandboxTimeoutError(
            f"Tool exceeded {config.timeout_seconds}s budget"
        )

    if proc.returncode != 0:
        error = stderr.decode(errors="replace")
        if "memory" in error.lower():
            raise SandboxTooMuchMemory(error)
        if "network" in error.lower():
            raise SandboxNetworkBlocked(error)
        raise SandboxError(f"Tool exited {proc.returncode}: {error}")

    result = json.loads(stdout.decode())

    # 3. Post-execution output filtering
    if result.get("output_bytes", 0) > config.max_output_bytes:
        raise SandboxError("Output exceeds max_output_bytes")

    yield result

Notice what is not in this code: authentication, authorization, logging, or retry logic. Those belong to the orchestration layer. The sandbox only answers one question: did this tool complete within its declared budget, and was the output structurally valid?

From Monolithic Orchestration to Decentralized Capability Routing

The oldest agent architecture is the central orchestrator: one agent reads a prompt, decides which tools to call, calls them, and returns a result. This pattern collapses under two conditions — high concurrency and heterogeneous tool ownership.

The Routing Problem

Imagine ten teams each maintain a set of tools. No team wants to expose a generic REST endpoint that any other team's agent can call with any payload. They want:

• Typed contracts they can test autonomously.
• Rate limits they control per caller.
• Auditable invocation records for compliance.
• The ability to rotate implementations without breaking consumers.

A central orchestrator cannot accommodate this without becoming a bottleneck and a single point of failure.

Capability Registry Architecture

The replacement is a capability registry — a service that agents publish to and discover from. Each agent registers its capabilities. Other agents query the registry for capabilities matching their needs. When an agent invokes a capability, the registry resolves the target and injects the invocation context.

# capability-registry.example.yaml
capabilities:
  - agent_id: payments-agent
    version: 2.1.0
    capabilities:
      - name: charge
        input: ChargeRequest
        output: ChargeResult
        error_schemas: [InsufficientFunds, CardDeclined, NetworkTimeout]
        rate_limit:
          calls_per_minute: 100
          burst: 20
        trust_policy:
          required_roles: [payments-service]
          allowed_origins:
            - order-agent
            - refund-agent

  - agent_id: research-agent
    version: 1.4.0
    capabilities:
      - name: search
        input: SearchRequest
        output: SearchResultList
        error_schemas: [RateLimited, QueryTooLong]
        rate_limit:
          calls_per_minute: 30
          burst: 5
        trust_policy:
          required_roles: [research-service]
          allowed_origins: [user-agent]

This file is not a configuration dump. It is a machine-readable contract. The registry enforces it at runtime by checking caller identity, applying rate limits, and short-circuiting invalid invocations before they reach the agent.

How A2A Fits the Registry

A2A messages carry a capability_id field instead of an address. The registry translates that identifier to the actual invocation target using the trust policy and rate-limit state. Consumers never know — and should not care — which deployment hosts the capability. This decoupling is what allows agents to scale horizontally without fragile routing tables.

Trustworthiness Through Explicit State Machines

The hardest part of building trustworthy agents is controlling state. An agent that mutates shared variables, sends side-channel messages, or silently retries failures will appear correct in tests and fail catastrophically in production.

Deterministic Agent State Models

Every production agent should expose its state as a finite state machine with explicit transitions. Consider a task agent that processes a user request:

                    ┌──────────┐
   user_request ──▶ │  PENDING │
                    └────┬─────┘
                         │ plan_generated
                    ┌────▼─────┐
                    │  PLANNING │
                    └────┬─────┘
                         │ plan_approved
                    ┌────▼─────┐
              ┌─────│  EXECUTING│─────┐
              │     └────┬─────┘     │
              │          │ tool_failed      tool_completed
        ┌─────▼─────┐  ┌─▼────────┐  ┌──▼──────────┐
        │ RETRYING  │  │COMPLETED │  │FAILED_MAX   │
        └─────┬─────┘  └──────────┘  └─────────────┘
              │
              │ retry_budget_exhausted
              └───────────────────────▶ FAILED_MAX

This diagram is not decoration. It drives four engineering decisions:

1. Serialization — Transitions are the only write path. Concurrency bugs become impossible if every handler is applied to an immutable snapshot and writes back atomically.
2. Observability — State transitions are events. Emit them to a durable log. Replay becomes free.
3. Recovery — On restart, the agent rebuilds state from the event log instead of trusting volatile memory.
4. Testing — Each transition is a unit test. You validate behavior, not luck.

Idempotent Tool Handlers

An agent that calls a tool twice with different results is an agent that is lying about its own state. Every tool handler must be idempotent or clearly non-idempotent with compensation logic.

interface ToolHandler {
  id: string;
  execute(ctx: InvocationContext, params: unknown): Promise<ToolResult>;
  /** True if re-invoking with the same params must return the same result */
  idempotent: boolean;
  /** Optional cancellation hook */
  cancel?(ctx: InvocationContext): Promise<void>;
}

When a tool is non-idempotent — writing to a database, posting to an API — the agent must associate a unique invocation ID with each call and check for prior completion before re-invoking after a failure. Without this, retry storms double costs and corrupt state.

Observability: Tracing Agent Behavior as First-Class Data

Agents are opaque by default. One bad iteration and you cannot tell whether the fault lies in the prompt, the tool, the LLM, or the routing logic. Observability is not an add-on — it is the lens that makes debugging possible.

The Four Required Signals

A production agent system must emit four signals consistently:

Structured invocation logs. Every A2A call logs correlation ID, caller, target, capability, input hash, start timestamp, end timestamp, and outcome. Inputs are hashed for deduplication but never logged in plaintext.

Tool execution traces. For each tool call, record: tool ID, sandbox boundary info, resource usage, network egress, and output size. This lets you detect sandbox escapes and resource abuse.

LLM cost and latency breakdown. Track tokens consumed, latency per model call, and cost per agent. Agents that appear cheap in isolation can be expensive in aggregate when you sum retries, fallback calls, and long context windows.

State transition audit log. Every FSM transition is recorded with the event that triggered it, the actor, and the resulting state. This log is the source of truth for postmortems.

Example: Structured Invocation Log

{
  "trace_id": "7f3a9b2c-4e1d-4f8b-b5a6-9c8d7e6f5a4b",
  "span_id": "01a2b3c4d5e6f7a8",
  "timestamp_ms": 1718496234567,
  "caller_agent_id": "order-agent",
  "target_agent_id": "payments-agent",
  "capability_id": "charge",
  "capability_version": "2.1.0",
  "input_hash": "sha256:e3b0c44298fc1c149afbf4c8996fb924",
  "start_ms": 1718496234567,
  "end_ms": 1718496234789,
  "duration_ms": 222,
  "outcome": "success",
  "sandbox": {
    "cpu_ms": 45,
    "memory_peak_mb": 87,
    "network_egress_bytes": 1024,
    "timeout_budget_ms": 30000,
    "timeout_used_ms": 0
  },
  "llm_usage": {
    "model": "claude-sonnet-4",
    "input_tokens": 1240,
    "output_tokens": 89,
    "cache_read_tokens": 320
  },
  "cost_usd": 0.00142,
  "state_transition": {
    "from": "EXECUTING",
    "to": "EXECUTING",
    "event": "tool_completed"
  },
  "retry_count": 0,
  "idempotency_key": "charge-7f3a9b2c-order-88291"
}

This log tells you everything a postmortem needs without requiring access to production memory or replaying conversations.

Scaling Patterns for Multi-Agent Systems

Architecture patterns for single-agent systems do not transfer cleanly to multi-agent systems. Here are the patterns that actually work.

Pattern 1: Fan-Out with Outcome Aggregation

When a task decomposes into independent subtasks, fan out to multiple agents and aggregate results. The aggregator waits for a quorum or best-response policy:

• Best-response: Return the first successful result.
• Quorum: Return when N agents succeed.
• Majority: Return the mode of successful outputs.

This pattern reduces latency linearly with agent count and provides natural fault tolerance.

Pattern 2: Pipeline with Backpressure

When tasks have sequential dependencies, pipeline agents with explicit backpressure. Each agent holds a bounded queue. When the queue is full, the upstream agent blocks instead of dropping work or spilling to disk. This keeps the system stable under load.

# High-level backpressure-controlled pipeline
async def run_pipeline(task: Task, agents: list[Agent]) -> Result:
    queue = asyncio.Queue(maxsize=100)  # bounded

    async def producer():
        await queue.put(task)

    async def consumer(agent: Agent):
        while True:
            item = await queue.get()  # blocks when empty
            try:
                result = await agent.process(item)
                await queue.put(result)  # blocks when full
            finally:
                queue.task_done()

    await asyncio.gather(
        producer(),
        *(consumer(agent) for agent in agents),
    )

The maxsize=100 queue is not arbitrary. It is a memory bound that prevents unbounded queue growth when downstream agents slow down.

Pattern 3: Fallback Chains with Degrading Capability

Agents should declare fallback hierarchies. If the primary agent is unavailable or returns an error, the system tries the next capability version or a simpler agent with fewer features:

primary-agent:1.0 ──failure──▶ primary-agent:1.0-fallback ──failure──▶ read-only-agent:2.3

Each fallback has a documented capability reduction. The orchestrator surfaces this to the caller so the UI can adapt — showing a degraded response instead of a hard error.

Pattern 4: Cost-Aware Routing

Not all invocations deserve the most capable model. Route requests by complexity:

• Simple tool lookup → small model or rule-based router
• Medium reasoning → medium model
• Complex multi-step planning → large model

A cost-aware router inspects the invocation context — input size, required reasoning depth, and output structure complexity — and selects the smallest model that meets the SLA. The savings compound across millions of invocations.

Security: The Non-Negotiable Baseline

Security in agent systems is not about perimeter defense. It is about assuming compromise at every boundary and designing accordingly.

Principle 1: Zero Trust Between Agents

Assume every agent is potentially compromised. Treat every A2A call as if it could be spoofed. Enforce:

• Mutual TLS between all agents.
• Short-lived, scoped tokens for each invocation.
• Caller verification at the registry, not at the capability implementor.

Principle 2: Least Privilege Per Capability

Each capability declares the minimum resources it needs. The sandbox enforces this. Capabilities that only read from a database should never be able to write. This is enforced at the capability registration level, not in code.

Principle 3: Prompt Injection as a Network Threat

Treat user input the same way you treat an untrusted network: it is adversarial. Validate and sanitize inputs at the sandbox boundary. Use structural parsing (JSON schema) rather than natural-language filtering. Natural-language filters fail against adversarial prompts; schema validation does not.

Principle 4: Secret Rotation as Default

Agents must never hold long-lived secrets. Use short-lived tokens issued by a secrets provider. Rotate automatically on each invocation or on a fixed schedule, whichever comes first. Store secrets in a vault, not in environment variables or configuration files.

Measuring What Matters

Prompt engineering metrics — token count, latency per call, success rate per prompt — are necessary but insufficient. Production agent systems need infrastructure metrics:

Frequently Asked Questions

Q: Do I need A2A protocols if I only have one agent?

No. A2A pays for itself when you have multiple agents that coordinate, share capabilities, or evolve independently. For a single agent with a fixed toolset, a monolithic orchestrator is simpler and sufficient. Introduce A2A when coordination complexity exceeds what a single orchestrator can manage.

Q: Can I start with a central orchestrator and migrate to A2A later?

Yes, but design your orchestrator to be stateless and your capabilities to be addressable independently. If every tool call is wrapped in a capability contract from day one, migration is a configuration change. If tool calls are hardcoded across the orchestrator, migration is a rewrite.

Q: How do I handle non-determinism if I am building an agent state machine?

Non-determinism lives in the LLM layer, not the agent layer. The LLM returns a probabilistic plan; the agent executes a deterministic state machine based on that plan. Log the plan, validate it against the schema, apply it as a state transition, and treat any deviation as an error. This separation of concerns keeps agents predictable even when their inputs are not.

The transition from prompts to infrastructure is not about abandoning prompt engineering — it is about recognizing that prompts are one component in a larger system. A2A protocols, agent sandboxes, capability registries, and state machine design form the skeleton. Prompts are the nervous system. Build the skeleton well, and the nervous system has something stable to control.