Why adding autonomy to a software problem can make the architecture worse
We have started calling almost everything an agent.
A system that reads an email and extracts a few fields becomes an agent. A system that summarizes a customer conversation becomes an agent. A system that calls an API and returns the result becomes an agent.
Sometimes the label is useful. Sometimes it is simply adding a new name to a problem that software already knows how to solve.
The interesting question is not whether a model can perform a task autonomously. It is whether the task actually requires autonomy in the first place.
That distinction matters because an agent does not only add intelligence. It adds planning, tool selection, state, retries, new failure modes, observability, permissions, evaluation, and more possible execution paths.
For some problems, that complexity is exactly what we need. For others, it is complexity we have introduced without getting much in return.
The goal should not be to make every system more autonomous.
The goal should be to make the system more capable without making it less predictable.
We Started Calling Everything an Agent
Consider a simple enterprise request:
“What is the status of this customer’s invoice?”
At first glance, this looks like an easy case for deterministic software:
Customer → CRM → Invoice Service → Response
The system knows what information it needs, where to get it, and how to return the result.
But the example has an important boundary. A user may not ask the question in exactly the form the system expects. They might say, “Has Acme paid the latest invoice?” or “Why is the March invoice still showing as open?”
That is where an AI component can be useful: understanding intent and translating natural language into a structured request.
But the workflow itself can still remain deterministic.
AI yes. Agent no.
An agentic version, by contrast, would receive the question, decide which tool to call, interpret the result, and potentially decide whether another tool is necessary. The application would then have to deal with a much larger set of possible execution paths.
The fact that an agent can perform the workflow does not mean that it should.
The important architectural question is:
Does the problem contain enough uncertainty to justify dynamic decision-making?
If the answer is no, an agent may simply be a more complicated way of implementing a predictable workflow.
An Agent Adds More Than Intelligence
One of the easiest mistakes in AI architecture is to think of an agent as simply a smarter function.
It is not.
A traditional function has a relatively clear contract:
Input → Logic → Output
An agent introduces a different execution model:
Input → Reason → Choose → Act → Observe → Reason Again
That difference has consequences.
The agent may need to select tools, maintain state, retry operations, recover from interruptions, ask for human approval, or determine what to do next based on information it did not have at the beginning.
Every one of those capabilities can be useful. Every one also creates another architectural responsibility.
This is why I think of an agent as:
Capability + complexity
The complexity is not automatically a problem. The mistake is pretending it does not exist.
The Autonomy Tax
There is a cost to giving a system freedom to decide what happens next.
More autonomy creates more possible paths. More paths create more states and failure modes. More failure modes require more testing, observability, recovery, and governance.
The relationship looks roughly like this:
More autonomy → More paths → More states → More failures → More testing → More observability → More governance
Consider two implementations of the same general task.
A deterministic workflow with AI for interpretation might look like:
Request
↓
Interpret (AI)
↓
Validate
↓
Call API
↓
Return Result
An agentic version might look like:
Request
↓
Model
↓
Select Tool
↓
Tool Call
↓
Interpret Result
↓
Decide Again
↓
Another Tool Call
↓
Final Response
The second architecture may be necessary for some problems. But every additional decision creates another path to test, another place to fail, and another source of latency and cost.
That is the autonomy tax.
It is not an argument against agents. It is a reminder that autonomy should be earned by the problem.
The engineering goal is not maximum autonomy. It is appropriate autonomy.
Choosing the Right Level of Autonomy
A useful way to make the decision is to classify the work by how much uncertainty it contains.
SignalDeterministic softwareAI componentAgentRules explicit?Rules are clear and stableAI interprets inputs before rules are appliedRules alone are insufficient to determine the pathPath predictable?YesMostly yesNo; the next step depends on what the system discoversInput ambiguous?Low ambiguityNatural language or unstructured information needs interpretationAmbiguous input affects a changing sequence of actionsNext step depends on previous results?No or known in advanceUsually noYes; each result can change what happens nextWhat controls the outcome?Application logicAI output is validated by the applicationDynamic decisions require stronger validation and control
A practical rule follows from this.
Deterministic Work
When the rules are explicit, the workflow is predictable, and the possible actions are known, use software.
For example:
“Move an opportunity from Proposal to Closed Won when the contract is signed.”
If the business rule is explicit, implement the rule. Do not ask an agent to decide whether the transition should happen.
Ambiguous Work
When the input requires interpretation, AI can create real leverage.
For example:
“Summarize this customer’s recent problems and identify the main concern.”
The model can interpret the language and produce a structured result. The application can validate it and continue through a deterministic workflow.
Dynamic Work
An agent becomes more interesting when the system genuinely needs to determine what to do next.
Consider:
“Investigate why this customer has stopped ordering, gather information from CRM and support systems, compare recent activity, and recommend what the sales team should do next.”
The system may not know which information will be useful before the investigation starts. One result may determine the next tool call, and the next result may change the plan again.
That is a fundamentally different problem.
Before introducing an agent, ask:
Is the path predictable?
Are the rules explicit?
Is the input genuinely ambiguous?
Does the next step depend on what the system discovers?
What is the cost of being wrong?
Does autonomy create enough value to justify the additional complexity?
The last question is often the one that gets skipped.
An Illustrative Example
Consider a hypothetical but common scenario.
A team is building an internal workflow for handling customer requests. The process is fairly predictable: understand the request, retrieve the customer record, check a business condition, and create an internal task.
The team initially builds an agent because requests arrive in natural language. The agent can interpret the request and decide which tools to call.
In testing, the system works, but debugging becomes harder. The same request can produce different tool sequences, some executions take longer than expected, and failures are difficult to trace because the model is deciding what to do at each step. Additional model calls also introduce latency and cost.
The team changes the design. A single AI component interprets the customer’s request and returns a structured result. The application then performs the CRM lookup, applies the business rules, validates the result, and creates the task through a deterministic workflow.
The important change is not that the AI disappeared.
The AI remained where interpretation was useful.
What disappeared was unnecessary autonomy.
The Strongest Argument for Agents
There is a legitimate argument on the other side.
Real users do not always follow predefined workflows. Their questions can be open-ended, their needs can change during an interaction, and the information required to solve a problem may not be known in advance.
In those situations, building a separate workflow for every possible variation can become expensive and brittle. An agent with access to a well-defined set of tools can sometimes handle a much wider range of situations without requiring engineers to explicitly encode every possible path.
That is a real advantage.
If the problem is genuinely dynamic, the agent can be the right abstraction.
But autonomy does not mean authority.
Even when an agent decides which tools to use and what to investigate next, the surrounding application still needs to control the boundaries around what the agent can do.
A recommendation is not an authorization.
The agent can recommend a transition. The application decides whether that transition is valid.
The Agent Is Not the Architecture
An agent does not have to own the entire workflow.
In many enterprise systems, a hybrid architecture makes more sense:
User
↓
Application
↓
Deterministic Workflow
↓
AI Agent
↓
Validation
↓
Business Rules
↓
Execution
The agent handles the part where interpretation and dynamic reasoning are useful.
The application continues to own the parts that require certainty.
For example, the agent might understand the customer’s request, decide which information is relevant, select a tool, investigate available data, and recommend the next step.
The application should still control:
Identity
Permissions
Business rules
Transaction boundaries
Workflow state
Validation
Final execution
Audit records
The model participates in the workflow.
It does not own the workflow.
Start With the Boundary, Not the Agent
Before building an agent, define its boundary:
What information can the agent see?
Which tools can it call?
Which actions can it recommend?
Which actions require validation?
Which actions require human approval?
What does the application actually execute?
This is particularly important when the agent interacts with CRM, ERP, finance, identity, support, or operational systems.
The model gets room to interpret and reason.
The application retains control over consequences.
An agent might be very good at identifying what should happen next. That does not mean it should have unrestricted authority to make it happen.
A production system should also make it possible to reconstruct what happened: what the model interpreted, which tools were called, which decisions were made, which validations passed, which actions were executed, and where the workflow ended up.
That is the difference between an AI demo and an AI system that can be operated.
Conclusion
Agents are useful when the problem is genuinely dynamic. AI components are useful when the problem is ambiguous. Deterministic software remains the right place for explicit rules, validation, authorization, and execution.
The difficult part is not deciding whether agents are powerful enough.
It is deciding where their power belongs.
The best AI architecture is not the one with the most autonomy. It is the one that uses exactly as much autonomy as the problem requires.
