Agent-first development is a fast way to discover a workflow. As the workflow becomes predictable, stable execution should move into code, while the model remains available for the cases where fixed rules and conventional software are no longer enough. But how do you tell when that’s happening?
We have all seen this movie while building an agent.
You begin with a question, refine the prompt, add another instruction, and try again. The answers improve until the system appears capable of completing useful work. It may still be rough, but it has moved beyond a demo and started to resemble an agent.
Then you let it run.
You change the prompt here, add a tool there, and introduce a skill somewhere in the middle. After a few iterations, the model is selecting tools, deciding how to sequence them, passing results between steps, and working out what to do when the expected output does not arrive.
I started calling this the LLM-as-orchestrator pattern.
It is often a very effective way to start the build because at the beginning of a project, you rarely understand the workflow well enough to define every branch, dependency, and failure mode. Letting the model attempt the task helps expose what the process actually requires.
The agent reveals missing tools, recurring decisions, awkward handoffs, and inputs that are less structured than everyone originally claimed, so you learn the workflow by watching an agent try to execute it.
The architecture, however, needs to evolve as that learning accumulates. Often, the agent begins following the same sequence across most runs, and much of its work becomes predictable execution. Keeping the LLM in control of every step then starts to add token cost, latency, and behavioral variance to a path the team already understands.
The useful production question is therefore fairly simple:
Which parts of the agent’s workflow have become ordinary software, and where does the code still run out of road?
And when that happens, what are the common signs, or smells, to look out for?
When the prompt starts looking suspiciously like code
You can usually see the transition by reading the prompt.
Early prompts describe an objective and provide enough context for the model to attempt the task. Later prompts contain increasingly precise instructions about which tool should run first, what fields should be extracted, how the output should be transformed, which conditions should trigger another action, and what should happen when a value falls outside an expected range.
The prompt gradually assumes responsibility for application logic.
At some point, it is worth asking whether much of that agent behavior would be clearer and safer in Python. Other languages remain available, apparently, although Python appears to have won the argument by refusing to leave.
This is where the phrase “prompt engineering” occasionally gives us more comfort than it deserves.
A prompt can contain substantial application logic while providing very few of the controls normally associated with software engineering: interfaces remain implicit and testing often focuses on the final answer rather than the route taken to produce it. A change in one instruction can affect behavior somewhere else, and the regression may remain hidden because the output still sounds plausible.
Plausibility is one of the less charming failure modes of language models. The system can produce a coherent answer after skipping a required check, choosing the wrong tool, or passing incomplete information downstream.
As the workflow becomes clearer, however, repeated behavior should start moving out of the prompt. That is when known transformations can become functions, stable payloads can become typed interfaces, and validation can move into schemas and assertions. At this point repeated sequences can be handled through deterministic orchestration and the model should remain where fixed rules, schemas, and ordinary control flow cannot resolve the case cleanly.
The accidental DAG
This transition normally happens one piece at a time.
A repeated instruction becomes a tool, a model-generated object becomes a structured payload, and one LLM call disappears because a rule can now produce the same result reliably.
Run it long enough, and eventually the traces show that the agent has called the same seven tools in the same order for the last twenty successful runs.
If you squint hard enough, this can resemble a self-improving loop.
It is, in a sense. You still have to inspect the traces, identify the pattern, extract it into code, test it, deploy it, and keep it alive while the system contributes mainly by generating the evidence and the invoice. Alternatively, you automate this with a “grade and trajectory analysis”, but in either case the agent is likely to converge into making ten tool calls back to back, passing the result of one into the next, before using a final model call to translate, classify, or summarize the output.
After that, you need to decide when the agent runs, so you add scheduling.
When it wakes up and there is nothing to process, you pay for the privilege of discovering that nothing happened.
When something important occurs between scheduled runs, processing is delayed.
When the workflow fails halfway through, you need state, retries, idempotency, and a way to recover from partial execution.
By this stage, the agent has known dependencies, defined stages, an execution schedule, and predictable failure modes.
You have accidentally converged on a directed acyclic graph.
Data and software engineers have spent decades solving this class of problem. Once the path is understood, conventional orchestration can execute it more cheaply and reliably, while giving the team clearer control over state, retries, monitoring, and deployment.
This is useful progress. The system has revealed enough of its structure to become ordinary software.
The only disappointment is that “scheduled workflow with a semantic transformation step” sounds considerably less magical than “autonomous agent”. Production systems have an unfortunate habit of losing some whimsy as they improve.
You can arrive from the other direction
You can also begin designing the workflow from the tools:
One deterministic function performs a defined operation. Another follows it.
The sequence is explicit, testable, and scheduled through conventional infrastructure.
You continue connecting steps until the path between two tools becomes difficult to express in code: perhaps the input is unstructured, or perhaps the next action depends on language that can reasonably be interpreted in several ways. Perhaps mapping one system into another would require a forest of brittle rules and regular expressions that nobody will want to maintain six months later (if you know Perl, I hear you may disagree).
That breakpoint is a natural place for an LLM.
The LLM receives the context, resolves the case, produces a structured result, and returns control to the deterministic workflow. It is being used because the software cannot cover the situation economically or reliably through fixed rules.
This route often produces a lighter agent because the LLM is invoked only where the normal runtime is insufficient while scheduling, retries, state, and observability are already handled by infrastructure designed for those responsibilities.
Agent-first and tool-first development therefore represent two different ways of discovering the same eventual boundary.
Agent-first development starts with broad flexibility and gradually extracts stable behavior. Tool-first development starts with a defined path and introduces the model at the points where the code is not enough.
Both can converge on the same production architecture.
The emerging architecture
The architecture I keep seeing emerge has three layers:
A deterministic backbone
An agentic exception path
A human escalation boundary
The deterministic backbone handles the normal path by owning stable sequencing, scheduling, validation, state management, retries, permissions, and observability.
The agentic exception path activates when the normal workflow reaches a case that the known rules cannot handle confidently. Here, “exception” covers more than a runtime error and includes any situation where the software lacks enough structure, context, or coverage to continue.
Consider, for example, all the cases where an input fails to match an existing schema, a document is incomplete, or two systems return conflicting information. And we could go on and on... a request may contain ambiguity that materially affects the next action, and a result may fall outside the range covered by the original rules.
In those “exception” cases, the model can inspect the context, call tools, gather additional evidence, and select a recovery path dynamically.
That is still agentic orchestration; it simply happens at the edge of the known workflow instead of supervising every routine step, and when the LLM resolves the issue, control returns to the deterministic path.
This approach paves the way for higher escalation. When the information remains insufficient or the action carries too much risk, the workflow escalates to a human in the loop.
The resulting three-step architecture gives the team a clear operating model where the known path is predictable, the exception path is adaptive and bounded, and the escalation boundary has a clear owner.
I quite like the approach as it is more useful than debating whether the system is sufficiently “agentic”: that term now covers everything from a loop around an LLM call to software with access to production systems and corporate credit cards. Some precision feels overdue.
Agent-first and tool-first solve different learning problems
Agent-first development works well when the workflow remains uncertain.
The model can attempt the task, expose missing tools, reveal hidden dependencies, and show which decisions actually recur. This reduces the cost of learning because the full process does not need to be formalized before anyone knows whether it works.
Its main risk is inertia: once the prototype begins producing useful results, delivery pressure rewards adding another instruction. The next stakeholder demonstration is approaching, someone needs the feature by Thursday, and restructuring the workflow feels slower than making the prompt slightly longer.
Over time, the prompt becomes the permanent runtime and the token cost, latency, and variance of the experimentation phase become production characteristics because nobody owns the industrialization.
Tool-first development works well when the execution path is already understood, or when reliability, auditability, and control matter from the beginning. The team implements the known workflow and introduces model calls where the software cannot cover the case cleanly.
That route has its own risk: teams can formalize the process too early and spend weeks implementing rules for a workflow they barely understand while several agent-driven attempts may have exposed the real structure much faster.
The starting point should reflect how much the team already knows and the final architecture should reflect how much uncertainty remains after the workflow has been exercised.
Four operating gates for every model call
When reviewing an agent workflow, I find these four questions useful.
Is the execution path predictable?
Look at the traces.
When similar inputs consistently produce the same tools in the same order, the agent is executing a known path and that sequence is ready to be considered for deterministic orchestration.
The length of the prompt is only a clue, while the agent’s behavior is stronger evidence. The useful questions are how often the path repeats, where it branches, and which steps continue to vary meaningfully.
Can the result be checked mechanically?
When an output must match a schema, satisfy a rule, reconcile against a known value, or pass an assertion, software should perform that check directly.
Asking a second model whether the first model followed a fixed structural rule adds another probabilistic step to a question with a deterministic answer, although it remains a surprisingly popular design, perhaps because one uncertain opinion feels lonely.
Model-based evaluation can still help when quality depends on meaning, completeness, or contextual relevance. Nonetheless, mechanical correctness should remain mechanical.
Is code enough to resolve the case?
Some steps are easy to express through fixed rules, typed interfaces, and normal control flow while others become expensive or brittle when forced into deterministic logic.
A model call earns its place when the software cannot cover the case cleanly, such as when it involves unstructured input, incomplete context, conflicting evidence, unfamiliar formats, or a large number of possible variations.
Moving data between known systems, applying fixed transformations, checking explicit conditions, and following a stable sequence rarely need a model. Token cost is only one part of the calculation, as even an inexpensive call introduces latency, variance, another failure mode, and another component that needs to be observed.
The LLM call should contribute something the runtime cannot provide more effectively.
Is the agentic path bounded?
Once the model can choose tools dynamically, the boundaries need to be explicit.
Which tools may it call? What data can it access? Which actions can it take without approval? How is recovery validated? When does control return to the deterministic path? Which conditions require human review?
These boundaries make agentic orchestration operable, and very importantly, make the system easier to explain to security, compliance, operations, and customers because the known path is predictable, the adaptive path is constrained, and the escalation point has an owner.
Without those controls, “agentic exception handling” can become a sophisticated phrase for granting broad permissions and hoping the traces are educational.
The organizational failure usually happens gradually
Most teams do not make a deliberate decision to keep an LLM responsible for predictable orchestration forever; they arrive there through a series of reasonable local decisions.
The prototype works → A customer asks for another case → A tool is added → The prompt receives another instruction → The deadline moves closer, and nobody owns the moment when stable behavior should be extracted into code!
At the speed of prototyping, each change makes sense on its own. Put together and left unchecked, however, those changes can create a production architecture whose cost and reliability remain tied to the way the team experimented.
Engineering leaders need an explicit review point for this transition. That review should examine path stability, token cost per successful run, latency, validation coverage, exception frequency, recovery behavior, and the proportion of model calls handling cases that software cannot yet cover.
The purpose is simple: as the team understands more of the workflow, the architecture should reflect that understanding.
Without this review, the agent remains the orchestrator by default. Stable behavior stays buried in prompts, and the organization continues paying for flexibility it no longer uses.
Let the architecture learn with you
Broad agentic orchestration can be a very effective way to explore a workflow early.
As the system runs, repeated patterns emerge. Known sequences can move into code. Common decisions can become explicit rules. Stable outputs can gain mechanical validation. Scheduling, state, retries, and observability can move into the runtime designed to handle them.
The agent remains available for the cases where code is no longer enough. It can handle unfamiliar inputs, reconcile conflicting information, coordinate recovery, and return the workflow to the known path.
That gives you the flexibility of an agent without charging the agent tax on every ordinary execution.
Use the model to discover the workflow. Move the parts you understand into code. Preserve agentic orchestration for the points where fixed rules and conventional software can no longer resolve the case efficiently.
Or, in terms my cloud bill can understand:
Thou shalt not spend tokens on software-engineering problems.