Skip to content
All posts

Do you need an AI agent, or a workflow with one smart step?

Agents are the most hyped way to build with LLMs and often the hardest to run in production. Here is how to decide whether your feature needs one.

Shubham Somani 3 min read

Plenty of first conversations about AI start with “we want to build an agent”. Sometimes they do need one. More often, what they describe is a workflow: a fixed sequence of steps where one or two of them need a language model.

The difference matters. Workflows are easier to test, cheaper to run and simpler to debug. Agents can handle problems a workflow can’t, but you pay for that flexibility every day they run in production.

What we mean by each

A workflow is code that calls a model at specific points. Read the ticket, classify it with the model, look up the account, draft a reply with the model, send it to a person for approval. You decide the steps; the model fills in the parts that need judgement.

An agent is a model that decides the steps itself. You give it a goal and a set of tools, and it chooses what to call, in what order, and when it’s done. The control flow lives in the model, not in your code.

Why workflows win by default

  • They’re predictable. The same input takes the same path, so a failure points to one step you can fix.
  • They’re testable. You can write evals for each step and for the whole run, as you would for any pipeline.
  • Their cost is bounded. You know how many model calls a run makes. An agent can loop, retry or wander, and every extra step costs tokens and time.
  • They’re easier to explain. When a customer asks why something happened, you can show the path it took.

If you can write the steps down on a whiteboard, build the workflow.

When an agent earns its place

Reach for an agent when the steps genuinely can’t be known in advance:

  • The path depends on what it finds. Investigating a bug, researching across many sources, or working through a codebase, where each step decides the next.
  • The number of steps varies a lot from one request to the next, and hard-coding every branch would be a maze.
  • A person would do this task by exploring, not by following a checklist.

Even then, start with a narrow agent: a few tools, a clear goal and a hard limit on steps.

If you build an agent, build the guardrails first

  • Small, well-described tools. The model picks tools by their names and descriptions. Vague tools lead to wrong calls.
  • Read-only by default. Let it look things up freely. Anything that writes, sends or spends goes through a person or a strict rule.
  • Step and cost limits. A hard cap on turns and tokens per run, with a clean failure when it’s hit.
  • Full traces. Log every tool call and its result so you can replay a run that went wrong.
  • Evals on whole runs, not just single answers: did it reach the goal, how many steps did it take, and did it do anything it shouldn’t?

A middle path that often wins

Many good AI features sit in between: a fixed workflow where one step is a small agent with a narrow job. The outer workflow keeps cost and behaviour predictable. The inner agent handles the one part that really needs flexibility. You can always widen its scope later, once your evals show it’s reliable.

The short version

  • If you can draw the steps, build a workflow and use the model where judgement is needed.
  • Use an agent only when the path depends on what it discovers.
  • Start agents narrow, read-only and capped, with full traces.
  • Evaluate whole runs, including cost and the number of steps.
  • Consider a workflow with one small agent inside before building a fully autonomous one.

Not sure which one your idea needs? Book a 30-minute call and we’ll sketch both on a whiteboard with you.