There is a moment early in building any agent system where a language model starts to look like a hammer, and every problem starts to look like a nail. You have a model that can read, reason, and write, so the temptation is to route everything through it: parse this, sort that, decide the order, apply the rule. It works in the demo. It is also, quietly, one of the most expensive architectural mistakes you can make.

The useful question is not "can the model do this step." It usually can. The question is "does this step have a correct answer that does not depend on interpretation." When the answer is yes, that step belongs in ordinary code, and handing it to a model makes it slower, costlier, and less reliable in exchange for nothing. The whole craft of deterministic vs LLM design is learning to see that line and build on the right side of it.

The line: judgment versus correctness

Draw the line at one question. Does the step require judgment over ambiguous or natural-language input, or does it have a single right answer that a function could compute?

Models earn their cost on the first kind of work. Reading a vague requirement and asking a clarifying question. Deciding whether a diff actually satisfies an acceptance criterion. Turning a messy description into a structured plan. These are judgment calls over fuzzy input, and there is no if statement that captures them. That is what a model is for.

The second kind of work is where teams overreach. Anything with a deterministic right answer belongs in code:

  • Sorting or ordering a list by a defined rule.
  • Validating a payload against a schema.
  • Deduplicating records by a key.
  • Retrying a failed call with backoff.
  • Applying a scoring formula or a fixed policy.
  • Parsing a known format, or routing by a rule you can write down.

None of these need a model. They need a function. A function that sorts is correct every time, costs nothing per call, runs in microseconds, and can be unit tested to exhaustion. Wrap that same sort in a language model and you have made it probabilistic, slow, metered by the token, and impossible to fully test. You took something that was already solved and un-solved it.

Why the LLM version is worse, concretely

It is worth being specific about what you lose, because "use code where you can" sounds obvious and gets ignored anyway.

You lose determinism. A function that dedupes a list returns the same answer on Tuesday that it returned on Monday. A model asked to dedupe a list will usually get it right, and the word "usually" is the whole problem. Occasionally it drops an item, invents one, or reorders in a way you did not ask for, and because the failure is rare and plausible, it slips through review and surfaces later as a bug you cannot reproduce.

You lose speed and money. Every model call is a network round trip metered in tokens. Doing arithmetic or string manipulation that way is paying premium latency and cost for an operation your CPU would do for free. Across a pipeline that runs thousands of times, that is not a rounding error.

You lose testability. Deterministic code has a property that is easy to undervalue until you do not have it: you can write a test that passes or fails. A step whose output varies run to run cannot be pinned down the same way, so your confidence in it comes from sampling and vibes rather than from a green check. For a system that ships to production, that is a real downgrade.

The gray zone, and how to handle it

Most real steps are not purely one or the other. A step might be mostly mechanical with a small judgment buried in it: process these items in a fixed way, but decide which ones are ambiguous enough to flag. The instinct is to hand the whole thing to a model because part of it needs judgment. The better move is to split it.

Extract the judgment into a small, well-scoped model call, and keep everything around it in code. Let the model make the one decision it is uniquely good at, return a structured answer, and let deterministic code do the rest: the iteration, the ordering, the validation, the side effects. This keeps the surface where things can go wrong small and legible, which is the same reason small specialized agents beat one autonomous mega-agent. A narrow model call with a clear contract is testable in a way a sprawling one is not, and designing agent handoffs and contracts is largely the discipline of drawing those boundaries cleanly.

This also reframes what "orchestration" should be. The glue that moves work between steps, decides what runs next, and enforces the rules is almost always better as deterministic code than as a model improvising control flow, which is the argument in orchestration vs autonomy. Let the model decide the hard content question; let code decide the sequence.

The honest limitation

The line is not fixed, and pretending it is has its own failure mode. Sometimes a model is the pragmatic bridge for something you could code but have not yet, or for input so varied that writing exhaustive rules would be its own tar pit. Natural-language parsing that used to demand a brittle mountain of regexes is a fair place to let a model earn its keep. Drawing the line too aggressively toward code can leave you maintaining rigid logic that a model would handle more gracefully, so this is a judgment, not a dogma.

And "put it in code" assumes the deterministic version is actually simpler. When the rules are genuinely complex and always shifting, a model may be the lower-maintenance option even though it is less predictable. The point is not that code always wins. It is that you should choose deliberately, on the axis of correctness versus judgment, instead of defaulting to the model because it is there.

This is a principle we lean on hard in Loopsfinity: the parts of the system with a right answer are ordinary, testable code, and the model is reserved for the judgment only a model can make. Where exactly we draw that line internally is our own work, but the discipline is not proprietary, and it is the one that separates an agent system you can trust in production from an impressive demo that flakes in ways you cannot explain. The wider picture is in AI agent architecture.