Rule, Recommend, Decide
Model selection comes up too early in most agent design reviews. Before choosing a provider, label every step in the production path with what it is actually allowed to do.
Model selection usually comes up too early in agent design reviews.
Which model? Which provider? How large should the context window be?
Before discussing any of that, I draw the production path and give each step one label: Rule, Recommend, or Decide.
The three labels
A step is a Rule when the correct next action is already known. If an amount exceeds a limit, send it for approval. No model is needed. A simple condition is cheaper, faster, easier to test, and easier to defend when someone asks why the system did what it did.
A step is Recommend when the input is ambiguous but the consequence of a wrong decision is significant. A model might review a contract, identify potential risks, and suggest what to do next. It should not necessarily execute that action. The output is an input to a person, and the design should make that obvious rather than burying a human check behind a default-accept button.
A step is Decide only when three conditions hold together: there are several acceptable paths, the choice genuinely depends on context, and the cost of a wrong decision is limited or reversible.
That third condition carries most of the weight, and it is the one most often waved through. Reversibility is what makes autonomy affordable. If a wrong decision cannot be undone, the step does not belong in Decide regardless of how capable the model is.
The two mistakes this exposes
Running this exercise on a real design tends to surface the same two errors.
Using a model where a few lines of code would be enough. This is the more common of the two and the easier to fix. It shows up as a model call that only ever produces one of three answers, or a prompt that reimplements a threshold comparison in natural language. It adds latency, cost, and a failure mode, in exchange for nothing.
Giving a model decision authority where it should only provide a recommendation. This is rarer and considerably more expensive. It usually happens by drift rather than decision: a step ships as Recommend with a person reviewing every output, the reviews are consistently fine, the review becomes a formality, and eventually someone removes it to reduce friction. Nothing was ever explicitly promoted from Recommend to Decide. The boundary simply eroded.
Labeling the steps makes that drift visible, because a promotion becomes a change to a document rather than a quiet change in behavior.
Why this has to come first
Only after those boundaries are clear does model selection become useful.
The labels tell you what you are actually shopping for. A path that is mostly Rules with one Recommend step does not need the largest available model; it needs a reliable one with predictable latency and a cost that survives your request volume. A path with a genuine Decide step needs stronger reasoning, but it also needs an evaluation harness, a permission boundary, and an escalation route, and those requirements shape the architecture far more than the choice of provider.
Choosing the model first inverts this. It fixes the most replaceable component before the constraints that determine everything else are known.
Running the exercise
It takes less time than it sounds. Draw the production path as it exists today. Give every step exactly one label. Where a step resists a single label, that is usually a sign it is really two steps that have been collapsed into one, and separating them is worthwhile on its own.
Then look at the Decide steps and check each against the three conditions, particularly reversibility. In my experience most designs come out of this with fewer Decide steps than they went in with, and the ones that remain are much easier to justify.