Juan Piaggio · 2026-08-12 · 7 min read · ai · architecture · cost
Most teams choose a model the way they choose a database: once, early, for everything. It is an understandable move, and for the first few months it is the right one. Then the bill arrives, and you notice that the same model reasoning about a governance policy is also the one deciding whether a string is a date.
Model routing is the fix, and it is less exotic than it sounds. It is the recognition that a product is made of steps, that the steps have wildly different difficulty, and that matching capability to difficulty is ordinary engineering rather than a compromise.
Take a single agentic workflow in a ticketing system. An agent reads an incoming issue, classifies it, checks for duplicates, evaluates whether a transition is permitted, drafts a summary, and proposes a state change. Six steps, one model, one price.
But the difficulty spread across those six steps is enormous. Classification into a fixed set of categories is nearly a lookup. Extracting a field from structured text is deterministic work wearing a probabilistic costume. Evaluating whether a transition violates policy, on the other hand, is genuine judgment, and drafting something a human will read without wincing is close behind.
Paying frontier rates for the lookup does not make the lookup better. It makes it slower and more expensive, and it burns latency budget that the judgment step actually needed.
Static tiering assigns a model to each step at design time. You know that duplicate detection runs on the small model and policy evaluation runs on the frontier one, and that mapping lives in configuration. It is predictable, trivially testable, and covers most of the value.
Dynamic escalation — the cascade — runs the cheap model first and promotes only the cases it cannot handle. The promotion signal is the interesting part: not the model's own claim of difficulty, but a check you control, such as a confidence score below a floor, an output that fails a schema validator, or a second attempt at the same step.
route(step) =
classify | extract | reformat → small, fast model
summarize | draft for a human → mid model
judge | plan | multi-step tool use → frontier model
escalate when confidence < floor
OR validator rejected the output
OR this is already a retry
The two compose. Static tiering sets the floor of the system; escalation handles the tail. Start with tiering, because it is one config file and no new failure modes, and add escalation on the specific steps where you can measure that the cheap model is wrong often enough to matter.
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 190" role="img">
<title>A cascade runs the cheap model first and escalates only when confidence falls short</title>
<rect x="1" y="68" width="140" height="52" rx="10" fill="var(--primary-50)" stroke="var(--primary-600)" />
<text x="71" y="99" text-anchor="middle" font-size="14" fill="var(--ink-1)">Incoming step</text>
<path d="M141 94 H180" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M180 89 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="190" y="68" width="152" height="52" rx="10" fill="var(--sand-1)" stroke="var(--border)" />
<text x="266" y="99" text-anchor="middle" font-size="14" fill="var(--ink-1)">Small model</text>
<path d="M342 94 H366 V36 H392 M342 94 H366 V152 H392" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M392 31 l10 5 -10 5 z" fill="var(--text-muted)" />
<path d="M392 147 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="402" y="8" width="186" height="56" rx="10" fill="var(--sand-1)" stroke="var(--warning)" />
<text x="495" y="30" text-anchor="middle" font-size="11" fill="var(--warning)">Confidence below floor</text>
<text x="495" y="49" text-anchor="middle" font-size="14" fill="var(--ink-1)">Frontier model retries</text>
<rect x="402" y="124" width="186" height="56" rx="10" fill="var(--sand-1)" stroke="var(--success)" />
<text x="495" y="146" text-anchor="middle" font-size="11" fill="var(--success)">At or above floor</text>
<text x="495" y="165" text-anchor="middle" font-size="14" fill="var(--ink-1)">Answer accepted</text>
<path d="M588 36 H612 V94 H636 M588 152 H612 V94 H636" stroke="var(--text-muted)" stroke-width="1.5" fill="none" />
<path d="M636 89 l10 5 -10 5 z" fill="var(--text-muted)" />
<rect x="646" y="68" width="148" height="52" rx="10" fill="var(--primary-50)" stroke="var(--primary-600)" />
<text x="720" y="99" text-anchor="middle" font-size="14" fill="var(--ink-1)">Decision ledger</text>
</svg>
There is a tempting version of routing where a model reads the request and decides which model should handle it. It adds a call, adds latency, and makes your routing logic itself non-deterministic — you have introduced a component whose failures are as hard to reproduce as the ones you were trying to manage.
Route on things you already know: which step of the workflow this is, how long the input is, whether tools are involved, whether the output is consumed by code or by a person. These are cheap, legible signals, and a plain function over them is auditable in a way a router model never will be.
Honesty about the downsides is what keeps this from being a slogan.
You now have more than one behavior to evaluate. A prompt tuned on a frontier model will not necessarily survive on a small one, so each route needs its own regression set. That is real work, and it is the reason to keep the number of routes small — three tiers is plenty, seven is a maintenance burden pretending to be optimization.
You also lose a little consistency. Two similar tickets handled by different tiers can produce differently-worded output, which users notice even when both answers are correct. Formatting the tail of every route through the same template is usually enough to hide the seam.
And you inherit a debugging obligation: when something looks wrong, the first question is which model produced it. That is only answerable if you wrote it down.
Routing is a governance surface, not just a cost lever. Every proposal an agent makes in Meshworq lands in the decision ledger under a correlationId, and the model that produced it belongs in that record alongside the confidence score and the eventual human verdict.
That one field turns several vague arguments into measurable ones. You can compare acceptance rates per tier and find the steps where the cheap model is quietly costing you human review time. You can see whether escalations actually rescue the cases that triggered them, or whether the frontier model is just producing a more confident version of the same mistake. And when a model version changes underneath you, the ledger is where the regression becomes visible before a customer finds it.
A route you cannot see in the audit trail is not an optimization. It is an untracked variable in production.
One model for everything is a starting position, not a design. Split your workflow into steps, assign the cheapest model that reliably clears each one, and add escalation only where you can measure that it pays. Route on facts you already have rather than on another model's opinion, keep the number of tiers small enough that each can be evaluated properly, and write the chosen model into the same audit record that carries the confidence score and the human decision. Done that way, routing gives you lower cost and lower latency without giving up the thing that matters more than both: knowing why the system did what it did.