Cost-based routing is the practice of sending each request to the cheapest model that still clears your quality bar — and it is the fastest way most teams can cut an LLM bill, because it changes which prices you pay instead of what you build. The idea is exactly what llm router means inside a routing layer: an AI router grades every prompt, applies a cost floor, and dispatches each request to the model that qualifies at the lowest price. OrcaRouter runs this out of the box, and the live rate card for GLM-5.3 shows what a strong non-flagship workhorse costs per million tokens — the per-model pricing the router is choosing between on every call.
Most teams still do the opposite. They pick one flagship model, wire it to a single endpoint, and accept the bill. The hidden cost is not that the flagship is overpriced — it is that a very large share of traffic does not need a flagship at all. A support query, a tag extraction, a summarization job: these have a quality bar, and almost always a lower one than “the most expensive model available.” Cost-based routing exists to capture that gap, per request, without letting quality slip through the cracks.
What cost-based routing actually is
Cost-based routing is a routing policy: for each incoming request, pick the cheapest model that still satisfies the task’s quality requirement. It is not “use the cheap model for everything,” and it is not “negotiate a better deal once.” It is a per-request decision. The prompt is graded first — is this hard reasoning or straightforward extraction? — and the grade determines a minimum acceptable model tier. The router then applies a cost floor: never spend more than the cheapest model in that tier. The request goes to the first model that clears the bar at the lowest price.
Three ingredients make the strategy work in practice: a prompt grader (to know what the request actually needs), a cost floor (to keep the choice honest), and a routing layer with enough models on tap that the cheapest qualifying choice is genuinely cheap. Without the third ingredient, cost-based routing is a thought experiment. If your gateway fronts only two models, the cheapest qualifying model is one of two, and the strategy has almost nothing to choose between.
Why it works: the price gap is the whole game
The reason this strategy pays is arithmetic. On input price alone, frontier flagships commonly run 10–25× more expensive than economy models, and output pricing for reasoning-heavy flagships widens the gap further [independent: published list prices on neutral leaderboards and vendor rate cards, August 2026]. When the cheapest qualifying model for a task is not the flagship, you are not shaving a few percent — you are moving spend across a gap measured in multiples.
The tiers line up roughly like this:
| Model tier | Relative input price | Typical fit under cost-based routing |
| Frontier flagship | 10–25× the economy baseline [independent] | Long-horizon reasoning, agent planning, hardest code |
| Mid-tier workhorse | A few × the baseline | Most general chat, tool calls, everyday generation |
| Economy / fast models | Baseline | Extraction, classification, summarization, routing |
The important column is the last one. A routing policy does not need to match a model to a task perfectly; it needs to be right at the boundaries. Every request that does not need the flagship and does not get the flagship is pure saving, and the majority of production traffic usually lands in that camp.
The mechanism: grade the prompt, set the floor, route
The mechanism is a three-step loop. First, grade the prompt. The router reads the request and scores what it needs — reasoning depth, tool use, context length. OrcaRouter performs this grading in under 1 millisecond per prompt [OrcaRouter], fast enough that the decision costs less than the latency you would lose on a single misrouted call. Second, apply the cost floor. The grade maps to a minimum model tier, and the floor says: do not spend above the cheapest model in that tier. Third, route. The request goes to the qualifying model with the lowest price, with load balancing and automatic failover behind it so a model going down becomes a retry, not an outage [OrcaRouter].
Two properties make this realistic rather than theoretical. One API key for 200+ models gives the router real choice — economy models, mid-tier workhorses, and flagships from every major lab behind a single endpoint [OrcaRouter]. And prefix-aware routing means a long repeated system prompt does not get re-graded from scratch on every turn, which keeps the per-request decision cheap at high volume [OrcaRouter].
What you actually get: blended cost down, quality held
The real-world effect of cost-based routing shows up in two places. Blended cost falls — the average price per request drops because the mix of models your traffic hits changes, not because anyone discounted anything. Quality holds — because the routing decision is governed by the quality bar, not by price alone. A prompt that genuinely needs the flagship still gets the flagship; the savings come from the majority of traffic that does not.
The predictable second-order effect is on planning. Once the router owns the decision, your bill stops being “flagship price × everything” and starts being a function of your workload mix. The reasoning-heavy slice becomes a visible line item instead of diffuse spend spread across all traffic. That is the difference between a bill you react to and a bill you budget.
The trap: cheap without a floor, and markup hiding the real price
There are two ways this strategy fails, and both are common.
The first is routing everything to the cheapest model. Cost-based routing without a quality floor is just cost-based cutting: the grader is ignored, the floor is removed, and the router sends hard reasoning to an economy model. You save money and lose the task — a silently wrong answer, a truncated agent loop, support traffic that comes back around because the first answer was bad. The cost floor exists precisely to prevent this. If you cannot grade the prompt, you cannot set a floor, and you should not attempt the strategy.
The second trap is buying routing through a layer that hides the real price. A “discounted” flagship is often a list price with a markup underneath it; the number you plan against is not the number you pay. That is why the pricing side matters as much as the routing side. If the routing layer marks up the models it fronts, every saving from the strategy is partially absorbed before you see it. Passing vendor list prices through at 0% markup — with glass-box receipts per request — keeps the floor honest [OrcaRouter]. The blended rate you compute is the blended rate you are billed.
The takeaway
Cost-based routing is for teams whose traffic does not uniformly need a flagship — which is nearly every team with a real workload. It pays most when your request mix spans extraction, chat, and occasional hard reasoning, because that spread is where the price gap does the work. It pays least when every request is genuinely hard, where the cheapest qualifying model is the flagship anyway. The decision rule is simple: grade the prompt, set the floor, route to the cheapest model that clears it. And buy the routing through a channel that passes list prices through unchanged, as OrcaRouter does, so the strategy’s savings are real savings.
Sourcing note: product facts — 0% markup with vendor list prices passed through unchanged, one API key for 200+ models, per-prompt grading in under 1 ms, adaptive routing, automatic failover, prefix-aware routing, and glass-box receipts — are from OrcaRouter’s official pages, verified 2026-08-22. The 10–25× input-price gap between frontier flagship and economy tiers is derived from published list prices on neutral leaderboards and vendor rate cards, checked August 2026. Vendor pricing changes without notice.
