model: "auto" and Orbitrage runs your prompt through a six-step pipeline.
The goal: the cheapest model that can actually handle this task.
auto vs. pinning a model
Let Orbitrage route
Pass
auto (or router, default, orbitrage). The engine scores the
prompt and selects a model for you.Pin a specific model
Pass any concrete model id (e.g.
glm-5.2, DeepSeek-V4-Flash, or a
BYOK model like claude-sonnet-4-6). Scoring is skipped —
the request goes straight there.The six-step pipeline
1
Normalize
The request (chat, Responses, or legacy completion shape) is normalized to a
common internal form, so the rest of the pipeline is uniform.
2
Score
The prompt gets a difficulty score in
[0.05, 0.95]. Two signals compete:- 70+ heuristics — fast regex rules (sub-millisecond, no model to load) that lower the score for extraction, formatting, classification, and classic exercises; raise it for reasoning, debugging, strategy, and long/complex prompts.
- Explicit annotation — a caller-supplied priority overrides the score.
3
Capability ceiling
If the call declares a capability type, the score is capped so trivial work
can’t escalate to an expensive tier. A
formatting task is capped low;
reasoning and planning are uncapped.4
Dial
A per-deployment dial (0.0–1.0) shifts tier thresholds. Lower =
conservative (stay cheap longer); higher = aggressive (escalate sooner).
5
Select tier + model
The score maps to a tier, then the engine picks a concrete model: a
vision-capable model when the prompt has images, a code-biased model for code,
the cheapest open model otherwise. Long prompts escalate automatically;
trivially simple code de-escalates.
6
Proxy + fallback
The request is proxied to the provider. On an infrastructure error (5xx, 429,
connectivity), a fallback chain of 2–5 models is tried across providers.
Client errors (4xx, content filters) do not cascade — they return
immediately.
Tiers
Models are grouped by capability and cost. Routing climbs only as high as the prompt needs.The frontier tier is BYOK-only. Auto routing considers
those models only when your organization has an enabled key for the vendor; with
no key it stays on the open-weight models Orbitrage serves and never returns
byok_key_required.Reading the routing decision
Every routed call records the model it chose and why. On the dashboard’s Routing page (and each span):- Requested → Routed to — the alias you sent vs. the model used
- Tier and priority score — what the prompt scored and where it landed
- Signals — the heuristics that fired (e.g.
code detected,long prompt) - Fallback chain — the models that would have been tried on failure
- Saved — the cost difference vs. a frontier baseline
X-Orbitrage-Overhead-Ms response header, so you can
see exactly how much latency it added on top of the provider.
Forcing behavior
Always use one model
Always use one model
Pin the model id on every call. A concrete id is treated as an explicit pin
and skips scoring entirely.
Use a frontier model (GPT, Claude, Gemini, Grok)
Use a frontier model (GPT, Claude, Gemini, Grok)
Save and enable a key for that vendor on the
Models page. The call is forwarded to the
real provider with your key, billed by them at your rate, and Orbitrage charges
$0. Without an enabled key those models return
403 byok_key_required. See
BYOK.Bias the whole project cheaper or smarter
Bias the whole project cheaper or smarter
The operator dial shifts tier thresholds for your deployment. Lower it to keep
traffic on cheaper tiers; raise it to escalate sooner.