Design · Orchestration
LLM Routing (RouteLLM)
A learned router scores each query and sends easy ones to a cheap model, hard ones to a strong model. Trained on human-preference signals to hold quality while cutting spend.
Verified against arxiv.org
How it works
A learned router scores each query and sends easy ones to a cheap model, hard ones to a strong model. Trained on human-preference signals to hold quality while cutting spend.
At a glance
TechniqueRouting
Cost leverMuch lower, up to 85% cost cut at GPT-4-level quality on MT-Bench
Latency leverLower on average, most queries hit the small model
Quality leverNear-frontier when routed well
ExampleLMSYS RouteLLM · Martian · OpenRouter auto
When to use
High-volume mixed-difficulty traffic where most queries are easy and a minority need a frontier model.
Basis
Verified against
https://arxiv.org/abs/2406.18665 The corpus does not rewrite vendor documentation. Where copy is quoted verbatim it is marked as such and attributed to its source.