Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems

Why This Matters

The prevailing consensus in frontier AI often pursues an "all-knowing, all-capable" unified foundation model. Yet in production, no single model dominates every discipline: a specialist diffusion model produces photo-realistic portraits while failing at logical code debugging; a Vision-Language-Action (VLA) policy excels at tidy tabletops but falls apart under randomized visual noise.

Meanwhile, deploying the largest frontier model (e.g. Claude Opus or GPT-4/5 level) for every single call is economically ruinous and computationally sluggish. While routing has been studied for text LLM cascades, researchers and practitioners lacked an empirical and theoretical foundation answering three pivotal questions across all modalities:

• How much headroom actually exists? If we perfectly routed every input to its best solver, how much better is that than just picking the single best model on the market?
• Can real learned routers achieve it? Or do classification errors wipe out the theoretical margin?
• When is a cheap model good enough? Can we guarantee that cost savings will not secretly degrade system quality?

The Big Idea

How It Works: The Theoretical Foundation

1. Value, Cost, and Dinkelbach Duality

Let an input $x$ arrive from distribution $\pi$. The router observes a state representation $s = \sigma(x)$ and selects a candidate model or policy $u \in \mathcal{U}(x)$. Running candidate $u$ yields a normalized benchmark quality value $V_u \in [0, 1]$ while consuming resource cost $D_u > 0$ (e.g., dollars or latency in seconds).

Under a resource price $\lambda \ge 0$, an optimal router $g_\lambda$ maximizes expected net surplus:

When maximizing resource efficiency $\eta(g) = \frac{\mathbb{E}[V_g]}{\mathbb{E}[D_g]}$ subject to a strict quality loss floor $Q = \mathbb{E}[V_g - Y_B] \ge -\epsilon$ against reference policy $B$, Theorem 2 proves using Dinkelbach's fractional programming method and LP duality that the constrained optimal policy is simply the price maximizer at an effective Lagrange multiplier:

This provides an indispensable realization: maximizing efficiency under a quality floor introduces no esoteric policy architectures; it is equivalent to standard Lagrangian routing where the market reference and user tolerance $\epsilon$ rigorously determine the exchange price $\lambda_\epsilon$.

2. The Information Ladder

What a router can achieve depends entirely on the granularity of what it observes ($s = \sigma(x)$). Proposition 3 establishes the fundamental ordering:

The left-most rung is the best fixed model per benchmark. The right-most rung is the theoretical per-input oracle. The gap between $V^\star(x)$ and the oracle is irreducible aleatoric noise (e.g. prompt variance where an LLM succeeds 50% of the time like a coin flip). Crucially, the authors discover that task-type labels capture 8% of the potential gain on benchmarks with such labels. This is not a variance decomposition across all modalities, and the oracle uses outcomes unavailable before execution.

Routers That Read the Scene: Embodied AI

In physical robot manipulation (evaluated on RoboTwin 2.0 with 50 bimanual tasks and 2,000 test scenes), candidate policies have diametrically opposed failure modes:

• PatchWAM: Dominates structured, clean laboratory environments (81.7% success), but collapses under visual distractor noise (45.6%).
• ME-Dex: Robust against heavy domain-randomization (62.5% success), but less precise on canonical scenes (78.4%).

Instead of forcing one policy to operate everywhere, the authors train a lightweight router reading three camera views ($240 \times 320$ pixels) plus proprioceptive state. It dynamically sends clean scenes to PatchWAM and noisy scenes to ME-Dex.

The Training Target Trap: Outcomes vs. Winners

A deceptively simple intuition is to formulate routing as multi-class classification: train a model to emit the categorical label of the winning candidate. The paper gives a counterexample to winner-label consistency and observes a large loss in its RoboTwin experiment.

In physical systems with binary success, ties are typically broken by latency. On RoboTwin, multiple policies succeed on most training scenes, causing a latency tie-break to decide 94% of the winner labels. As a result, 24,556 out of 31,408 successful runs were treated as "negative/wrong-class" samples! The winner classifier learned to select brittle 1.9-second policies that failed repeatedly, causing success to crash to 42.8% compared to 71.6% for per-candidate outcome regression.

Empirical Results Summary

Table 5 reports different testbeds, resource units and quality tolerances. The 8.82× arithmetic mean combines 25.5× and 7.1× dollar efficiency, 1.35× image efficiency at a half-point fitted tolerance, and 1.33× robot time efficiency; it is not a single certified zero-loss result:

Limitations & Open Questions

Glossary

Citation

@article{zhou2026routeanything,
  title={Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems},
  author={Zhou, Pengfei and Ma, Yixing and Wang, Hexin and Tang, Weidong and Wan, Zhenglin and Zhao, Wangbo and You, Yang},
  journal={arXiv preprint arXiv:2611.05275},
  year={2026}
}