Model routing is becoming a key component of the enterprise AI stack, dynamically sending directions to the correct AI model to optimize speed and costs. However, current frameworks primarily treat routing as a static classification problem, which severely limits its potential.
A new open source framework called Agent as router addresses this bottleneck by treating the router as a dynamic agent that creates memory. It uses a Context-Action-Feedback (CAF) loop to track model successes and failures and update the router’s behavior.
The researchers also released ACRouter, a concrete implementation of this paradigm. In its tests, ACRouter significantly outperformed static routers and the costly strategy of opting for premium models by default, all without requiring teams to train massive models or write endless heuristics.
For real-world applications, this framework offers the option of replacing hard-coded AI infrastructure with self-optimizing systems that can adapt to changes in user behavior and the base models used in the enterprise AI stack.
The economics of routing and the information deficit
Single-model setups are useful for experiments, but detrimental for scaling AI applications. AI engineers use model routing assign tasks to cheaper and faster open models when possible, while reserving expensive frontier models for complex reasoning.
Currently, developers rely on two main mechanisms for this task. The first is heuristics-based routing, which relies on hard-coded manual rules. For example, a developer could write a rule that states that if a message contains certain keywords, it is routed to GPT-5.5. Otherwise, you opt for a self-hosted open source model like Kimi K2.7.
The second mechanism is trained static policies. These are machine learning classifiers trained on historical data sets that analyze message embeddings and predict the best model based on previous training data.
Both approaches are static. When researchers tested these existing mechanisms in real-world coding and agent workflows, they found a hard limit on accuracy. The key finding shows that static routers suffer from a serious information deficit. Because they only evaluate the input text and never see whether the model actually managed to execute the task, they guess blindly when faced with complex edge cases.
This results in three distinct points of failure. First, static routers suffer from a frozen information state, meaning they cannot accumulate new execution feedback during deployment. Second, they fail to generalize out of distribution (OOD). They break during second-day operations, when business data or user behavior changes because your training data no longer matches reality. Finally, they are very vulnerable to model abandonment. A static classifier trained with current models may become obsolete when a better model comes out next week.
Agent as router: a self-evolving system
The central thesis of the agent as a router is that a truly effective router must acquire and accumulate execution-based information during deployment, essentially learning on the job.
The researchers achieved this through the CAF circuit. When a new message arrives, the router examines the message and the task’s metadata, such as programming language or difficulty. Then you search your historical memory for similar tasks to see which models succeeded or failed in the past. The router uses this context to select the destination model and execute the task. Finally, the system observes the real-world outcome, extracts a success or failure signal, and writes this feedback to its memory to inform future routing decisions.
Consider an automated business data analysis process. The router receives a SQL generation task and sends it to an open source model such as Kimi. The model is confused by the name of a column and cannot compile the SQL. The CAF loop observes the compiler error, records it as feedback, and logs it. The next time a similar obscure SQL query arrives, the router checks its context and routes the task to a more advanced model like Claude Opus 4.8.
AC router
The researchers developed ACRouter as the concrete instance of this framework. It is made up of three main components: the orchestrator, the verifier and the memory. This architecture is supported by a tooling layer to physically execute the CAF loop.
The Memory module powers the context phase. Built on a vector store, it retrieves relevant past interactions and updates the historical database with new results. The orchestrator manages the action phase. Processes the user message along with the recovered memory to select the most capable target model from the available pool. The Verifier manages the feedback phase by evaluating the outcome of the chosen model to generate a clear signal of success or failure.
The tooling layer connects the Verifier to real-world execution environments, such as a Python code interpreter, an agent sandbox, or a database engine. The tooling layer allows the system to run the generated code or query and observe the exact result, providing the verifiable signal the router needs to learn.
The Orchestrator itself is lightweight. Instead of a massive, computationally heavy language model, the researchers trained a billion-parameter adapter based on Qwen 3.5 (0.8B parameters), meaning it can self-host on a device of your choice.
ACRouter in action: surpassing frontier baselines
To test the framework, the researchers introduced CodeRouterBench, an evaluation environment comprising approximately 10,000 tasks with scores verified on eight frontier models, including Claude Opus 4.6, GPT-5.4, Qwen3-Max, and GLM-5. The evaluation was split between in-distribution (ID) tests (covering nine single-round coding dimensions such as algorithm design and test generation) and an out-of-distribution (OOD) agent programming testbed. The OOD tasks were qualitatively different and required multi-step planning, file browsing, and iterative debugging to see if the router could adapt to fundamentally new domains.
The baseline results revealed why a single-model strategy is flawed: no model dominates all categories. For example, although Claude Opus 4.6 achieved the highest average performance, it was outperformed in algorithm design by GLM-5 (an 86% relative improvement) and in test generation by Qwen3-Max (a 111% improvement), even though Opus costs approximately 12 times more than smaller models such as Kimi-K2.5.
In benchmarks, static routers continually failed to send a specific niche encoding task to a model ill-equipped for that exact syntax. The static router had no way of knowing that the code was not running. In contrast, ACRouter adjusted its strategy after receiving a negative feedback signal from the execution environment.
According to the researchers’ benchmarking, ACRouter sits firmly on the Pareto frontier in terms of cost and performance. In both identification task flows and complex OOD agent tests, ACRouter achieved the lowest cumulative regret, a metric that measures suboptimal routing decisions over time. On the distribution test set, ACRouter cost $13.21 over the entire task run, compared to $34.02 for always using Opus by default – a 2.6x savings.
It dynamically matched tasks to the most capable model for that specific niche, suggesting that companies can achieve or exceed frontier-level accuracy across diverse workloads without paying a premium price for each query.
Warnings, limitations and how to get started
While the agent-as-router paradigm solves the information deficit, it is not a general solution for all AI workflows.
The framework shines in verifiable tasks where the Verifier gets a clear signal of success or failure from the environment, such as encoding or data retrieval. It is effective for applications with changing distributions and domains where different models excel in completely different niches.
In contrast, the configuration is overkill for trivial tasks where any model will suffice, or for low-volume applications that do not justify the engineering overhead. It is also not suitable for subjective domains, such as creative writing, where it is not easy to verify a correct answer and feedback signals are impossible to standardize.
Open Source Researchers the code on GitHub and released the Orchestrator Model Weights in Hugging Face under the Apache 2.0 license. The router supports Claude Code, Codex and OpenCode.
