Router
Decide which model a message history goes to.
A router decides which model a message history goes to. It is constructed
with an inference stack and can be used anywhere a model id is accepted —
chat({ model: router }), generateObject, streamText, and as an agent's
model.
The interface
interface Router {
readonly name: string;
readonly stack: InferenceStack;
pickModel(messages: Message[], options?): string | Promise<string>;
}pickModel receives the message history and returns the model id (from the
stack) to use.
Factories
import { simpleRouter, roundRobinRouter, llmRouter } from "yoke";
// first matching rule wins, else the fallback
const cheap = simpleRouter({
name: "cheap",
stack,
rules: [
{
model: "gpt-4o-mini",
when: (messages) =>
messages.filter((m) => m.role === "user").length <= 2,
},
{
model: "deepseek/deepseek-v4-flash-0731",
when: (messages) => messages.some((m) => m.role === "tool"),
},
],
fallback: "gpt-4o",
});
// round-robin across candidates
const balanced = roundRobinRouter({
name: "balanced",
stack,
models: ["gpt-4o-mini", "claude-3-5-haiku"],
});
// a cheap judge model picks among candidates
const smart = llmRouter({
name: "smart",
stack,
judgeModel: "gpt-4o-mini",
models: ["gpt-4o", "claude-sonnet-4", "deepseek/deepseek-v4-flash-0731"],
});Using a router as a model
const { message } = await ctx.inference.chat({
model: cheap, // a Router, not a model id
messages,
});Routers compose with everything that accepts a model:
chat({ model: router })generateObject({ model: router })streamText({ model: router })Agent({ model: router })
Building custom routers
Implement the Router interface when the built-ins don't fit:
import type { Router, InferenceStack } from "yoke";
function visionAwareRouter(stack: InferenceStack): Router {
return {
name: "vision-aware",
stack,
pickModel(messages) {
const hasImage = messages.some((m) =>
m.content.some((c) => c.type === "image"),
);
return hasImage ? "gpt-4o" : "gpt-4o-mini";
},
};
}