Inference

Inference is a primitive like any other: providers, models, and a stack, with the Vercel AI SDK hidden behind an internal translator.

The pieces

Provider — the contract for a model provider. Provider packages (@yoke/provider-openai, @yoke/provider-openrouter, @yoke/provider-mock) implement it with their SDK. The resolver is attached under an internal symbol, so Vercel types never leak to consumers.
ModelSpec — { id, provider, model, capabilities? }. The model id is what agents use in ctx.inference.
InferenceStack — the mapping of models → which provider serves them.

Building an inference stack

import { inferenceStack } from "yoke";
import { openaiProvider } from "@yoke/provider-openai";
import { openrouterProvider } from "@yoke/provider-openrouter";

const stack = inferenceStack({
  providers: [
    openaiProvider({ apiKey: process.env.OPENAI_API_KEY }),
    openrouterProvider({ apiKey: process.env.OPENROUTER_API_KEY }),
  ],
  models: [
    { id: "gpt-4o", provider: "openai", model: "gpt-4o" },
    { id: "deepseek/deepseek-v4-flash-0731", provider: "openrouter", model: "deepseek/deepseek-v4-flash-0731" },
  ],
});

Hand the stack to the runtime:

const runtime = Runtime({ inference: stack })     // at creation
  .withInference(stack);                          // or later

Using ctx.inference

Model calls are stateless: messages in → result out.

// chat — messages in, assistant message out
const { message, finishReason, usage } = await ctx.inference.chat({
  model: "gpt-4o",
  messages: [
    systemMessage("You are helpful."),
    userMessage("What is the capital of France?"),
  ],
});

// structured output — generate an object from a zod schema
const { object } = await ctx.inference.generateObject({
  model: "gpt-4o",
  schema: z.object({ city: z.string(), country: z.string() }),
  messages: [userMessage("Paris")],
});

// embeddings
const { embeddings } = await ctx.inference.embed({
  model: "text-embedding-3-small",
  values: ["yoke", "primitives"],
});

// streaming
const stream = await ctx.inference.streamText({
  model: "gpt-4o",
  messages: [userMessage("Count to ten.")],
});

// available models
const models = await ctx.inference.listModels();

Inference uses the same Message type as the chat store (with a role: system / user / assistant / tool). Chat history is fed straight to ctx.inference.chat; the assistant reply is a Message ready to persist.

chat({ tools }) accepts a tool, a toolchain, or a list. chat({ model }) accepts a model id or a Router.

The agent-owned tool loop

The SDK never executes tools and never loops. Loop control lives on the agent:

Declare tools in chat({ tools }).
The model returns tool calls in the response.
You execute them yourself via ctx.tools / ctx.callTool.
Append toolResultMessage(...) and call again.

const reply = await ctx.inference.chat({
  model: "gpt-4o",
  messages,
  tools: [forecastToolchain],
});

for (const call of reply.message.content.filter((c) => c.type === "tool_call")) {
  const result = await ctx.callTool(call.toolName, call.arguments);
  messages.push(
    toolResultMessage(call.toolCallId, result, call.toolName),
  );
}

inferenceLoop

inferenceLoop codifies this loop — including error recovery. If executing a tool throws, the error is fed back to the model as a tool_result so it can correct and retry:

import { inferenceLoop } from "yoke";

const { message, steps } = await inferenceLoop({
  ctx,
  model: "gpt-4o",
  messages,
  tools: [forecastToolchain],
  maxSteps: 5,
});

The high-level Agent({ ... }) primitive wraps this loop (plus error recovery) for you.

Providers

Providers live in separate packages implementing the Provider interface. Never expose Vercel types from the framework.

import { openaiProvider } from "@yoke/provider-openai";
import { openrouterProvider } from "@yoke/provider-openrouter";
import { mockProvider } from "@yoke/provider-mock";