Global Stream

The global stream is Yoke's single sequencing authority. Every action that matters — actor work, portal ingress, instance lifecycle, conversation messages — becomes a durable row in stream_events.

User code can append only. There is no tail, subscribe, or read API exposed to TypeScript. The Rust kernel tails internally for fan-out catch-up and audit.

Event shape

Events stored in Postgres match the SDK StreamEvent type:

priority and group_id propagate from stream events into actor_queues on fan-out.

PgStream

PgStream implements the Stream trait. It holds a Postgres connection pool and a TenantId.

Per-tenant sequencing

Sequence numbers are not global across tenants. Each tenant has a row in tenant_stream_counters. next_seq atomically increments and returns the assigned sequence:

INSERT INTO tenant_stream_counters (tenant_id, next_seq)
VALUES ($tenant, 2)
ON CONFLICT (tenant_id) DO UPDATE
SET next_seq = tenant_stream_counters.next_seq + 1
RETURNING next_seq - 1 AS seq

The INSERT ... ON CONFLICT DO UPDATE pattern ensures the first event for a tenant gets seq = 1. The primary key on stream_events is (tenant_id, seq).

Append implementation

On append, the runtime:

1. Generates a unique event id and current timestamp.
2. Calls next_seq() to allocate the tenant-scoped sequence number.
3. Inserts the full row into stream_events (type, payload, source, correlation, priority, group_id, tenant_id).
4. Returns AppendResult { seq, id } to the caller.

Before each query, the runtime sets the Postgres session variable used by row-level security policies for tenant isolation.

Append path end-to-end

sequenceDiagram
    participant SDK as Deno SDK
    participant Op as RuntimeOpHandler
    participant SWF as StreamWithFanout
    participant PG as PgStream
    participant DB as Postgres
    participant FO as FanoutEngine

    SDK->>Op: stream_append
    Op->>SWF: append_and_fanout
    SWF->>PG: append
    PG->>DB: tenant_stream_counters bump
    PG->>DB: INSERT stream_events
    PG-->>SWF: AppendResult
    SWF-->>Op: AppendResult
    Op->>DB: record_dispatched (metadata)
    Op-->>SDK: { seq, id }
    SWF->>FO: dispatch (async)

Steps in order:

1. Allocate seq — tenant_stream_counters UPSERT.
2. Insert row — full event JSON into stream_events.
3. Return to caller — AppendResult before fan-out completes.
4. Fan-out — FanoutEngine::dispatch evaluates filters and enqueues (see Fan-out).
5. Record cursor — stores fanout_last_seq:{tenant_id} in runtime_metadata.

StreamWithFanout

Callers use StreamWithFanout rather than raw PgStream. It wraps append with metrics and fan-out dispatch:

• Increments yoke_stream_appends_total.
• Builds a StreamEvent from the append result + input fields.
• Dispatches to Fanout trait impl (FanoutEngine in production, NoOpFanout on control plane).

When the op handler is processing a nested Deno call (e.g. instances_create during portal bootstrap), fan-out is deferred and spawned on a separate task to avoid re-entering Deno synchronously.

Internal read API

list_events_after(after_seq) is kernel-only. Stream catch-up uses it to replay events the executor missed while cold:

SELECT ... FROM stream_events
WHERE tenant_id = $tenant AND seq > $after_seq
ORDER BY seq ASC

This is not exposed to user TypeScript — it is the security boundary between author code and durable history.

Schema reference

Initial table:

CREATE TABLE stream_events (
    seq BIGSERIAL PRIMARY KEY,  -- later: (tenant_id, seq) composite PK
    id TEXT NOT NULL UNIQUE,
    at BIGINT NOT NULL,
    type TEXT NOT NULL,
    payload JSONB NOT NULL DEFAULT '{}',
    source JSONB,
    correlation TEXT
);

Indexes:

• idx_stream_events_type — type lookups
• idx_stream_events_tenant_seq — per-tenant tail
• uq_stream_events_tenant_seq — unique (tenant_id, seq)

Environment variables

SDK surface

Application code appends via ctx.append():

await ctx.append({ type: "work.assigned", payload: { role: "coder" } });

Optional fields priority and groupId on the event input flow through to queue ordering and scheduler concurrency scopes.

There are no targets — routing is entirely filter-based on the consumer side.