Fan-out and consensus

Press Return in Polycode and the prompt does not go to one model. It goes to every model you have configured, in parallel, at the same instant. That shape is the whole point of the app — fan-out gives you a comparison surface in the place where a single-vendor chat client gives you one opinion. This article walks through how that runs end-to-end and what each piece is actually doing.

What fan-out means

When you submit, the consensus engine inside Polycode dispatches one request per configured peer: a Claude call, a GPT-5 call, a Gemini call, an Apple Intelligence call, an Ollama call, and so on for whatever set you set up in Settings → Providers. The peers run concurrently. None of them blocks on the others. The fastest reply lands first; the slowest closes the round when it finishes or hits its timeout.

The dispatch is per-peer, not per-provider — if you’ve configured two models from the same vendor (say, Claude Sonnet and Claude Opus), they each get their own dispatch slot with their own pricing and their own latency budget. Same prompt, different model, separate row in the trace.

Why parallel inference, not routing

A common alternative shape is to route: pick one provider per prompt based on classifier heuristics, send to that one. Routing optimizes for the average case; it hides the cases where the routing decision was wrong.

Fan-out optimizes for the times you actually wanted a second opinion. Three peers strongly agreeing on a fact is a signal you didn’t have when one peer said it. Three peers diverging on an interpretation is a signal you definitely didn’t have. The cost is more tokens per turn; the payoff is calibration — you know which answers to trust and which to dig into.

Per-peer streaming

Each peer streams its response on its own track. The chat surface renders a fan-out card the moment the first peer starts emitting tokens, with a per-peer progress track and a live status indicator while it streams; once the peer settles, its pill shows the final latency (e.g. 180ms).

The per-peer timer records two values: first-token latency (when did this peer start saying anything) and total latency (when did it finish). The consensus engine stores both — first-token in a lock-protected map keyed by peer, total at finalize time — and the inspector reads them back later. First-token is the live-feel signal; total is the one you compare across peers in a trace.

Synthesis: one primary peer writes the reply

When the slowest peer either finishes or times out, the engine ends the fan-out round and hands the full set of responses to your primary peer — the model you trust most, set in Settings → Providers and overridable on a given send via the per-window model pick or a Mode. That peer reads every other peer’s reply, then composes a single synthesized answer that either agrees with the consensus, surfaces a dissent, or chooses between competing options.

The synthesis prompt instructs the primary peer to cite a peer only when that peer’s contribution is load-bearing — a fact only one model surfaced, a disagreement worth flagging, a peer that was unavailable. Universal-agreement sentences carry no citation; that’s the cleanest signal that the peers all said the same thing.

When the panel splits

Synthesis is not averaging. If one peer says one thing and three peers say another, the synthesizer is instructed to surface the disagreement, not paper over it. You’ll see language like “Three peers recommend X; Gemini suggests Y because…” in the rendered reply, and the inspector shows the raw text from each peer so you can read the originals yourself.

For a deep dive on what that trace looks like and how to navigate it, see the inspector.

Cost: roughly N× a single model, plus a synthesis pass

Every peer is a full inference call, billed by that provider at their listed price. A multi-peer fan-out costs roughly N× one model, where N is the number of peers — plus one synthesis pass billed at your primary model’s rate, since the primary runs a separate turn to compose the final reply (free if your primary is Apple Intelligence or a local endpoint). The math varies because pricing is per-token-per-model and response lengths differ. Each row of the inspector trace shows the exact tokens in / out and dollars for that peer — and the synthesis lane is metered too — so the per-turn spend is never a mystery.

Two practical mitigations sit inside the app. Apple Intelligence on-device is free and counts as a fan-out peer — the inspector marks it Free — so every fan-out with on-device enabled gets one free vote. Local OpenAI-compatible endpoints (Ollama, LM Studio, an internal gateway) carry no per-token cost either; they show as unpriced (—) in the trace rather than a labeled Free. The unpriced peers make the comparison surface workable for everyday questions; the paid peers stay in for the questions that warrant them.

Tracking spend over time

Per-turn cost lives in the inspector, but Settings → Usage rolls it up over a date range: a running total, broken down by provider and by project, plus a top-sessions list — all computed locally from the exchange-time costs already stored with each response. A Reset baseline control rebases the dashboard’s start date without deleting any history. See data lifecycle § Usage for exactly what’s counted.

The “consensus reached” header

When the fan-out finishes, the inspector’s consensus header reports X/Y providers · TOTALms · NN% agreement. Y is the number of peers dispatched, X is the number that returned a successful response, and the percentage is the simple ratio of successful peers — a cheap proxy for “did this round complete.” It is not a semantic-similarity score. Polycode favors a measurement that is easy to reason about (every peer either replied or it didn’t) over a measurement that is more sophisticated but harder to verify. A richer semantic agreement score is on the table for a later release.

Tools live alongside fan-out

A peer can decide to call a tool — read a file, run a query, hit an MCP server — instead of (or before) producing its final response. The tool-call step happens inside the peer’s run, the result feeds back into that peer’s context, and the peer continues. Different peers may make different tool calls on the same prompt; the inspector shows which peer called what. The approval model is covered in tools and approvals.

What’s next

• The inspector — open the per-provider trace, read what each peer actually said and what it cost.
• Setting up providers — control which peers participate in fan-out and how each one is configured.
• Tools and approvals — connect MCP servers and manage the approvals that peers ask for during a run.