The People's Supercomputer

The open models worth running are bigger than the machines most people own, while the laptops and gaming PCs in a house or an office sit idle most of the day. Splitting a model's layers across them is an old idea; doing it with machines that close their lids, drop off the Wi-Fi and belong to strangers is the hard part.

Two things have to hold at once. A dead node can't cost the user their answer — or quietly change it. And a pool that pays contributors in credits has to assume some of them will lie about the work they did.

A coordinator splits a Llama-family model's layers across whoever joins, in proportion to the memory each offers, and serves the pool as an OpenAI-compatible API. Activations stream hop to hop over persistent websockets in a plain binary frame — a JSON header and raw tensor bytes, no pickle — and every node keeps its own KV cache. Each node downloads only the safetensors shards that hold its layers.

When a hop can't reach the next one it names the dead node. The coordinator confirms with a probe, re-plans on the survivors, and replays the request step by step. Sampling is seeded by absolute position and the replay feeds the same steps as the original run, so the rebuilt caches are bitwise identical and the stream continues exactly where it stopped, with no repeated text.

Nodes earn credits per token-layer, and only for steps whose attestation chain verifies end to end: each hop signs the digest of the reply it got from the next, so a break pins the fault on one node or an adjacent pair, which is banned. Joining mints nothing, joiners run a real attested step on probation before they serve, and every control request has a deadline and a reply cap. Three independent attack rounds were run against it.

• make demo kill -9s one of three node processes after 40 tokens; the pool re-plans onto two, replays, and the streamed text matches a single-process run of the whole model on all 140 tokens.
• Logits parity-tested against transformers for Llama, Llama-3 RoPE, tied Llama, Qwen2 and Qwen3; pooled output matches a local run across crashes and in bf16, including a sharded Qwen2 checkpoint.
• Three adversarial review rounds — credit minting, overdraw, stalling and lying nodes, SSRF and DNS rebinding — each finding fixed with a regression test; 98 tests in CI on Python 3.10–3.13.
• Profiled at 4.5 ms per token on one node against 7.8 ms across three, in-process: pipeline parallelism buys capacity, not speed.