Research Engineer - Decentralized Training and Inference Verification

Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report). While many of the core research problems have been solved, Protocol Learning unlocks a series of new challenges. For the mission in full, read A Third Path: Protocol Learning.

Our training and inference network is trustless, and the workers are GPUs scattered across the world. Many things can go wrong in this system. An inference worker can return tokens from a cheaper model, or a highly quantized version of the one it's supposed to run. Even with the right model, it can sample with the wrong parameters. Training faces its own attacks, and Agora showed what a permissionless run deals with in practice: participants who disrupt training by dropping updates or flooding the system, free-riders who submit trivial work and collect the rewards, poisoned updates