How Poolside Engineers Build a 118B AI Model with…

Poolside creates a massive AI model using synthetic data pipelines that generate, police, and reuse code, while rigorously testing for hidden failures.

Ever wonder how AI models keep getting smarter when they run out of real code to study?

⚡ TL;DR: Poolside builds a 118B AI model using synthetic data pipelines that generate, police, and reuse code patterns, while rigorously testing every step to catch hidden failures.

The Simple Version

Think of teaching a robot to code by giving it a huge library of example recipes, then mixing and matching them in new ways so it learns without needing every single recipe from scratch.

How It Actually Works

Synthetic Data Pipeline

🎯 Think of it this way: It’s like a chef creating new dishes by remixing existing recipes with fresh ingredients, then tasting each version to keep only the good ones.

Orchestrator Policing

🎯 Think of it this way: An editor reads every story and throws away the ones that don’t follow the rules, ensuring only clean output makes it to the final book.

Multistage Pipelines

🎯 Think of it this way: It’s a relay race where each runner hands off a baton, changing the format each time — text, chat, or plot — so the model sees many flavors of the same skill.

Technical Deep Dive

The team runs two identical model replicas on the same data; if their outputs differ in any numeric field, the run is aborted to expose silent errors.

Broken GPUs reveal themselves as spiky loss curves, while a subtle precision bug in tensor‑parallel accumulation can flatten gradients until patched.

Race‑condition‑induced corrupted gradients were previously invisible, but the new checkpointing logic now catches them before scaling.

The final model contains 118 billion parameters, purpose‑built for agentic coding, and its early benchmark beats GLM 4.5 Air on a scaled recipe.

Why This Matters

Synthetic data lets developers train massive models without exhausting real‑world code, cutting costs and accelerating iteration.

Rigorous self‑checks uncover hidden hardware and software bugs that would otherwise slip into production, improving reliability.

Agentic coding models can autonomously write, test, and refactor code, freeing engineers to focus on higher‑level design.

Key Takeaways

Poolside uses synthetic data pipelines to generate diverse training examples.
An orchestrator discards low‑quality generations to maintain data integrity.
Dual‑run validation catches numerical mismatches early.
Hardware bugs like precision errors and race conditions are now visible.
The resulting 118B model excels at autonomous code generation.
Early results outperform GLM 4.5 Air on benchmark recipes.
These techniques pave the way for cheaper, more reliable LLMs.

How Poolside Engineers Build a 118B AI Model with Synthetic Data & Self‑Checking Pipelines