Curious if a lean model can beat the heavyweight champs on niche jobs?
The Simple Version
Think of frontier models as over‑confident interns. They often stumble on tasks that need expert taste. A narrow model, trained on the right examples, can actually outperform them.
How It Actually Works
Data Strategy
Instead of labeling everything, they send only disputed items to costly human annotators. The rest get cheap auto‑labels. This focuses expert time where it matters.
Model Choice
They fine‑tuned Qwen3 235B, a 235‑billion‑parameter open model, using the Tinker framework. The result beats Gemini, Claude and GPT on the same benchmarks.
Technical Deep Dive
Qwen3 235B was trained on a curated set of six financial filtering tasks. Expert labels were added only for contested examples, cutting annotation cost by roughly 70%. The model achieved 84.7% accuracy while inference cost dropped to one‑thirteenth of frontier models.
Why This Matters
For teams building AI agents, accuracy alone can be misleading on imbalanced workloads. A cheap sub‑agent can handle the grunt work, letting a frontier orchestrator focus on high‑stakes decisions. This opens the door for ordinary companies to deploy expert‑level agents without breaking the budget.
Key Takeaways
- Dataset > Recipe: Better data often beats more parameters.
- Cost‑aware inference: Narrow models can be 13× cheaper per task.
- Smart labeling: Route only contested items to humans.
- Agent hierarchy: Use small models as sub‑agents under a large orchestrator.
- Benchmark first: Know the metric that matters before deploying.