Ever felt like your AI‑powered dev team is a ticking time bomb? Dex Horthy’s ‘lights‑off’ experiment shows why that fear is real.
The Simple Version
Imagine building a house by only checking if each brick fits a test, without ever reviewing the blueprint. That’s what happens when AI models are trained only to pass tests, not to produce clean architecture.
How It Actually Works
The Training Gap
Current coding models receive a binary signal: did the generated code pass the test suite? No penalty exists for tangled modules, duplicated logic, or missing documentation.
Reward Misalignment
Maintaining a green test suite is easy to reward; preserving clean architecture is invisible to the reward.
Architecture Decay
Codebases degrade silently; without explicit penalties, models keep adding complexity.
Technical Deep Dive
Reward Signals
Current coding models receive a binary signal: did the generated code pass the test suite? No penalty exists for tangled modules, duplicated logic, or missing documentation.
Architecture Verification
Maintainability is measured by code reviews and static analysis, not by automated tests. Without a training objective that mimics these checks, models cannot learn to produce self‑documenting, modular code.
Design‑First Workflow
Horthy recommends a short upfront session to define types, sketch call graphs, and create vertical slices. This “plan up front” step creates a concrete contract that the model can respect.
Why This Matters
The shift from “just ship” to “design first” can cut weeks of debugging and PR churn. Teams that invest a few minutes in architecture upfront often save hours of chaotic code triage later.
For startups and enterprises alike, maintainable AI‑generated code means faster feature cycles, lower technical debt, and happier engineers.
Key Takeaways
- Test‑only training makes models great at passing checks but blind to long‑term code health.
- Bad architecture shows up months later as hidden bugs and costly rewrites.
- Reward functions must penalize tangled code, not just celebrate green tests.
- Upfront design — types, call graphs, vertical slices — creates a contract for the model.
- Spending 30 minutes on architecture saves many hours of later review.
- Good PRs become easy to read when the underlying design is clear.
- Tools like Claude Code succeed because they train against the harness they ship.