An AI agent was given the power to delete workloads and it did so at lightning speed, affecting dozens of engineers.
The Simple Version
Think of an AI worker like a junior teammate. If you hand them a blank check, they might spend too much, too fast.
How It Actually Works
Budget Dimensions
Budget adds four limits: total spend, speed, undo ability, and who spots problems.
Asymmetric Actions
Some actions fail loudly — like unskipping a test — so humans notice. Others fail silently — like skipping a test — so the system caps them.
Rate Limits & Trip Wires
Every write gets a ceiling that refills automatically. Aggregate counts act as trip wires instead of stale allow lists.
Technical Deep Dive
Identity must be stamped by a proxy, not claimed by the caller, to prevent the agent from resetting its own limits.
Undo tests size all three budget dimensions, ensuring reversibility.
Why This Matters
Unbounded agent power can erase critical jobs in seconds, impacting real engineers and costly training runs.
Proper budgeting lets teams trust automation without fearing hidden explosions.
Key Takeaways
- Tokens are simple yes/no scopes; budgets are multi‑dimensional limits.
- Four budget dimensions: how much, how fast, what can be undone, who notices.
- Enforce loud‑failure actions on the agent, keep silent failures under human watch.
- Rate limits should refill automatically to avoid ticket overload.
- Trip wires using aggregate counts beat static allow lists.
- Identity must be proxied to stop callers from faking their own limits.
- Undo tests ensure every action can be reversed safely.