How are you all handling cost enforcement on agents — not just
monitoring but a hard gate before tool calls fire?
Working on something for this and curious what patterns people are using.
We’re treating it as two separate layers:
- Hard ceiling on tokens/spend (budget middleware or pre-tool check that aborts before the call).
- Reducing unnecessary tokens before they ever count against that budget.
On the second part, a lot of the spend in longer runs comes from retrieved context and tool results that are already stale. We’ve been using a thin proxy that scores those blocks on temporal decay and drops high-decay ones before they reach the model. It doesn’t replace a proper spending limit, but it lowers the baseline consumption the limit has to enforce.
Curious what patterns others are using for the actual hard gate & are you doing it inside middleware, at the tool wrapper level, or outside the agent entirely?
We ended up putting the hard gate at plan time instead of wrapping individual tool calls. Before anything executes, walk the agent’s planned actions and estimate worst-case spend - static amounts where you have them, multiplied by fan-out counts where a step runs per-item. Three outcomes: under the ceiling, run it. Over, refuse to start. And the important third case: if the spend can’t be bounded statically (dynamic amounts, unbounded loops over retrieved lists), don’t auto-run at all - route it to a human instead of trying to meter it live.
What pushed us there: mid-run kills are messier than they sound. If the budget trips after the payment call but before the reconciliation step, you’ve got real side effects in the world and half a workflow. Refusing before anything fires avoids the whole compensation problem. We still keep runtime metering as a backstop, but treating “can’t bound it” as “needs a human” rather than “monitor and kill” removed the worst failure mode for us.
Agree with the two-layer framing above - trimming stale context lowers what the ceiling has to police. The piece I’d add is just that the ceiling itself works much better as a precondition than as a tripwire.