Something I’ve been thinking about that I haven’t seen addressed in LangChain yet — there’s no standard execution-layer hook for blocking spend before a tool call fires.
The callback system (on_tool_start, on_llm_start) is observability-only. You can log what’s happening but you can’t block it. So when an agent enters a retry loop — low confidence, tool timeout, hallucination cascade — there’s nothing that stops the next API call from firing. The loop runs until you notice, or until your provider sends a bill.
Built a simulator to show exactly what this looks like at different models/loop rates: Agent Loop Cost Simulator — Valta | Valta
What I think the interface should look like:
class BaseSpendController(ABC):
@abstractmethod
def check_spend(self, amount: float, category: str | None = None) -> bool:
"""Return False to block. Called before any paid tool execution."""
...
agent = initialize_agent(
tools=[...],
llm=llm,
spend_controller=MyController(limit=10.00),
)
The controller runs at the execution layer — below the LLM, above the external call. Not a prompt instruction. The model can’t reason past it.
Has anyone thought about where this would actually live in AgentExecutor? Curious if the callback chain is the right place or if it needs its own integration point.