I’ve been looking at multi-agent failure states, and it seems like everyone is relying on basic Python step_counters or LLM prompt-engineering (“do not repeat yourself”) to prevent agents from falling into infinite tool-calling loops or stampeding APIs.
Coming from a low-level systems/C++ background, handling loop-prevention at the application layer feels insane. It adds latency, and when a timeout triggers, it just leaves the state half-executed.
Has anyone tried intercepting tool calls deterministically via a dedicated proxy or trajectory ledger? E.g., a lightweight interceptor that hard-kills the process the exact microsecond an identical state or argument repeats?
Curious what the actual production solution is, or if everyone is just accepting the latency and hoping the prompt holds?
Deterministic interception is the right instinct, but hard-killing on the first repeat recreates the half-executed state you’re trying to avoid. Gate before execution with an idempotency-keyed dedup ledger, then return either the cached result or a structured refusal as the tool observation.
Add orchestrator-side budgets or tick caps, and the loop dies by starvation while state stays intact. Identical calls aren’t automatically bugs either, since retries, polling, and pagination can all repeat legitimately.
We hit this building MeshKore and put the gate before subagent dispatch, returning a structured refusal instead of killing in-flight work. So yes, move this outside the model, just don’t make process termination the primitive.
One distinction I’d add: a single agent looping and many agents overwhelming the same API need different limits. Every run can stay within its step budget while the combined traffic still overloads the downstream service.
I’d put a shared concurrency limit and retry budget at the outbound boundary, scoped to the downstream service or credential. Rejected calls should back off rather than immediately trigger another attempt. That still leaves already-dispatched calls to reconcile: stopping the agent doesn’t establish whether the external action completed.
Are you mainly seeing repeated calls within one run, or multiple workers hitting the same service?
hi
The production answer is layered, and each layer lives at a different boundary:
| Failure |
Boundary |
LangGraph / LangChain primitive |
| One run spins forever |
graph scheduler |
`recursion_limit` (super-step cap) → `GraphRecursionError`; `RemainingSteps` managed value for a graceful exit |
| One run calls the model/tools too often |
agent loop |
`ModelCallLimitMiddleware`, `ToolCallLimitMiddleware` (`run_limit` / `thread_limit`, per-tool) |
| Same call with same args repeats |
tool execution |
`wrap_tool_call` middleware: dedup against the message history, return cached result or structured refusal |
| Many parallel tasks in one run hammer an API |
task executor |
`config={“max_concurrency”: N}` → `asyncio.Semaphore` / thread-pool bound per run |
| Many runs/workers hammer one provider |
outbound client |
`rate_limiter=InMemoryRateLimiter(…)` on the chat model (per process); external token bucket for cross-process |
| Stuck call never returns |
node |
`TimeoutPolicy` / `RetryPolicy` on nodes and `@task`s, `ToolRetryMiddleware` |
| Was the side effect executed before we stopped? |
checkpoint |
superstep checkpoints + idempotency keys inside `@task` / tools |
A step counter is only one row of that table. If that is the only thing you have, it does feel like garbage, because it stops the loop without telling you why it looped and without protecting the downstream service.