Full disclosure up front: I run an autonomous AI dev studio, and one thing that kept surprising us was how quickly long agent runs get expensive. So I instrumented 66 real coding-agent sessions and broke the spend down by turn, role, and session. Sharing the numbers because the mechanism applies to any agent loop — LangChain AgentExecutor, LangGraph, tool-calling chains, anything that re-sends state each step.
The core issue: every step, the model is billed for the entire context it receives — system prompt + accumulated message history + every prior tool result + the new input. Not just the new tokens. In an agent loop that keeps appending tool outputs, the per-step input grows monotonically, so cost per step climbs even though the “question” isn’t getting harder.
What I measured across the 66 sessions:
| Metric | Value |
|---|---|
| Total input tokens | ~89M |
| Total output tokens | ~47M |
| Input:output ratio | ~1.9:1 |
| Median session length | 23 turns |
| Longest session | 1,270 turns |
| Cost of the single longest session | ~$1,278 (more than all other sessions combined) |
Input dwarfs output ~1.9:1 — you pay far more to feed the loop than to get answers out of it. And the cost curve per step is roughly linear in accumulated context:
| Step | Approx context | Cost/step (at $3/M input) |
|---|---|---|
| 1 | ~2K tokens | ~$0.006 |
| 10 | ~12K tokens | ~$0.036 |
| 50 | ~55K tokens | ~$0.165 |
| 100 | ~110K tokens | ~$0.33 |
| 200 | ~200K tokens | ~$0.60 |
By step 200 each step costs ~100x step 1 — same model, same task, just carrying more history.
The three multipliers that hurt most in practice:
- Tool results are the biggest line item. In tool-heavy agents, raw tool output (search dumps, file reads, API JSON) was often the single largest share of input. Trimming/summarizing tool results before they re-enter context was the highest-ROI lever we found.
- History never self-prunes unless you make it. A naive loop re-sends turn 1 at turn 200.
- The tail is brutal. A tiny fraction of runaway-long sessions dominated total spend.
Genuine questions for people running long chains/agents here:
- How are you bounding context in long
AgentExecutor/ LangGraph runs — windowing, running summaries,trim_messages, hard step caps, or something smarter? - Do you summarize or truncate tool outputs before they go back into the prompt, and how do you avoid dropping something the agent later needs?
- Has anyone found a clean way to detect a runaway loop early rather than eating the whole bill?
Happy to share more of the breakdown methodology if it’s useful. Curious whether others see the same ~1.9:1 input:output and the same “tool results dominate” pattern.