I’ve been writing react agents and MCP servers for a while now, and a recurring problem is that MCP tool output is often large JSON — which gets expensive in tokens fast. As far as I’ve found, LangChain has two ways to handle this today:
1. Middleware that clears past conversation history / tool outputs to reclaim context (`ClearToolUsesEdit` in `context_editing.py`).
2. Middleware that exports tool output to a file so Deep Agents can use file tools on it (`FilesystemMiddleware`).
Option 1 fails outright when a single tool output already exceeds the model’s context window — but the deeper issue is it’s lossy even when it doesn’t fail: once a result is cleared, it’s gone, with no way to ask for it back.
Option 2 is a better instinct — nothing is destroyed — but the outputs I work with are usually too large for context while still small in absolute terms, maybe a few hundred KB to a few MB. Exporting those to a file means the agent needs several LLM calls (`ls`, `read_file`, `grep`, pagination) just to discover the shape before it can even ask a real question. It’s also solving a memory/disk problem that mostly isn’t the one being hit — being too big for context (tokens) isn’t the same as being too big to hold in memory (bytes).
Two open threads on this forum ask versions of the same question with no settled answer: [large tool outputs in LangGraph]( Best way to handle very large tool outputs in LangGraph (avoid LLM + LangSmith overload) ) and [large tool outputs needed for subsequent calls]( Handling Large Tool Outputs Required for Subsequent Tool Calls in LangGraph ).
What I’m building: a middleware similar in spirit to `SmartCrusher` in [headroom](GitHub - headroomlabs-ai/headroom: Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server. · GitHub) — it derives a schema from any arbitrary JSON tool output (field names, token cost, cardinality) and hands that to the react agent instead of the raw data, along with an id. The agent writes a small query against the schema and gets back just what it needs. Most of the time that’s one extra LLM call, not several, and nothing is ever discarded — a bad query costs a round trip, not silent data loss.
I considered `jq`/JSONPath as the query language instead of a small Mongo-style pipeline (`$match/$project/$sort/$limit/$skip/$unwind/$group/$count`). Went with the smaller, stricter surface on purpose — `jq` can express almost anything, which also makes it easy for a model to write something confidently wrong with no signal that it’s wrong.
I’ve tested this two ways: a correctness harness against a real 20-query corpus, checking that the specific rows a later aggregation depends on survive every transform; and live in a stock `create_agent` react loop with zero custom middleware, where it correctly grouped instead of dumping raw rows, preserved the right grouping key, passed small results through untouched, and once caught itself on a value that looked inconsistent with its own schema summary and re-verified instead of trusting it.
Later this could extend to chaining tool output into further tool calls, or rendering tool output into the final agent output — e.g. a user asking for a CSV export could get simple tool-based data munging instead of a full python sandbox.
Is anything like this already in progress? If not, I’d like to contribute it.