Handling Large Tool Outputs Required for Subsequent Tool Calls in LangGraph
Problem Statement
I’m building an agent in LangGraph using a ReAct-style Chain-of-Thought approach with multiple tools.
Some tools produce very large outputs — for example, thousands of text lines or large JSON objects.
Currently, I append each tool’s output to the message history so that it’s available for subsequent tool calls.
However, this causes a problem:
- These large outputs consume a significant number of tokens.
- When the message history is sent in and out multiple times for other tool calls, token usage becomes excessive.
I need a solution to make these large outputs available to other tools without inflating the token count.
How I am trying to solve this
My idea is to store all tool outputs in a separate state (or an external database if the data is too large) that works like a key-value store.
- When a tool produces an output, I store it with metadata and a unique ID.
- In the message history, I only keep the metadata (not the full output).
- When another tool requires this output, I pass the unique ID as a parameter instead of the full data.
- I believe the LLM will then pass this ID automatically to tools that need it, similar to how it handles other parameters from the message history.
Request for Advice
- Is this a correct approach?
- Does LangGraph already have a built-in mechanism for handling large tool outputs in this way?
- Are there any alternative workarounds that might be simpler or more effective?