I’m trying to better understand how to use the return_direct=True option in LangGraph tools, especially when using them inside a ToolNode.
What I’m doing
I have several tools defined with @tool(return_direct=True).
These tools are used inside a ToolNode in my agent graph.
However, when the graph executes, I still see AI messages being generated after those tools run, even though I expected the tool output to be returned directly and stop message generation.
The issue
Some of my tools return very large results (e.g., get_all_products with full product details), and I don’t want those outputs:
To be added to the state messages, or
To trigger an extra AI message that just repeats the same tool output.
On the other hand, I have some tools where I do want the results appended to the state (for context), but still don’t want an AI message generated afterward.
What I know
I understand I could create a router node to detect specific tool names and handle this behavior manually (deciding when to return directly or not).
However, I’d prefer to keep using ToolNode for its built-in advantages and simplicity.
My questions
How can I make return_direct=True actually prevent AI message generation after tool execution when using ToolNode?
Is there a way to handle mixed behavior — for example:
Some tools return directly (display result only, no state append),
Some tools append their output to the state but skip the AI message?
Any examples or best practices for handling this pattern would be greatly appreciated!
I modified the quickstart example by adding an edge from tool_node to END.
# Step 1: Define tools and model
from langchain.tools import tool
from langchain.chat_models import init_chat_model
model = init_chat_model(
"ollama:qwen3",
temperature=0
)
from langgraph.prebuilt.tool_node import ToolNode
from langchain_core.tools import tool
@tool
def add(a: int, b: int) -> int:
"""Add two numbers."""
return a + b
tool_node = ToolNode([add])
# Augment the LLM with tools
model_with_tools = model.bind_tools([add])
# Step 2: Define state
from langchain.messages import AnyMessage
from typing_extensions import TypedDict, Annotated
import operator
class MessagesState(TypedDict):
messages: Annotated[list[AnyMessage], operator.add]
llm_calls: int
# Step 3: Define model node
from langchain.messages import SystemMessage
def llm_call(state: dict):
"""LLM decides whether to call a tool or not"""
return {
"messages": [
model_with_tools.invoke(
[
SystemMessage(
content="You are a helpful assistant tasked with performing arithmetic on a set of inputs."
)
]
+ state["messages"]
)
],
"llm_calls": state.get('llm_calls', 0) + 1
}
# Step 4: Define tool node
from langchain.messages import ToolMessage
# Step 5: Define logic to determine whether to end
from typing import Literal
from langgraph.graph import StateGraph, START, END
# Conditional edge function to route to the tool node or end based upon whether the LLM made a tool call
def should_continue(state: MessagesState) -> Literal["tool_node", END]:
"""Decide if we should continue the loop or stop based upon whether the LLM made a tool call"""
messages = state["messages"]
last_message = messages[-1]
# If the LLM makes a tool call, then perform an action
if last_message.tool_calls:
return "tool_node"
# Otherwise, we stop (reply to the user)
return END
# Step 6: Build agent
# Build workflow
agent_builder = StateGraph(MessagesState)
# Add nodes
agent_builder.add_node("llm_call", llm_call)
agent_builder.add_node("tool_node", tool_node)
# Add edges to connect nodes
agent_builder.add_edge(START, "llm_call")
agent_builder.add_conditional_edges(
"llm_call",
should_continue,
["tool_node", END]
)
agent_builder.add_edge("tool_node", END)
# Compile the agent
agent = agent_builder.compile()
# Invoke
from langchain.messages import HumanMessage
messages = [HumanMessage(content="Add 3 and 4.")]
messages = agent.invoke({"messages": messages})
for m in messages["messages"]:
m.pretty_print()
If this isn’t what you meant, could you please share your code so we can understand better?