Hi everyone,
I’m running into a weird behavior from my react agent where it basically makes a tool call, tells the user that it needs more information before proceeding to make the tool call, and then proceeds to make the tool call, all under a single request.
Setup
-
My agent uses a custom tool (
log_summarizer) that:-
Parses a log file,
-
Chunks the logs based on optional time ranges,
-
Stores those chunks in an Elasticsearch index,
-
Then passes the relevant chunks + job context to a prompt for summarization via a Qwen3 model.
-
The implementation can be found below:
@tool(
"log_summarizer",
args_schema=SummarizeLog,
description="Summarize the log file for a specified job. Use this to summarize a report of either an individual log file, or a combination of the log files."
)
def summarize_log_tool(
filename: str,
config: RunnableConfig,
start_time: Optional[str] = None,
end_time: Optional[str] = None,
):
page_context = config["configurable"]["page_context"]
print(f"filename is {filename}, start_time is {start_time}, end_time is {end_time}")
...
chunks = fetch_chunks(es_store=es_store, job_ctx_text=job_ctx_text)
prompt_content = construct_log_summarizer_prompt(
job_context=job_ctx_text, chunks=chunks
)
try:
response = qwen3_model.summary_model.invoke(
[
(
"system",
"You are a senior SSD storage analyst who is responsible for analyzing and summarizing log files from hardware regression tests",
),
(
"user",
f"""
Give me an elaboate description based on the log contents and the structured job information below, perform the following:
- Write down a detailed test execution summary to summarize the test steps.
- Provide the time sequence and the source of the log contents.
- Highlight some of the original logs if necessary.
- If any error logs are detected, write a section to analyze the error and write another section to recommend some potential resolutions.
Job information:
{job_context}
Job log contents:
{chunks}
""",
),
]
)
if hasattr(response, "content") and isinstance(response.content, str):
return response.content
elif isinstance(response, str):
return response
else:
return str(response)
except Exception as e:
raise HTTPException(
status_code=500, detail=f"Failed to generate assistant response: {e}"
)
log_tools = [summarize_log_tool]
summarize_log_tool_description = f"""
- Tool name: {summarize_log_tool.name}, Description: {summarize_log_tool.description}
"""
And this is how the tool is binded to the react agent:
import os
from typing import AsyncGenerator, Optional
from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware
from langchain_core.messages import AIMessage
from langfuse import Langfuse, propagate_attributes
from langfuse.langchain import CallbackHandler
from langgraph.checkpoint.mongodb import AsyncMongoDBSaver
from langgraph.graph.state import RunnableConfig
from app.config import settings
from app.helpers.api_spec import get_api_spec
from app.models import qwen3_model
from app.models.chat import ChatPayload
from app.tools import fa_tool, jira_tool, requests_tool, user_guide_tool
from app.tools.log_tool import log_tool
...
def _initialize_langfuse() -> None:
"""Initialize Langfuse client and callback handler"""
...
def _get_system_message() -> str:
"""Generate the system message with tool descriptions"""
api_spec = get_api_spec()
return """
# Instructions
## Output Formatting Guidelines
Always structure your final response clearly and cleanly using markdown-style formatting.
- Use **bold headings** to introduce sections.
- Organize information in **bullet points** or **numbered lists** for readability.
- Do not present the thought process, but just the answer.
- **DO NOT PREFACE WHAT YOU ARE GOING TO DO**.
- Highlight key terms or values with **backticks** (e.g., `field_name`).
- For logs or JSON snippets, wrap them in ```code blocks```.
- Avoid raw JSON dumps unless specifically requested.
- Keep paragraphs concise and well-spaced.
## Tool Access
### Jira tool
You have access to the following tools for Jira-specific requests:
{jira_tools_description}
### Failure analysis tool
...
### User Guide tool
...
### Log tool
You have access to the following log tool to summarize and generate a summary of the requested test log file.
{summarize_log_tool_description}
### Backend service tool
...
Here is OpenAPI documentation on the API endpoints:
{api_spec}
""".format(
api_spec=api_spec,
jira_tools_description=jira_tool.jira_tools_description,
fa_tools_description=fa_tool.fa_tool_description,
user_guide_tools_description=user_guide_tool.user_guide_tools_description,
summarize_log_tool_description=log_tool.summarize_log_tool_description,
)
# Prepare tools and system message
tools = [
*requests_tool.request_tools,
log_tool.summarize_log_tool,
*jira_tool.jira_tools,
*user_guide_tool.user_guide_tools,
*fa_tool.fa_tools,
]
system_message = _get_system_message()
def _build_config(payload: ChatPayload):
config: RunnableConfig = {
"configurable": {
"thread_id": payload.thread_id,
"page_context": payload.page_context,
},
"metadata": {"user_id": payload.user_id},
}
return config
async def _generate_response(
payload: ChatPayload, config: RunnableConfig, checkpointer: AsyncMongoDBSaver
):
agent_executor = create_agent(
model=qwen3_model.chatModel,
tools=tools,
system_prompt=system_message,
checkpointer=checkpointer,
middleware=[
SummarizationMiddleware(
model=qwen3_model.chatModel, # TODO: Find optimal settings for max_tokens_before_summary and messages_to_keep
max_tokens_before_summary=40000,
messages_to_keep=15,
),
],
)
userQuery = {"role": "user", "content": str(payload.query)}
events = agent_executor.astream(
{"messages": [userQuery]},
stream_mode="messages",
config=config,
)
async for chunk, metadata in events:
if isinstance(chunk, AIMessage):
if isinstance(chunk.content, str):
yield chunk.content
async def stream_answer(payload: ChatPayload) -> AsyncGenerator[str, None]:
"""
Takes a user query and yields the new token each time the LLM adds something.
Tool-call messages are ignored completely
"""
async with AsyncMongoDBSaver.from_conn_string(settings.mongo_uri) as checkpointer:
if langfuse is not None:
with langfuse.start_as_current_observation(
as_type="span", name="langchain-call"
) as observation:
with propagate_attributes(
session_id=payload.thread_id, user_id=payload.user_id
):
config = _build_config(payload)
config["callbacks"] = [langfuse_handler]
full_response = ""
async for token in _generate_response(
payload, config, checkpointer
):
full_response += token
yield token
observation.update(input=payload.query, output=full_response)
else:
config = _build_config(payload)
async for token in _generate_response(payload, config, checkpointer):
yield token
Issue
When I invoke the agent with a query requiring the log_summarizer tool, the agent first claims it needs more information to make the tool call, possibly triggering an intermediate response like:
“To summarize the Iog file, I need to know the specific job ID or any additional details to narrow down the log file. Could you please provide more information?”
But then, in the same session, it actually proceeds to call the tool successfully and returns a complete summary — all without asking the user for anything else. The entire response would look like the following:
To summarize the ICET log, I need to know the specific job ID or any additional details to narrow down the log file. Could you please provide more information?
Test Execution Summary & Analysis
1. Detailed Test Execution Summary
This test, titled “Telemetry Logs”, was designed as a White Box validation to verify the behavior of Telemetry Log Page (Log ID 8) under core dump trigger conditions with an unsafe power loss, per Specification Revision— Telemetry Controller-Initiated Log. The goal was to validate that telemetry log size increases after a core dump is triggered and that the telemetry data remains persistent across power cycles.
…
Trace
Discussion
From tracing through logs, I’ve confirmed that the tool does get called correctly and returns valid output. So this seems to be a miscommunication between how the agent interprets or presents the tool usage internally.
Would anyone know why this sort of behavior/response occurs within the react agent?
Thanks in advance!