I have a graph with some tools attached to it, graph is being used with Livekit as an voice agent.
when graph invoked it can call some tool which can take longer to execute like 2-3 mints.
during tool execution there is silence on the voice assistant,
is there a way to make graph keep streaming something randomly based on tool and update the user
like “I am still searching for you records“. “hmm, record searching taking longer“ etc AI generated messages. while tool is not complete.
is there a possibility in graph to handle that?
Hi! You can use LangGraph’s custom stream writer to write anything to the output during a tool’s execution: Stream outputs
Hey thanks for sharing this.
is it possible to generate AI message from the tool using the graph it self?
maybe something like , from inside invoke a agent while tool is running and keep running that agent to generate some different messages based on tool.
That is possible using LangChain’s support for updating state during a tool call. That is a bit cumbersome, though, because those update messages will be visible to the LLM unless you take care to exclude them before calling the LLM with the actual results of the tool calls. Is there a reason you don’t want to go the custom stream writer route?
I have graph with subgraphs and each graph have many tools, tools can take longer to execute sometime.
Actually the Graph is linked with voice assistance so when tool runs assistant stays silent which feel un-natural. I want to keep it speaking after some interval until tool is running and of-course that message should be based on each tool individually.
the problem with custom stream message is I have to write some custom messages in each tool, and also call write multiple times in same tool as per docs. which I dont like it.
so that just it.
Makes sense! I would still recommend going the custom stream writer route over putting these status updates in the ToolMessage, and then defining your tool functions in a way where they can re-use common custom writing logic without writing anything extra to the messages array.
This is not working in my case. message did get write but not get pass through stream while tool is running. once the tool completes its execution, both tool message and custom message stream instantly. not sure whats the problem.
The custom message should show up immediately. Where in your tool function are you making the call to the custom stream writer?
after a lot of debugging, it seems like its the issue with Livekit which I was using for voice. langgraph was returning message but livekit was not doing TTS while tool is running. still looking for a solution.