Custom middleware after tool fail

OSS Product Help
1

Hey everyone :waving_hand:
we are working on a custom middleware in langchain.js and ran into something I’d love your input on.

createMiddleware({
  name: "blablaMiddleware",
  contextSchema: blablaSchema,
  wrapToolCall: async (request, handler) => {
    if (something...) {
      return handler(request);
    } else {
      console.log("found a problem");
      return handler(request);
    }
  },
});

The question is: what’s the proper way to handle a failing tool call (e.g., an API error)?
We’ve noticed that sometimes the whole process crashes because the tool result doesn’t match the expected schema.

While digging through the source code, I saw that in some cases LangChain throws an exception, and in others it returns a ToolMessage.
Is there any rule of thumb for when we should catch and convert errors into a tool message vs. letting them bubble up?

Right now I’m considering wrapping the tool call in a try/catch and returning an explicit error message, e.g.:

return new ToolMessage({
  content,
  tool_call_id: toolCallId,
  name: toolName,
  status: "error",
});

Curious how others are handling this :folded_hands:

2

this is a issue with a promblem after an error in a tool with middleware

3

hi @yishai-stern_zoominf

  1. you are getting exactly that “Error in middleware “DynamicSystemPromptMiddleware”: 400 An assistant message with ‘tool_calls’ must be followed by tool messages responding to” error?
  2. could you share your code? Right now it is hard to infer what is actually happening to your agent

Hint:

instead of returning a simple ToolMessage, you have to return Command to update the state messages:

    return new Command({
      update: {
        messages: [
          new ToolMessage({
            content: `An error in... bla bla bla`,
            tool_call_id: config.toolCall?.id ?? "tool-call-id",
          }),
        ],
      },
    });
4

imho the most elegant way to handle error is using StructuredTool and ToolNode with their native error handling mechanizm

5

Hey @yishai-stern_zoominf ,

What I have been following is to return the error so the LLM knows how to heal itself. This is what I do in my production app.

export const toolErrorMiddleware = createMiddleware({

name: "ToolErrorMiddleware",

wrapToolCall: async (request, handler) => {

try {

return await handler(request);

    } catch (error) {

// Return a custom error message to the model

return new ToolMessage({

content: `[TOOL FAILED]: ${error}`,

tool_call_id: request.toolCall.id!,

      });

    }

  },

});
1 Like