In LangChain JS, it looks like the logic to use p-retry is pretty hard-coded in the LangChain runnables. For example here. This seems very non-customizable. p-retry supports additional parameters, such as a shouldRetry function, but there’s no way to pass that to LangChain to customize the behavior.
Use case example: Use a different retry interval when getting a 429 response vs. some other error code.
Lacking the above configurability, what I’m trying to do is set maxRetries: 0 on the model and do my own retrying when the error bubbles up to my application. However, this doesn’t work well when the model is used as a part of a longer chain, for example when passing it to an agent (for example with createReactAgent). In this case, any error will fail the whole agent, and I have no possibility to retry an individual operation.
A solution could be to wrap individual methods of the model in a retry handler, for example overriding invokewith something that wraps the original in a retry:
export class RetryingAzureChatOpenAI extends AzureChatOpenAI {
constructor(fields?: AzureChatOpenAIFields) {
super(fields)
}
invoke(input: BaseLanguageModelInput, options?: ChatOpenAICallOptions | undefined): Promise<AIMessageChunk> {
return withRetry(() => super.invoke(input, options))
}
}
I guess the above could work, but I’m not sure what’s the complete interface to override. There’s at least stream and batch methods in addition to invoke. I suppose wrapping all of them could achieve what I want - but what’s the complete set of methods that can be used to invoke the model?
Any help on this is appreciated. Many thanks