How to use clients with custom URL and authentication?

Product Help LangChain
1

Hello,

I have a service that exposes an AWS Bedrock anthropic model via REST API, but:

  • it has a custom URL format, which doesn’t end in “/converse” (it’s more or less “https://company-url/model/{model}”)
  • it has a different auth scheme than AWS (i.e. I would like to pass a custom Authentication header, instead of the one the client is creating for me based on the access & secret keys)

Is it possible to use ChatBedrockConverse to integrate with this API? If not, what would be the best recommendation?

I looked over how to create custom LLMs, but I would like to benefit from langchain clients exception handling, prompt creation and other features.

Thanks in advance.

1 Like
2

+1 Also interested in this

3

In short, no. The fundamental issue is that the ChatBedrock class is a wrapper around the boto3 AWS SDK. This SDK is hardwired to communicate with official AWS endpoints using the AWS Signature Version 4 signing process. It is not designed to be a generic HTTP client that can be reconfigured to hit an arbitrary URL with a custom authentication header.

Your service, by implementing a custom URL and authentication scheme, is essentially a new, distinct API. The approach to solving this problem would be to create a custom chat model class.

4

Thank you for your reply, I understand.

Let’s assume I somehow overcome these issues. Could you please let me know if it’s possible to send additional headers to ChatBedrockConverse, like it is possible to ChatOpenAI (via extra_headers) and ChatVertexAI (via additional_headers). I only found additional_model_request_fields, but they are sent in the body, not as HTTP headers.

5

Leaving my workaround here, in case someone else needs it:

I managed to bypass above limitation by registering a new event to trigger right before the http call is sent to AWS

method = "ConverseStream" if streaming else "Converse"
client.meta.events.register(f'before-send.bedrock-runtime.{method}', add_custom_headers)

and in the function add_custom_headers I altered the request object by modifying request.url and request.headers

The client still requires AWS credentials, but they do not have to be valid.