LangChain V1 Agentic Flow on Bedrock Fails for Non-Claude Models Available in Bedrock (Llama4, Nova, GPT OSS)

LangChain Academy
,
1

Hi everyone :waving_hand:

I’m using a LangChain V1 agent (create_agent) integrated with AWS Bedrock (ChatBedrockConverse). The setup works perfectly when using Claude models, but the same agentic flow fails or throws random errors when I switch to other models like:

  • us.meta.llama4-maverick-17b-instruct-v1:0

  • us.amazon.nova-premier-v1:0

  • openai.gpt-oss-120b-1:0

I’m using an async stream handler that invokes the agent like this:

async for token, metadata in agent.astream(
    {"messages": [{"role": "user", "content": request.query}]},
    stream_mode="messages",
):
    print(f"node: {metadata['langgraph_node']}")
    print(f"content: {token.content_blocks}")

And here’s how I configure the agent:

async def get_bedrock_agent():
    model = get_bedrock_model()
    tools = get_mcp_tools()

    agent = create_agent(
        model=model,
        tools=tools,
        system_prompt=SYSTEM_PROMPT,
    ).with_config({"recursion_limit": settings.agent_recursion_limit})

    yield agent

Observed Errors:

With non-Claude models, I get random Bedrock/streaming errors like these: (I never got any of these errors with Claude models)

ValidationException: An error occurred (ValidationException) when calling the Converse operation:
ErrorEvent { error: APIError { type: "BadRequestError", code: Some(400),
message: "Unexpected token 173781 while expecting start token 200006", param: None } }

Or:

Mantle streaming error: Unexpected token 200012 while expecting start token 200006

Or sometimes tool execution-related errors such as:

{"statusCode":500,"error":"Error executing tool: class java.util.LinkedHashMap cannot be cast to class java.lang.String"}

Or:

ERROR - ❌ Error: {"statusCode":500,"error":"Error executing tool: Unexpected character (':' (code 58)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 1, column: 1]"}

Or:

'{"statusCode":500,"error":"Error executing tool: Entity repository for index not found. Is the ENTITY_TYPE_MAP initialized?"}'

What I Suspect:

Could it be that:

  • The MCP servers (tool servers) or LangGraph tool calls cause issues because these models don’t fully support the same streaming structure or function-calling schema as Claude?

  • Or does ChatBedrockConverse internally expect a certain tokenization or response structure that only Claude adheres to for tool calling?

Questions:

  1. Has anyone successfully used LangChain agents with Bedrock + non-Claude models (like Llama 4, Nova, GPT-OSS)?

  2. Do these models support tool calling or function-calling via Bedrock Converse API in the same way as Claude does?

  3. Is there a recommended workaround or configuration tweak to make these models compatible with ChatBedrockConverse and LangChain’s agentic flow?

Environment:

  • langchain==1.0.2

  • langchain-core==1.0.0

  • langchain-aws==1.0.0

  • langchain-mcp-adapters>=0.1.11

  • fastapi==0.119.1 (Running async with FastAPI)

  • uvicorn[standard]==0.38.0

  • pydantic==2.12.3

  • pydantic-settings==2.11.0

  • Bedrock models tested: Claude 3.5, 4, 4.5 Sonnet :white_check_mark:, Llama 4 :cross_mark:, Nova :cross_mark:, GPT OSS :cross_mark:

Any insights or confirmations from others facing the same issue would be super helpful :folded_hands:
Thanks in advance!