Claude-sonnet-5 and chat.with_structured_output issues

OSS Product Help LangChain
1

i recently switched to try

claude-sonnet-5

and now i getting intermittent issues with

class Classification(BaseModel):
    transaction_id: str = Field(description="transaction id")
    transaction_type: str = Field(description="classification of the transaction")
    reasoning: str = Field(description="reason of the classification, including any applicable special instructions")

class Classifications(BaseModel):
    classifications: list[Classification] = Field(description="list of transaction classifications")


        structured_llm = self.chat.with_structured_output(Classifications)

while im expecting a Classification object from the call the llm seems to return a structure that’s can be transformed into the pydantic object due to double enveloping response.

[{"type":"tool_call","id":"toolu_011fM8mzebeLLaxUPRDccHqa","name":"Classifications","args":{"classifications":"{\"classifications\": [{\"transaction_id\": \"5491661-01-20241205\", \"transaction_type\": \"water\", \"reasoning\": \"This transaction represents a water bill invoice for the property at 34 Karoro Rd Flat Bush, including water consumption, wastewater consumption, and wastewater fixed charges. Although the amount is shown as positive, the details indicate this is a water utility invoice due to be paid (due_date 27/12/2024), consistent with a payment for water services rather than income.\"}]}"}}]

classification is double enveloped, causing

1 validation error for Classifications
classifications
  Input should be a valid list [
  type=list_type,
  input_value='{"classifications": [{"t...rather than income."}]}',
  input_type=str
]
    For further information visit https://errors.pydantic.dev/2.12/v/list_typeTraceback (most recent call last):


  File "/root/.cache/pypoetry/virtualenvs/in-the-black-47xJI4oC-py3.12/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 2060, in _call_with_config
    context.run(


  File "/root/.cache/pypoetry/virtualenvs/in-the-black-47xJI4oC-py3.12/lib/python3.12/site-packages/langchain_core/runnables/config.py", line 461, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
           ^^^^^^^^^^^^^^^^^^^^^


  File "/root/.cache/pypoetry/virtualenvs/in-the-black-47xJI4oC-py3.12/lib/python3.12/site-packages/langchain_core/output_parsers/base.py", line 208, in <lambda>
    lambda inner_input: self.parse_result(
                        ^^^^^^^^^^^^^^^^^^


  File "/root/.cache/pypoetry/virtualenvs/in-the-black-47xJI4oC-py3.12/lib/python3.12/site-packages/langchain_core/output_parsers/openai_tools.py", line 369, in parse_result
    pydantic_objects.append(
  tool(
    **res[
      "args"
    ]
  )
)
                            ^^^^^^^^^^^^^^^^^^^


  File "/root/.cache/pypoetry/virtualenvs/in-the-black-47xJI4oC-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


pydantic_core._pydantic_core.ValidationError: 1 validation error for Classifications
classifications
  Input should be a valid list [
  type=list_type,
  input_value='{"classifications": [{"t...rather than income."}]}',
  input_type=str
]
    For further information visit https://errors.pydantic.dev/2.12/v/list_type
2

Hi @darthShana!

Yeah, looks like Claude occasionally returns the nested field as a JSON string instead of a real array.

Have you tried the json_schema method?

structured_llm = self.chat.with_structured_output(Classifications, method="json_schema")

It’s generally more reliable than the default tool calling method, so I’d give it a try. You’ll just want a recent langchain-anthropic installed.

More detail in the docs: ChatAnthropic integration - Docs by LangChain

Hope that helps!

3

hi @dariel.datoon

thanks for your response.. I did try this, and i found that it ignored a lot of the descriptions in the pydantic description

    account_number: str = Field(description="the bank account number of the transaction")

so it ignored the instructions “the bank account number of the transaction” have you found this to be the case?

4

Typically, I’ve seen the model honor the description. What version of anthropic and langchain-anthropic are you on?

5

im using

anthropic-0.84.0

and

langchain_anthropic-1.3.4

6

Thanks for the versions!

I ran some tests on anthropic 0.117.0 and langchain-anthropic 1.5.0 and couldn’t reproduce it. The field descriptions came through to the model fine.

Do you have a minimal repro of a failing case? Or if you’re tracing in LangSmith, any chance you could share the trace? Either one would help me see what’s different in your setup.

In the meantime, one option is to go back to function calling (drop method="json_schema") and add a small guard: if a field comes back as a string, parse it as JSON before you validate. Something like this, reusing your Classifications model:

import json

def coerce_args(args: dict) -> dict:
    """Repair Claude's occasional stringified / double-enveloped tool args."""
    fixed = {}
    for key, value in args.items():
        if isinstance(value, str):
            try:
                decoded = json.loads(value)       # field came back as a JSON string
            except (json.JSONDecodeError, ValueError):
                fixed[key] = value                # genuinely a string, leave it
                continue
            if isinstance(decoded, dict) and key in decoded:
                decoded = decoded[key]            # unwrap the repeated wrapper key
            fixed[key] = decoded
        else:
            fixed[key] = value
    return fixed

# force the tool call, then validate through the guard
llm = self.chat.bind_tools([Classifications], tool_choice="Classifications")

def classify(prompt: str) -> Classifications:
    msg = llm.invoke(prompt)
    return Classifications(**coerce_args(msg.tool_calls[0]["args"]))

The guard walks the tool-call args, and for anything that came back as a JSON string it decodes it (and unwraps the repeated key if the model doubled it up). Well-formed responses pass straight through.

7

thanks @dariel.datoon

ill follow this pattern