Make a llm.with_structured_output call a tool

[ignacio]

I’m implementing a workflow where one of the nodes invokes an LLM that:

1. Returns a structured output (written to state), and
2. Conditionally calls a tool via a conditional edge, based on that output, and comes back to the same node in a React-like architecture.

I wrote something like this:

llm_with_tools = llm.bind_tools(tools)
structured_llm = llm_with_tools.with_structured_output(A_given_class)

The issue:
When the LLM tries to call a tool, it mistakenly uses fields from the structured output class (A_given_class) as tool arguments—instead of using the correct arguments expected by the tool.

How can I ensure the LLM uses the proper tool input schema rather than mixing it with the structured output class?

Thanks!

[niilooy]

Hey Ignacio! Its best not to combine structured output and tool calling simultaneously.

So combining bind_tools() and with_structured_output() creates schema confusion - the LLM sees multiple competing function schemas and gets confused about which one to use for which purpose. The best approach would be to use separate nodes - one with llm.with_structured_output(A_given_class) for structured data, another with llm.bind_tools(tools) for tool calling, connected via conditional edges.

Sequential processing in one node would also work (which I feel is what you’re aiming for), first call llm.with_structured_output(), then conditionally call llm.bind_tools() based on the structured result but this would still require two LLM calls to achieve reliable results.

[ignacio]

Hey niilooy,
It’s actually the other way around, I have a node that first conditionally calls a tools, and then that data needs to be passed to the next node in a structured format. The red arrow below depicts where the structure output is expected.

[elango5292]

This is a common challenge when combining structured output with tool calling in LangGraph. The issue is that with_structured_output() constrains the LLM’s output schema, which can conflict with tool argument schemas.

The solution is to use the tools parameter within with_structured_output() itself, rather than chaining bind_tools() separately. This allows the LLM to understand both schemas independently:

from langgraph.graph import StateGraph, END

# Don't do this:
# llm_with_tools = llm.bind_tools(tools)
# structured_llm = llm_with_tools.with_structured_output(A_given_class)

# Do this instead:
unified_model = llm.with_structured_output(
    A_given_class,
    method="json_schema",
    include_raw=True,
    strict=True,
    tools=tools,  # Pass tools here, not via bind_tools
)

Then in your chatbot node, check what the LLM returned:

def chatbot_node(state):
    messages = state.get("messages", [])
    result = unified_model.invoke(messages)
    
    raw_msg = result.get('raw')
    parsed_output = result.get('parsed')
    
    if parsed_output:
        # Got structured output - store it and signal completion
        return {
            "messages": messages + [raw_msg],
            "final_structured_output": parsed_output
        }
    else:
        # Got tool calls - continue the loop
        return {"messages": messages + [raw_msg]}

The key insight: when you pass tools directly to with_structured_output(), the LLM treats them as separate decision paths. It will either return a structured response matching your class OR make tool calls with proper tool schemas—never mixing the two.

This creates a clean ReAct-style loop where the LLM can call tools as needed, then produce the final structured output when it has enough information.

[alonahmias]

That actually doesn’t work

[auslei]

I did the following and it seemed to work. firstly I wrote pydantic helper functions below:

def build_structured_prompt(prompt: str, schema: type[BaseModel]) -> str:
    """
    Append JSON schema instructions to a prompt so the LLM understands what
    structure to output.
    """
    schema_json = json.dumps(schema.model_json_schema(), indent=2)

    return f"""
{prompt}

You MUST respond ONLY with a single valid JSON object.

The JSON MUST strictly follow this schema:
{schema_json}

Rules:
- Do not wrap the JSON in code blocks.
- Do not add explanations or text outside the JSON.
- Every field in the schema MUST be present.
- If a value cannot be determined, set it to null.
- Ensure the JSON is strictly valid and parseable.
""".strip()

def parse_structured_output(
    message: str,
    schema,
    llm: BaseChatModel | None = None,
):
    """
    Parse an LLM output into a Pydantic model.

    Steps:
        1. Direct JSON parse
        2. Bracket extraction
        3. Optional LLM repair (LLM must produce only JSON)
    """

    # ---- 1. Direct JSON load ----
    try:
        data = json.loads(message)
        return schema(**data)
    except Exception:
        pass

    # ---- 2. Naive bracket extraction ----
    try:
        start = message.index("{")
        end = message.rindex("}") + 1
        candidate = message[start:end]
        data = json.loads(candidate)
        return schema(**data)
    except Exception:
        pass

    # ---- 3. Optional LLM repair ----
    if llm:
        try:
            repair_prompt = f"""
            The following content is intended to be valid JSON matching this schema:

            {schema.model_json_schema()}

            Please fix the JSON structure. 
            Respond ONLY with valid JSON. Do not add explanations.

            Content:
            {message}
            """

            resp = llm.invoke(repair_prompt)
            fixed = resp.content if hasattr(resp, "content") else resp
            data = json.loads(fixed)
            return schema(**data)

        except Exception:
            pass

    # ---- 4. Final error ----
    raise ValueError(
        f"Could not parse message into schema {schema.__name__}: {message}"
    )
```

Then I formated the prompt like so:

sys_msg = SystemMessage(content=build_structured_prompt(sys_prompt,CompanyContextModel))
user_msg = HumanMessage(content="what ever")

response = self.llm_with_tools.invoke(messages)    

if hasattr(response, "tool_calls") and response.tool_calls:           # tool call    
    return {"messages": response}
else:
   your_data = parse_structured_output(get_message_content(response), CompanyContextModel)


It seemed to work. 

[kylebeni]

This worked perfectly, thank you!

[kylebeni]

Just to follow up for anyone in the future who may try to do this, it wasn’t exactly straightforward. After switching from llm.bind_tools() to llm.with_structured_output(tools=TOOLS, strict=True), I ran into an error with tool-calling:

ValueError: `my_func_tool` is not strict. Only `strict` function tools can be auto-parsed

The issue ended up being that my tool had a parameter type (dict[str, Any]) that wasn’t compatible with OpenAI’s allowable types for a strict schema enforcement. Setting strict to False was not allowed, so I had to change the tool parameter to list[dict] which then worked.

I also faced an issue with the LLM failing to use certain tool parameters at all, which was also related to the parameter typing not being enforceable so OpenAI was silently dropping the parameters, making it look like the LLM wasn’t using them. This was also resolved by the typing change on the tool parameter.

[pawel-twardziak]

hi @kylebeni

from my investigation:

The error isn’t raised by LangChain - it comes from the OpenAI Python SDK’s structured-output parse helper (openai/lib/_parsing/_completions.py → validate_input_tools()). Whenever response_format is in the request payload, langchain-openai routes the call through client.chat.completions.parse(), and that helper requires every tool to have "strict": true:

# langchain_openai/chat_models/base.py
if "response_format" in payload:
    raw_response = self.root_client.chat.completions.with_raw_response.parse(**payload)

Since with_structured_output(schema, tools=...) defaults to method="json_schema", response_format is always set - so with strict=False/None your tools are converted without the strict flag and the SDK throws the ValueError. That’s also why setting strict=False is a dead end: json_schema method + non-strict tools is inherently incompatible.

With strict=True, OpenAI requires your tool schemas to follow the strict subset: additionalProperties: false on every object, all properties in required (optional fields as anyOf with null, e.g. int | None), and only supported types. A free-form dict[str, Any] can’t be expressed there - replace it with an explicit Pydantic model:

class MyFuncArgs(BaseModel):
    entries: list[Entry]  # instead of params: dict[str, Any]

@tool(args_schema=MyFuncArgs)
def my_func_tool(entries: list[Entry]) -> str: ...

The “silently dropped parameters” symptom has the same root cause: when a type isn’t expressible in the strict subset, the schema sent to the model is degraded, so the model literally never sees those params.

Alternatives if you can’t make your tools strict-compatible:

1. Use create_agent with response_format - the recommended v1 pattern. It auto-sets strict=True on tools for OpenAI models, and ToolStrategy produces the structured output via a tool call, so strict-schema rules don’t apply to your tools:

from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

agent = create_agent(model="gpt-5.5", tools=[my_func_tool],
                     response_format=ToolStrategy(ContactInfo))
result["structured_response"]

2. method="function_calling" in with_structured_output - no response_format in the payload, so the strict requirement never triggers (you lose the provider-side guarantee on the final answer).
3. Two-step: plain bind_tools() agent, then a separate with_structured_output() call (no tools) on the final text.
4. Upgrade langchain-openai - newer versions auto-default strict=True on tools when response_format is present, so the ValueError goes away by itself. You still need strict-compatible schemas - that’s an OpenAI API constraint, not a LangChain bug.

Docs: LangChain structured output, OpenAI supported schemas