yes - that exact pattern works, with one important caveat about ToolStrategy that the docs don’t spell out.
What the source code actually does after your middleware runs
After wrap_model_call fires, _get_bound_model re-reads request.response_format (langchain/agents/factory.py:1198) and rebuilds the bound model from it. So mutating request.response_format inside the handler is enough - no other plumbing is required.
Caveat: ToolStrategy swaps must be a subset of the upfront declaration
This is buried in the factory but it will bite you if you don’t know it. When the final (post-middleware) format is a ToolStrategy, the factory validates that every schema variant was declared when the agent was built:
# sources/langchain/libs/langchain_v1/langchain/agents/factory.py
# Middleware is allowed to change the response format
# to a subset of the original structured tools when using ToolStrategy,
# but not to add new structured tools that weren't declared upfront.
for tc in effective_response_format.schema_specs:
if tc.name not in structured_output_tools:
msg = (
f"ToolStrategy specifies tool '{tc.name}' "
"which wasn't declared in the original "
"response format when creating the agent."
)
raise ValueError(msg)
Concretely:
- If you built the agent with
response_format=ToolStrategy(A) and the middleware swaps in ToolStrategy(B), you get ValueError - B wasn’t declared up front.
- The fix: declare the superset as a Union at construction time, and let the middleware pick one variant per invocation.
- This check does not apply to
ProviderStrategy. If you swap to ProviderStrategy(X) where the model supports native structured output, the new schema is accepted without the upfront-declaration requirement.
The full pattern for a predetermined, per-invocation format
Since you said you already know which format you want at call time, pass it via context_schema - that’s the read-only per-invocation channel the middleware docs recommend, and it’s exactly what runtime.context is for (see docs/src/oss/langchain/context-engineering.mdx):
from dataclasses import dataclass
from typing import Callable, Union
from pydantic import BaseModel
from langchain.agents import create_agent
from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse
from langchain.agents.structured_output import ToolStrategy
class ProductReview(BaseModel): ...
class CustomerComplaint(BaseModel): ...
class SupportTicket(BaseModel): ...
SCHEMAS = {
"review": ProductReview,
"complaint": CustomerComplaint,
"ticket": SupportTicket,
}
@dataclass
class Context:
response_format_key: str # predetermined by the caller
@wrap_model_call
def select_format(
request: ModelRequest,
handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
key = request.runtime.context.response_format_key
request.response_format = ToolStrategy(SCHEMAS[key]) # subset of the Union
return handler(request)
agent = create_agent(
model="gpt-5",
tools=tools,
# Declare the superset upfront so the subset swap above is always valid.
response_format=ToolStrategy(Union[ProductReview, CustomerComplaint, SupportTicket]),
middleware=[select_format],
context_schema=Context,
)
agent.invoke(
{"messages": [...]},
context=Context(response_format_key="complaint"),
)
Why this shape:
- The Union on
create_agent pre-registers every tool that any invocation might need - that clears the factory’s subset check.
context_schema is the right carrier for “the caller already decided”: it’s read-only, per-invocation, and doesn’t pollute state.wrap_model_call runs once per model step, so retries inside the same invoke keep the format you picked.
If instead you want to skip the Union gymnastics and your model has native structured output, swap to ProviderStrategy(SCHEMAS[key]) in the middleware - no upfront declaration needed, though you lose the Union fallback for providers without native support.
Sources
- Source (git repo):
langchain/libs/langchain_v1/langchain/agents/factory.py
lines ~1196–1255 - response_format re-read after middleware, and the ToolStrategy subset validation.
- Source (git repo):
langchain/libs/langchain_v1/langchain/agents/middleware/types.py
ModelRequest.response_format is a writable dataclass field.
- Docs: Overview - Docs by LangChain -
wrap_model_call
signature, editing the request, custom context_schema.
- Docs: Context engineering in agents - Docs by LangChain -
context_schema + runtime.context as the per-invocation channel.
