# Truncated JSON & Hidden Generations - GPT-4.1 + PydanticOutputParser **URL:** \ **Category:** LangChain\ **Tags:** python-help\ **Created:** [February 11, 2026, 10:31pm UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930 "2026-02-11T22:31:51Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![weipienlee](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/weipienlee/32/245_2.png) [@weipienlee](https://forum.langchain.com/u/weipienlee)\ **Post date:** [February 11, 2026, 10:31pm UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/1 "2026-02-11T22:31:51Z")
Using PydanticOutputParser with GPT-4.1, I keep hitting truncated JSON issues—turns out, the model sometimes generates extra messages (generations) after a failure, but the `ainvoke` API only returns one. Switching to the generations API just for this feels clunky. Anyone know how to deal with this? (langchain (1.2.10), langchain-core (1.2.7), AzureChatOpenAI from langchain-openai (1.1.7)) ![Screenshot 2026-02-11 at 22.15.55](https://canada1.discourse-cdn.com/flex007/uploads/langchain/original/2X/a/a8767a2bb2b8b9d4ee41d620fa2da3d31ed8ab42.png) ---
**Author:** ![weipienlee](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/weipienlee/32/245_2.png) [@weipienlee](https://forum.langchain.com/u/weipienlee)\ **Post date:** [February 12, 2026, 1:18am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/2 "2026-02-12T01:18:37Z")
The ainvoke of the BaseChatModel literally return the first message of the first generation. Maybe it makes sense have an option to return the last message? ```python class BaseChatModel(BaseLanguageModel[AIMessage], ABC): ... async def ainvoke( ... return cast( "AIMessage", cast("ChatGeneration", llm_result.generations[0][0]).message ) ``` ---
**Author:** ![keenborder786](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/keenborder786/32/1757_2.png) [@keenborder786](https://forum.langchain.com/u/keenborder786)\ **Post date:** [February 12, 2026, 2:33am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/3 "2026-02-12T02:33:02Z")
Are you using `with\_structured\_output`? If yes then use strict=true.` `Doc Reference: `[Models - Docs by LangChain](https://docs.langchain.com/oss/python/langchain/models#structured-output) ---
**Author:** ![weipienlee](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/weipienlee/32/245_2.png) [@weipienlee](https://forum.langchain.com/u/weipienlee)\ **Post date:** [February 12, 2026, 3:03am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/4 "2026-02-12T03:03:33Z")
I’m aware of this option, but one would except more consistency with the current models and any configuration by now. Nevertheless, multiple messages in a single generation(-option) is really a weird one. Must be an api thing. I’m intentionally not using the `with_structured_output` for the reason that i want reasoning before the structure data to increase performance/quality. gpt-4.1 is the most intelligent model, but not a reasoning model. Furthermore, the reasoning models only provide reasoning summaries. That’s no good in my use case. ---
**Author:** ![keenborder786](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/keenborder786/32/1757_2.png) [@keenborder786](https://forum.langchain.com/u/keenborder786)\ **Post date:** [February 12, 2026, 3:08am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/5 "2026-02-12T03:08:12Z")
You can include a `reasoning` parameter in the Pydantic class that you are using. That always works well for me. ---
**Author:** ![weipienlee](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/weipienlee/32/245_2.png) [@weipienlee](https://forum.langchain.com/u/weipienlee)\ **Post date:** [February 12, 2026, 9:41pm UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/6 "2026-02-12T21:41:54Z")
I use to do that too, but the model has no chance to reflect anymore since that’s the final “answer”. What really improves quality for complex tasks is to let it reason on a “scratch pad” before writing the final answer. One could also prompt chain it, but this way you can do it in 1 invocation. Any any case, thank for sparring @keenborder786 ---
**Author:** ![Bitcot\_Kaushal](https://avatars.discourse-cdn.com/v4/letter/b/59ef9b/32.png) [@Bitcot\_Kaushal](https://forum.langchain.com/u/Bitcot_Kaushal)\ **Post date:** [February 16, 2026, 5:38am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/7 "2026-02-16T05:38:23Z")
Hi @weipienlee Yeah, I totally get the frustration here — this one feels more like an abstraction leak than something you’re doing wrong. What’s really happening is: GPT-4.1 sometimes “tries again” internally if it messes up the JSON (like truncating it). The generations API actually contains those retries. But `ainvoke()` only returns the **first message of the first generation** , so if that first one is broken, you never see the corrected version. So it’s not really a Pydantic problem — it’s just how `ainvoke()` is currently implemented. Since you’re intentionally not using `with_structured_output(strict=True)` (which makes sense if you want scratchpad-style reasoning before the final structured output), here are a couple of practical ways people handle this: **1. Add a small retry on parse failure** If JSON parsing fails, just re-prompt with something like: “Return only valid JSON matching the schema. No extra text.” It’s simple, but surprisingly reliable in production — and you still keep your reasoning-first approach. **2. Return the last generation instead of the first** If you’re comfortable subclassing, you can override `ainvoke()` to return: ```python llm_result.generations[0][-1].message ``` Instead of `[0][0]`. In cases where the model internally corrected itself, that usually gives you the fixed version without switching everything to `agenerate()`. **3. Structured reasoning field (if you ever reconsider)** You can keep reasoning by explicitly including it in your Pydantic model: ```python class Output(BaseModel): reasoning: str answer: FinalSchema ``` Then instruct the model to think in `reasoning` and put the final structured output in `answer`. That preserves reflection but still constrains the final output shape. * * * If I were optimizing for stability while keeping scratchpad reasoning, I’d probably combine: - returning the last generation - plus a lightweight retry on JSON parse failure That tends to eliminate most truncation issues without making the architecture feel clunky. ---
**Author:** ![weipienlee](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/weipienlee/32/245_2.png) [@weipienlee](https://forum.langchain.com/u/weipienlee)\ **Post date:** [February 16, 2026, 6:33pm UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/8 "2026-02-16T18:33:05Z")
@Bitcot_Kaushal, my experience is that you need much effort to engineer the reasoning field description to get a comparable quality of a “scratch pad” reasoning. It really makes difference, with the current OpenAI models at least. I ended up overriding the chatmodel. you have to peel it from the content and create an new message instance , something like: ```auto org_msg = llm_result.generations[0][0].message # type: ignore if (org_msg.content and isinstance(org_msg.content, list)): return AIMessage( content=[org_msg.content[-1]], additional_kwargs=org_msg.additional_kwargs, ) # fallback to base behavior return await super().ainvoke(input, config=config, stop=stop, **kwargs) ``` tnx, for sparring, though. ---
**Author:** ![Bitcot\_Kaushal](https://avatars.discourse-cdn.com/v4/letter/b/59ef9b/32.png) [@Bitcot\_Kaushal](https://forum.langchain.com/u/Bitcot_Kaushal)\ **Post date:** [February 20, 2026, 2:46am UTC](https://forum.langchain.com/t/truncated-json-hidden-generations-gpt-4-1-pydanticoutputparser/2930/10 "2026-02-20T02:46:55Z")
Hi @weipienlee That makes a lot of sense actually — and you’re absolutely right. In practice, getting a “reasoning” field to behave like a true scratchpad **does** take real prompt engineering effort. With current OpenAI models especially, the quality difference between free-form internal reasoning vs. schema-constrained reasoning can be noticeable. I like your approach of peeling the last content item and reconstructing a new `AIMessage`. It’s a pretty clean way to: - Preserve the internal retry behavior - Avoid switching to `agenerate()` everywhere - Keep scratchpad-style reasoning intact - Not force strict structured output Your override pattern is pragmatic and minimal — which I honestly prefer over over-engineering the schema just to simulate reasoning.