Is it possible to stream structured output?

OSS Product Help LangChain
1

I am trying to stream a response, so it can be delivered in chuncks to the user. I want to stream the ‘answer‘ part from the structured output. Is this possible? I can’t find anything in the documentations about this.


private readonly agent = createAgent({
    model: new ChatGoogleGenerativeAI({
      model: 'gemini-2.5-flash-lite',
      streaming: true,
    }),
    tools: [retrieveDocuments],
    responseFormat: z.object({
    answer: z.string().describe('A markdown repsonse to the user query.'),
    resources: z.array(z.object({ name: z.string(), url: z.string() })),
    }),
    contextSchema: z.object({ organisationId: z.number() }),
  });

  const messages = [
      new SystemMessage(systemPrompt.replace('[company]', organisation.name)),
      new HumanMessage(query),
    ];

    for await (const [token] of await this.agent.stream(
      { messages },
      {
        streamMode: 'messages',
        context: { organisationId: session.organisationId },
      },
    )) {
    }
2

Have you checked if this documentation page is helpful: Streaming - Docs by LangChain?

3

Yes, I have. Might be missing something, but it doesn’t seem to document how to stream structured output.

4

Hi @Younes

afaik you can stream the model’s message tokens while it’s generating, but you cannot incrementally stream the parsed fields of a structured output (e.g., the answer property of your responseFormat object). The schema is validated once the model finishes and a complete, valid JSON/tool-call payload is available. This is by design in LangChain v1/LangGraph v1.

5

@pawel-twardziak is correct. You can either use agent.stream and get a stream of individual tokens that you can assemble into the final response (which will adhere to the specified responseFormat). Or you can run agent.invoke to get a properly typed object (but this will not support streaming the response).

6

Thank you!