# Does langchain provides classes for multimodal output models or for simplicity models with image only output like GPT-Image-2? Is the ChatOpenAI() going to work for GPT-Image-2 model?

**URL:** <https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698>\
**Category:** Forum Feedback\
**Tags:** python-help\
**Created:** [October 5, 2026, 3:56pm UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698 "2026-10-05T15:56:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![shubhamkala06](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/shubhamkala06/32/3652_2.png) [@shubhamkala06](https://forum.langchain.com/u/shubhamkala06)\
**Post date:** [October 5, 2026, 3:56pm UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698/1 "2026-10-05T15:56:42Z")

</div>

I have been trying to figure out that is it possible that a model response can be a image block or any other block other than the text block.

So can a model response give an image block like this-  
`{`  
` "type": "image_url",`  
` "image_url": {`  
` "url": f"data:image/png;base64,{base64_str}",`  
` },`

---

<div class="post-metadata">

**Author:** ![pawel-twardziak](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/pawel-twardziak/32/960_2.png) [@pawel-twardziak](https://forum.langchain.com/u/pawel-twardziak)\
**Post date:** [October 5, 2026, 5:36pm UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698/2 "2026-10-05T17:36:24Z")

</div>

hi @shubhamkala06

Yes, a model response can contain image blocks. LangChain has no separate class for image-output models. Images come back in the regular AIMessage. Read them from msg.content\_blocks, where they’re normalized to {“type”: “image”, “base64”: …, “mime\_type”: …}. msg.content keeps the provider’s raw format.

ChatOpenAI(model=“gpt-image-2”) won’t work. ChatOpenAI only calls the Chat Completions and Responses endpoints, and GPT Image models are served from the Images API (/v1/images/generations). The model profile shipped with langchain-openai says the same thing: text\_outputs False, max\_output\_tokens 0.

There are two ways that do work:

1. Use a chat model with OpenAI’s built-in image\_generation tool – ChatOpenAI(model=“gpt-5.5”).bind\_tools([{“type”: “image\_generation”}], tool\_choice=“image\_generation”). ChatOpenAI switches to the Responses API automatically, and the image\_generation\_call output is converted into a standard image block. Multi-turn editing works if you pass the previous AIMessage back in. Streaming partial images („partial\_images”) isn’t supported yet and raises NotImplementedError.

2. Call the Images API yourself – wrap client.images.generate(model=“gpt-image-2”, prompt=…) from the OpenAI SDK in a @tool. Have it return [{“type”: “image”, “base64”: result.data[0].b64\_json, “mime\_type”: “image/png”}]. Tools can return standard content blocks. Don’t use DallEAPIWrapper from langchain-community for this. It only reads image URLs, and GPT Image models return base64.

The exact block from your example, {“type”: “image\_url”, “image\_url”: {“url”: “data:image/png;base64,…”}}, is what ChatGoogleGenerativeAI returns. Use a Gemini image model (e.g. gemini-2.5-flash-image) with response\_modalities=[Modality.TEXT, Modality.IMAGE].

Docs: [Models - Docs by LangChain](https://docs.langchain.com/oss/python/langchain/models#multimodal) and [Tools - Docs by LangChain](https://docs.langchain.com/oss/python/langchain/tools#return-multimodal-content)

---

<div class="post-metadata">

**Author:** ![shubhamkala06](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/shubhamkala06/32/3652_2.png) [@shubhamkala06](https://forum.langchain.com/u/shubhamkala06)\
**Post date:** [October 6, 2026, 7:11am UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698/3 "2026-10-06T07:11:28Z")

</div>

Thanks for the earlier response. Could you also help me with this?

I’m currently working on a **proof-of-concept** where I’m using a RAG subagent as part of my **Deep Agent (using the Deep Agents library)**. The retrieved chunks can contain text, images, tables, etc. Ideally, I’d like to pass the **complete retrieved content, including images, to the model** and have it generate the final response.

My current model (`gemini-3.1-flash-lite`) is text-output only, so whatever it generates is represented as text content. Even if the model outputs JSON describing an image, that JSON is still a text block and therefore isn’t rendered as an image by LangSmith Studio.

As a workaround, I keep the image reference IDs in the agent state and use `@after_agent` middleware to resolve them and add `image_url` blocks to the final `AIMessage`.

The images in my canonical representation are stored as **base64-encoded data** , and I have been able to render them in LangSmith Studio using:

```auto
{
    "type": "image_url",
    "image_url": {
        "url": f"data:image/png;base64,{base64_str}",
    },
}

```

**Is there a native/recommended Deep Agents pattern where the retrieved multimodal content can be passed to the model and the resulting final `AIMessage` can contain both the generated text and the retrieved image blocks, without the `@after_agent` post-processing?**

Passing the images to the model may be less efficient with a text-output model, but I’d prefer this architecture since it would also work naturally with a multimodal output model later.

I’m using LangGraph Dev/Agent Server with LangSmith Studio, and I’m not manually constructing the graph with LangGraph nodes.

---

<div class="post-metadata">

**Author:** ![pawel-twardziak](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/pawel-twardziak/32/960_2.png) [@pawel-twardziak](https://forum.langchain.com/u/pawel-twardziak)\
**Post date:** [October 6, 2026, 11:21am UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698/4 "2026-10-06T11:21:34Z")

</div>

hi @shubhamkala06

afaik there’s no native Deep Agents pattern for this, and your after\_agent approach is a reasonable one.  
An AIMessage only contains what the model actually generated. A text-output model like gemini-3.1-flash-lite can’t emit image blocks. Even a multimodal-output model would generate new images rather than return your retrieved ones byte-for-byte. So if the final message must contain the original retrieved images, some deterministic post-processing is always needed. Middleware (after\_agent, after\_model, wrap\_model\_call) is the supported place for it.

A few things from the deepagents source that matter for your architecture:

1. The subagent result is text-only. The task tool builds its ToolMessage from the last non-empty AIMessage.text of the subagent, or from structured\_response serialized to JSON (\_return\_command\_with\_state\_update in deepagents/middleware/subagents.py). Image blocks your RAG subagent retrieves never reach the main agent’s model.

2. Custom state keys do flow back. The same function merges every subagent state key into the parent, except messages, todos, structured\_response, skills\_metadata and a few internal ones. So a key like „retrieved\_images” set inside the subagent ends up in the main agent’s state. This is basically what you’re doing already, and it’s the supported route. Declare the key in both state schemas (e.g. via a middleware state\_schema). Give it a reducer if the main agent can call the subagent several times.

3. If you want the main model to actually see the images, make retrieval a tool on the main agent instead of a subagent. Have it return standard content blocks, i.e. text plus {“type”: “image”, “base64”: …, “mime\_type”: “image/png”}. Deep Agents passes those to the model as a multimodal ToolMessage: [Multimodal inputs and outputs - Docs by LangChain](https://docs.langchain.com/oss/python/deepagents/multimodal#custom-tool-outputs). Your model supports this, since its profile has image\_inputs and image\_tool\_message set to True.

4. One small improvement to your post-processing: ask the model to cite image IDs inline (e.g. [img:abc123]). Then have the middleware split the text at those markers and interleave the image blocks, so each image appears where it’s referenced instead of all at the end.

5. Image blocks you add to the final AIMessage are sent back to the model on the next turn as part of the history. Gemini accepts that. Some providers don’t - for example, OpenAI Chat Completions rejects images in assistant messages. Deepagents’ unsupported-content middleware only sanitizes HumanMessage and ToolMessage. If you might switch providers later, strip those blocks in wrap\_model\_call, or keep display-only images in state.

---

<div class="post-metadata">

**Author:** ![shubhamkala06](https://yyz1.discourse-cdn.com/flex007/user_avatar/forum.langchain.com/shubhamkala06/32/3652_2.png) [@shubhamkala06](https://forum.langchain.com/u/shubhamkala06)\
**Post date:** [October 7, 2026, 6:35am UTC](https://forum.langchain.com/t/does-langchain-provides-classes-for-multimodal-output-models-or-for-simplicity-models-with-image-only-output-like-gpt-image-2-is-the-chatopenai-going-to-work-for-gpt-image-2-model/4698/5 "2026-10-07T06:35:35Z")

</div>

Thanks, that’s very helpful.