How can i create a tool, which will allow agent to read and understand the image, What should the tool return so that agent can read the image or any other binary formats. Note that i don’t want to make the tool which describes the image, instead i want to create the tool which will allow agents to …

Hi @hottered10 ! Correct me if I’m wrong but you are trying to create a tool that takes as input an image (by url or file path) and returns some data for the agent to comprehend. Usually, you can call a multimodal model by passing a message that specifies the file format and the source type (check …

I came across a similar problem, and I only found tools that describe images by returning a string. I want my agent to be able to recognize when an image contains a table and understand its contents, or for an example it’s an image of nature, to recognize the objects in it, not just receive a descri…

Hey @rako and @hottered10 you could do something similar to the following: from pydantic import BaseModel, Field from langchain_openai import ChatOpenAI from langchain_core.messages import SystemMessage, HumanMessage from langchain_core.tools import tool class ImageInfo(BaseModel): description…

You can generally create ToolMessages containing an image content block, e.g., ToolMessage( content=[ { "type": "image", "source_type": "base64", "data": image_data, "mime_type": "image/jpeg", }, ], tool_call_id="...", ) T…

@chester-lc Thanks for the help. I tried test exaple from LangChain you referenced. Here is the code: import base64 from langchain_core.messages import HumanMessage, ToolMessage, AIMessage from langchain_openai import ChatOpenAI def random_image() -> dict: "Return a image" return "" llm…

OpenAI does not support images in tool responses. Refer to their API reference : [Screenshot 2025-07-10 at 10.22.31 AM] The same appears true for their Responses API. One option is to instead use structured outputs to generate your tool payload, and then return the response as a user message inste…

On that note, I’ve actually created a small Image Tool that helps with quick tasks like converting and optimizing images without unnecessary features. If you’re into lightweight, practical tools, you can check it out — I’d genuinely love feedback.

@Papadopulus I am running into the same problem, its because langchain ToolMessage is treating the output of the tool as string, even if the tool returns JSON it concatenating putting it under content of ToolMessage, which is making llm to see it as a text rather than a Image. I tried with the bas…

@Bandarakesh @Papadopulus You can use response API which does support image data in Tool Message: from langchain_openai import ChatOpenAI from langchain_core.tools import tool import base64, httpx @tool def get_image() -> list: """Fetch and return an image.""" image_data = base64.b64encode…

How to make an image tool?

OSS Product Help LangChain

rako July 10, 2025, 6:50am 6

Hello @marco and @chester-lc! Thank you for the help, I’ll try the suggestions you sent me, and if you think of anything else, please send it my way

Topic		Replies	Views
Tool call and structured ouput LangChain python-help	2	437	October 23, 2025
File uploads using langgraph models LangChain python-help	1	1183	November 11, 2025
Image editing inside langchain ts LangChain js-help	0	239	August 6, 2025
Feature Request: Image Input support for ChatMistralAI LangChain python-help , feature-request	1	47	April 19, 2026
Image input support for Grok LangChain python-help	6	430	October 27, 2025

How to make an image tool?

Related topics