Prompt_cache_retention: '24h' supported in langchain agents and where to provide it, inside invoke or while creating client?

OSS Product Help LangChain
1

OpenAI documentation says, so it goes inside invoke? -

Configure per request

If you don’t specify a retention policy, for most models the default is in_memory. For gpt-5.5, gpt-5.5-pro, and all future models, the default is 24h and in_memory is not supported. Allowed values are in_memory and 24h.

{
  "model": "gpt-5.5",
  "input": "Your prompt goes here...",
  "prompt_cache_retention": "24h"
}
2

hi @Samarth_03

Yes, prompt_cache_retention: "24h" is supported - LangChain’s OpenAI integration forwards it to the OpenAI SDK via {**self._default_params, **kwargs} in _get_request_payload (Python), and it’s a first‑class promptCacheRetention field in JS (OpenAICacheRetentionParam = "in-memory" | "24h" | null).

Three placement options:

  • Client/constructor: Python ChatOpenAI(model_kwargs={"prompt_cache_retention": "24h"}); JS new ChatOpenAI({ promptCacheRetention: "24h" }).
  • Per invoke: llm.invoke(messages, prompt_cache_retention="24h") (Python) or llm.invoke(messages, { promptCacheRetention: "24h" }) (JS).
  • For create_agent: put it on the model - agent.invoke({"messages": …}, prompt_cache_retention=…) does not forward to the LLM. Use ChatOpenAI(...).bind(prompt_cache_retention="24h") then create_agent(model=…, tools=…).
3

Hi @pawel-twardziak, thanks for your help. I think it can also be passed in model_kwargs of ChatOpenAI. Is that correct?

4

yes, as mentioned above :slight_smile: should work