Feature Request: Support OpenAI GPT-5.6 Explicit Prompt Cache Breakpoints in @langchain/openai

OSS Product Help LangChain
,
1

Feature Request: Support OpenAI GPT-5.6 Explicit Prompt Cache Breakpoints in @langchain/openai

Summary

OpenAI GPT-5.6 introduces explicit prompt cache breakpoints, allowing applications to precisely control which parts of a prompt become cached. This can significantly reduce costs for agentic workloads with large, stable prompt prefixes.

Currently, @langchain/openai supports promptCacheKey, but I couldn’t find a way to use OpenAI’s explicit prompt caching features exposed by the native SDK.

It would be great if LangChain.js exposed this functionality.

Use case

I’m building a LangGraph.js agent that performs evaluations repeatedly throughout the day.

Each evaluation consists of multiple cases, and each case performs several LLM → tool → LLM iterations.

Across all runs, a large portion of the prompt remains identical:

  • tool definitions

  • system prompt

  • shared evaluation instructions

  • few-shot examples

OpenAI’s explicit prompt caching is designed exactly for this scenario and can substantially reduce input token costs.

While promptCacheKey already improves cache hit rates, it doesn’t expose explicit cache breakpoints, which provide much finer control over what gets cached.

Current behavior

Today I can configure:

const model = new ChatOpenAI({
  model: "gpt-5.6",
  promptCacheKey: "evaluation-v1",
});

However, I don’t see a way to express the native OpenAI request fields described here:

Specifically:

  • prompt_cache_options

  • prompt_cache_breakpoint

Expected behavior

It should be possible to configure OpenAI explicit prompt caching through ChatOpenAI without dropping down to the raw OpenAI SDK.

I don’t have a strong opinion on the public API, as long as the OpenAI functionality is fully accessible.

Possible approaches include:

  • exposing the native OpenAI fields directly

  • provider-specific metadata

  • another LangChain abstraction if that’s preferred

The important part is that the adapter forwards OpenAI’s prompt caching configuration unchanged.

Why this matters

This is especially valuable for:

  • LangGraph agents

  • evaluation pipelines

  • repeated benchmark runs

  • tool-heavy agents

  • applications with large static prompt prefixes

These workloads often execute hundreds of nearly identical requests where only the latest user message differs.

Without explicit cache breakpoints, applications cannot fully leverage the prompt caching capabilities offered by GPT-5.6.

Questions

  1. Is this functionality already available through an undocumented passthrough?

  2. If not, would the maintainers be open to adding support?

  3. If this feature would be accepted, I’d be happy to help test or contribute an implementation.

2

Hi, @KurtzOmer! You can do a direct passthrough using modelKwargs. Whatever you put in modelKwargs is forwarded directly to the OpenAI SDK. Here’s a code example for how you might configure prompt_cache_options and prompt_cache_breakpoint:

const model = new ChatOpenAI({
  model: "gpt-5.6",
  promptCacheKey: "evaluation-v1",
  modelKwargs: { prompt_cache_options: { mode: "explicit" } },
});

const system = new SystemMessage({
  content: [
    { type: "text", text: LARGE_STABLE_PREFIX, prompt_cache_breakpoint: { mode: "explicit" } } as any,
  ],
});

const res = await model.invoke([system, new HumanMessage("...")]);
console.log(res.usage_metadata?.input_token_details?.cache_read); // > 0 on repeat calls

Hope that helps!

3

Thanks @dariel.datoon , this was very helpful!

We confirmed that passing prompt_cache_options through the constructor’s modelKwargs works with @langchain/openai 1.4.7.

Our agent intentionally has no system prompt because we are evaluating whether the tool names, descriptions, and schemas are sufficient without additional coaching. We only want to cache the stable tool definitions.

Because OpenAI’s explicit breakpoints must be attached to a supported content block, we added an effectively empty `SystemMessage` after the tools:

const OPENAI_EMPTY_SYSTEM_PROMPT_FOR_TOOL_CACHING = new SystemMessage({
  // LangChain omits a system message whose standard text is empty. The second,
  // provider-native block preserves OpenAI's breakpoint while both whitespace
  // blocks keep this agent promptless.
  content: [
    { type: 'text', text: ' ' },
    {
      type: 'input_text',
      text: ' ',
      prompt_cache_breakpoint: { mode: 'explicit' },
    },
  ] as never,
});

The first standard text block is necessary because LangChain’s agent implementation omits a SystemMessage when its .text value is empty. The provider-native input_text block preserves prompt_cache_breakpoint during Responses API serialization.

We configure the model with:

new ChatOpenAI({
  model: 'gpt-5.6-sol',
  useResponsesApi: true,
  promptCacheKey: 'explorium-evals-clean-agent-tools-v1',
  modelKwargs: {
    prompt_cache_options: {
      mode: 'explicit',
      ttl: '30m',
    },
  },
});

This works end-to-end. In a test run, the first request wrote to cache, and all subsequent 51 requests read the same 20,480-token tools prefix from cache.

Is there a more accurate or intended way to place an explicit breakpoint immediately after the tool definitions without introducing a synthetic system/developer message?

Thanks in advance :folded_hands:

4

Hey, @KurtzOmer!

It looks like cache breakpoints must be in the message block, and cannot be attached to tools.

Also, as you noted, we drop empty system messages in LangChain.

It is rare that we see agents with no system message. Using a whitespace in the prompt is an interesting but reasonable workaround. Personally, I would probably just use a generic "You are a helpful assistant” system message.