Fetching prompts from prompt registry at runtime on a thread level

Product Help LangGraph
1

Hi everyone,

I’m new to LangGraph and LangSmith frameworks, I work on a chatbot project, and I struggle to find a solution for dynamically getting the prompts from the registry at a thread level.

Problem Overview

I deployed my graph with LangSmith, but all prompts were hardcoded. Now I’d like to use different prompts for different threads (conversations) — i.e., specify a prompt name in the registry, fetch it once when the thread is created, and reuse it for all subsequent messages in that thread.

I saw the Context example in the docs (https://docs.langchain.com/oss/python/use-graph-api#extended-example-specifying-llm-at-runtime), but that pattern still requires static definitions (Anthropic / OpenAI) and doesn’t solve the dynamic registry prompt issue.

What I’ve tried

One of the possible solutions would be to add prompt names into the context, and then inside the node get the name and pull the prompt:

@dataclass
class BotContext:
    system_prompt_name: str = "default-system-prompt"

from langsmith import Client
client = Client()

def node(state):
        runtime = get_runtime(BotContext)
        system_prompt = client.pull_prompt(runtime.context.system_prompt_name)

But this pulls the prompt every run, meaning for each message in the conversation. I only want to fetch it once per thread, when it’s first created.

Is there a recommended way to:

  • Fetch a prompt from the LangSmith registry once when a thread starts

  • Cache / store it in thread state or context

  • Reuse it for all subsequent messages without re-pulling?

Any advice or examples would be greatly appreciated — thank you!

2

Hi! The best method depends on how your structuring the code that calls your LangGraph, but most likely, you’ll want to pull the prompt outside of a node function, and either a) Put it in the graph state when invoking the graph b) Put the prompt in some other context you define and that your node functions can access.

Option b will most likely be the cleanest for you.