I was messing around with the InMemoryCache and saw that the docs don’t mention too much about the invocation of it. The docs use it like this:
print(graph.invoke({"x": 5}, stream_mode='updates'))
# [{'expensive_node': {'result': 10}}]
print(graph.invoke({"x": 5}, stream_mode='updates'))
# [{'expensive_node': {'result': 10}, '__metadata__': {'cached': True}}]
Is it necessary to specify the stream_mode? If so how to use it with other modes. If we can’t use it with other modes how can I get the actual value of the State at the end as the returned value in the given example is not the complete state (check docs for full code)?
Another problem I faced was when trying to use InMemoryCache and InMemorySaver together. When introducing an InMemorySaver, the InMemoryCache just stops working and no caching is being performed no matter what stream_mode I choose. There is no explanation for this behavior anywhere. Here is the code I am using to test this behavior.
from langgraph.graph import StateGraph
from langgraph.cache.memory import InMemoryCache
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import CachePolicy
from typing import TypedDict
import time
class State(TypedDict):
x: int
result: int
def expensive_node(state: State) -> State:
# expensive computation
time.sleep(2)
return {"result": state["x"] * 2}
graph = (
StateGraph(State)
.add_node(expensive_node, cache_policy=CachePolicy())
.set_entry_point("expensive_node")
.set_finish_point("expensive_node")
.compile(cache=InMemoryCache(), checkpointer=InMemorySaver())
)
CONFIG = {'configurable': {'thread_id':1}}
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))
# [{'expensive_node': {'result': 10}}]
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))
# [{'expensive_node': {'result': 10}}]
What am I missing? Could anyone let me know of the invocation conditions of the cache and any potential issues with checkpointers?