I ran the checkpoint-conformance suite from main against three savers:
InMemorySaverAsyncPostgresSaver3.1.2- one I wrote that deliberately accepts writes from any number of workers on the same thread
All three pass. All three report passed_all_base() = True and conformance_level = FULL.
That is fair, because the suite never tests two writers. All 89 clauses across the 9 capabilities use a single saver instance writing in sequence. There is no isolation clause anywhere. The base class docstrings do not say who is allowed to write either.
The failure this allows. Two processes, one thread_id, real Postgres, durability="sync":
- Worker A is paused with
SIGSTOPduring a model call. - Worker B takes over the thread with
invoke(None, config)and finishes. - Worker A is resumed and writes.
Both workers ran the same task from the same checkpoint, so both call put_writes with the same task_id. INSERT ... ON CONFLICT DO NOTHING means the first write wins: A’s write is dropped, but A is told it succeeded. A then writes a new checkpoint under the same parent, so the chain forks and A’s branch becomes the tip.
The result: B’s final checkpoint is unreachable, and the answer B already returned to its caller no longer matches the thread. Both processes exit 0. Nothing raises and nothing is logged.
Reproduction: GitHub - bharatnpti/langgraph-write-ownership: A reproduction of concurrent-writer checkpoint corruption in LangGraph, and a proposed write_ownership capability for langgraph-checkpoint-conformance · GitHub — one command, about 25 seconds, no API key and no Docker.
This is LangGraph plus PostgresSaver. A plain MessagesState graph forks the same way. I also saw a malformed message list that one provider rejected with HTTP 400, but that part comes from deepagents’ DeltaChannel, not from LangGraph.
A saver cannot refuse a write today. aput returns a config, aput_writes returns None, and langgraph.errors has no rejection type. Raising something anyway is unreliable. I tested 56 configurations; in 28 the exception disappeared with nothing recorded anywhere, because a guard in pregel’s background executors drops it when another exception is already in flight. In 4 the caller saw nothing at all.
What I would like to ask for. One extended capability in checkpoint-conformance: if a saver claims write ownership, it must visibly refuse a write from a superseded worker.
This is additive only. _is_overridden returns True for a method BaseCheckpointSaver does not define, so a new capability is detected just by a saver defining it. No base class change, no runtime change, no effect on existing savers. Extended rather than base, because test_validate_memory.py asserts passed_all_base() for InMemorySaver.
What this is not:
- Not write ordering. #6985’s lock test stays correct.
- Not idempotency.
test_put_writes_idempotentstays correct. - Not silent. #7207 drops stale writes and returns normally. A silent refusal leaves the caller believing it still owns the thread, which is what causes the fork above.
- Not an Agent Server feature.
libs/checkpoint-*is what you use when you are not running Agent Server.
What I have not shown: this is one machine, two processes, SIGSTOP. No real network partition.
I wrote the capability to check it was possible without disturbing anything, and it is: 15 clauses, 3 new files, 21 lines of registration in 4 existing files. A reference implementation is about 40 lines over InMemorySaver. InMemorySaver still scores FULL. A saver that ignores supersession fails 8 of the 15. #6883 is already adding conformance tests for checkpoint-postgres, so the saver this affects most is heading for coverage anyway.
It is in a branch, not a PR: I understand an issue needs to exist and be assigned first.
Is a write-ownership clause something you would want in the suite? And if so, is an extended capability the right place for it?