TL;DR - 2nd tool call of a loop is returning partial data like a pseudo JSON Patch. So I’m looking for examples of using JSON patches with Langchain.
This a little deep, but this is a hacking bot designed to play CTF games.
I recently had to switch to using Claude 4.5 Sonnet (from Claude 3.7 Sonnet) and the tool calls started getting a bit odd, but the gist of it is I’m sending it a pydantic model of an array representing a list of steps it’s supposed to perform, each element is an individual ExploitStep object.
It goes through a REAct style loop and during the first pass of the ExploitStepRevizornode it works perfect, but for subsequent steps it’s only returning a partial amount of data, almost like a JSON patch but not really? And occasionally just returning a list. So i wanted to lean into that idea and just have it return a JSON patch that could be applied to the array.
Exploit Agent Trace: LangSmith
Here is a graph of the Exploit Agent:
The entire reason for this change is the agent would sometimes mangle the list because it only saw one step and could only add a single step at a time as part of the error correction, so now it has access to the entire steps array to modify as needed. I realize while writing this I could handle the output more cleanly, but
A common pattern I will use is NOT using structured output to capture thinking steps from the LLM for making adjustments. This will eventually become a teaching tool so having that output is very handy. The code is also very amateur hour. I work in offensive security as a red teamer and this is my first jump into crafting a large codebase.
Full Trace for anyone that’s interested: LangSmith