Idea: A Return Node for Recovering from Empty AIMessage Outputs

OSS Product Help LangGraph
1

Problem:

In some cases, the LLM returns an AIMessage with:

  • finish_reason = "STOP"

  • content = "" (empty)

  • sometimes with or without tool_calls

Even when a response is expected, the graph silently ends without producing any meaningful output.
Currently, users must write custom router logic and retry nodes to recover from this state.

Proposed Solution: Route + Recovery Node (separate concerns)

This feature consists of two parts:

1. Empty Output Route (Router Function)

A built-in route that detects empty AIMessage outputs and decides what to do next:

if (
    isinstance(last, AIMessage)
    and last.response_metadata.get("finish_reason") == "STOP"
    and not last.content
    ):
        if last.tool_calls:
            return "tool"   #send it to ToolNode
        else:
            return "recover"

It also supports a retry limit (e.g., trial > MAX_TRIALS) to avoid infinite loops.

2. Recovery Node (Optional, User-Configurable)

A node that defines how to recover from the empty output. For example:

  • Inject a corrective SystemMessage, or

  • Directly reroute back to ChatNode without modifying messages.

Example:

def recovery_node(state):
    state["messages"] = [SystemMessage(
        "Use backend tool responses as the primary source of truth."
    )]
    state["trial"] += 1
    return state

Users can customize this behavior depending on their application.

Example Flow:

ChatNode → EmptyOutputRoute
→ ToolNode (if tool_calls exist)
→ RecoveryNode → ChatNode (retry)
→ End (once a non-empty output is produced)

Benefits:

  • Handles a real LLM failure mode: empty outputs

  • Prevents silent termination of graphs

  • Reduces boilerplate routing/retry logic

  • Keeps routing and recovery logic separate

  • Gives users flexibility in how recovery is implemented

I believe this could be helpful for many developers building graph-based workflows.
I’d really appreciate feedback from the maintainers and the community.

2

hi @ayushkumar802

This is a real use case - empty AIMessage outputs do happen in practice, particularly with certain models, providers, or under high load.
However, IMhO LangGraph already provides all the building blocks to solve this problem cleanly, and adding a dedicated built-in for this specific failure mode would likely be the wrong level of abstraction.

You can use RetryPolicy with validation inside the node

The most straightforward approach is to treat empty output as an exception inside your model-calling node and let LangGraph’s built-in RetryPolicy handle retries automatically:

from langgraph.graph import StateGraph, START, END
from langgraph.types import RetryPolicy
from langchain_core.messages import AIMessage

class EmptyResponseError(Exception):
    pass

def call_model(state):
    response = model.invoke(state["messages"])
    if isinstance(response, AIMessage) and not response.content and not response.tool_calls:
        raise EmptyResponseError("LLM returned empty response")
    return {"messages": [response]}

builder = StateGraph(State)
builder.add_node(
    "agent",
    call_model,
    retry_policy=RetryPolicy(
        max_attempts=3,
        retry_on=EmptyResponseError,
        backoff_factor=2.0,
    ),
)

So one clean pattern is: treat empty final output as an error*(raise a dedicated exception), and let users configure retries via RetryPolicy / bounded loops. That keeps retry semantics consistent (and observable) instead of creating a new “silent retry” subsystem.

This gives you automatic retries with exponential backoff, without needing any additional routing or recovery nodes. Source: LangGraph RetryPolicy documentation.

When to use this: When the empty response is likely a transient provider issue and simply retrying the same request will produce a valid response.

Or you can implement your own middleware or node with a route for that

I think the functionality is valuable, but it probably belongs as an opt-in guardrail, not a mandatory built-in route.