Difficulty Adding a Google ADK Run Trace to a Dataset

LangSmith Product Help
1

Hi,

I have some google adk (gemini-3.5-flash) traces run that I am trying to add to a dataset but it always fail.
Get message:
Failed to convert example to dataset schema automatically. Please edit the example manually.

The dataset was created clicking in the Apply button:
"Apply chat model schema

Automatically convert tracing project LLM runs into industry standard OpenAI formats that can be used downstream with any model for testing. Compatible with LangChain ChatModel and OpenAI client traces."

Any suggestions?

Thanks

2

Hi @Bonadio that error is expected with Google ADK traces on a dataset that uses Apply chat model schema.

That schema’s transformations (convert_to_openai_message / tools) are built for LangChain BaseChatModel runs and LangSmith OpenAI-wrapper traces. They extract messages/tools into the standard OpenAI-shaped inputs.messages / outputs.message format. Google ADK (via configure_google_adk()) traces fine in LangSmith for observability, but its run payload shape is different, so automatic conversion fails with:

Failed to convert example to dataset schema automatically. Please edit the example manually.

Docs: Dataset transformations, Compatibility and Create a dataset schema.

What to do

1. Edit the example manually (quickest for a few runs)
When you Add to Dataset from a run’s detail page, the modal reports the transformation/schema failure and lets you edit before saving. Shape the example to the chat schema:

  • Inputs: { "messages": [ { "role": "user", "content": "..." }, ... ], "tools": [ ... ] } (tools optional)
  • Outputs: { "message": { "role": "assistant", "content": "..." } }

See Manage datasets in the application and the schema specs.

2. Prefer a dataset without the chat-model schema (best for ADK-native data)
Create a new dataset without “Apply chat model schema”, or define a custom JSON schema that matches your ADK run inputs/outputs. Then add traces as-is. Use chat-model schema only when you need OpenAI-standard messages for cross-model evals.

3. Build examples programmatically
Fetch ADK runs and map fields yourself:

from langsmith import Client

client = Client()
dataset = client.create_dataset("adk-eval-set")

runs = client.list_runs(project_name="your-adk-project", is_root=True, error=False)

# Map ADK run.inputs / run.outputs into whatever schema you want
examples = [
    {"inputs": run.inputs, "outputs": run.outputs or {}}
    for run in runs
]
client.create_examples(dataset_id=dataset.id, examples=examples)

More detail: Create a dataset from traces.

If you specifically need automatic OpenAI conversion for ADK, open a request at support.langchain.com, the docs say they can extend chat-model schema support for incompatible LLM run formats.

Hope that helps!

3

Hi @keenborder786
Thanks for all the information, I tried option 2 (create a dataset with no schema) and I tried to add a run. I was able to save in the dataset the input and output messages, but my run have a tools definitions and those were not included. The tool calls appear just fine.

Any idea on how to include the tools too?

Thanks

4

Hi, glad the no-schema dataset worked for messages.

What you’re seeing is expected: tool calls ≠ tool definitions.

  • Tool calls live on the assistant message in inputs / outputs, so they get copied when you add the run.
  • Tool definitions (name, description, parameters) are not in inputs for Google ADK LLM runs. The ADK integration stores them under: extra.invocation_params.tools

That’s the same pattern as LangChain chat models (defs in invocation params, not inputs). The chat-model schema’s convert_to_openai_tool transformation is what normally lifts those into inputs.tools, without that schema, “Add to Dataset” only persists inputs/outputs, so defs are dropped.

Docs: Dataset transformations – convert_to_openai_tool, Tool JSON type.

How to include the tools

1. Confirm you’re adding the LLM span
Tools are on the nested llm run (model name / google_adk_llm), not the root google_adk.session chain. Open that child run → check Extra / invocation_params.tools.

2. UI (one-off)
When adding to the dataset, edit the example and set:

{

“messages”: [ /* already saved */ ],

“tools”: [ /* paste from the run’s extra.invocation_params.tools */ ]

}

ADK already formats those as OpenAI-style { "type": "function", "function": { ... } } objects.

3. Programmatic (recommended for many runs)

from langsmith import Client

client = Client()

*# Prefer the LLM child runs, not only root chain runs*

runs = client.list_runs(

project_name="your-adk-project",

run_type="llm",

error=False,

)

examples = \[\]

for run in runs:

    tools = (run.extra or {}).get("invocation_params", {}).get("tools") or \[\]

    examples.append(

{

"inputs": {

\*\*(run.inputs or {}),

"tools": tools,

},

"outputs": run.outputs or {},

}

)

client.create_examples(dataset_name="your-dataset", examples=examples)

See Create a dataset from traces.

If those tools still don’t show up on the LLM run’s Extra panel, share a public trace link and we can dig in further.

5

Thanks so much, now I have everything.