Hello @alexk,
Your logging is working at the run level. The experiments table reads a different layer: pre-aggregated session stats, not raw run metadata.
Why this happens
Per the run data format, costs and TTFT live on first-class run fields (prompt_cost, completion_cost, total_cost, first_token_time). The experiments table aggregates those into session stats like prompt_cost, completion_cost, first_token_p50 (fetch perf metrics). Arbitrary fields like time_to_first_token_seconds or nested metadata.usage_metadata are not rolled up into experiment columns.
Fix 1: Cost breakdown in the experiments table
Attach costs on the LLM span using run.set(usage_metadata=...) (cost tracking):
from langsmith import traceable, get_current_run_tree
@traceable(run_type="llm", metadata={"ls_provider": "my_provider", "ls_model_name": "my_model"})
def chat_model(messages: list):
# ... your logic ...
run = get_current_run_tree()
run.set(usage_metadata={
"input_tokens": 170915,
"output_tokens": 1584,
"total_tokens": 172499,
"input_cost": 0.12819,
"output_cost": 0.00713,
"total_cost": 0.13532,
})
return assistant_message
Important:
- Use
run_type="llm" on the span where costs are attached
- Include both
input_cost/output_cost and token counts
- Prefer
run.set(usage_metadata=...) over nesting under generic metadata (log LLM trace)
After re-running, verify the run has populated prompt_cost / completion_cost (not just total_cost) via the API. Session stats should then expose prompt_cost and completion_cost (fetch perf metrics).
If breakdown still only shows total_cost, that’s likely a LangSmith aggregation bug. worth reporting with a run ID.
Fix 2: Time-to-first-token in the experiments table
time_to_first_token_seconds is not a supported experiment metric. LangSmith expects:
Option A: Native TTFT (recommended for streaming): emit a new_token event on the LLM run (TTFT docs). This populates first_token_time on the run and first_token_p50 / first_token_p99 in session stats.
Option B: Custom metric via evaluator (works for non-streaming): return it as feedback so it appears as an experiment column (analyze an experiment, multiple scores):
from langsmith.schemas import Run
def ttft_evaluator(run: Run, example) -> dict:
# read from your run outputs/metadata however you stored it
ttft = run.outputs.get("time_to_first_token_seconds")
return {"key": "time_to_first_token", "score": ttft}
evaluate(target, data="my-dataset", evaluators=[ttft_evaluator])
What won’t work
- Putting custom metrics only in run
metadata, experiment tables don’t aggregate arbitrary metadata (experiment metadata is for grouping experiments, not per-run custom columns)
- Expecting
/sessions?include_stats=true to return your custom fields. it only returns the fixed stats schema above
TL;DR: Use run.set(usage_metadata=...) on run_type="llm" spans for cost breakdown; use new_token events or a feedback evaluator for TTFT. If prompt_cost/completion_cost are still missing after that, it’s a product bug worth escalating to LangSmith support.
