Hi team, we have set up some alerts based on metadata filters.
We are getting falsely alerted. When the alert triggers, we enter the tracing list using the same filters and no traces show up. The alert preview also shows no occurrances.
We had a similar issue with the error field filter in the past and was fixed Alert triggering and ignoring filters - #10 by davidfanggg .
Could you please take a look on the metadata field filtering?
The specific filter we are configuring is:
Thanks in advance!
Hi @davidfanggg
I’m looking into this one. ls_run_depth is a system-injected metadata key (LangSmith adds it during ingestion to mark root vs child runs) and it’s stored as a JSON number, not a string. Your filter has the numeric type (#) selected. Under the hood, alert firing and the alert chart preview + trace list go through two separate filter implementations, and one of the places they tend to disagree is exactly this: number vs string comparison on metadata values. We shipped a fix in this same area last week for a related metadata filter inconsistency, and your case looks like an adjacent edge it didn’t cover.
In the same filter if you switch the value type from # to T and set the value to “0” as text, does the alert preview start showing occurrences and the alert stops misfiring? If you can share the alert ID and the rough timestamp of a recent false positive, we can do some additional digging on our side as well.
Best,
Chad
Hi @lc-chad ,
Thanks for looking into this. We’ve tried with both T and #, also quoting the 0, but the preview always shows zero occurrences.
One example alert id is 7c412336-8ee5-43c5-a038-2334c20dcaab, which triggered 13/05 at 9:02 UTC.
Best,
David
