Hello, recently there has been a limit added to the number of traces you can bulk-add to an annotation queue (100 traces limit). This is really problematic for my workflow. We collect 100s of traces every week. Adding them in batches of <100 takes ages and means the queue is not randomised.
Any workarounds to this? Or fixes on the horizon?
I know I can build automations to add traces to annotation queues as they come in, but this doesn’t work for me as I need to do them in weekly batches.
Apologies for the delay here. The 100-item limit applies to a single add action, including the newer annotation queue items API, so batching is currently unavoidable there.
Two things that should help. First, the older “add runs to annotation queue” SDK method does not declare the same per-request cap, so it is worth trying your full weekly batch through it: client.add_runs_to_annotation_queue(queue_id, runs=[...]), where each entry carries run_id, session_id, and start_time. If a large request is rejected or times out, fall back to sequential batches of 100.
Second, on randomization - shuffle the full weekly set client-side before you add anything. The sample stays randomized even if the adds go out in chunks of 100, which removes the manual UI work either way.