refactor(playground): use tracer in playground clients #11244
Conversation
axiomofjoy
force-pushed
the
xander/playground-traces-v2
branch
7 times, most recently
from
7848e72 to
533c979
Compare
dosubot
bot
added
the
size:XXL
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
axiomofjoy
force-pushed
the
version-13
branch
from
cf33097 to
1d82b46
Compare
axiomofjoy
force-pushed
the
xander/playground-traces-v2
branch
from
533c979 to
0f8e96f
Compare
anticorrelator
left a comment
anticorrelator
left a comment
There was a problem hiding this comment.
looks good! I think the main thing is that some evaluator codepaths were removed
anticorrelator
left a comment
anticorrelator
left a comment
There was a problem hiding this comment.
lgtm after discussion
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| dataset_example_id=None, | ||
| repetition_number=repetition_number, | ||
| ) | ||
| await results.put((tracer, repetition_number)) |
There was a problem hiding this comment.
Evaluator logic removed from single chat completion subscription
High Severity
The entire evaluator flow has been removed from _stream_single_chat_completion and the chat_completion subscription method no longer calls get_evaluators. The ChatCompletionInput schema still exposes an evaluators field, so clients can pass evaluators, but they're silently ignored. The chat_completion_over_dataset path still processes evaluators correctly, suggesting this removal from the single completion path was unintentional.
Additional Locations (1)
There was a problem hiding this comment.
This was on purpose
2 participants
Note
Medium Risk
Touches core playground LLM execution/tracing paths and span persistence, so regressions could affect observability data and experiment runs across providers; changes are mostly refactor/instrumentation but broad in surface area.
Overview
Routes playground LLM tracing through the shared
Tracer/OpenTelemetry span pipeline instead of the bespokestreaming_llm_spanimplementation, including persisting spans viaTracer.save_db_traces()for both chat completions and dataset experiment runs.Unifies message handling by replacing tuple-based chat messages with a typed
PlaygroundMessagedict (plus helpers), and centralizes OpenInference attribute emission (input/output, tools, invocation params with sensitive-key filtering) insidePlaygroundStreamingClient.chat_completion_create(); call sites (chat_mutations,subscriptions,evaluators) now pass atracerinto clients.Updates dependency pins (notably
anthropicandarize) and tightens type-check requirements by pinning OpenTelemetry package versions; adjusts and adds tests/cassettes to validate the new span name (ChatCompletion) and recorded attributes (including request URL, input payload, tools, and invocation parameters).Written by Cursor Bugbot for commit 8432b71. This will update automatically on new commits. Configure here.