fix: deserialize JSON in message function/tool call arguments for experiment run outputs #11299
Conversation
dosubot
bot
added
the
size:L
RogerHYang
changed the title
RogerHYang
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
axiomofjoy
force-pushed
the
version-13
branch
from
cf33097 to
1d82b46
Compare
RogerHYang
force-pushed
the
feat/dataset-helpers-deserialize-message-arguments
branch
from
c89c9af to
cd34463
Compare
RogerHYang
force-pushed
the
version-13
branch
from
595e9f3 to
d77e250
Compare
RogerHYang
force-pushed
the
feat/dataset-helpers-deserialize-message-arguments
branch
from
cd34463 to
d4d0340
Compare
dosubot
bot
added
size:XL
| const prefix = evaluatorName + "."; | ||
| if (annotationName.startsWith(prefix)) { | ||
| const configName = annotationName.slice(prefix.length); | ||
| matchedConfig = outputConfigs.find((c) => c.name === configName); |
There was a problem hiding this comment.
Preview optimization matching fails with spaced names
Low Severity
computePositiveOptimization matches multi-output annotations using evaluatorName from store state, but preview annotation names are generated from trimmed payload.name. If the evaluator name has leading/trailing spaces, the prefix check fails and positiveOptimization is omitted in ExperimentAnnotationButton.
Additional Locations (1)
RogerHYang
force-pushed
the
feat/dataset-helpers-deserialize-message-arguments
branch
from
346b3f8 to
5e89eaf
Compare
… arguments - Add _deserialize_json_if_possible() and use it in _get_message() for MESSAGE_FUNCTION_CALL_ARGUMENTS_JSON and TOOL_CALL_FUNCTION_ARGUMENTS_JSON so stored JSON strings are parsed to dicts; invalid JSON is left as string. - Add tests for _get_message() using unflatten (role, content, name, function_call with valid/invalid JSON, tool_calls with valid/invalid JSON, empty/None tool_calls). - Update test_get_dataset_example_input expected messages to use deserialized arguments.
RogerHYang
force-pushed
the
feat/dataset-helpers-deserialize-message-arguments
branch
from
5e89eaf to
477080e
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| if function_arguments is None: | ||
| function_arguments = arguments | ||
| function = _Function(name=function_name, arguments=function_arguments) | ||
| tool_calls.append(_ToolCall(function=function)) |
There was a problem hiding this comment.
Tool calls dropped without function names
Medium Severity
The new tool_calls loop only appends entries when TOOL_CALL_FUNCTION_NAME is truthy. Tool calls that have TOOL_CALL_FUNCTION_ARGUMENTS_JSON but no name are now discarded, causing loss of captured tool-call data in dataset_helpers.py.
| tool_calls.append(_ToolCall(function=function)) | ||
| role = get_attribute_value(message, MESSAGE_ROLE) or "assistant" | ||
| content = get_attribute_value(message, MESSAGE_CONTENT) | ||
| msg = _Message(role=role) |
There was a problem hiding this comment.
Missing role now mislabeled as assistant
Medium Severity
_get_message now defaults role to "assistant" when MESSAGE_ROLE is absent. This changes previously missing/unknown roles into concrete assistant messages, which can misrepresent conversation data produced by dataset_helpers.py.
| function_call_arguments = _safely_json_decode(arguments) | ||
| if function_call_arguments is None: | ||
| function_call_arguments = arguments | ||
| function_call = _Function(name=function_call_name, arguments=function_call_arguments) |
There was a problem hiding this comment.
JSON null arguments stay as strings
Low Severity
_get_message treats a decoded None as a parse failure and falls back to the original string. Because _safely_json_decode("null") returns None, valid JSON null in MESSAGE_FUNCTION_CALL_ARGUMENTS_JSON or TOOL_CALL_FUNCTION_ARGUMENTS_JSON is emitted as "null" instead of None.
Additional Locations (1)
RogerHYang
deleted the
feat/dataset-helpers-deserialize-message-arguments
branch
2 participants
Before
After
Note
Low Risk
Localized change to message-shaping logic plus added test coverage; main risk is minor downstream expectations changing because arguments may now be dicts instead of JSON strings.
Overview
Improves
dataset_helpers._get_messageto JSON-decodefunction_call.argumentsandtool_calls[].function.argumentswhen they are stored as JSON strings, while leaving invalid JSON as the original string and omitting empty tool call lists.Adds typed message/tool/function structures (
TypedDict) and expands unit tests to cover deserialization behavior and updates expected dataset input snapshots accordingly.Written by Cursor Bugbot for commit 477080e. This will update automatically on new commits. Configure here.