fix: preserve tool call/response pairing in forceCompression#871
fix: preserve tool call/response pairing in forceCompression#871QuietyAwe wants to merge 2 commits intosipeed:mainfrom
Conversation
The forceCompression function was cutting messages from the middle without considering tool call/response pairs, causing orphaned 'tool' role messages that trigger API 400 errors: "messages with role 'tool' must be a response to a preceeding message with 'tool_calls'" Changes: - Add findSafeCutPoint() to find a safe cut point after a user message, which ensures tool call/response pairs are kept together - Add removeOrphanedToolMessages() as a safety net to remove any orphaned tool messages at the start of the kept conversation - Add comprehensive unit tests for both helper functions Fixes sipeed#870
nikolasdehor
left a comment
There was a problem hiding this comment.
Good approach with findSafeCutPoint — finding a user message boundary is a clean prevention strategy. A few observations:
-
Coverage gap: This fix only covers
forceCompression()but the same bug exists insummarizeSession()viaTruncateHistory(key, 4). PR #665 covers both paths. -
Fallback to mid:
findSafeCutPointfalls back tomidwhen no user message is found, which reintroduces the original bug. Edge case: conversations with very few user messages (e.g., one long user prompt followed by many tool calls). -
Orphaned assistant messages:
removeOrphanedToolMessagesstrips leadingtoolrole messages but does not handle orphanedassistantmessages withToolCallsat the end of the kept conversation. These would cause the provider to expect tool results that were cut away. -
PR #730: Already merged, addresses the same class of bug in
sanitizeHistoryForProvider(). Consider whether this PR and #665 are still needed as additional layers of defense. -
Issue #870 vs #475: Issue #870 appears to be a duplicate of #475 (filed 2026-02-19), which is the issue our PR #665 closes.
nikolasdehor
left a comment
There was a problem hiding this comment.
Good fix for a real production issue. The root cause is well-explained: forceCompression was naively cutting at mid = len/2 without considering tool call/response pairing, which would leave orphaned tool role messages that trigger API 400 errors.
The approach is sound:
findSafeCutPoint()searches for a user message boundary (always safe since user messages have no tool call dependencies)removeOrphanedToolMessages()acts as a safety net for edge cases
Code review notes:
Tests are thorough -- unit tests for findSafeCutPoint, removeOrphanedToolMessages, and an integration test for the full scenario. The test cases cover forward search, backward search, fallback, and tool pair preservation.
One edge case to consider (non-blocking): if findSafeCutPoint falls back to mid (no user messages found), the cut still happens at a potentially unsafe location, and removeOrphanedToolMessages only strips leading tool messages. If the cut happens in the middle of a multi-tool-call sequence (assistant with multiple tool_calls followed by multiple tool responses), some tool responses could survive while their corresponding tool_calls are removed. In practice this is extremely unlikely since conversations almost always have user messages, so the fallback path should never trigger in real usage.
Nit: test file is missing a trailing newline.
LGTM. This prevents a class of 400 errors that are otherwise very confusing to debug.
|
Thanks for the thorough review @nikolasdehor! Great points. Let me address each: 1. Coverage gap & PR #665: 2. Fallback to 3. Orphaned assistant messages with ToolCalls: 4. PR #730: 5. Issue #870 vs #475: 6. Nit (trailing newline): I'll push updates soon to address #2, #3, and #6. Let me know if you have additional suggestions! |
- Enhanced findSafeCutPoint to handle fallback cases by finding safe cut points after complete tool sequences when no user message exists - Added removeOrphanedAssistantWithToolCalls to handle orphaned assistant messages with ToolCalls at the end of kept conversation - Added comprehensive tests for new edge cases - All tests pass Addresses review feedback from @nikolasdehor on PR sipeed#871
Updates pushed ✅I've addressed the review feedback: 1. Improved
|
Problem
The
forceCompressionfunction was cutting messages from the middle without considering tool call/response pairs, causing orphanedtoolrole messages that trigger API 400 errors:Root Cause
When context limit is hit,
forceCompressionwould simply cut atmid = len(conversation) / 2, which could break a tool call/response pair:Before compression:
After compression (mid=2, keeps [2], [3], [4]):
Solution
findSafeCutPoint()- Finds a safe cut point after a user message, ensuring tool call/response pairs are kept together. User messages are always safe to cut after because they don't have tool call dependencies.removeOrphanedToolMessages()- Safety net that removes any orphaned tool messages at the start of the kept conversation (edge case handling).Testing
Added comprehensive unit tests:
TestFindSafeCutPoint- Tests various scenarios for finding safe cut pointsTestRemoveOrphanedToolMessages- Tests orphaned message removalTestForceCompressionPreservesToolPairs- Integration test for tool pair preservationAll tests pass ✅
Fixes #870