Add streaming support to ToolCallAdvisor#5383
Closed
tzolov wants to merge 7 commits intospring-projects:mainfrom
Closed
Add streaming support to ToolCallAdvisor#5383tzolov wants to merge 7 commits intospring-projects:mainfrom
tzolov wants to merge 7 commits intospring-projects:mainfrom
Conversation
tzolov
force-pushed
the
toolcalladvisor-streaming
branch
from
7b133d3 to
b76b263
Compare
This was referenced Feb 2, 2026
Implement the adviseStream method in ToolCallAdvisor to enable streaming tool calling with recursive loop support. The streaming implementation mirrors the synchronous adviseCall pattern, supporting returnDirect tool execution and conversation history management. - Implement adviseStream with full tool calling loop for streaming - Add copy(StreamAdvisor) method to StreamAdvisorChain interface - Refactor DefaultAroundAdvisorChain to share copy logic between call and stream advisor chains - Add hook methods for streaming: doInitializeLoopStream, doBeforeStream, doAfterStream, doFinalizeLoopStream, and doGetNextInstructionsForToolCallStream - Execute tool calls on boundedElastic scheduler for blocking ops - Properly handle ToolCallReactiveContextHolder for reactive context Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
… streaming behavior - Add `streamToolCallResponses` configuration option to ToolCallAdvisor that controls whether intermediate tool call responses are emitted during streaming operations - Add `streamWithToolCallResponses()` method for real-time streaming of all chunks including tool call responses using publish() for multicast - Add `streamWithoutToolCallResponses()` method that buffers chunks and only emits for final non-tool-call responses (previous default behavior) - Add `suppressToolCallStreaming()` builder method as convenient shorthand - Refactor streaming implementation with improved code organization and comments - Add unit tests for new streamToolCallResponses configuration - Update AbstractToolCallAdvisorIT with improved test coverage: - Add returnDirect tool callback tests for both call and stream - Add external memory integration tests - Rename test methods for consistency - Remove Google GenAI specific ToolCallAdvisor integration test (consolidated into abstract test class) The main feature in this commit is the new `streamToolCallResponses` option that allows users to control whether intermediate tool call responses are streamed in real-time (for better UX feedback) or buffered until the final response (previous behavior). Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Replace dual-path streaming implementation (buffering vs. parallel) with a single unified approach that always uses parallel streaming and applies a terminal filter to control tool call response visibility. Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
tzolov
force-pushed
the
toolcalladvisor-streaming
branch
from
070e89f to
fb69a3f
Compare
ilayaperumalg
approved these changes
Feb 5, 2026
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement the adviseStream method in ToolCallAdvisor to enable
streaming tool calling with recursive loop support. The streaming
implementation mirrors the synchronous adviseCall pattern,
supporting returnDirect tool execution and conversation
history management.
call and stream advisor chains
doBeforeStream, doAfterStream, doFinalizeLoopStream, and
doGetNextInstructionsForToolCallStream