Questions about DashscopeLLMClient Streaming ToolCall

[imhansai]

When using DashscopeLLMClient to execute StreamingAgentWithTools，

An anomaly has occurred:

Caused by: kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 0: Cannot read Json element because of unexpected end of the object '}' at path: $ JSON input: }

After investigation, two potential issues were found in the code:

1. When the returned id is "", appendArgumentsDelta should be executed.but it is not.

2. When function.arguments is null, the generated JSON has issues.

================================================================================
Since DashscopeLLMClient is final, it cannot be inherited.The only option is to write a similar "repeated" client, mimicking DashscopeLLMClient.

I am puzzled, is there an issue with the data format returned by aliyun? Or do we need to adjust upsertToolCall? Thanks:)

[BahelanBaishan]

+1, I always thought this was the default behavior, but when I switched to OpenAI, Message.Tool.Call was merged correctly. This seems to be an incorrect implementation in DashscopeLLMClient.

[aozherelyeva]

Hi @imhansai @BahelanBaishan, thank you for the detailed investigation with screenshots. I've created an issue to fix the DashscopeLLMClient streaming tool call handling. Please feel free to subscribe to it – we'll keep you updated on its progress!

[imhansai]

Hi @imhansai @BahelanBaishan, thank you for the detailed investigation with screenshots. I've created an issue to fix the DashscopeLLMClient streaming tool call handling. Please feel free to subscribe to it – we'll keep you updated on its progress!

Thank you! I actually created an issue before, and I will link it to KG-658. I look forward to a fix soon🫰

[957447668]

Hi @imhansai @BahelanBaishan, thank you for the detailed investigation with screenshots. I've created an issue to fix the DashscopeLLMClient streaming tool call handling. Please feel free to subscribe to it – we'll keep you updated on its progress!

When will this problem be resolved?

[1006245347]

meet a same problem ,streaming tool can not useful in alibaba, Message: Unexpected JSON token at offset 247: Unexpected 'null' value instead of string literal at path: $.choices[0].delta.toolCalls[0].type
JSON input: .....index":0,"id":null,"type":null,"function":{"arguments":" {".....
Status code: 200

[957447668]

meet a same problem ,streaming tool can not useful in alibaba, Message: Unexpected JSON token at offset 247: Unexpected 'null' value instead of string literal at path: $.choices[0].delta.toolCalls[0].type JSON input: .....index":0,"id":null,"type":null,"function":{"arguments":" {"..... Status code: 200

koog说是支持的，但实际并不支持，一时半会他们也不会解决，最好的办法就是用DashscopeLLMClient，然后自己拼接tool。

[aozherelyeva]

Hi @957447668, unfortunately, I have no deadline so far. However, please feel free to add more descriptions of the problem to the YouTrack issue – it will help us in investigation and defining the priority level. Thanks!