⚡️Improve AI requests

[AntoLC]

Feature Request

We can improve the AI requests with a few steps:

• Translation seems to be an easy task, we can use meta-llama/Llama-3.1-8B-Instruct, the request will be faster
• add a maxToken to the call API, to do so we have to calculate the amount of token in our request, see https://github.com/belladoreai/llama-tokenizer-js or tiktoken (python)
• we can improve the frontend message errors by displaying what is the real problem ?
• We can stream the response, by streaming the response we will be able to display content the the user gradually.

[virgile-dev]

Hey @AntoLC is this still relevant ? If yes I'll let you add a date (quarter) that feels more relevant to you. I'm removing the current date for now.

[AntoLC]

Hey @AntoLC is this still relevant ? If yes I'll let you add a date (quarter) that feels more relevant to you. I'm removing the current date for now.

Actually this PR could close this one: #1016
Let's wait to be merged.

[hodyroff]

Is there still progress on getting the Blocknote AI feature merged? Somehow nothing happened since July, is there a reason?

[AntoLC]

Hi, yes there is a reason, Blocknote were doing lot of refactoring on the AI feature, we preferred to wait before starting:

• https://github.com/TypeCellOS/BlockNote/releases/tag/v0.44.0
• feat: AI autocomplete [render preview] TypeCellOS/BlockNote#2191
• feat: Agent Demo TypeCellOS/BlockNote#2216
• fix: AI menu not updating position on new line TypeCellOS/BlockNote#2233
• fix(ai): upgrade prosemirror-suggest-changes TypeCellOS/BlockNote#2235