Interleaving of thinking blocks with regular output should be preserved for models to continue conversation without errors

[Slava-256]

What's happening?

Chatting with Anthropic Claude Opus 4.6, I received a response with 2 thinking blocks with short regular text output in between them.

When (or before) I submitted my next message in a chat, thinking blocks were moved next to each other and the text that was between them was pushed below:

(Show Reasoning)
(Show Reasoning)
<response text>

When I submitted my next message to the chat, I received the following error:

[Service Issue] Anthropic: Upstream responded with HTTP 400  -
messages.7.content.2: thinking or redacted_thinking blocks in the latest assistant message cannot be modified. These blocks must remain as they were in the original response.

If possible, regular output text and reasoning/thinking blocks order and interleaving (if any) should be preserved.

Where does this happen?

Big-AGI Pro (big-agi.com)

Impact on your workflow

High - Major feature broken

Environment (if applicable)

No response

Additional context

No response

[Slava-256]

@enricoros Hi Enrico! Could you please look at this issue, in case it is a quick and simple fix?

[enricoros]

@Slava-256 right on time, I was about to sign out. Looking now.

[enricoros]

Found the root cause: interleaved blocks disallow back-to-back thinking blocks. Even if produced by the API, they will crash the next request.

Solution: inject a newline separator between signed thinking blocks in the rare case they're back to back.

Rolling out to production now.

[enricoros]

@Slava-256 Fixed. Reasoning with encrypted signatures is enabled. In case of emergency going forward there's a new option in Settings > Chat AI > Reasoning Traces to turn them off altogether, but shall not be needed.

[Slava-256]

@enricoros Thank you very much! I indeed was using both search and browse features, so that may be a key factor in triggering the issue.

[enricoros]

@Slava-256 I should have the new Anthropic Fast Mode working today, which is a new feature to speed up inference by 2.5x for 6x the cost (sometimes the task can justify the option).

To enable, requires a joining an Anthropic wait-list:
https://platform.claude.com/docs/en/build-with-claude/fast-mode

I've registered, waiting for being enabled to test the implementation.

[enricoros]

Anthropic Fast mode is deployed. Requires whitelisting from Anthropic.

[enricoros]

Note for Anthropic on Claude 4.6. There seem to be 2 'newline' (\n\n) characters generated at every response, before the first reasoning block. I would like to remove them, purely from a cosmetic point of view, but their removal would after the message which will then not be the same generated. I need to get in touch with Anthropic to understand the behavior. I suspect a bug that's hopefully gonna be fixed soon in the Anthropic protocol, as it doesn't make much sense to always inject 2 newline characters before reasoning blocks, and seems more like some stray code on the Anthropic end.

[enricoros]

Update: interleaving is being preserved across chats, however the prevailing mechanics according to Anthropic and other vendors is to keep the reasoning traces across an agentic loop, (i.e. keeping all the reasoning blocks until the model is fully done) and then removing them before the next agentic loop.

For Big-AGI we need to apply per-loop conservation (starts with no reasoning and interleaved tool calls and text) while independently having an UI setting that controls whether we want to display or not reasoning traces (show none, show all, show last - they shall all be about visibility, while agentic loops shall work as per above).

Note: very important to keep native blocks across calls for the same vendor, both for correct perfomance and maximize cache hits.