Skip to content

Conversation

@fangchenli
Copy link
Contributor

@fangchenli fangchenli commented Feb 3, 2026

Problem

When diff-based evolution is enabled, the "Previous Attempts" section of prompts shows changes like:

Change 1: Replace 15 lines with 18 lines

This gives the LLM no visibility into what the actual edits were, making it harder to:

  • Learn from successful patterns
  • Avoid repeating failed exact matches
  • Understand what format produces valid SEARCH blocks

This contributes to the high rate of "apply diff fail" errors (see issue #346) where SEARCH patterns don't exactly match the original code.

Solution

Update format_diff_summary() to show actual content for multi-line blocks:

Change 1: Replace:
  def old_function():
      return False
with:
  def new_function():
      return True

Single-line changes remain compact:

Change 1: 'x = 1' to 'x = 2'

Add _format_block_lines() helper with configurable truncation limits.

Configuration

New options in prompt: config section:

prompt:
  diff_summary_max_line_len: 100  # Truncate lines longer than this
  diff_summary_max_lines: 30      # Max lines per SEARCH/REPLACE block

Files Changed

  • openevolve/config.py - Add PromptConfig options
  • openevolve/utils/code_utils.py - Update format_diff_summary
  • openevolve/iteration.py - Pass config to format_diff_summary
  • openevolve/process_parallel.py - Pass config to format_diff_summary
  • tests/test_code_utils.py - Add tests for new behavior

Some files got reformatted, I'll leave those changes there.

## Problem

When diff-based evolution is enabled, the "Previous Attempts" section
of prompts shows changes like:

    Change 1: Replace 15 lines with 18 lines

This gives the LLM no visibility into what the actual edits were,
making it harder to:
- Learn from successful patterns
- Avoid repeating failed exact matches
- Understand what format produces valid SEARCH blocks

This contributes to the high rate of "apply diff fail" errors
(see issue algorithmicsuperintelligence#346) where SEARCH patterns don't exactly match the
original code.

## Solution

Update `format_diff_summary()` to show actual content for multi-line
blocks:

    Change 1: Replace:
      def old_function():
          return False
    with:
      def new_function():
          return True

Single-line changes remain compact:

    Change 1: 'x = 1' to 'x = 2'

Add `_format_block_lines()` helper with configurable truncation limits.

## Configuration

New options in `prompt:` config section:

```yaml
prompt:
  diff_summary_max_line_len: 100  # Truncate lines longer than this
  diff_summary_max_lines: 30      # Max lines per SEARCH/REPLACE block
```

## Files Changed

- `openevolve/config.py` - Add PromptConfig options
- `openevolve/utils/code_utils.py` - Update format_diff_summary
- `openevolve/iteration.py` - Pass config to format_diff_summary
- `openevolve/process_parallel.py` - Pass config to format_diff_summary
- `tests/test_code_utils.py` - Add tests for new behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@CLAassistant
Copy link

CLAassistant commented Feb 3, 2026

CLA assistant check
All committers have signed the CLA.

@fangchenli fangchenli marked this pull request as ready for review February 3, 2026 21:52
@codelion codelion merged commit e13e7f6 into algorithmicsuperintelligence:main Feb 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants