Skip to content

Conversation

@Jon-Becker
Copy link
Owner

What changed? Why?

Notes to reviewers

How has it been tested?

@github-actions
Copy link
Contributor

❌ Eval Report for 5928291

Test Case CFG Decompilation
SimpleLoop 100 15
TransientStorage 100 15
WhileLoop 45 15
WETH9 100 45
NestedMappings 100 25
SimpleStorage 100 35
Events 100 55
Mapping 100 35
NestedLoop 35 15
NestedMapping 100 0
Average 88 25
⚠️ 10 eval(s) scoring <70%

SimpleLoop (CFG: 100, Decompilation: 15)

Decompilation

{
  "score": 15,
  "summary": "Decompilation fails to capture the fundamental loop behavior and state-changing operations. The function is incorrectly marked as 'view' instead of state-modifying, the loop structure is completely missing, and the storage write operations (number++) are not represented.",
  "differences": [
    "Function mutability is incorrect: marked as 'view' instead of non-view/state-modifying",
    "Loop control flow structure is completely missing - no for-loop or equivalent iteration logic",
    "Storage write operation (number++ inside loop) is not preserved",
    "Contains nonsensical require statements that don't match original logic: 'require(arg0 == arg0)' is always true, 'require(!0 < arg0)' appears to be inverted loop condition",
    "The increment operation repeated 'loops' times is not represented at all",
    "The decompiled code would not modify state when called, while the original increments 'number' by 'loops' amount"
  ]
}

TransientStorage (CFG: 100, Decompilation: 15)

Decompilation

{
  "score": 15,
  "summary": "Severe decompilation failure. Only 1 of 6 functions decompiled, and that function has incorrect logic. Five functions are completely missing. The setTempOwner function uses incorrect bitwise operations instead of simple storage assignment. Transient storage access is malformed.",
  "differences": [
    "Missing incrementCounter() function - counter increment logic not present",
    "Missing lock() function - cannot set locked flag to true",
    "Missing unlock() function - cannot set locked flag to false",
    "Missing getCounter() view function - cannot read counter value",
    "Missing isLocked() view function - cannot read locked state",
    "setTempOwner() uses incorrect bitwise operations (address multiplication and OR with uint96 cast) instead of simple transient storage assignment",
    "Transient storage accessed via array-like transient[0x01] notation with bit manipulation instead of proper variable assignment",
    "Functions incorrectly declared as public constants with empty/invalid values instead of being recognized as callable functions"
  ]
}

WhileLoop (CFG: 45, Decompilation: 15)

Decompilation

{
  "score": 15,
  "summary": "Decompilation fails to capture the core while loop functionality. The loop structure, state modifications, and iteration logic are completely missing. Function is incorrectly marked as view instead of state-modifying.",
  "differences": [
    "While loop control structure is not preserved - replaced with static require statements",
    "State variable increment operation (number = number + 1) is missing",
    "Loop counter initialization and increment logic (i = 0; i = i + 1) is not present",
    "Function incorrectly marked as 'view' when it should modify state",
    "The iterative behavior that runs 'loops' times is not captured",
    "No actual computation or state changes occur in decompiled version"
  ]
}

CFG

{
  "score": 45,
  "summary": "CFG captures function dispatch, loop entry and exit, but missing the critical loop back-edge for iteration",
  "missing_paths": [
    "Loop iteration back-edge: After executing the loop body (nodes 8-9), control should jump back to the loop condition check (node 7) to continue iteration. This back-edge is completely missing from the CFG.",
    "The while loop structure requires: entry -> condition check -> body -> back to condition. The CFG shows entry and exit but not the cyclic iteration path."
  ],
  "extra_paths": [
    "Callvalue check in constructor (node 0->1)",
    "Calldatasize validation (node 2->3->14)",
    "Function selector dispatch logic (nodes 3, 12)",
    "Parameter validation for loop function (nodes 4->5->6)",
    "Overflow check for addition operations (node 8->9)",
    "Various stack management and validation paths throughout"
  ],
  "observations": [
    "The CFG correctly identifies the loop entry point at node 7 (JUMPDEST at 0x77) with the loop condition check (LT, ISZERO, JUMPI)",
    "The loop body is represented in node 8 with SLOAD, ADD operations for 'number = number + 1' and 'i = i + 1'",
    "The loop exit path (node 7->10) is correctly shown for when condition 'i < loops' becomes false",
    "Critical missing element: Node 9 has no outgoing edges, meaning the CFG doesn't show how control returns to node 7 for the next iteration",
    "Without the back-edge, the CFG represents the loop as executing at most once, fundamentally misrepresenting the program's control flow",
    "The function selector routing correctly identifies both public functions: loop() and the number getter"
  ]
}

WETH9 (CFG: 100, Decompilation: 45)

Decompilation

{
  "score": 45,
  "summary": "Decompilation captures basic structure but contains critical logic errors in transfer/transferFrom functions and incorrect allowance/balanceOf storage mappings. Core WETH functionality (deposit/withdraw) is mostly preserved but transfer logic is fundamentally broken.",
  "differences": [
    "transfer() function has corrupted control flow with unreachable code blocks and incorrect condition logic (uses '!storage_map_c[var_a] < arg1' instead of proper >= check, and checks msg.sender == msg.sender tautology instead of src != msg.sender)",
    "transfer() should call transferFrom(msg.sender, dst, wad) but decompiled version implements inline logic incorrectly",
    "transferFrom() has same corrupted control flow issues: unreachable code blocks, incorrect conditions, and duplicate balance deductions (deducts from msg.sender then from arg0, instead of checking allowance correctly)",
    "transferFrom() allowance check logic is completely wrong: checks 'storage_map_c[var_a] == max_uint256' instead of properly reading from allowance mapping and checking/updating allowance[src][msg.sender]",
    "approve() writes to wrong storage mapping: uses 'storage_map_c[var_a]' instead of nested mapping 'storage_map_d[msg.sender][arg0]'",
    "allowance() returns from wrong mapping structure: uses single-level 'storage_map_d[var_b]' instead of nested 'storage_map_d[arg0][arg1]'",
    "balanceOf() returns from wrong mapping: uses 'storage_map_d[var_b]' instead of 'storage_map_c[arg0]'",
    "withdraw() uses '.transfer()' return value capture '(bool success, bytes memory ret0)' which is incorrect - transfer() doesn't return values, it reverts on failure",
    "Multiple unreachable code paths in transfer() and transferFrom() after 'return 0x01' statements suggest failed control flow reconstruction"
  ]
}

NestedMappings (CFG: 100, Decompilation: 25)

Decompilation

{
  "score": 25,
  "summary": "Decompilation fails to capture nested mapping storage logic and view function return values. Critical functional differences prevent correct program behavior.",
  "differences": [
    "The approve function does not compute the correct storage slot for nested mapping allowances[msg.sender][spender] - uses simple single-key storage instead of properly hashed nested mapping key",
    "The public allowances mapping getter (0xdd62ed3e) accepts only one address parameter instead of two and does not return the uint256 value from storage",
    "The allowance view function (0x55b6ed5c) accepts only one address parameter instead of two and does not return the uint256 value from storage",
    "Both view functions are missing storage read operations - they only validate address parameters without accessing the mapping data"
  ]
}

SimpleStorage (CFG: 100, Decompilation: 35)

Decompilation

{
  "score": 35,
  "summary": "Decompilation fails to correctly handle storage layout with packed variables. The boolean 'initialized' field is misidentified as part of the 'owner' address field, causing incorrect state mutations. Only setValue() preserves correct logic.",
  "differences": [
    "setOwner() uses bitwise OR with uint96(owner) which preserves lower 96 bits instead of completely replacing the owner address",
    "initialize() modifies the owner variable with a bitwise operation instead of setting a separate boolean 'initialized' field to true",
    "reset() fails to properly zero the owner address (preserves lower 96 bits) and incorrectly attempts to clear initialized boolean through owner manipulation",
    "Missing proper handling of the 'initialized' boolean state variable - it's never properly represented as a separate field"
  ]
}

Events (CFG: 100, Decompilation: 55)

Decompilation

{
  "score": 55,
  "summary": "Core event emission logic is preserved for 5 of 7 functions, but two critical functions (emitTransfer and emitApproval) are completely missing from the decompilation. The preserved functions contain the correct event emissions, though they include spurious require statements and incorrect mutability specifiers.",
  "differences": [
    "emitTransfer function is completely missing - fails to capture Transfer event emission with three parameters",
    "emitApproval function is completely missing - fails to capture Approval event emission",
    "All decompiled functions incorrectly marked as 'pure' when they should allow state modifications (event emissions)",
    "Spurious require statements added that don't exist in original code (e.g., 'require(arg0 == (address(arg0)))')",
    "Two unresolved functions (0x5687f2b8, 0x23de6651) present that don't correspond to any original contract functions"
  ]
}

Mapping (CFG: 100, Decompilation: 35)

Decompilation

{
  "score": 35,
  "summary": "Decompilation fails to preserve functional logic due to incorrect storage mapping and unnecessary bitwise operations. Storage reads and writes are inconsistent, making the decompiled contract non-functional.",
  "differences": [
    "setBalance writes to storage_map_a but balances() reads from storage_map_b, breaking the getter-setter relationship",
    "setOwner uses bitwise packing operations instead of simple address assignment to storage",
    "register uses bitwise packing operations instead of simple boolean assignment",
    "owners() getter reads from storage_map_b with division operation, but setOwner writes to storage_map_a with packing",
    "registered() getter reads from storage_map_b with complex bit manipulation, inconsistent with register() which writes to storage_map_a",
    "getBalance reads from storage_map_a (correct pairing with setBalance), but this contradicts the balances() public getter which reads from storage_map_b",
    "Storage slot mapping is fundamentally broken - three distinct mappings in source are collapsed into two storage maps with inconsistent read/write patterns"
  ]
}

NestedLoop (CFG: 35, Decompilation: 15)

Decompilation

{
  "score": 15,
  "summary": "Decompilation fails to capture the fundamental nested loop structure and state-modifying behavior. The loop logic is completely missing, replaced with incorrect static require statements. The function mutability is wrong (view instead of state-changing).",
  "differences": [
    "Nested loop structure completely missing - no iteration logic preserved",
    "State modification (number += 1) not captured - function incorrectly marked as 'view' instead of state-changing",
    "Loop iteration count calculation missing - should increment number by loops^2 total times",
    "Incorrect require statements that don't represent the original loop conditions",
    "No storage write operations present despite original function modifying storage variable 'number'",
    "Function behavior fundamentally different: original performs loops^2 increments, decompiled performs static checks and reverts"
  ]
}

CFG

{
  "score": 35,
  "summary": "CFG fails to capture loop iteration structure - missing all back edges for both nested loops, making it appear loops execute at most once instead of iteratively",
  "missing_paths": [
    "Inner loop back edge: from loop body (node 9) back to inner loop condition check (node 8) after j increment",
    "Inner loop exit path: from inner loop condition (node 8 at 0xb1) to outer loop increment code",
    "Outer loop back edge: from outer loop increment back to outer loop condition check (node 7)",
    "Complete path for outer loop increment: when inner loop completes, i should be incremented and flow back to outer condition"
  ],
  "extra_paths": [
    "Overflow check path (node 10) - compiler-added arithmetic safety check",
    "Function selector dispatch logic (nodes 0-3, 13-15)",
    "Calldata decoding and validation (nodes 4-6, 12)",
    "Constructor payable check (node 1)"
  ],
  "observations": [
    "The CFG correctly identifies loop entry points and condition checks",
    "Loop body operation (number += 1) is captured in node 9",
    "Both loop exit conditions are present (node 7->11 for outer, node 8->? for inner)",
    "Critical failure: no back edges present for either loop, making the CFG acyclic where it should be cyclic",
    "Without back edges, the CFG suggests linear execution rather than iterative loop behavior",
    "This is a fundamental structural deficiency - the most important characteristic of loops (repetition) is absent",
    "The nested structure is somewhat visible (node 7 contains node 8) but iteration paths are completely missing"
  ]
}

NestedMapping (CFG: 100, Decompilation: 0)

Decompilation

{
  "score": 0,
  "summary": "Complete decompilation failure - no functional logic preserved. All storage operations, state changes, and return values are missing. Functions contain only meaningless tautological require statements instead of actual logic.",
  "differences": [
    "All nested mapping storage write operations are completely missing (setAllowance, setGrid, setDeepNested have no storage writes)",
    "All nested mapping storage read operations are completely missing (getAllowance returns nothing, public mapping getters have no read logic)",
    "Function mutability is incorrect - all functions marked as 'pure' when they should be state-changing (3 functions) or 'view' (4 functions)",
    "Functions contain only tautological require statements (e.g., 'require(arg0 == arg0)') with no actual business logic",
    "No return values present - getAllowance should return uint256 from storage but has no return statement",
    "Control flow completely absent - no conditional logic, loops, or branches from original preserved",
    "State variable access patterns for nested mappings not captured at all"
  ]
}

@github-actions
Copy link
Contributor

✅ Coverage Report for 5928291

Metric Value
Base branch 66.57%
PR branch 67.35%
Diff +0.78%

@github-actions
Copy link
Contributor

Benchmark for 5928291

Click to view benchmark
Test Base PR %
heimdall_cfg/complex 9.6±0.03ms 9.5±0.09ms -1.04%
heimdall_cfg/simple 978.7±6.63µs 994.3±4.70µs +1.59%
heimdall_decoder/seaport 39.7±2.37µs 40.7±2.65µs +2.52%
heimdall_decoder/transfer 2.9±0.22µs 3.0±0.27µs +3.45%
heimdall_decoder/uniswap 11.0±0.44µs 11.2±0.44µs +1.82%
heimdall_decompiler/abi_complex 39.8±0.50ms 39.4±1.44ms -1.01%
heimdall_decompiler/abi_simple 1018.2±11.57µs 1033.6±6.89µs +1.51%
heimdall_decompiler/sol_complex 55.5±6.57ms 55.2±0.72ms -0.54%
heimdall_decompiler/sol_simple 1597.5±16.13µs 1536.8±35.64µs -3.80%
heimdall_decompiler/yul_complex 43.7±0.94ms 43.2±1.10ms -1.14%
heimdall_decompiler/yul_simple 1188.2±19.10µs 1123.6±13.28µs -5.44%
heimdall_disassembler/complex 916.5±11.46µs 928.7±42.58µs +1.33%
heimdall_disassembler/simple 47.2±2.61µs 46.8±3.07µs -0.85%
heimdall_vm/erc20_transfer 182.8±5.31µs 180.4±8.71µs -1.31%
heimdall_vm/fib 607.6±5.18µs 624.5±6.33µs +2.78%
heimdall_vm/ten_thousand_hashes 4.5±1.02s 4.5±0.82s 0.00%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant