[Task] Re‑validate hand evaluation using Tenhou phoenix logs

[Apricot-S]

Summary

The README currently states:

The code was validated on tenhou.net phoenix replays in total on 11,120,125 hands.

Since v2 introduces a new block‑decomposition algorithm, this validation is no longer valid.
A full re‑validation must be performed using Tenhou phoenix logs.

Scope

• Obtain Tenhou phoenix .db log files
  Logs are available at: https://github.com/NikkeTryHard/tenhou-to-mjai/releases
• Export Tenhou mjlog format (.xml) from the .db files
  Export tool: https://github.com/Apricot-S/houou-logs
• Run the v2 hand evaluation pipeline on all available logs
• Compare results with expected outcomes to confirm correctness
• Update the README with new validation statistics

[Apricot-S]

@Nihisil

Since the block‑decomposition algorithm has been updated in v2, the validation result currently shown in the README:

The code was validated on tenhou.net phoenix replays in total on 11,120,125 hands.

is no longer applicable.

Because this repository does not include the validation tools,
I would like to kindly ask you to run the re‑validation for v2.

Timing

Please perform this after all issues in the v2.0.0 milestone except the documentation tasks (Wiki, API docs) have been closed.

The data source and export tool are described in the issue description above.

Thank you in advance for your help.

[Apricot-S]

Just to confirm: in the validation using Tenhou logs, were you checking the cost, han, fu, and yaku fields of the HandResponse returned by HandCalculator.estimate_hand_value()?

[Nihisil]

I'm comparing han, fu and yaku. And will add cost here as well just to be safe

The current code:

def compare_result(hand: ParsedHand, calc_result: CalculationResult) -> ComparisonResult:
    """
    Compare calculated result with expected values.
    """
    if not calc_result.success:
        return ComparisonResult(status="ERROR", error=calc_result.error)

    fu_match = calc_result.fu == hand.expected_fu
    is_yakuman = calc_result.han >= 13

    calculated_yaku_ids = _get_calculated_yaku_ids(calc_result.yaku)
    expected_yaku_ids = _normalize_expected_yaku_ids(hand.expected_yaku)

    missing_yaku = expected_yaku_ids - calculated_yaku_ids
    extra_yaku = calculated_yaku_ids - expected_yaku_ids

    # fu doesn't matter for yakuman
    if not missing_yaku and not extra_yaku and (fu_match or is_yakuman):
        return ComparisonResult(status="PASS")

    return ComparisonResult(
        status="MISMATCH",
        fu_match=fu_match or is_yakuman,
        missing_yaku=missing_yaku,
        extra_yaku=extra_yaku,
    )

[Nihisil]

Where is calc_result is:

def calculate_hand(calculator: HandCalculator, hand: ParsedHand) -> CalculationResult:
    """
    Calculate hand value using the mahjong library.
    This function is isolated for performance measurement.
    """
    try:
        result = calculator.estimate_hand_value(
            tiles=hand.tiles,
            win_tile=hand.win_tile,
            melds=hand.melds if hand.melds else None,
            dora_indicators=hand.dora_indicators if hand.dora_indicators else None,
            config=hand.config,
        )

        if result.error:
            return CalculationResult(success=False, error=result.error)

        return CalculationResult(
            success=True,
            fu=result.fu,
            han=result.han,
            cost=result.cost,
            yaku=result.yaku,
        )
    except Exception as e:
        return CalculationResult(success=False, error=str(e))