Skip to content

stats: add defensive guard for interval calculation#65889

Merged
ti-chi-bot[bot] merged 1 commit intopingcap:masterfrom
0xPoe:poe-patch-defensive
Feb 3, 2026
Merged

stats: add defensive guard for interval calculation#65889
ti-chi-bot[bot] merged 1 commit intopingcap:masterfrom
0xPoe:poe-patch-defensive

Conversation

@0xPoe
Copy link
Member

@0xPoe 0xPoe commented Jan 28, 2026

What problem does this PR solve?

Issue Number: close #65815

Problem Summary:
Negative durations from mysql.analyze_jobs (e.g. clock skew/bad records) can happen and should be handled defensively without skewing retry logic.

What changed and how does it work?

Based on #65821.

  • Document why negative average durations return NoRecord.
  • Add unit tests for negative finished-job durations.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. component/statistics sig/planner SIG: Planner and removed do-not-merge/needs-linked-issue labels Jan 28, 2026
@codecov
Copy link

codecov bot commented Jan 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.6447%. Comparing base (258332a) to head (1365b14).
⚠️ Report is 22 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #65889        +/-   ##
================================================
+ Coverage   77.7820%   78.6447%   +0.8626%     
================================================
  Files          2001       1932        -69     
  Lines        545618     537652      -7966     
================================================
- Hits         424393     422835      -1558     
+ Misses       119563     114372      -5191     
+ Partials       1662        445      -1217     
Flag Coverage Δ
integration 44.6222% <0.0000%> (-3.5377%) ⬇️
unit 77.0866% <100.0000%> (+0.6672%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 56.7974% <ø> (ø)
parser ∅ <ø> (∅)
br 48.8835% <ø> (-12.0906%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@0xPoe 0xPoe force-pushed the poe-patch-defensive branch from 86fc05c to 2b18cf5 Compare January 30, 2026 17:07
@ti-chi-bot ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 30, 2026
@0xPoe 0xPoe changed the title WIP: stats: add defensive guard for interval calculation stats: add defensive guard for interval calculation Jan 30, 2026
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 30, 2026
refactor: rename

docs: add comments

test: add more cases
@0xPoe 0xPoe force-pushed the poe-patch-defensive branch from 5e8397b to 1365b14 Compare January 30, 2026 17:46
Copy link
Member Author

@0xPoe 0xPoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback)

  • Code compiles successfully

  • Unit tests added

  • All tests pass

  • Bazel files updated

  • Comments added where necessary

  • PR title and description updated

  • Documentation PR created (or confirmed not needed)

  • PR size is reasonable

/cc @terry1purcell @henrybw

@ti-chi-bot ti-chi-bot bot requested review from henrybw and terry1purcell January 30, 2026 17:53
@0xPoe 0xPoe requested a review from Copilot January 30, 2026 17:53
@0xPoe 0xPoe added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Jan 30, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the auto-analyze priority queue’s interval calculations against bad or skewed mysql.analyze_jobs duration records, preventing negative durations from corrupting retry/skip logic. It also clarifies the NoRecord sentinel semantics and adds regression tests around negative durations.

Changes:

  • Update GetAverageAnalysisDuration and GetLastFailedAnalysisDuration to treat negative durations as non-usable (either NoRecord or a bounded default wait time) instead of flowing them into scheduling logic.
  • Switch last-failed duration reading to a signed integer and document that both functions return NoRecord when no usable history exists.
  • Add unit tests that insert synthetic negative-duration records to validate the new defensive behavior for both average and last-failed analysis durations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pkg/statistics/handle/autoanalyze/priorityqueue/interval.go Adds defensive guards for negative average/failed analysis durations, uses signed integer for TIMESTAMPDIFF results, and updates comments to state that NoRecord is returned when there is no usable history.
pkg/statistics/handle/autoanalyze/priorityqueue/interval_test.go Extends tests to cover negative finished and failed job durations, ensuring they map to NoRecord or the default failed-analysis wait time instead of influencing skip logic.

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Feb 2, 2026
@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 2, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: henrybw, terry1purcell

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Feb 2, 2026
@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 2, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-02-02 17:14:27.915241804 +0000 UTC m=+114339.016640523: ☑️ agreed by terry1purcell.
  • 2026-02-02 18:51:18.657019501 +0000 UTC m=+120149.758418220: ☑️ agreed by henrybw.

@0xPoe
Copy link
Member Author

0xPoe commented Feb 2, 2026

/retest

@0xPoe
Copy link
Member Author

0xPoe commented Feb 3, 2026

/test all

1 similar comment
@0xPoe
Copy link
Member Author

0xPoe commented Feb 3, 2026

/test all

@ti-chi-bot ti-chi-bot bot merged commit 995d6f9 into pingcap:master Feb 3, 2026
36 of 37 checks passed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #66009.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved component/statistics lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto analyze skip due to negative lastFailedAnalysisDuration (-7h55m)

4 participants