Skip to content

Display ok rate#89

Open
lolpack wants to merge 2 commits intomainfrom
display_ok_rate
Open

Display ok rate#89
lolpack wants to merge 2 commits intomainfrom
display_ok_rate

Conversation

@lolpack
Copy link
Owner

@lolpack lolpack commented Jan 2, 2026

No description provided.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new "ok_rate" metric to the LSP benchmark system to distinguish between request reliability (whether requests complete without timeout/error) and accuracy (whether requests return valid definition locations). The ok_rate metric provides visibility into timeout/error rates separately from the existing success_rate metric.

Key Changes

  • Added ok_rate calculation and tracking to aggregate statistics alongside success_rate
  • Created a backfill script to retroactively add ok_rate to existing benchmark JSON files
  • Updated the web dashboard to display ok_rate with a new chart and table column

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/test_lsp_benchmark_daily_runner.py Added comprehensive test cases validating ok_rate calculation for various scenarios including timeouts, multiple checkers, and edge cases
tests/test_backfill_ok_rate.py New test suite for the backfill script covering calculation logic, file updates, and error handling
lsp/benchmark/daily_runner.py Updated AggregateStats TypedDict and compute_aggregate_stats function to calculate and include ok_rate; added exception handling to _benchmark_single_package
lsp/benchmark/backfill_ok_rate.py New utility script to backfill ok_rate into existing benchmark JSON files by recalculating from stored results
lsp/benchmark/scripts/lsp-benchmark.js Added createOkChart function and integrated ok_rate display into table with new OK % column
lsp/benchmark/index.html Added OK Rate chart section and updated table headers to include OK % column; updated metrics documentation
lsp/benchmark/results/*.json Backfilled ok_rate values into historical benchmark data files (latest.json and dated benchmark files)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant