experiment: hammer/premise selection benchmarks#106
Open
kim-em wants to merge 3 commits intonightly-testing-greenfrom
Open
experiment: hammer/premise selection benchmarks#106kim-em wants to merge 3 commits intonightly-testing-greenfrom
kim-em wants to merge 3 commits intonightly-testing-greenfrom
Conversation
kim-em
force-pushed
the
hammer_measurements
branch
3 times, most recently
from
5ae212a to
b1793a3
Compare
kim-em
force-pushed
the
hammer_measurements
branch
from
2012f87 to
e0a6b41
Compare
kim-em
force-pushed
the
hammer_measurements
branch
from
393dc77 to
3f56141
Compare
kim-em
force-pushed
the
hammer_measurements
branch
from
a02a589 to
06ff22e
Compare
Add elapsed time measurement to the "can be replaced with" messages generated by tryAtEachStep linters. Messages now include timing in milliseconds, e.g.: `simp` can be replaced with `grind` (45ms) This enables timing analysis of hammer tactics without external instrumentation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add infrastructure for testing arbitrary tactics without modifying Mathlib: - `parseTacticString`: Parse a string into tactic syntax at runtime - `tryAtEachStepFromStrings`: Generic entry point taking label and tactic strings - `tryAtEachStepFromEnvImpl`: Reads TRY_AT_EACH_STEP_TACTIC from env var (TRY_AT_EACH_STEP_LABEL is optional, defaults to tactic string) Example usage: ``` TRY_AT_EACH_STEP_TACTIC="omega" \ lake build Mathlib -Klinter.tacticAnalysis.tryAtEachStepFromEnv=true ``` This generic entry point is used by the hammer-bench benchmarking tool (https://github.com/leanprover-community/hammer-bench) to test arbitrary tactics without requiring Mathlib code changes for each new tactic variant. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.