-
Notifications
You must be signed in to change notification settings - Fork 862
fix: limit depth of index discovery to prevent slow scans #196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix: limit depth of index discovery to prevent slow scans #196
Conversation
Fixes yichuan-w#122 The `leann list` command was scanning entire directory trees using `rglob()`, causing extremely slow performance when run in large directories like $HOME. Changes: - Add `_find_meta_files_limited()` method with max_depth parameter - Skip common large directories (node_modules, .venv, .git, etc.) - Apply limited search in `_discover_indexes_in_project()` and `_find_all_matching_indexes()` - Add `_has_app_indexes_limited()` in registry.py for faster checks - Add comprehensive tests for the new functionality Signed-off-by: majiayu000 <1835304752@qq.com>
|
Thanks for working on this! A few suggestions: 1. Add
|
Address reviewer feedback by making the directory scan depth configurable instead of hardcoding it to 3. Users with deeply nested project structures can now increase the depth limit as needed. - Add --max-depth argument to list command (default: 3) - Update list_indexes() and _discover_indexes_in_project() to accept max_depth - Add tests for the new CLI option and custom depth behavior 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Thanks for the feedback @andylizf! I've added the # Default depth (3)
leann list
# Scan deeper directories
leann list --max-depth 5For the global registry suggestion (making |
a8e6f17 to
ba91644
Compare
|
Update: I've also implemented the long-term global registry solution in a separate PR #199.
This PR can be merged independently. PR #199 builds on top of this one. |
, yichuan-w#196, yichuan-w#199 - Fix unused variable `zmq_port` in warmup() method (F841) - Remove unused imports in test_cli_list_performance.py (F401) - Sort imports in test files (I001) - Fix test_warmup.py: correct attribute name `_warmup_enabled` → `_warmup` - Add skipif for DiskANN test when backend is not installed - Apply ruff format to all modified files Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Summary
leann listscans all files in large directories like $HOME_find_meta_files_limited()method with configurable max_depth (default: 3)Fixes #122
Test plan
tests/test_cli_list_performance.py