Skip to content

Force index rebuild when a new repo is pulled #216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arjunsuresh merged 4 commits into from
Jan 14, 2026
Merged

Force index rebuild when a new repo is pulled #216

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9148f77
Update repo_action.py to force index rebuild when a new repo is pulled
sujik18 Jan 2, 2026
ae44d44
Update AI PR review workflow by adding fetch step for PR head
sujik18 Jan 4, 2026
067b815
Update index.py to incrementally add and remove index entries of a repo
sujik18 Jan 4, 2026
3e2cf7b
Update repo_action.py to use the index from the Action class
sujik18 Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 27 additions & 3 deletions mlc/repo_action.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
from . import utils
from .logger import logger
from urllib.parse import urlparse
from .repo import Repo
from .index import Index

class RepoAction(Action):
"""
Expand Down Expand Up @@ -164,9 +166,31 @@ def register_repo(self, repo_path, repo_meta, ignore_on_conflict=False):

with open(repos_file_path, 'w') as f:
json.dump(repos_list, f, indent=2)
logger.info(f"Updated repos.json at {repos_file_path}")



logger.debug("Forcing Index rebuild ...")
# reload repos list
with open(repos_file_path, 'r') as f:
repos_list = json.load(f)

repos = []
for p in repos_list:
meta = {}
yaml_file = os.path.join(p, "meta.yaml")
json_file = os.path.join(p, "meta.json")

if os.path.exists(yaml_file):
meta = utils.read_yaml(yaml_file)
elif os.path.exists(json_file):
meta = utils.read_json(json_file)
else:
logger.info(f"No meta file found in {self.repos_path}")
continue

repos.append(Repo(path=p, meta=meta))

# rebuild index via constructor
Index(self.repos_path, repos)
Copy link
Contributor

@arjunsuresh arjunsuresh Jan 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are building the index for all the repos here right? So, if a repo has a dependency we'll rebuild the index twice right? Is it possible to rebuild the index per repo? And we also need to fix the index on unregister_repo right?

Copy link
Member Author

@sujik18 sujik18 Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are building the index for all the repos here right? So, if a repo has a dependency we'll rebuild the index twice right? Is it possible to rebuild the index per repo?

I think that would require refactoring of index.py and all the places where Index class is being accessed, because currently index gets build whenever Index class is being accessed as build_index() is called in init function of Index class.

And we also need to fix the index on unregister_repo right?

If we intend to continue script execution after unregister_repo then yes we would need that, but I guess ideally that wont be the case and otherwise whenever the next time a mlc command is run which access Index class, index will be forcefully rebuilded as it will pickup that repo.json is modified

Copy link
Contributor

@arjunsuresh arjunsuresh Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sujik18 we can have a custom update function in the Index class to selectively update the entries for a given repo right?

The requirement for handling unregister_repo is when we do pull or a MLC repo which has a fork already registered in mlcflow. Then mlcflow automatically unregisters the fork and pull the new one. But here, the Index entries may not be consistent anymore.

Copy link
Member Author

@sujik18 sujik18 Jan 4, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requirement for handling unregister_repo is when we do pull or a MLC repo which has a fork already registered in mlcflow. Then mlcflow automatically unregisters the fork and pull the new one. But here, the Index entries may not be consistent anymore.

@arjunsuresh I have added a remove_repo_from_index function to remove index entries of a repo when the repo is deleted, currently called only in mlc rm command, if it looks good same function can be reused to update index entries while unregistering a repo.

logger.debug("repos.json and index file has been updated")
return {'return': 0}

def unregister_repo(self, repo_path):
Expand Down
Loading