Skip to content

Conversation

@prototaxites
Copy link
Contributor

Running nfcoreInstall on a bunch of modules takes time. If you are caching the same library between multiple tests, this means calling nf-core install for modules that are already installed. Instead, now we write a state file for each module after a successful install (hashed from name, git commit SHA and remote), and check if it exists before trying an install later.

Not sure how to test this though as I don't think I can access the nf-test stdout in its own test...

@prototaxites prototaxites changed the title fix(nfcoreInstall): use statefiles to track installs in a library feat(nfcoreInstall): use statefiles to track installs in a library Feb 10, 2026
@muffato
Copy link
Member

muffato commented Feb 10, 2026

Essentially, I'd do:

  1. fake a statefile for a given module
  2. check that nfcoreInstall doesn't reinstall that module

To fake the statefile:

  1. Craft one manually
  2. Or install a module, remove all its files but leaving the statefile intact

To check nfcoreInstall:

  1. Run the process and check no file was created
  2. Or (if possible), modify and empty $PATH and check no sub-process exception is raised.

@muffato
Copy link
Member

muffato commented Feb 10, 2026

Also, what is it you're trying to optimise ? The time it takes for nf-core modules install to load the module list, lookup in modules.json and decide there's nothing to do ? Is it that slow ?

@prototaxites
Copy link
Contributor Author

Also, what is it you're trying to optimise ? The time it takes for nf-core modules install to load the module list, lookup in modules.json and decide there's nothing to do ? Is it that slow ?

It was running very slowly for me on the farm, for reasons I'm not sure why. It does have to query github each time, I think, which is not so ideal in any case?

@prototaxites
Copy link
Contributor Author

Added a test, using the install -> delete -> reinstall -> check for files method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants