reformat.sh randomly hangs on certain samples when running multiple jobs in parallel

[ypchan]

Dear Authors,

When running multiple instances of reformat.sh (BBMap's ReformatReads) in parallel, the program intermittently hangs on random FASTQ samples. The issue occurs without any error messages or warnings — the process simply stalls and produces no further output, even i had been waiting for several hours. This happens with different samples on different runs, so it does not appear to be data-specific.

version information:

$reformat.sh --version
java -ea   -Xmx5000m -Xms300m -cp /user-storage/data/a902366200/software/bbmap/current/ jgi.ReformatReads --version
BBTools version 39.55

batch content:

reformat.sh in=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA226001_se_illumina/00_fq/SRR1544623.fastq.gz.fastq.gz  out=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA226001_se_illumina/00_fq/SRR1544623.fastq.gz.1m.fastq.gz reads=1000000 sampleseed=33
reformat.sh in=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA219042_se_illumina/00_fq/SRR981224.fastq.gz.fastq.gz  out=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA219042_se_illumina/00_fq/SRR981224.fastq.gz.1m.fastq.gz reads=1000000 sampleseed=33
reformat.sh in=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA1182308_se_illumina/00_fq/SRR31230451.fastq.gz.fastq.gz  out=/user-storage/home/tm902982941970000/a902366200/bioproject/02_amplicon/unfinished/PRJNA1182308_se_illumina/00_fq/SRR31230451.fastq.gz.1m.fastq.gz reads=1000000 sampleseed=33
...

codes:

cat reformat_se.list | rush -j 4 -c -C reformat_se.list.finished --verbose --eta 'eval {}'

Hangs on here

I will appreciate it if you can help fixing this.

Kind regards,
Yanpeng