about the memory

[Wanli-HE]

Hi there,

really thanks for the tool, which is really helpful.

and I am trying to use ct3 for 45Gb phages contigs, but it raises error about out of the memory, i am try to split into 'smaller part', each of which about 770Mb, but still showed out of memory, so do you have any idea how to estimate the running memory?

best,
wanli

[Wanli-HE]

my machine is running cpu is 250GB

[mtisza1]

Hi @Wanli-HE

Sorry for the slow reply as I've been otherwise engaged. I'm looking into this now.
Other than input .fasta file size, the key parameter that effects peak memory is number of CPUs. Please try your 770Mb inputs again with only one CPU (-t 1). This should drastically reduce memory footprint and not affect the wall time too much.

I will follow up with a more thorough benchmark as time allows.

Best,

Mike

[Wanli-HE]

Hi @Wanli-HE

Sorry for the slow reply as I've been otherwise engaged. I'm looking into this now. Other than input .fasta file size, the key parameter that effects peak memory is number of CPUs. Please try your 770Mb inputs again with only one CPU (-t 1). This should drastically reduce memory footprint and not affect the wall time too much.

I will follow up with a more thorough benchmark as time allows.

Best,

Mike

hi mike,

i am not using the tool but the 5 hmm databases, and write script by myself, and i have another questions, is the mmseqDB and HmmDB same?

[mtisza1]

@Wanli-HE,

Some database elements are for functional gene annotation and some are for taxonomic assignment.

functional gene annotation (all HMM databases):

|____hmmscan_DBs
| |____v3.1.1
| | |____DNA_rep_HMMs.h3m
| | |____Useful_Annotation_HMMs.h3m
| | |____phrogs_for_ct.h3m
| | |____Virion_HMMs.h3m
| | |____RDRP_HMMs.h3m

functional gene annotation (CDD formatted for mmseqs):

|____mmseqs_DBs
| |____CDD
| |____CDD.index
| |____CDD.lookup
| |____cddid_all.tbl
| |____CDD_h.dbtype
| |____CDD_h
| |____CDD.dbtype
| |____CDD.source
| |____CDD.version
| |____CDD_h.index

taxonomy assignment (AAs formatted for mmseqs):

|____mmseqs_DBs
| |____ct3_hallmark.taxDB_h.dbtype
| |____ct3_hallmark.taxDB.source
| |____ct3_hallmark.taxDB.dbtype
| |____ct3_hallmark.taxDB_h
| |____ct3_hallmark.taxDB_mapping
| |____ct3_hallmark.taxDB.lookup
| |____ct3_hallmark.taxDB.index
| |____ct3_hallmark.taxDB
| |____ct3_hallmark.taxDB_h.index
| |____ct3_hallmark_nr_cd90_refseq.prot_taxids.mmseqs_fmt.tsv
| |____ct3_hallmark_nr_cd90_refseq.faa
| |____ct3_hallmark.taxDB_taxonomy

Let me know if this helps,

Mike

[Wanli-HE]

@Wanli-HE,

Some database elements are for functional gene annotation and some are for taxonomic assignment.

functional gene annotation (all HMM databases):

|____hmmscan_DBs
| |____v3.1.1
| | |____DNA_rep_HMMs.h3m
| | |____Useful_Annotation_HMMs.h3m
| | |____phrogs_for_ct.h3m
| | |____Virion_HMMs.h3m
| | |____RDRP_HMMs.h3m

functional gene annotation (CDD formatted for mmseqs):

|____mmseqs_DBs
| |____CDD
| |____CDD.index
| |____CDD.lookup
| |____cddid_all.tbl
| |____CDD_h.dbtype
| |____CDD_h
| |____CDD.dbtype
| |____CDD.source
| |____CDD.version
| |____CDD_h.index

taxonomy assignment (AAs formatted for mmseqs):

|____mmseqs_DBs
| |____ct3_hallmark.taxDB_h.dbtype
| |____ct3_hallmark.taxDB.source
| |____ct3_hallmark.taxDB.dbtype
| |____ct3_hallmark.taxDB_h
| |____ct3_hallmark.taxDB_mapping
| |____ct3_hallmark.taxDB.lookup
| |____ct3_hallmark.taxDB.index
| |____ct3_hallmark.taxDB
| |____ct3_hallmark.taxDB_h.index
| |____ct3_hallmark_nr_cd90_refseq.prot_taxids.mmseqs_fmt.tsv
| |____ct3_hallmark_nr_cd90_refseq.faa
| |____ct3_hallmark.taxDB_taxonomy

Let me know if this helps,

Mike

Hi Mike!

thanks for your answering, my case is i just want to see whether my contigs have the virus hallmarking genes to make should these contig are belong to virus, so which database i should use? what y metioned functional genes are they virus specific genes?

best,
wanli

[mtisza1]

Sure, so Virion_HMMs.h3m, RDRP_HMMs.h3m, and/or DNA_rep_HMMs.h3m can be used depending on your goals.

My advice: for most cases, use Virion_HMMs.h3m and RDRP_HMMs.h3m. Including dsDNA sequencing libraries and RNA sequencing libraries. For ssDNA virus discovery from MDA-amplified samples, use Virion_HMMs.h3m and DNA_rep_HMMs.h3m.

The -db flag has info on this as well:

  -db {virion,rdrp,dnarep} [{virion,rdrp,dnarep} ...], --virus_domain_db {virion,rdrp,dnarep} [{virion,rdrp,dnarep} ...]
                        default: virion rdrp -- Hits to which domain types should count as hallmark genes? 'virion' database: genes encoding virion
                        structural proteins, packaging proteins, or capsid maturation proteins (DNA and RNA genomes) with LOWEST false discovery rate.
                        'rdrp' database: For RNA virus-derived RNA-dependent RNA polymerase. 'dnarep' database: replication genes of DNA viruses. mostly
                        useful for small DNA viruses, e.g. CRESS viruses