Memory use

[sanniruo]

Hi! What have been your memory settings for running trajgwas for larger data sets? I tried to run it with sample size of ~60k and ~1.5M measurements, and for ~30k variants chunks, but even 500G of memory gives me Killed. Thanks in advance!

[Hua-Zhou]

Are there ~1.5M measurements per sample or ~1.5M measurements in total?

…

On Tue, May 28, 2024 at 7:12 AM sanniruo ***@***.***> wrote: Hi! What have been your memory settings for running trajgwas for larger data sets? I tried to run it with sample size of ~60k and ~1.5M measurements, and for ~30k variants chunks, but even 500G of memory gives me Killed. Thanks in advance! — Reply to this email directly, view it on GitHub <#43>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGAPMNHZZRAI7KXUF4GDS3ZESGEBAVCNFSM6AAAAABINA4OTWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDCMJYHAYDOMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[sanniruo]

~1.5M measurements in total

[sanniruo]

An update on this: the issue seems to be in covariates rather than the file size. When trying with only one binary covariate (sex) there was no problem even with a bit smaller memory, but with one quantitative covariate it got 'killed'...

[Hua-Zhou]

The issue sounds very data-specific. Can you generate a minimal working example (MWE) that reproduces the issue?

…

On Mon, Jun 10, 2024 at 12:37 AM sanniruo ***@***.***> wrote: An update on this: the issue seems to be in covariates rather than the file size. When trying with only one binary covariate (sex) there was no problem even with a bit smaller memory, but with one quantitative covariate it got 'killed'... — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGAPMKYNAZFER43M7TRL4TZGVJUZAVCNFSM6AAAAABINA4OTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXGU2TSNBSGM> . You are receiving this because you commented.Message ID: ***@***.***>