Skip large memory tests in ASan

[sarthakaggarwal97]

We have been seeing github actions runners being OOM when large memory tests are run with ASan. The operation eventually is being canceled during the test.

To begin with, we will skip the large memory tests in the scheduled daily runs. If someone wants to trigger manually / on PR, they should be able to do so.

Some recent failure examples:

1. https://github.com/valkey-io/valkey/actions/runs/21573028934/job/62155215331#step:10:425
2. https://github.com/valkey-io/valkey/actions/runs/21553319484/job/62105370976#step:10:425

[sarthakaggarwal97]

@madolson I skipped the tests right now for scheduled runs. Let me know if you think we should skip the tests altogether. Thanks!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.94%. Comparing base (839d0c6) to head (641e70f).
⚠️ Report is 3 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #3161      +/-   ##
============================================
- Coverage     74.96%   74.94%   -0.03%     
============================================
  Files           129      129              
  Lines         71318    71318              
============================================
- Hits          53467    53450      -17     
- Misses        17851    17868      +17     

see 17 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[dvkashapov]

One of the reasons to keep running large memory with asan is to keep new bugs from being introduced, here #1748 large string bug was fixed because we did not have those runs before. Also if someone will see failure in extra tests they will not have reference to wether this change introduced new bug or it was already present.

[sarthakaggarwal97]

@dvkashapov that's a good example but over there it affected a specific platform. But currently ASan tests with daily are only run on single machine / platform. So we are anyways not covering all the platforms.
I am more worried if these large memory tests might hide important failures since the operation just gets canceled due to machines going OOM.

Alternate is that I saw couple of specific tests which were affecting these runs, and maybe we can change the params (like with lesser memory) of those based on if the run is in ASan mode.

What do you think?

[dvkashapov]

Alternate is that I saw couple of specific tests which were affecting these runs, and maybe we can change the params (like with lesser memory) of those based on if the run is in ASan mode.

Yes, this makes sense, what tests did affect the most?

[zuiderkwast]

What if we skip the large-memory tests and then add a new separate job that only runs the large-memory tests?

To use less memory, we can run with --clients 1 (or another small number) instead of the default 10, to avoid parallelizing the tests.

[sarthakaggarwal97]

What if we skip the large-memory tests and then add a new separate job that only runs the large-memory tests?

Like as a part of daily itself? In another workflow? Can try it out.

To use less memory, we can run with --clients 1 (or another small number) instead of the default 10, to avoid parallelizing the tests.

This can slow down the execution time significantly. It takes about an hour right now.

Yes, this makes sense, what tests did affect the most?

test_quicklistCompressAndDecompressQuicklistListpackNode

[sarthakaggarwal97]

Also, multiple sanitizer runs failed again yesterday: https://github.com/valkey-io/valkey/actions/runs/21693609685/job/62558879812#step:10:423

[zuiderkwast]

To use less memory, we can run with --clients 1 (or another small number) instead of the default 10, to avoid parallelizing the tests.

This can slow down the execution time significantly. It takes about an hour right now.

Yes, but if this new job (a separate job in Daily) only runs the large-memory tests and nothing else, then it shouldn't take too long I guess.

[sarthakaggarwal97]

@zuiderkwast I can try doing this and maybe run the tests and observe.

[sarthakaggarwal97]

@zuiderkwast I have separated them into different jobs. Let me give it a few tries to run daily in my forked repo and see if it works.