Skip to content

Fix lime CUDA OOM #811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Logiquo wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Fix lime CUDA OOM #811

Logiquo wants to merge 3 commits into from

Conversation

@Logiquo
Copy link
Collaborator

@Logiquo Logiquo commented Feb 4, 2026 •
edited
Loading

Contributor: Yongda Fan (yongdaf2@illinois.edu)

Contribution Type: Interpretability

Description
The old implementation suffers from

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 5 has a total capacity of 47.53 GiB of which 17.06 MiB is free. Including non-PyTorch memory, this process has 47.51 GiB memory in use. Of the allocated memory 47.20 GiB is allocated by PyTorch, and 2.27 MiB is reserved by PyTorch but unallocated.

The new code accumulate tensor in cpu while maintain most of the compute on gpu.

@Logiquo Logiquo requested a review from jhnwu3 February 4, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jhnwu3 jhnwu3 Awaiting requested review from jhnwu3

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Logiquo