-
Notifications
You must be signed in to change notification settings - Fork 105
Experimental Feature: Function delegation to rapids_singlecell when gpu is available #1093
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
selmanozleyen
wants to merge
82
commits into
scverse:main
Choose a base branch
from
selmanozleyen:feat/device-settings
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 8 commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
41189a5
init
selmanozleyen e058886
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] a0bfcd4
add cooccurance demo
selmanozleyen fa33967
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen 5c67afd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 73c1b15
move import location
selmanozleyen 4896c93
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen 2474ac5
remove unused DeviceType
selmanozleyen 20625c4
Merge branch 'main' into feat/device-settings
selmanozleyen 1eca173
Merge branch 'main' into feat/device-settings
selmanozleyen 464ae3c
save wip to test in the clusters
selmanozleyen 037b6a8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 0820a0e
adjust deps but need prerelease of spatialdata for unpining dask
selmanozleyen 70af5c7
fix the set_default_colors_for_categorical_obs
selmanozleyen 74c70d9
add ligrec and cooccurence
selmanozleyen a1d2d10
reduce bloat code
selmanozleyen a331134
updates for the new dispatch approach
selmanozleyen 0dffb8b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] de83264
use try except for compar
selmanozleyen 1d7eb48
interphinx mapping
selmanozleyen 17d34bd
rsc link refer workaround
selmanozleyen 89be752
ad gpu notes
selmanozleyen 5a7cb83
ok fix docs
selmanozleyen 7478f62
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 74dc1ff
update docs
selmanozleyen f1b506e
refactor
selmanozleyen 403f08f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 3aed99e
save wip
selmanozleyen ab5af39
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] a77f555
Merge branch 'main' into feat/device-settings
selmanozleyen b05f2bb
Merge branch 'main' into feat/device-settings
selmanozleyen c0aa5da
save wip
selmanozleyen 8d03b5b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 08f7af6
remove kwargs from ligrec
selmanozleyen 4885195
refactor and formatting
selmanozleyen ebb03c2
arg refactor
selmanozleyen eb58e0e
remove n_splits
selmanozleyen 3568694
update parallelize args
selmanozleyen 7b2aad3
apply suggestion
selmanozleyen 4453d2b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 0482932
update docs
selmanozleyen 8cd3829
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen f854794
fux docs
selmanozleyen 26e8ddc
make gpu_availible an attr
selmanozleyen 5034343
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 8aae2ee
revisions
selmanozleyen 40f534a
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen 25ebbb0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 52e0b4e
cleanup
selmanozleyen 233ff07
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen 0993068
Merge branch 'main' into feat/device-settings
selmanozleyen 60f1eba
revert to old file
selmanozleyen bae11a6
Merge branch 'main' into feat/device-settings
selmanozleyen 99d55e9
remove device and rely on context manager
selmanozleyen fb23269
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen 22edd86
add device_kwargs
selmanozleyen 5bc8ca4
pass args directly
selmanozleyen 1f118da
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 97f5355
non-thread version of gpu dispatch
selmanozleyen b192fbe
gpu func cache
selmanozleyen 6359577
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 9dca51b
context manager
selmanozleyen ac5421e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 06c001a
update
selmanozleyen b7ac76b
Merge branch 'feat/device-settings' of https://github.com/selmanozley…
selmanozleyen ba9e7e0
remove redundant test
selmanozleyen ae54591
redundant docstrings
selmanozleyen a137673
remove redundancy
selmanozleyen 0ebb219
add uv.lock to gitignore
selmanozleyen a9517a9
remove device leftover
selmanozleyen 4ebda67
spatial_neighbors doesnt match rsc much
selmanozleyen 93b06dc
remove redundant test
selmanozleyen f19bdc7
change the order of filtering
selmanozleyen c0a10cb
make _SqSettings private
selmanozleyen 0276e47
error when device_args is given
selmanozleyen 53650c0
update tests
selmanozleyen 49ed2fb
update regex patternf or test
selmanozleyen 64fc33c
update gpu tests
selmanozleyen 4f540ad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] e3b331e
clean up test settings
selmanozleyen 50ece87
simplify
selmanozleyen 97fbb00
Merge branch 'main' into feat/device-settings
selmanozleyen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| """Squidpy settings and configuration.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from squidpy.settings._settings import DeviceType, settings | ||
|
|
||
| __all__ = ["settings", "DeviceType"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from contextvars import ContextVar | ||
| from typing import Literal, get_args | ||
|
|
||
| __all__ = ["settings", "DeviceType"] | ||
|
|
||
| DeviceType = Literal["auto", "cpu", "gpu"] | ||
|
|
||
| _device_var: ContextVar[DeviceType] = ContextVar("device", default="auto") | ||
|
|
||
|
|
||
| class _SqSettings: | ||
| """Global settings for squidpy.""" | ||
|
|
||
| @property | ||
| def device(self) -> DeviceType: | ||
| """Current compute device setting.""" | ||
| return _device_var.get() | ||
|
|
||
| @device.setter | ||
| def device(self, value: DeviceType) -> None: | ||
| valid = get_args(DeviceType) | ||
| if value not in valid: | ||
| raise ValueError(f"Invalid device {value!r}. Must be one of: {valid}") | ||
| if value == "gpu" and not self.gpu_available(): | ||
| raise RuntimeError( | ||
| "Cannot set device='gpu': rapids-singlecell not installed. " | ||
| "Install with: pip install squidpy[gpu-cuda12] or squidpy[gpu-cuda11]" | ||
| ) | ||
| _device_var.set(value) | ||
|
|
||
| @staticmethod | ||
| def gpu_available() -> bool: | ||
| """ | ||
| Check if GPU acceleration is available. | ||
|
|
||
| Returns | ||
| ------- | ||
| bool | ||
| True if rapids-singlecell is installed and importable. | ||
| """ | ||
| try: | ||
| import rapids_singlecell # noqa: F401 | ||
|
|
||
| return True | ||
| except ImportError: | ||
| return False | ||
|
|
||
|
|
||
| settings = _SqSettings() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| """Tests for GPU functionality (skipped in CI without GPU).""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import pytest | ||
|
|
||
| from squidpy.settings import settings | ||
|
|
||
| # Skip all tests in this module if GPU is not available | ||
| pytestmark = pytest.mark.skipif( | ||
| not settings.gpu_available(), | ||
| reason="GPU tests require rapids-singlecell to be installed", | ||
| ) | ||
|
|
||
|
|
||
| class TestGPUCoOccurrence: | ||
| """Test GPU-accelerated co_occurrence function.""" | ||
|
|
||
| def test_co_occurrence_gpu(self, adata): | ||
| """Test co_occurrence with GPU device.""" | ||
| import squidpy as sq | ||
|
|
||
| # Run with explicit GPU device | ||
| result = sq.gr.co_occurrence( | ||
| adata, | ||
| cluster_key="leiden", | ||
| copy=True, | ||
| device="gpu", | ||
| ) | ||
|
|
||
| assert result is not None | ||
| arr, interval = result | ||
| assert arr.ndim == 3 | ||
| assert arr.shape[1] == arr.shape[0] == adata.obs["leiden"].unique().shape[0] | ||
|
|
||
| def test_co_occurrence_gpu_vs_cpu(self, adata): | ||
| """Test that GPU and CPU results are approximately equal.""" | ||
| import numpy as np | ||
|
|
||
| import squidpy as sq | ||
|
|
||
| # Run on CPU | ||
| cpu_result = sq.gr.co_occurrence( | ||
| adata, | ||
| cluster_key="leiden", | ||
| copy=True, | ||
| device="cpu", | ||
| ) | ||
|
|
||
| # Run on GPU | ||
| gpu_result = sq.gr.co_occurrence( | ||
| adata, | ||
| cluster_key="leiden", | ||
| copy=True, | ||
| device="gpu", | ||
| ) | ||
|
|
||
| cpu_arr, cpu_interval = cpu_result | ||
| gpu_arr, gpu_interval = gpu_result | ||
|
|
||
| # Results should be close (allow for floating point differences) | ||
| np.testing.assert_allclose(cpu_interval, gpu_interval, rtol=1e-5) | ||
| np.testing.assert_allclose(cpu_arr, gpu_arr, rtol=1e-5) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| """Tests for squidpy.settings module.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import pytest | ||
|
|
||
| from squidpy.settings import settings | ||
|
|
||
|
|
||
| class TestSettings: | ||
| """Test the settings module.""" | ||
|
|
||
| def test_default_device(self): | ||
| """Test that default device is 'auto'.""" | ||
| # Reset to default | ||
| settings.device = "auto" | ||
| assert settings.device == "auto" | ||
|
|
||
| def test_set_device_cpu(self): | ||
| """Test setting device to 'cpu'.""" | ||
| settings.device = "cpu" | ||
| assert settings.device == "cpu" | ||
| settings.device = "auto" # reset | ||
|
|
||
| def test_set_device_invalid(self): | ||
| """Test that invalid device raises ValueError.""" | ||
| with pytest.raises(ValueError, match="Invalid device"): | ||
| settings.device = "invalid" | ||
|
|
||
| def test_set_device_gpu_without_rsc(self): | ||
| """Test that setting device to 'gpu' without rapids-singlecell raises RuntimeError.""" | ||
| # This will fail if rapids-singlecell is not installed | ||
| if not settings.gpu_available(): | ||
| with pytest.raises(RuntimeError, match="rapids-singlecell not installed"): | ||
| settings.device = "gpu" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this can generally be wrapped up in a decorator where we:
effective_devicen_jobsdoesn't apply, this can probably be done with python'sinspectmodule)