[Feature]: Add SSD offloading support #169
Description
Problem Statement
Add SSD offloading support can inference of extremely large models at the cost of performance
Proposed Solution
Add SSD offloading support.
Alternatives Considered
🎨 Feature Category
Model support (new formats, engines, LoRA)
Priority
Low - Nice to have
Use Case
can inference of extremely large models at the cost of performance.
Technical Details (optional)
No response
Examples
Implementation
- I am willing to submit a pull request for this feature
- I can help with testing this feature
- I can help with documentation for this feature
Additional Context
No response