[enhancement]: Support for SD.Next Quantizer

[iwr-redmond]

Is there an existing issue for this?

• I have searched the existing issues

Contact Details

No response

What should this feature add?

SDNQ is a Diffusers-compatible quantizer that works with all major platforms. It includes support for SVDQuant, previously introduced in the CUDA-only Nunchaku, which allows models to run at 4-bit precision with minimal quality loss.

SDNQ may be suitable as the default quantizer for Invoke, as it is compatible with more end user systems than Bitsandbytes while avoiding the need for upcasting to FP16 inherent to GGUF quantization. A range of prequantized checkpoints is available here.

As part of adopting SDNQ, it would be helpful to install triton-windows for Windows CUDA and ROCm installations.

Alternatives

No response

Additional Content

Updated 2/3: Triton for Windows now experimentally supports AMD (triton-windows#188)

[lstein]

@JPPhoto Tagging you on this. It sounds like the sort of thing you might want to dig into. Feel free to suggest another assignee.

[JPPhoto]

@JPPhoto Tagging you on this. It sounds like the sort of thing you might want to dig into. Feel free to suggest another assignee.

This is neat. Are you envisioning a conversion tool as well as integration into generation? Wonder if this is something @dunkeroni would be interested in?