Add ROCm GPU backend support for stable-diffusion.cpp#980
Add ROCm GPU backend support for stable-diffusion.cpp#980
Conversation
- Add sdcpp_backend option with --sdcpp CLI flag and LEMONADE_SDCPP env var - Support 'cpu' (default) and 'rocm' backends for sd.cpp - Download ROCm-enabled sd.cpp binaries for AMD GPU acceleration - Add backend-specific versioning in backend_versions.json - Set PATH for ROCm DLLs on Windows for HIP runtime loading - Add api_image_gen_rocm.py example demonstrating ROCm image generation - Fix example script to use correct API parameters (model_name, model)
makn87amd
requested review from
danielholanda,
jeremyfowers and
ramkrishna2910
|
The example needs to be pruned. It was auto-generated and has a ton of helper functions |
src/cpp/server/recipe_options.cpp
Outdated
| {"envname", "LEMONADE_LLAMACPP_ARGS"}, | ||
| {"help", "Custom arguments to pass to llama-server (must not conflict with managed args)"} | ||
| }}, | ||
| // ADDED: sd.cpp backend selection option |
There was a problem hiding this comment.
pretty noisy/pointless comment, no?
| {"option_name", "sdcpp_backend"}, | ||
| {"type_name", "BACKEND"}, | ||
| {"allowed_values", {"cpu", "rocm"}}, | ||
| {"envname", "LEMONADE_SDCPP"}, |
There was a problem hiding this comment.
please also add this new environment variable to data/lemonade.conf
ramkrishna2910
left a comment
ramkrishna2910
left a comment
There was a problem hiding this comment.
Added some minor comments, looks great overall!
| #ifdef _WIN32 | ||
| filename = "sd-" + short_version + "-bin-win-rocm-x64.zip"; | ||
| #elif defined(__linux__) | ||
| filename = "sd-" + short_version + "-bin-linux-rocm-x64.zip"; | ||
| #else | ||
| throw std::runtime_error("ROCm sd.cpp only supported on Windows and Linux"); | ||
| #endif | ||
| std::cout << "[SDServer] Using ROCm GPU backend" << std::endl; | ||
| } else { |
There was a problem hiding this comment.
The indentation feels a little off here. This should be inside the backend_==rocm?
There was a problem hiding this comment.
Yes, you're right. Somewhere in a source file a diff showed up with the indentation you mention. So, I thought this was your preference.
I was probably comparing it to nonsense. Let me fix it
src/cpp/server/system_info.cpp
Outdated
| // stable-diffusion.cpp - Windows/Linux x86_64 | ||
| {"sd-cpp", "default", {"windows", "linux"}, { | ||
| {"sd-cpp", "rocm", {"windows", "linux"}, { | ||
| {"amd_igpu", {"gfx1150", "gfx1151"}}, |
There was a problem hiding this comment.
gfx1150 on stx did not work for me and I dont think its supported by rocm.
There was a problem hiding this comment.
Yes, you're right. Thomas was able to get this to run, but he had to disable his IGPU.
There was a problem hiding this comment.
Should I just take out "linux" support for rocm? We had discussed this. Change would be:
{"sd-cpp", "rocm", {"windows"}, {
{"amd_igpu", {"gfx1151"}},
{"amd_dgpu", {"gfx110X", "gfx120X"}},
}},
There was a problem hiding this comment.
Yeah, just remove linux from the list.
@superm1 stablediffusion.cpp doesn't publish Linux ROCm binaries on release, only Windows. We'll need to work with them to get them building for Linux too. I don't suppose that's something you'd be interested in...?
There was a problem hiding this comment.
@superm1 stablediffusion.cpp doesn't publish Linux ROCm binaries on release, only Windows. We'll need to work with them to get them building for Linux too. I don't suppose that's something you'd be interested in...?
There was a problem hiding this comment.
These binaries support more architectures than Windows HIP do. Here's what I added:
gfx1151
gfx1150
gfx1100
gfx1101
gfx1102
gfx1200
gfx1201
| @@ -0,0 +1,66 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
Do we need a new example file or can we use the existing example and add a cmd line param?
|
Not only does this work on my Radeon 9070 XT, it is SO FREAKING FAST. Love it! RDNA 4 is supported :) |
4 participants
This PR adds ROCm support to sd.cpp.
Only one GPU configuration (stx-halo) has been tested. Help is needed testing other GPU configurations