CUDA image fails to load ONNX Runtime due to outdated CUDNN/CUDA versions

[ss1gohan13]

Problem Description

When running the ghcr.io/imagegenius/obico:cuda container, I'm getting errors that indicate the CUDA version is incompatible with the current ONNX Runtime requirements.

Error Message

[E:onnxruntime:Default, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.9: cannot open shared object file: No such file or directory

[W:onnxruntime:Default, onnxruntime_pybind_state.cc:1055 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

Analysis

Looking at the Dockerfile in the CUDA branch, I can see that it's currently installing CUDNN 8.2.4 for CUDA 11.4:

curl -o /tmp/libcudnn.deb -L https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/libcudnn8_8.2.4.15-1+cuda11.4_amd64.deb

However, the latest ONNX Runtime now requires:

• CUDNN 9.*
• CUDA 12.*

Possible Solutions

Either:

1. Pin the ONNX Runtime version to one compatible with CUDA 11.4 and CUDNN 8 (like onnxruntime-gpu==1.14.1)
2. Update the Dockerfile to install CUDA 12.x and CUDNN 9.x

Since ONNX Runtime is regularly updated and may have important fixes and improvements, option 2 would be preferable for a long-term solution.

Additional Information

• ONNX Runtime GPU requirements: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements

[hydazz]

Thanks for pointing this out, did you want to open a PR yourself and push a fix?

[ss1gohan13]

Thanks for pointing this out, did you want to open a PR yourself and push a fix?

I had been chatting with CoPilot about this and discovered the potential fix via CoPilot. I'm really inexperienced with GitHub and even less with Docker. I simply run a unRAID server and install the wonderfully designed packages people make for unRAID.

If you could implement this in some fashion to help resolve the error, that would be wonderful. I'm not in any big hurry myself, I am currently able to run the container without issues.

I am running into some caveat issues with video bitrates, but that's not a discussion to have here. I can look into that elsewhere.

[hydazz]

Try ghcr.io/imagegenius/igpipepr-obico:cuda-e3993e83-pkg-f537188b-dev-041f158a35ff5e7f7ea6700855573adb7ffb83ac-pr-34 and see if cuda acceleration works

[hydazz]

@johnsmoses @ss1gohan13

^

[ss1gohan13]

@johnsmoses @ss1gohan13

^

Apologies for the long delay here. I've been working on other projects.

I did try this image and I am no longer getting any image to display and no indication that the docker image is trying to utilize the GPU.