[Detector Support]: No TensorRT (AMD Machine), and no Onnx Support in Frigate 16.*

[quantum77]

Describe the problem you are having

It's in the title. Looks like I am out of luck with Frigate, after a week of work. I was assembling a HowTo for this fantastically complex app, but dead end.

What a fussy mess Docker is. But I can't use Podman because Debian 13.3 is too new for the nVidia drivers.

Version

16.3

[inkeep-ai-assistant[bot]]

The error indicates your detector configuration is invalid. Based on your setup with an NVIDIA GPU, you need to use the correct Frigate image and detector configuration(1).

For NVIDIA GPUs with ONNX models, use the -tensorrt image instead of stable(1):

image: ghcr.io/blakeblackshear/frigate:stable-tensorrt

(1)

Your detector configuration has issues. The ONNX detector will automatically use TensorRT when the correct image is used(1). Update your detector config:

detectors:
  onnx:
    type: onnx

model:
  model_type: yolo-generic
  width: 640
  height: 640
  input_tensor: nchw
  input_dtype: float
  path: /models/yolov9c.onnx
  labelmap_path: /labelmap/coco-80.txt

(1)

Key points:

• TensorRT will automatically be detected and used with the ONNX detector in the -tensorrt Frigate image(1)
• The model configuration should be at the root level, not nested under detectors.onnx(1)
• Ensure the model path matches your volume mount (/models/yolov9c.onnx)(1)
• The width and height must match your YOLO model size(2)

Your hardware acceleration for video decoding looks correct with preset-nvidia-h264(3).

📚 Sources:

1. Supported Hardware
2. [Detector Support]: Fatal Python error: Segmentation fault
3. Video Decoding

---

Have another question? Just tag @inkeep.

[hawkeye217]

The AI is correct. Your config is incorrect, the model key should not be indented under detectors, it should be at the global level.

[hawkeye217]

You should also be using presets rather than manual ffmpeg arguments.

[quantum77]

Onnx crashes:

[hawkeye217]

How did you generate your YOLOv9 ONNX model? The official docs suggest the method that works with the correct opset version.

Frigate is also crashing because you have specified invalid manual ffmpeg arguments. The recommended ffmpeg preset that has all of the appropriate arguments for your GPU is preset-nvidia.

[quantum77]

Please advise the 'bot that I can't run TensorRT because this is an AMD machine and Frigate has removed that functionality.

[hawkeye217]

You are not running the TensorRT detector, your config is set up for the ONNX detector, which is correct, as suggested by the AI and the official docs. You just need to generate the model as also suggested by the docs.

[quantum77]

[NickM-27]

your code is improper, onnxruntime always defaults to CPU provider unless you explicitly give it another option. I don't see any reason to think it is running on the CPU in Frigate's case

[lack0fcode]

you mean about the compose file? sorry about I'm being so dumb here, I thought the onnx were always using GPU. any suggetions on where to "explicitly give" the GPU option?

docker-compose.yml
services:
frigate:
container_name: frigate
privileged: true
image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
restart: unless-stopped
stop_grace_period: 30s
shm_size: "512mb"

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          capabilities: [gpu]

[NickM-27]

I am referring to your python, you did not initialize onnxruntime properly. That is entirely irrelevant to the actual question, and why do you think it is not running on the GPU?

[lack0fcode]

[NickM-27]

That doesn't mean it is running on the CPU, 0.16 is less efficeint with Nvidia GPUs and 0.17 uses CUDA graphs to reduce CPU usage when launching each kernel, so 0.17 will be better in this regard.

The only way to actually ensure it is running on the GPU is check nvidia-smi on the host

[quantum77]

[hawkeye217]

We appreciate the feedback. Frigate began as a weekend project and has been purposely aimed at a more technical audience while we build out core features and improve performance before wider audience adoption.

The main contributors are just a small team of unpaid volunteers passionate about open source. We're continually improving the configuration/onboarding features and we recently added the "Ask AI" button on the official docs site to help "mere mortals" walk through many basic and advanced topics.

Frigate is not for everyone, and it shouldn't be for everyone. We're always seeking to make it better, though.

[quantum77]

[hawkeye217]

0.17 is currently in beta. If you choose to test, we recommend using a separate instance if your production installation is critical.