Skip to content

Develop tensorrt sam faster model inference#10213

Open
Yakuho wants to merge 4 commits intocvat-ai:developfrom
Yakuho:develop-tensorrt-sam
Open

Develop tensorrt sam faster model inference#10213
Yakuho wants to merge 4 commits intocvat-ai:developfrom
Yakuho:develop-tensorrt-sam

Conversation

@Yakuho
Copy link

@Yakuho Yakuho commented Jan 22, 2026

Motivation and context

Improve SAMEncoder inference speed for interactor mode

How has this been tested?

Test condition: GPU: RTX-3090, CPU: Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz, pytorch-cu121==2.5.1,
tensorrt-cu12==10.6.0, onnx==1.20.1, onnxsim==0.4.36

exported onnx model version: opset_version==18, dtype==float32

/ pth trt
gpu-memory ~7000m ~7000m
speed ~420ms ~250ms

totally, trt will bring about a 40% improvement

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@Yakuho Yakuho requested a review from bsekachev as a code owner January 22, 2026 11:37
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant