[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission #33313

tovinkere · 2025-12-18T20:07:57Z

Enables default ITT markers for higher level operations such as inference pass, op preparation and submission
Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu
Supports both synchronous and asynchronous operations

Enabling default GPU ITT markers using standard convention - Part 3

This PR is the third of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation.

The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play PR#33639.
The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution PR#33312.
This third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2
The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity.

Summary of the current PR (PR#3)

Use the same convention standardized in PR#33639
Ensures the namespace for GPU Plugin activity falls under:
ov::phases::gpu::inference
ov::op::gpu

Details:
GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used.

Tickets:
CVS-179230

@isanghao Please review this as you are generally aware of what was discussed

- Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

+ Adding base markers for node execution instead of a prepare() and execute() Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

isanghao · 2026-01-06T08:35:07Z

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

 void SyncInferRequest::infer() {
-    OV_ITT_SCOPED_TASK(itt::domains::intel_gpu_plugin, "SyncInferRequest::infer");
+    // String can be constructed once in the constructor
+    OV_ITT_SCOPED_TASK_BASE(itt::domains::intel_gpu_inference, "SyncInferRequestGPU::infer" +  m_graph->get_runtime_model()->get_name());


out of curiosity, is it OK to change the name here? you commented not to change the name in other place.. [Warning] The strings in ITT_SCOPED_TASK_BASE should NOT be deleted or edited!

@isanghao Please suggest what changes you need. I tried to make the strings in TASK_BASE to be similar for all plugins.

I mean, you changed the name from SyncInferRequest to SyncInferRequestGPU. If it is OK to change the name, why do you leave comment that we should not change the name?

@isanghao Previous task definition was incomplete. Also, it is not enabled by default. I enable the task by default using OV_ITT_SCOPED_TASK_BASE macro and renamed the string to be indicative of being executed by the GPU plugin. This new string cannot be changed as it acts as a SW contract and all tools that use this data will rely on the strings being invariant.

p-durandin · 2026-01-12T09:24:02Z

build_jenkins

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

- This update fixes the GPU runtime performance regressions due to increased submission times. By caching the required string for each inference, the overheads are eliminated. Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk · 2026-01-21T09:11:06Z

build_jenkins

src/plugins/intel_gpu/include/intel_gpu/plugin/sync_infer_request.hpp

isanghao · 2026-01-22T06:17:37Z

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

-#include "intel_gpu/runtime/debug_configuration.hpp"

 #include <algorithm>
+#include <functional>


Please do not include style fix in the PR. It makes review difficult.

@isanghao Previous PRs had clang-format failures on changed files, so not I run clang-format as recommended by @praasz. Can defer this till the end in the future.

You don't need to reformat the whole file. You can just change the problematic line to pass the test..

It should consistent with current style in GPU, @isanghao has greater experience in this component I would follow his suggestions

src/plugins/intel_gpu/src/plugin/graph.cpp

src/plugins/intel_gpu/src/graph/network.cpp

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

praasz · 2026-01-22T14:27:15Z

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp

    }
+
+    // Helper function to return the model name for ITT tracing
+    std::string get_model_name() const {


Suggested change

std::string get_model_name() const {

std::string_view get_model_name() const {

Be used instead make copy of string the copy can be created where needed.

@isanghao
Is it required or model name can be get via different member?

@isanghao Please suggest other ways to get the model name, if there is a way. Using get_runtime_model()->get_name() caused the perf regression earlier, so I am getting the compiled model name with the assumption that it does change with new models being loaded.
@praasz In this particular case, the number of string constructors being invoked should be the same, so we don't gain much?

The std::string_view better hide internal implementation.

@isanghao
compiled_model->get_runtime_model()->get_friendly_name()
can it be used instead introduce this member? or it will give different name?

@praasz If there is a way of getting the "friendly name" with low cost, it makes sense. compiled_model->get_runtime_model() goes through a chain of calls, so I am not sure it is low cost.

The std::string_view better hide internal implementation.

It has been changed to string_view :)

- Moving string constructor to where it is used Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere · 2026-01-27T05:30:38Z

@praasz @aobolensk @isanghao Any progress or next steps?

isanghao · 2026-01-27T06:29:41Z

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp

+    // Helper function to return the model name for ITT tracing
+    std::string_view get_model_name() const {
+        return m_model_name;
+    }


is it possible to reuse compiled_model->get_runtime_model()->get_friendly_name() instead of adding this new method?

@isanghao get_runtime_model() call is what causes the performance regressions, so the answer is no. Are there alternatives?

@aobolensk @isanghao If there are no other changes, can this be merged?

@praasz do you have any further comment?

aobolensk · 2026-01-27T08:22:58Z

build_jenkins

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

- Also update the strings to be consistent across all plugins Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere · 2026-01-30T06:40:55Z

@isanghao The style changes have been removed as requested. Should be easily reviewable now.

aobolensk

Overall LGTM

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk · 2026-01-30T08:50:52Z

build_jenkins

isanghao

LGTM for GPU

eparshut · 2026-02-02T18:24:25Z

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

+    OV_ITT_DOMAIN(intel_gpu_inference, "ov::phases::gpu::inference");
+    // Domain namespace for all of the operators
+    OV_ITT_DOMAIN(intel_gpu_op, "ov::op::gpu");
    OV_ITT_DOMAIN(intel_gpu_plugin);


It would be beneficial to define this domain name as ov::intel_gpu_plugin to maintain consistency with other domains. This ensures the ov:: prefix is present, making it easier for profiling tools to filter and capture all ov events

Suggested change

OV_ITT_DOMAIN(intel_gpu_plugin);

OV_ITT_DOMAIN(intel_gpu_plugin, "ov::intel_gpu_plugin");

tovinkere requested review from a team as code owners December 18, 2025 20:07

github-actions bot added the category: GPU OpenVINO GPU plugin label Dec 18, 2025

sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Dec 18, 2025

[OV][ITT][GPU Plugin] Reduce ITT events in half

4e989a7

+ Adding base markers for node execution instead of a prepare() and execute() Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

mg-intel assigned isanghao and aobolensk Dec 23, 2025

aobolensk reviewed Dec 23, 2025

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp Outdated Show resolved Hide resolved

isanghao reviewed Jan 6, 2026

View reviewed changes

isanghao reviewed Jan 13, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp Show resolved Hide resolved

tovinkere and others added 3 commits January 20, 2026 13:54

Merge branch 'openvinotoolkit:master' into itt_markers_gpu

cf51b56

[OV][ITT] ITT markers for GPU - regression fix

3c51087

- This update fixes the GPU runtime performance regressions due to increased submission times. By caching the required string for each inference, the overheads are eliminated. Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

[OV][ITT] Making ITT string for GPU consistent

2e8591d

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/plugin/sync_infer_request.hpp Show resolved Hide resolved

isanghao reviewed Jan 22, 2026

View reviewed changes

aobolensk reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/src/plugin/graph.cpp Show resolved Hide resolved

aobolensk reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/src/graph/network.cpp Outdated Show resolved Hide resolved

[OV][ITT] Address reviewer feedback

0e7d052

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

praasz reviewed Jan 22, 2026

View reviewed changes

[OV][ITT] Addressing reviewer comments

79a0773

- Moving string constructor to where it is used Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao reviewed Jan 27, 2026

View reviewed changes

tovinkere added 2 commits January 29, 2026 22:13

[OV][ITT] Undo style changes

34f6696

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

[OV][ITT] Add the pref regression fix

480cb06

- Also update the strings to be consistent across all plugins Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk approved these changes Jan 30, 2026

View reviewed changes

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp Outdated Show resolved Hide resolved

[OV][ITT] Fixed string separator in naming convention

62272d3

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao approved these changes Feb 2, 2026

View reviewed changes

eparshut reviewed Feb 2, 2026

View reviewed changes

	std::string get_model_name() const {
	std::string_view get_model_name() const {

	OV_ITT_DOMAIN(intel_gpu_plugin);
	OV_ITT_DOMAIN(intel_gpu_plugin, "ov::intel_gpu_plugin");

[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission #33313

Are you sure you want to change the base?

[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission #33313

Uh oh!

Conversation

tovinkere commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

p-durandin commented Jan 12, 2026

Uh oh!

Uh oh!

aobolensk commented Jan 21, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

praasz Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tovinkere Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tovinkere commented Jan 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aobolensk commented Jan 27, 2026

Uh oh!

tovinkere commented Jan 30, 2026

Uh oh!

aobolensk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aobolensk commented Jan 30, 2026

Uh oh!

isanghao left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tovinkere commented Dec 18, 2025 •

edited

Loading

praasz Jan 22, 2026 •

edited

Loading

tovinkere Jan 22, 2026 •

edited

Loading