opentelemetry-util-genai: add support for emitting inference events and enrich message types (#3994)

* Add support for emitting inference events and enrich message types

Change-Id: I8fd0b896fc103a986f78c7351ce627611e545a62
Co-developed-by: Cursor <noreply@cursor.com>

* Add change log

Change-Id: I5c4c93613e3e1084245b7298955a08cbc7c9708d
Co-developed-by: Cursor <noreply@cursor.com>

* Fix unit tests

Change-Id: If34cfce0e7eb130db6a1e8e30a5f4be7c215285f
Co-developed-by: Cursor <noreply@cursor.com>

* Fix linting failure

Change-Id: I847f75259e01729db88129a44b241afb0ea2aca4
Co-developed-by: Cursor <noreply@cursor.com>

* Fix readme

Change-Id: I818a042d275d3c8e3348647d73e34560e7d92f54
Co-developed-by: Cursor <noreply@cursor.com>

* Format codes

Change-Id: I40b8e01bbe4fa9c182e99085a7c71d4536042247
Co-developed-by: Cursor <noreply@cursor.com>

* Fix missing trace context in events

Change-Id: Ie07c495002143fb2f0cf88033206290eb85386ad
Co-developed-by: Cursor <noreply@cursor.com>

* feedback

Change-Id: Ida0c2305d950d978c31eb04a80e21e947fabdfba
Co-developed-by: Cursor <noreply@cursor.com>

* fix type check

Change-Id: I1da48b52b76042a9efd124057681f579cc93fb6e
Co-developed-by: Cursor <noreply@cursor.com>

* Fix the span name of LLM invocations

Change-Id: I1b9f40e5576e699b1f61fa3d7e7790ee4b1448a5
Co-developed-by: Cursor <noreply@cursor.com>

* Fix span name

Change-Id: I7adf6fc9cf0bcbed3927862ff87d441bb2a1f78b
Co-developed-by: Cursor <noreply@cursor.com>

* Fix operation name of llm span

Change-Id: I4a6f48d0f66b8ad00a6ce4be8dcf4a46aba68e33
Co-developed-by: Cursor <noreply@cursor.com>

* Adjust event emission switches to optimize the integration experience

Change-Id: Ied7182fe2f3493a1eba96d7eaae5fd1b790c16e7
Co-developed-by: Cursor <noreply@cursor.com>

* Resolve conflicts

Change-Id: I95472013fe59b2b7a7448bb4374fe103021ec8a1
Co-developed-by: Cursor <noreply@cursor.com>

* Fix lint error

Change-Id: I78008c9b812d81a705eb7e7c75bd622b47d9da5a
Co-developed-by: Cursor <noreply@cursor.com>

* Refactor span_utils.py to streamline attribute collection

Change-Id: I243cc757454c085818d621011435076bf501108d
Co-developed-by: Cursor <noreply@cursor.com>

* Fix the lint error

Change-Id: If8bfaf1fde5594f7918c195243db9ec83f156156
Co-developed-by: Cursor <noreply@cursor.com>

---------

Co-authored-by: Aaron Abbott <aaronabbott@google.com>

Author: Cirilla-zmh

util/opentelemetry-util-genai/CHANGELOG.md

@@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+- Add support for emitting inference events and enrich message types. ([#3994](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3994))
 - Add support for `server.address`, `server.port` on all signals and additional metric-only attributes
   ([#4069](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4069))
 - Log error when `fsspec` fails to be imported instead of silently failing ([#4037](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4037)).

util/opentelemetry-util-genai/README.rst

@@ -9,7 +9,18 @@ while providing standardization for generating both types of otel, "spans and me
 This package relies on environment variables to configure capturing of message content. 
 By default, message content will not be captured.
 Set the environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` to `gen_ai_latest_experimental` to enable experimental features.
-And set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to `SPAN_ONLY` or `SPAN_AND_EVENT` to capture message content in spans.
+Set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to one of:
+- `NO_CONTENT`: Do not capture message content (default).
+- `SPAN_ONLY`: Capture message content in spans only.
+- `EVENT_ONLY`: Capture message content in events only.
+- `SPAN_AND_EVENT`: Capture message content in both spans and events.
+
+To control event emission, you can optionally set `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` to `true` or `false` (case-insensitive).
+This variable controls whether to emit `gen_ai.client.inference.operation.details` events.
+If not explicitly set, the default value is automatically determined by `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`:
+- When `NO_CONTENT` or `SPAN_ONLY` is set: defaults to `false`
+- When `EVENT_ONLY` or `SPAN_AND_EVENT` is set: defaults to `true`
+If explicitly set, the user's value takes precedence over the default.
 
 This package provides these span attributes:
 
@@ -23,6 +34,13 @@ This package provides these span attributes:
 - `gen_ai.usage.output_tokens`: Int(7)
 - `gen_ai.input.messages`: Str('[{"role": "Human", "parts": [{"content": "hello world", "type": "text"}]}]')
 - `gen_ai.output.messages`: Str('[{"role": "AI", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')
+- `gen_ai.system_instructions`: Str('[{"content": "You are a helpful assistant.", "type": "text"}]') (when system instruction is provided)
+
+When `EVENT_ONLY` or `SPAN_AND_EVENT` mode is enabled and a LoggerProvider is configured, 
+the package also emits `gen_ai.client.inference.operation.details` events with structured 
+message content (as dictionaries instead of JSON strings). Note that when using `EVENT_ONLY` 
+or `SPAN_AND_EVENT`, the `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` environment variable defaults 
+to `true`, so events will be emitted automatically unless explicitly set to `false`.
 
 
 Installation

util/opentelemetry-util-genai/src/opentelemetry/util/genai/_upload/completion_hook.py

@@ -174,7 +174,7 @@ def _calculate_ref_path(
         if is_system_instructions_hashable(system_instruction):
             # Get a hash of the text.
             system_instruction_hash = hashlib.sha256(
-                "\n".join(x.content for x in system_instruction).encode(  # pyright: ignore[reportUnknownMemberType, reportAttributeAccessIssue, reportUnknownArgumentType]
+                "\n".join(x.content for x in system_instruction).encode(  # pyright: ignore[reportUnknownMemberType, reportAttributeAccessIssue, reportUnknownArgumentType, reportCallIssue, reportArgumentType]
                     "utf-8"
                 ),
                 usedforsecurity=False,

util/opentelemetry-util-genai/src/opentelemetry/util/genai/environment_variables.py

@@ -16,6 +16,15 @@
     "OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"
 )
 
+OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT = "OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT"
+"""
+.. envvar:: OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT
+
+Controls whether to emit gen_ai.client.inference.operation.details events.
+Must be one of ``true`` or ``false`` (case-insensitive).
+Defaults to ``false``.
+"""
+
 OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK = (
     "OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK"
 )

util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py

@@ -65,10 +65,11 @@
 from typing import Iterator
 
 from opentelemetry import context as otel_context
-from opentelemetry.metrics import MeterProvider, get_meter
-from opentelemetry.semconv._incubating.attributes import (
-    gen_ai_attributes as GenAI,
+from opentelemetry._logs import (
+    LoggerProvider,
+    get_logger,
 )
+from opentelemetry.metrics import MeterProvider, get_meter
 from opentelemetry.semconv.schemas import Schemas
 from opentelemetry.trace import (
     Span,
@@ -80,7 +81,8 @@
 from opentelemetry.util.genai.metrics import InvocationMetricsRecorder
 from opentelemetry.util.genai.span_utils import (
     _apply_error_attributes,
-    _apply_finish_attributes,
+    _apply_llm_finish_attributes,
+    _maybe_emit_llm_event,
 )
 from opentelemetry.util.genai.types import Error, LLMInvocation
 from opentelemetry.util.genai.version import __version__
@@ -96,6 +98,7 @@ def __init__(
         self,
         tracer_provider: TracerProvider | None = None,
         meter_provider: MeterProvider | None = None,
+        logger_provider: LoggerProvider | None = None,
     ):
         self._tracer = get_tracer(
             __name__,
@@ -106,6 +109,12 @@ def __init__(
         self._metrics_recorder: InvocationMetricsRecorder | None = None
         meter = get_meter(__name__, meter_provider=meter_provider)
         self._metrics_recorder = InvocationMetricsRecorder(meter)
+        self._logger = get_logger(
+            __name__,
+            __version__,
+            logger_provider,
+            schema_url=Schemas.V1_37_0.value,
+        )
 
     def _record_llm_metrics(
         self,
@@ -129,7 +138,7 @@ def start_llm(
         """Start an LLM invocation and create a pending span entry."""
         # Create a span and attach it as current; keep the token to detach later
         span = self._tracer.start_span(
-            name=f"{GenAI.GenAiOperationNameValues.CHAT.value} {invocation.request_model}",
+            name=f"{invocation.operation_name} {invocation.request_model}",
             kind=SpanKind.CLIENT,
         )
         # Record a monotonic start timestamp (seconds) for duration
@@ -148,8 +157,9 @@ def stop_llm(self, invocation: LLMInvocation) -> LLMInvocation:  # pylint: disab
             return invocation
 
         span = invocation.span
-        _apply_finish_attributes(span, invocation)
+        _apply_llm_finish_attributes(span, invocation)
         self._record_llm_metrics(invocation, span)
+        _maybe_emit_llm_event(self._logger, span, invocation)
         # Detach context and end span
         otel_context.detach(invocation.context_token)
         span.end()
@@ -164,10 +174,11 @@ def fail_llm(  # pylint: disable=no-self-use
             return invocation
 
         span = invocation.span
-        _apply_finish_attributes(invocation.span, invocation)
-        _apply_error_attributes(span, error)
+        _apply_llm_finish_attributes(invocation.span, invocation)
+        _apply_error_attributes(invocation.span, error)
         error_type = getattr(error.type, "__qualname__", None)
         self._record_llm_metrics(invocation, span, error_type=error_type)
+        _maybe_emit_llm_event(self._logger, span, invocation, error)
         # Detach context and end span
         otel_context.detach(invocation.context_token)
         span.end()
@@ -201,6 +212,7 @@ def llm(
 def get_telemetry_handler(
     tracer_provider: TracerProvider | None = None,
     meter_provider: MeterProvider | None = None,
+    logger_provider: LoggerProvider | None = None,
 ) -> TelemetryHandler:
     """
     Returns a singleton TelemetryHandler instance.
@@ -212,6 +224,7 @@ def get_telemetry_handler(
         handler = TelemetryHandler(
             tracer_provider=tracer_provider,
             meter_provider=meter_provider,
+            logger_provider=logger_provider,
         )
         setattr(get_telemetry_handler, "_default_handler", handler)
     return handler