Added invoke workflow operation name to agent span#3249
Added invoke workflow operation name to agent span#3249wrisa wants to merge 21 commits intoopen-telemetry:mainfrom
Conversation
zhirafovod
left a comment
There was a problem hiding this comment.
LGTM with some conditional notes left on the PR.
|
@wrisa Is there any active discussions or doc about conditional nodes and how they can be represented, especially with gen ai workflows. For example conditional edges in langgraph. |
@singankit Not exactly, but we briefly presented step for intermediate steps like langchain's chains or CrewAi's tasks. According to SIG discussions, it seemed expensive as all the info could be captured on agent span as well and also it seemed more framework specific. If we can have more use cases like you suggested we can start the discussion maybe again. |
| This span SHOULD be reported by the instrumentations when they can | ||
| reliably determine that invocation is a workflow (i.e. groups several agent | ||
| invocations) and SHOULD NOT be reported by instrumentations that | ||
| can't distinguish it `invoke_workflow` from `invoke_agent`. |
There was a problem hiding this comment.
How would you recommend instrumenting ADK?
There was a problem hiding this comment.
@lmolkova @aabmass ADK has workflow agents which can be a differentiator to invoke_agent.
There was a problem hiding this comment.
I think we can actually tell if it's a workflow agent and do this, but it seems duplicative to have both.
Have you tried out an ADK agent with this @wrisa? It actually has native instrumentation already and I believe it would output invoke_agent with the workflow name.
There was a problem hiding this comment.
So I tried it out in ADK to see what happens. With this agent
from google.adk.agents import LlmAgent, SequentialAgent
hello_agent = LlmAgent(
model="gemini-2.5-flash",
name="hello_agent",
description="A helpful assistant for user questions.",
instruction="Your job is to creatively say hello",
)
bye_agent = LlmAgent(
model="gemini-2.5-flash",
name="bye_agent",
description="A helpful assistant to say bye.",
instruction="Your job is to creatively reply to the greeting and reply with a quip",
)
root_agent = SequentialAgent(
name="root_agent",
description="An agent that says hello and then says goodbye.",
sub_agents=[hello_agent, bye_agent],
)You get this span tree, which is noticeably missing any info about the kind of workflow (parallel, sequential, loop).
I'm trying to get someone from ADK to look at this PR, but maybe we can do one of two things:
- Create an extra workflow span under the workflow agent. A little unnecessary in terms of timings
- Make the workflow span attributes available on the invoke_agent, or say they are "compatible" similar to what we did for MCP tool calls.
There was a problem hiding this comment.
- would it be fair to ask, ADK instrumentation to adapt to this new operation
invoke_workflowand not produce span withinvoke_agentfor workflow agents likeSequentialAgent/ParallelAgent/LoopAgent. Maybe becauseinvoke_workflowoperation name did not exist earlier and so adk is usinginvoke_agent?
Also if you see the tree above, the root agent is not capturing input messages from the user and output messages after finishing the workflow which can be captured for span with operation name 'invoke_workflow' gen_ai.workflow.typeattribute can be added to record sequential/parallel/loop.
There was a problem hiding this comment.
My 2 cents from ADK perspective:
would it be fair to ask, ADK instrumentation to adapt to this new operation invoke_workflow and not produce span with invoke_agent for workflow agents like SequentialAgent/ParallelAgent/LoopAgent. Maybe because invoke_workflow operation name did not exist earlier and so adk is using invoke_agent ?
I'd be a little hesitant to do that because in ADK those workflow agents are still modeled as agents. So having different span types for different types of agents could be confusing.
gen_ai.workflow.type attribute can be added to record sequential/parallel/loop.
This makes sense as an optional attribute to be used only in case of workflow-like agents.
There was a problem hiding this comment.
in ADK those workflow agents are still modeled as agents
@jawoszek thanks for looking into this. Do workflow agents use LLM internally ? If they are then it makes sense. But if they are not using LLM would there still be any issue to have workflow span for them ?
| This span SHOULD be reported by the instrumentations when they can | ||
| reliably determine that invocation is a workflow (i.e. groups several agent | ||
| invocations) and SHOULD NOT be reported by instrumentations that | ||
| can't distinguish it `invoke_workflow` from `invoke_agent`. |
Fixes #
Changes
Workflow proposal https://github.com/open-telemetry/semantic-conventions/blob/b6bfc44631e28e87a0252011a8898e8098cb0ff9/docs/gen-ai/Session-and-Workflow-for-GenAI-Observability.md discussion led to suggestion of invoke workflow operation name in agent span.
Important
Pull requests acceptance are subject to the triage process as described in Issue and PR Triage Management.
PRs that do not follow the guidance above, may be automatically rejected and closed.
Merge requirement checklist
[chore]