.NET:[Breaking] Add support for structured output

[SergeyMenshykh]

Motivation and Context

Today, ChatClientAgent has RunAsync<T> instance methods to work with structured output in AF. However, if ChatClientAgent is decorated by, let's say, an Open Telemetry decorator, the agent users lose access to the RunAsync<T> methods because they are instance methods of ChatClientAgent rather than members of the AIAgent class.

Additionally, there are no RunAsync overloads that can work with SO type represented by System.Type.

Description

This PR adds the following:

• The ChatResponse property to the AgentRunOptions to communicate the agent's SO configuration.
• RunAsync<T> and RunAsync(Type type,...) non-virtual methods to the AIAgent base abstract class.
• Removes the bool? useJsonSchemaResponseFormat = null parameter from existing RunAsync<T> signatures.
• Adds the Clone method to AgentRunOptions, ChatClientAgentRunOptions, DurableAgentOptions.

Out of Scope

The following will be added in the next PRs:

• Wrapping primitives and arrays into a JSON object schema.
• A decorator to add structured output support to agents that do not support structured output natively.
• Functionality to check if an agent supports structured output and throw an exception if it does not.

Note This PR targets feature-so feature branch.

[Copilot]

Pull request overview

This PR centralizes structured output support in the AIAgent abstraction and threads response format configuration through AgentRunOptions, so consumers can use RunAsync<T> and RunAsync(Type, ...) even when agents are wrapped or decorated. It also adds conformance/integration tests across multiple backends (Azure AI Agents, AI Projects, OpenAI chat/completions/assistants) to verify consistent structured-output behavior.

Changes:

• Add AgentRunOptions.ResponseFormat and a new NewChatResponseFormat/NewChatResponseFormatJson abstraction to describe structured JSON output, plus update AgentResponse<T> to deserialize structured responses based on that configuration.
• Introduce non-virtual AIAgent.RunAsync<T> and AIAgent.RunAsync(Type, ...) overloads that set up JSON-schema response formats and return AgentResponse<T>, with ChatClientAgent and Durable agents wired to honor response formats from both initialization and per-run options.
• Add shared conformance tests (StructuredOutputRunTests<TAgentFixture>) and concrete integration tests for Azure AI Agents (including persistent) and OpenAI backends, along with new unit tests verifying ChatClientAgent structured-output behavior and propagation/overrides of response formats.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseStructuredOutputRunTests.cs	Adds conformance-based structured-output tests for an OpenAIResponse-backed agent using the shared StructuredOutputRunTests harness.
dotnet/tests/OpenAIChatCompletion.IntegrationTests/OpenAIChatCompletionStructuredOutputRunTests.cs	Adds structured-output conformance tests for an OpenAI Chat Completions agent, opting out of reasoning model usage.
dotnet/tests/OpenAIAssistant.IntegrationTests/OpenAIAssistantStructuredOutputRunTests.cs	Adds structured-output conformance tests for an OpenAI Assistants-based agent fixture.
dotnet/tests/Microsoft.Agents.AI.UnitTests/Models/Animal.cs	Introduces a reusable Animal/Species model used across new structured-output unit tests.
dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgent_SO_WithRunAsyncTests.cs	Adds unit tests verifying that ChatClientAgent.RunAsync<T> and RunAsync(Type, ...) set JSON-schema response formats and correctly deserialize structured output.
dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgent_SO_WithFormatResponseTests.cs	Adds tests ensuring ChatClientAgent respects response formats from agent initialization, per-run options, and precedence between ChatOptions and AgentRunOptions.ResponseFormat, and that structured JSON remains available via AgentResponse.Text.
dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgentTests.cs	Removes the old inlined Animal/Species and previous structured-output test in favor of the new dedicated SO tests.
dotnet/tests/AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentStructuredOutputRunTests.cs	Plugs the Azure AI Agents persistent fixture into the shared StructuredOutputRunTests suite.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientFixture.cs	Adds overloads to initialize ChatClientAgent with ChatClientAgentOptions and to initialize the fixture from those options, enabling structured-output configuration at agent creation time.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientAgentStructuredOutputRunTests.cs	Adds AI Project client tests for structured output (both via AgentRunOptions.ResponseFormat and generic RunAsync<T>), and overrides/skips base tests where the backend cannot yet take a type at invocation time.
dotnet/tests/AgentConformance.IntegrationTests/Support/SessionCleanup.cs	Makes SessionCleanup public so it can be reused across integration test projects for consistent session cleanup.
dotnet/tests/AgentConformance.IntegrationTests/Support/Constants.cs	Makes retry Constants public for reuse in other integration tests.
dotnet/tests/AgentConformance.IntegrationTests/StructuredOutputRunTests.cs	Introduces a shared conformance test base class for structured-output runs and a simple CityInfo DTO, driving consistent tests across different agent implementations.
dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgentStructuredOutput.cs	Removes the ChatClientAgent-specific structured-output implementation in favor of the new generic AIAgent structured-output APIs.
dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgentRunResponse{T}.cs	Removes the ChatClientAgentResponse<T> wrapper class now that AgentResponse<T> handles structured deserialization generically.
dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgentCustomOptions.cs	Updates ChatClientAgent extension overloads to return AgentResponse<T> and to bridge ChatClientAgentRunOptions into the base AIAgent structured-output APIs, and adds non-generic RunAsync(Type, ...) overloads wired through AgentRunOptions.
dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs	Wires AgentRunOptions.ResponseFormat into ChatOptions.ResponseFormat in CreateConfiguredChatOptions, and keeps ChatClientAgentRunOptions.ChatOptions.ResponseFormat as a lower-precedence default that can be overridden at the AgentRunOptions level.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableAgentRunOptions.cs	Simplifies durable run options by removing its own ResponseFormat in favor of the new base AgentRunOptions.ResponseFormat while preserving durable-specific knobs (EnableToolCalls, EnableToolNames, IsFireAndForget).
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableAIAgentProxy.cs	Adjusts the durable proxy to derive responseFormat either from ChatClientAgentRunOptions.ChatOptions.ResponseFormat or from AgentRunOptions.ResponseFormat, with the latter taking precedence.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableAIAgent.cs	Similarly updates the durable orchestration agent to compute responseFormat from chat-client options or the new AgentRunOptions.ResponseFormat, giving the base option precedence.
dotnet/src/Microsoft.Agents.AI.Abstractions/MEAI/NewChatResponseFormatJson.cs	Adds NewChatResponseFormatJson, capturing schema metadata (schema element, type, name/description, serializer options) and providing implicit conversion to ChatResponseFormatJson.
dotnet/src/Microsoft.Agents.AI.Abstractions/MEAI/NewChatResponseFormat.cs	Adds NewChatResponseFormat as a polymorphic base with helpers to build JSON-schema formats from a JsonElement, a Type, or T, using AIJsonUtilities.CreateJsonSchema.
dotnet/src/Microsoft.Agents.AI.Abstractions/AgentRunOptions.cs	Extends agent run options with a ChatResponseFormat? ResponseFormat property and ensures it is copied in the copy constructor so per-run format requests can be cloned and forwarded to implementations.
dotnet/src/Microsoft.Agents.AI.Abstractions/AgentResponse{T}.cs	Reworks AgentResponse<T> into a concrete class that performs JSON deserialization of Text into T, honoring an optional NewChatResponseFormatJson (type and serializer options) and handling the “multiple top-level JSON objects” quirk on newer .NET targets.
dotnet/src/Microsoft.Agents.AI.Abstractions/AgentResponse.cs	Adds a copy constructor from another AgentResponse, used to wrap an existing response when layering structured-deserialization semantics.
dotnet/src/Microsoft.Agents.AI.Abstractions/AIAgentStructuredOutput.cs	Introduces non-virtual AIAgent.RunAsync<T> and RunAsync(Type, ...) overloads that apply NewChatResponseFormat.ForJsonSchema(...), set AgentRunOptions.ResponseFormat, invoke the existing RunAsync pipeline, and wrap the result into AgentResponse<T>.
dotnet/src/Microsoft.Agents.AI.Abstractions/AIAgent.cs	Marks AIAgent as partial so the new structured-output partial can extend it without changing the existing core run/streaming APIs.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.