feat(go/plugins/anthropic): support native structured output#4701
feat(go/plugins/anthropic): support native structured output#4701hugoaguirre wants to merge 2 commits intomainfrom
Conversation
Summary of ChangesHello @hugoaguirre, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces native support for structured JSON output when interacting with Anthropic models. By leveraging Anthropic's tool use capabilities, it allows developers to specify a JSON schema for the desired output, ensuring that model responses adhere to a predefined structure. This enhancement streamlines the integration of Anthropic models into applications that require reliable and parseable data formats, improving developer experience and reducing post-processing efforts. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for native structured output in the Anthropic Go plugin. The changes look good overall. I've identified a few areas for improvement:
- Error Handling: There's a place where an error from
json.Marshalis ignored, which could lead to silent failures. - Maintainability: There's some duplicated code for checking structured output conditions and a hardcoded string that could be extracted into a constant.
- Testing: The new test for structured output could be more robust in its schema verification.
I've left specific comments with suggestions for these points.
| func handleStructuredOutput(r *ai.ModelResponse) { | ||
| for i, part := range r.Message.Content { | ||
| if part.IsToolRequest() && part.ToolRequest.Name == "return_json_output" { | ||
| // Convert input to JSON | ||
| jsonBytes, _ := json.Marshal(part.ToolRequest.Input) | ||
| r.Message.Content[i] = ai.NewTextPart(string(jsonBytes)) | ||
| r.FinishReason = ai.FinishReasonStop | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The error returned by json.Marshal on line 144 is being ignored. This could lead to silent failures where an invalid tool input results in an empty text part without any indication of an error. The function should be modified to return an error, which should then be handled in the Generate function.
For example, you could update the call sites in Generate like this:
// In Generate function (non-streaming):
if isStructured {
if err := handleStructuredOutput(r); err != nil {
return nil, err
}
}
return r, nilfunc handleStructuredOutput(r *ai.ModelResponse) error {
for i, part := range r.Message.Content {
if part.IsToolRequest() && part.ToolRequest.Name == "return_json_output" {
// Convert input to JSON
jsonBytes, err := json.Marshal(part.ToolRequest.Input)
if err != nil {
return fmt.Errorf("failed to marshal structured output: %w", err)
}
r.Message.Content[i] = ai.NewTextPart(string(jsonBytes))
r.FinishReason = ai.FinishReasonStop
}
}
return nil
}|
|
||
| req.Model = anthropic.Model(model) | ||
|
|
||
| isStructured := input.Output != nil && input.Output.Format == "json" && input.Output.Schema != nil |
There was a problem hiding this comment.
This condition to check for structured output is also used in toAnthropicRequest (line 217). To avoid duplication and improve maintainability, consider extracting this logic into a helper function, for example:
func isStructuredOutput(req *ai.ModelRequest) bool {
return req.Output != nil && req.Output.Format == "json" && req.Output.Schema != nil
}You can then use isStructured := isStructuredOutput(input) here and if isStructuredOutput(i) in toAnthropicRequest.
| } | ||
| req.Tools = append(req.Tools, anthropic.ToolUnionParam{ | ||
| OfTool: &anthropic.ToolParam{ | ||
| Name: "return_json_output", |
There was a problem hiding this comment.
The string "return_json_output" is used in multiple places in this file (lines 142, 231) and in the test file. It would be better to define it as a constant to avoid typos and improve maintainability.
For example:
const structuredOutputToolName = "return_json_output"Then use this constant here and in other places where this string appears.
| // Verify schema | ||
| inputSchemaBytes, _ := json.Marshal(foundTool.InputSchema) | ||
| expectedSchemaBytes, _ := json.Marshal(schema) | ||
| if len(inputSchemaBytes) == 0 { | ||
| t.Errorf("tool input schema is empty") | ||
| } | ||
| t.Logf("Schema found: %s", string(inputSchemaBytes)) | ||
| t.Logf("Expected: %s", string(expectedSchemaBytes)) | ||
| } |
There was a problem hiding this comment.
The schema verification in this test is incomplete. It logs the schemas but doesn't actually assert that they are equal. Additionally, errors from json.Marshal are ignored.
A more robust test would be to unmarshal the schemas into comparable maps and then use reflect.DeepEqual for comparison.
// Verify schema
inputSchemaBytes, err := json.Marshal(foundTool.InputSchema)
if err != nil {
t.Fatalf("failed to marshal found tool schema: %v", err)
}
expectedSchemaBytes, err := json.Marshal(schema)
if err != nil {
t.Fatalf("failed to marshal expected schema: %v", err)
}
var inputSchemaMap, expectedSchemaMap map[string]any
if err := json.Unmarshal(inputSchemaBytes, &inputSchemaMap); err != nil {
t.Fatalf("failed to unmarshal input schema: %v", err)
}
if err := json.Unmarshal(expectedSchemaBytes, &expectedSchemaMap); err != nil {
t.Fatalf("failed to unmarshal expected schema: %v", err)
}
if !reflect.DeepEqual(inputSchemaMap, expectedSchemaMap) {
t.Errorf("schema mismatch:\ngot: %s\nwant: %s", string(inputSchemaBytes), string(expectedSchemaBytes))
}
}1 participant
Checklist (if applicable):