Add text and image document support for Ballerina OpenAI model provider#18
Merged
SasinduDilshara merged 22 commits intoballerina-platform:mainfrom Jul 26, 2025
Merged
Conversation
SasinduDilshara
requested review from
MohamedSabthar and
shafreenAnfar
as code owners
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #18 +/- ##
==========================================
+ Coverage 30.46% 35.74% +5.27%
==========================================
Files 5 5
Lines 407 442 +35
Branches 100 112 +12
==========================================
+ Hits 124 158 +34
- Misses 283 284 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for text and image documents in the Ballerina OpenAI model provider, enabling multimodal AI interactions. It refactors the prompt processing to use structured content parts instead of simple strings and adds comprehensive test coverage for the new document types.
- Adds support for
ai:ImageDocumentandai:ImageDocument[]types with binary data and URL content - Implements validation for image URLs and proper base64 encoding for binary image data
- Refactors prompt processing from string concatenation to structured content parts array
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| ballerina/types.bal | Adds new GPT model variant for audio preview support |
| ballerina/provider_utils.bal | Core implementation of document processing with image support and URL validation |
| ballerina/tests/tests.bal | Comprehensive test suite for image document functionality and error handling |
| ballerina/tests/test_values.bal | Test data and expected content parts for multimodal document tests |
| ballerina/tests/test_utils.bal | Updated mock response handlers to support new document content structure |
| ballerina/tests/test_services.bal | Modified test service to handle structured content parts instead of strings |
| ballerina/Dependencies.toml | Updated dependencies for constraint validation and array utilities |
| } else if doc is ai:ImageDocument { | ||
| return contentParts.push(check buildImageContentPart(doc)); | ||
| } | ||
|
|
| if content.length() == 0 { | ||
| return; | ||
| } | ||
|
|
Contributor
Author
|
Will fix the comments with the next PR(PR for labels) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add text and image document support for Ballerina OpenAI model provider
Related to ballerina-platform/ballerina-library#7868