Skip to content

Add Google Vertex AI skill for video and image generation#10

Draft
Copilot wants to merge 9 commits intomainfrom
copilot/add-google-vertex-ai-skill
Draft

Add Google Vertex AI skill for video and image generation#10
Copilot wants to merge 9 commits intomainfrom
copilot/add-google-vertex-ai-skill

Conversation

Copy link
Contributor

Copilot AI commented Dec 24, 2025

Implements a PromptWar̊e ØS skill enabling agents to generate videos (Veo 3.1) and images (Imagen) via Google Cloud Vertex AI REST API.

Implementation

Core Tool (vertex-ai.ts, 471 lines)

  • Three commands: generate-video, generate-image, check-auth
  • ADC authentication via gcloud CLI (service account JWT not implemented)
  • Input validation: 1-10 images per request, integer-only counts
  • Type-safe API responses: VideoPrediction, ImagePrediction interfaces
  • Model constants: MODEL_VEO_3_1, MODEL_IMAGEN, MAX_NUM_IMAGES

Architecture

  • Zero-footprint: remote execution via deno run <url>
  • JSR imports: jsr:@std/cli/parse-args
  • Stateless, microservice-based per PromptWar̊e standards
  • Error messages include troubleshooting guidance

Documentation (1,062 lines)

  • SKILL.md: Skill spec with usage patterns
  • EXAMPLES.md: Prompt engineering, cost management, troubleshooting
  • INTEGRATION.md: Agent workflow patterns, multi-step generation
  • CHANGELOG.md: Version history, roadmap

Usage

# Generate video
deno run --allow-net --allow-env --allow-read \
  https://raw.githubusercontent.com/.../vertex-ai.ts \
  generate-video \
  --prompt "Robot in forest" \
  --project "my-project" \
  --duration "8s"

# Batch image generation
vertex-ai generate-image \
  --prompt "Futuristic car" \
  --num-images 4 \
  --style "photorealistic"

Known Limitations

  • Service account JWT authentication not implemented (requires gcloud CLI)
  • Maximum 10 images per request
  • Video generation is asynchronous (job ID returned for status polling)
Original prompt

Now let's design a new skill that will cover the Google Vertex AI Module API. The goal of this skill is to be able to use the Google Cloud Vertex AI and leverage all the AI modules from it and provide a powerful skill for our AI agent.

First version, let's target to the most powerful veo 3.1 and Nano Banana Pro.

The UX is like when an agent enables this skill, and the user says something like "Generate a video", "Generate an image", or "Design an image". This skill will be activated, and in this skill, it should have a tool which is a Deno script to convert the Google Cloud Vertical AI module's RESTful API calls to map them to the tool which can be used by the skill.
And for the authentication part, if the skill identified that the current system has not authenticated, it can require users to enable application default authentication or providing users an OAuth login URL with the additional information that users should follow.

Based on this design concept and direction, follow it to design the framework, I mean the most important skeleton to this skill as the first developing version, and let's prepare to continue developing and keep improving.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 8 commits December 24, 2025 23:21
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Co-authored-by: huan <1361891+huan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add Google Vertex AI Module skill for video and image generation Add Google Vertex AI skill for video and image generation Dec 24, 2025
Copilot AI requested a review from huan December 24, 2025 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants