Skip to content

Add Google Vertex AI skill for video and image generation #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 9 commits into
base: main
Choose a base branch
from
+1,533 −0
Draft

Add Google Vertex AI skill for video and image generation #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a453950
Initial plan
Copilot Dec 24, 2025
5eff46d
Add Google Vertex AI skill with video and image generation support
Copilot Dec 24, 2025
f33399b
Add examples and tests for Vertex AI skill
Copilot Dec 24, 2025
92c04db
Add integration guide and changelog for Vertex AI skill
Copilot Dec 24, 2025
e4dfb9b
Fix code review issues: improve error handling and use model constants
Copilot Dec 24, 2025
8a18b33
Improve validation, type safety, and service account error handling
Copilot Dec 24, 2025
42b9ddd
Remove duplicate interface and simplify validation logic
Copilot Dec 24, 2025
6cb4278
Document service account authentication limitation prominently
Copilot Dec 24, 2025
01e6d2f
Improve error logging and remove redundant model parameter
Copilot Dec 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions os/skills/vertex-ai/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Changelog - Google Vertex AI Skill

All notable changes to the Vertex AI skill will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.0] - 2025-12-24

### Added
- Initial release of Google Vertex AI skill for PromptWar̊e ØS
- Video generation support using Veo 3.1 model
- Text-to-video generation
- Duration control (customizable video length)
- Style customization (cinematic, vibrant, etc.)
- Aspect ratio support (16:9, 9:16, 1:1)
- Image generation support using Imagen model
- Text-to-image generation
- Style options (photorealistic, artistic, product-design)
- Resolution control
- Multiple image generation (batch processing)
- Authentication handling
- Application Default Credentials (ADC) detection
- Service account support via GOOGLE_APPLICATION_CREDENTIALS
- gcloud CLI integration
- OAuth 2.0 guidance and setup instructions
- Error handling and user guidance
- Clear error messages with troubleshooting steps
- Authentication status checking
- API error interpretation
- Permission and quota guidance
- Command-line tool (`vertex-ai.ts`)
- Standard `--help` flag support
- Three main commands: `generate-video`, `generate-image`, `check-auth`
- JSR-based imports (jsr:@std/cli/parse-args)
- Remote execution support (no download required)
- Documentation
- SKILL.md: Complete skill specification
- README.md: Quick start guide
- EXAMPLES.md: Comprehensive examples and best practices
- INTEGRATION.md: Agent integration patterns
- CHANGELOG.md: Version history
- Testing
- vertex-ai.test.ts: Unit tests and behavior documentation
- Structure validation tests
- Help message format verification

### Technical Details
- Language: TypeScript
- Runtime: Deno
- Architecture: Stateless, microservice-based
- API Integration: Google Cloud Vertex AI REST API
- Authentication: Google Cloud OAuth 2.0 / ADC
- License: Public Prompt License - Apache Variant (PPL-A)

### API Endpoints
- Video: `publishers/google/models/veo-3.1:predict`
- Image: `publishers/google/models/imagegeneration:predict`

### Supported Models
- Veo 3.1 (Video generation)
- Imagen (Image generation)

### Dependencies
- jsr:@std/cli@^1.0.0 (parseArgs)
- Deno standard library
- Google Cloud Vertex AI API

### Known Limitations
- Requires active Google Cloud project with billing
- Subject to Vertex AI API quotas
- Video generation is asynchronous and may take several minutes
- Regional availability varies by model
- **Service account JWT authentication not fully implemented** (uses gcloud CLI as fallback)
- Maximum 10 images per generation request

### Future Roadmap
See INTEGRATION.md for planned enhancements:
- Additional model support
- Advanced parameter controls
- Automatic status polling
- Integrated file management
- Prompt templates library
- Cost estimation features

## [Unreleased]

### Planned for 0.2.0
- Full service account JWT authentication implementation
- Job status polling with automatic completion detection
- File download and storage integration
- Enhanced prompt templates
- Cost estimation before generation
- Support for additional Imagen model variants

### Under Consideration
- Video editing capabilities (trim, merge)
- Image post-processing (resize, filter)
- Prompt suggestion engine
- Result caching mechanism
- Multi-region failover
- Quota management tools

---

## Version History

- **0.1.0** (2025-12-24): Initial release with video and image generation

---

## Contributing

When contributing to this skill:

1. Update this CHANGELOG with your changes
2. Follow [Semantic Versioning](https://semver.org/)
3. Add entries under [Unreleased] until release
4. Move changes to a new version section on release
5. Update version in SKILL.md frontmatter

### Changelog Categories

- **Added**: New features
- **Changed**: Changes in existing functionality
- **Deprecated**: Soon-to-be removed features
- **Removed**: Removed features
- **Fixed**: Bug fixes
- **Security**: Security vulnerability fixes

---

## License

Copyright (c) 2025 Ship.Fail
Licensed under the Public Prompt License - Apache Variant (PPL-A)

See the LICENSE file in the repository root for details.
158 changes: 158 additions & 0 deletions os/skills/vertex-ai/EXAMPLES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Google Vertex AI Configuration Examples

This file contains example configurations for using the Vertex AI skill.

## Environment Variables

```bash
# Set your Google Cloud project ID
export GCP_PROJECT_ID="your-project-id"

# Set default location/region
export GCP_LOCATION="us-central1"

# For service account authentication
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
```

## Example Prompts

### Video Generation Examples

#### Example 1: Nature Scene
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-video \
--prompt "A serene sunset over snow-capped mountains with a lake reflecting the golden sky" \
--project "$GCP_PROJECT_ID" \
--duration "8s" \
--style "cinematic"
```

#### Example 2: Technology Demo
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-video \
--prompt "A humanoid robot assembling electronic components in a modern factory" \
--project "$GCP_PROJECT_ID" \
--duration "10s" \
--aspect-ratio "16:9"
```

#### Example 3: Urban Scene
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-video \
--prompt "Time-lapse of a busy city street at night with neon lights and traffic" \
--project "$GCP_PROJECT_ID" \
--duration "5s" \
--style "vibrant"
```

### Image Generation Examples

#### Example 1: Photorealistic
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-image \
--prompt "A photorealistic portrait of a person in a futuristic spacesuit on Mars" \
--project "$GCP_PROJECT_ID" \
--style "photorealistic" \
--resolution "1024x1024"
```

#### Example 2: Artistic Style
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-image \
--prompt "An abstract painting of a city skyline at sunset in vibrant colors" \
--project "$GCP_PROJECT_ID" \
--style "artistic" \
--num-images 4
```

#### Example 3: Product Design
```bash
deno run --allow-net --allow-env --allow-read vertex-ai.ts generate-image \
--prompt "A sleek minimalist smartwatch with a transparent display" \
--project "$GCP_PROJECT_ID" \
--style "product-design" \
--resolution "512x512"
```

## Common Use Cases

### 1. Marketing Content
Generate promotional videos and images for products or services.

### 2. Concept Visualization
Create visual representations of ideas, concepts, or designs.

### 3. Storyboarding
Generate scenes for video storyboards or presentations.

### 4. Social Media Content
Create engaging visual content for social media platforms.

### 5. Prototyping
Quickly visualize design concepts without manual creation.

## Best Practices

### Prompt Engineering

1. **Be Specific**: Include details about subject, setting, style, and mood
- ❌ "A car"
- ✅ "A sleek red sports car driving on a coastal highway at sunset"

2. **Use Descriptive Language**: Add adjectives and context
- ❌ "A building"
- ✅ "A modern glass skyscraper with curved architecture reflecting clouds"

3. **Specify Style**: Mention the desired artistic or visual style
- Examples: "photorealistic", "cinematic", "artistic", "minimalist"

4. **Include Composition Details**: Describe camera angles, lighting, etc.
- "viewed from above", "soft lighting", "wide angle shot"

### Performance Optimization

1. **Start Small**: Begin with shorter videos and lower resolutions to test
2. **Choose Appropriate Regions**: Use regions closest to your location
3. **Batch Requests**: Group similar generation requests together
4. **Monitor Quotas**: Keep track of API usage and quotas

### Cost Management

1. **Use Development Projects**: Test with separate GCP projects
2. **Set Budget Alerts**: Configure billing alerts in Google Cloud Console
3. **Cache Results**: Save generated content to avoid regeneration
4. **Optimize Prompts**: Refine prompts to reduce iteration count

## Troubleshooting

### Authentication Issues

**Problem**: "No authentication token available"
**Solution**: Run `gcloud auth application-default login` or set `GOOGLE_APPLICATION_CREDENTIALS`

**Problem**: "Permission denied"
**Solution**: Ensure your account has the `aiplatform.endpoints.predict` permission

### API Errors

**Problem**: "API not enabled"
**Solution**: Enable Vertex AI API at https://console.cloud.google.com/apis/library/aiplatform.googleapis.com

**Problem**: "Quota exceeded"
**Solution**: Check your quota limits in the GCP Console and request increases if needed

### Generation Issues

**Problem**: "Generation failed"
**Solution**: Review your prompt for policy violations or clarity issues

**Problem**: "Long generation time"
**Solution**: Video generation can take several minutes; check job status with returned job ID

## Additional Resources

- [Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs)
- [Veo Model Documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/video/overview)
- [Imagen Documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/image/overview)
- [Google Cloud Pricing](https://cloud.google.com/vertex-ai/pricing)
Loading