Skip to content

ci: Add GitHub Actions workflows for automated testing and validation#80

Open
chris-schra wants to merge 7 commits intodevelopfrom
gh/workflows/ci
Open

ci: Add GitHub Actions workflows for automated testing and validation#80
chris-schra wants to merge 7 commits intodevelopfrom
gh/workflows/ci

Conversation

@chris-schra
Copy link
Owner

@chris-schra chris-schra commented Sep 30, 2025

Summary

Adds comprehensive CI/CD workflows for automated testing and validation on pull requests and pushes.

New Workflows

1. Test Workflow (.github/workflows/test.yml)

  • Purpose: Run test suite with coverage reporting
  • Triggers: Pull requests and pushes to main/develop
  • Features:
    • Uses Vitest's GitHub Actions reporter for inline test failure annotations
    • Generates coverage report on PRs using vitest-coverage-report-action
    • Shows test results and coverage metrics directly in PR view
    • Runs yarn test with GitHub Actions reporter integration

2. Validate Workflow (.github/workflows/validate.yml)

  • Purpose: Run TypeScript/ESLint/Prettier validation
  • Triggers: Pull requests and pushes to main/develop
  • Features:
    • Uses new --github-actions flag for PR annotations
    • Shows validation errors inline in PR file view
    • Reports TypeScript errors, ESLint violations, and formatting issues
    • Runs yarn validate --github-actions

Enhanced ts-validate Command

New --github-actions flag:

  • Outputs validation errors in GitHub Actions annotation format
  • Supports three severity levels:
    • ::error - TypeScript errors, ESLint errors
    • ::warning - ESLint warnings
    • ::notice - Info-level messages (like auto-formatting)
  • Reports tool failures/skips as warnings for visibility
  • Proper exit codes (1 for failures, 0 for success)

Example annotation output:

::error file=packages/auth/src/foo.ts,line=42,col=10,title=typescript(TS2304)::Cannot find name 'Foo'
::warning file=packages/core/src/bar.ts,line=15,col=5,title=eslint(max-lines)::File has too many lines
::notice file=test.ts,line=1,col=1,title=prettier(no-rule)::File was automatically formatted

Workflow Benefits

Parallel Execution:

  • Both workflows run concurrently as separate checks
  • Faster feedback - test and validation results appear independently
  • Clear separation of concerns in PR status checks

PR Integration:

  • ✅ Test results appear inline at failing test locations
  • ✅ Validation errors appear inline at error locations
  • ✅ Coverage metrics displayed in PR comments
  • ✅ Individual status checks for test and validate

Developer Experience:

  • See exactly where tests fail without leaving GitHub
  • See exactly which lines have validation errors
  • Coverage trends visible on every PR
  • No need to run CI locally to understand failures

Configuration Details

Both workflows:

  • Use Node.js 20 with yarn caching for faster runs
  • Set proper permissions for PR comments/checks
  • Fail the build if issues are found
  • Run on both pull_request and push events for main/develop

Testing

Workflows have been validated:

  • ✅ Workflow syntax is valid (passes GitHub Actions linter)
  • ✅ Test workflow uses correct vitest reporters
  • ✅ Validate workflow uses new --github-actions flag
  • ✅ Permissions are properly scoped
  • ✅ Exit codes propagate correctly

Breaking Changes

None - This only adds CI workflows and a new optional flag to ts-validate.

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

chris-schra and others added 2 commits September 30, 2025 23:27
@chris-schra @claude
feat(ts-validate): Add GitHub Actions annotation support
2d1a427 
Add --github-actions flag to output validation results in GitHub Actions
workflow command format. Implements complete annotation support with:

- Severity mapping: error → ::error, warning → ::warning, info → ::notice
- File paths relative to process.cwd() for GitHub Actions
- Tool status reporting for failed/skipped tools as ::warning
- Title format: tool(ruleId) for easy identification
- Exit code 1 for validation failures or tool failures

Implementation:
- Extracted formatter to util/github-actions-formatter.ts for maintainability
- Integrated into CLI flow with proper exit code handling
- Uses console.info for output (linter-approved)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@chris-schra @claude
ci: add GitHub Actions workflows for test and validation
0832660 
Added two parallel CI workflows:

1. **test.yml** - Runs test suite with coverage reporting
   - Uses vitest github-actions reporter for inline annotations
   - Generates coverage report on PRs using vitest-coverage-report-action
   - Runs on both push and pull_request events

2. **validate.yml** - Runs TypeScript/ESLint/Prettier validation
   - Uses new --github-actions flag for PR annotations
   - Shows validation errors inline in PR file view
   - Runs on both push and pull_request events

Both workflows:
- Run in parallel as separate checks in PR view
- Use Node.js 20 with yarn caching
- Set proper permissions for PR comments/checks
- Fail the build if issues are found

**Enhanced ts-validate command:**
- Added --github-actions flag for CI integration
- Outputs errors in GitHub Actions annotation format
- Supports ::error, ::warning, and ::notice severity levels
- Reports tool failures/skips as warnings
- Proper exit codes (1 for failures, 0 for success)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@chris-schra
fix(ci): fix test failures in GitHub Actions
885bfb7 
- Add Playwright chromium installation for browser tests
- Fix file:// URL handling in session-manager.test.ts for CI
- Use 'npx tsx' instead of 'tsx' for CI compatibility

Tests now work in both local and CI environments.
chris-schra and others added 2 commits September 30, 2025 23:58
@chris-schra @claude
perf(ci): add Playwright browser caching
88b55fc 
- Cache Playwright browsers using actions/cache
- Key cache by Playwright version for consistency
- Only download browsers on cache miss
- Install system dependencies separately when using cache
- Significantly reduces CI time by avoiding repeated chromium downloads

Typical savings: 30-60 seconds per CI run

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@chris-schra
fix(ci): fix more tests
e476cd5
@chris-schra chris-schra force-pushed the gh/workflows/ci branch 2 times, most recently from a11b12d to 4749592 Compare September 30, 2025 22:29
@chris-schra @claude
fix(test): wait for WebSocket connection before evaluating
5d91406 
The test was failing because it tried to evaluate immediately after
session creation, before the WebSocket connection was fully established.

Fixed by:
- Add waitForPause(5000) after session creation to ensure connection ready
- Add try-finally for proper cleanup
- Keep reasonable timeouts (no need for excessive waits)

This ensures the WebSocket is connected before we try to use it,
preventing the 'WebSocket connection closed' race condition.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@chris-schra chris-schra force-pushed the gh/workflows/ci branch 2 times, most recently from 512cb71 to 5599bc1 Compare September 30, 2025 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@chris-schra