Skip to content

Latest commit

 

History

History
460 lines (353 loc) · 10.9 KB

TESTING_GUIDE.md

File metadata and controls

460 lines (353 loc) · 10.9 KB

Testing Guide 🧪

This guide covers the comprehensive testing infrastructure implemented in Phase 5: Testing & Validation, including enhanced integration tests, performance benchmarks, chaos testing, and multi-environment validation.

Overview

The testing suite provides multiple layers of validation:

  • Unit Tests - Individual component testing with Jest
  • Integration Tests - Authentication and workflow testing
  • End-to-End Tests - Complete workflow validation
  • Performance Tests - Benchmarking and performance monitoring
  • Chaos Tests - Resilience and error handling validation
  • Multi-Environment Tests - Cross-project and configuration validation

Test Structure

tests/
├── unit/                    # Unit tests with Jest
│   ├── validation.test.ts   # Schema validation tests
│   └── *.test.ts           # Component-specific tests
├── integration/             # Integration tests
│   ├── auth-methods.test.ts # Authentication method testing
│   └── e2e-workflow.test.ts # End-to-end workflow tests
├── performance/             # Performance benchmarks
│   └── benchmark.test.ts    # Performance testing suite
├── chaos/                   # Chaos testing
│   └── resilience.test.ts   # Resilience and error handling
├── validation/              # Multi-environment validation
│   └── multi-environment.test.ts
├── fixtures/                # Test fixtures and mock data
└── setup.ts                # Global test setup

Running Tests

Quick Test Commands

# Run all enhanced tests
npm run test:all-enhanced

# Run specific test suites
npm run test:auth          # Authentication tests
npm run test:e2e           # End-to-end workflow tests
npm run test:performance   # Performance benchmarks
npm run test:chaos         # Chaos/resilience tests

# Traditional Jest tests
npm test                   # All Jest tests
npm run test:unit          # Unit tests only
npm run test:integration   # Integration tests (Mocha)
npm run test:coverage      # With coverage reporting
npm run test:watch         # Watch mode

Advanced Test Commands

# Coverage with thresholds
npm run test:coverage:check

# Type checking
npm run type-check

# Validation tests
npm run validate:examples
npm run docs:test-links

Test Categories

1. Authentication Tests (test:auth)

For detailed information on authentication methods and testing, refer to the Authentication Implementation Guide.

2. End-to-End Workflow Tests (test:e2e)

Tests complete operational workflows:

Complete Cluster Lifecycle

  • Cluster creation workflow
  • Status monitoring
  • Lifecycle management

Hive Query Execution Workflow

  • Query submission
  • Job monitoring
  • Result retrieval

Parameter Validation Workflow

  • Input validation
  • Error handling
  • Validation feedback

Multi-Environment Configuration

  • Environment-specific configurations
  • Cross-environment compatibility
  • Configuration validation

Error Handling and Resilience

  • Invalid input handling
  • Network error simulation
  • Recovery mechanisms

Performance and Timeout Handling

  • Concurrent operation testing
  • Timeout validation
  • Performance thresholds

3. Performance Benchmarks (test:performance)

Measures and validates performance metrics:

Schema Validation Performance

  • Target: <5ms average, >1000 ops/sec
  • Memory impact: <1MB
  • Validation accuracy

Parameter Injection Performance

  • Target: <2ms average, >2000 ops/sec
  • Memory impact: <512KB
  • Injection accuracy

Credential Validation Performance

  • Target: <50ms average, >100 ops/sec
  • Memory impact: <2MB
  • Validation thoroughness

MCP Tool Call Performance

  • Target: <100ms average, >50 ops/sec
  • Memory impact: <5MB
  • End-to-end latency

Concurrent Operations Performance

  • Parallel processing capability
  • Resource utilization
  • Scalability metrics

4. Chaos Testing (test:chaos)

Tests system resilience under adverse conditions:

Network Failures

  • Connection timeouts
  • Intermittent connectivity
  • DNS resolution failures

Authentication Failures

  • Invalid credentials
  • Expired tokens
  • Permission denials

Resource Exhaustion

  • Memory pressure
  • CPU saturation
  • Disk space limitations

Configuration Errors

  • Invalid configurations
  • Missing parameters
  • Malformed data

External Service Failures

  • Google Cloud API failures
  • Service unavailability
  • Rate limiting

5. Multi-Environment Validation

Tests across different deployment scenarios:

Environment Configurations

  • Development environment
  • Staging environment
  • Production environment

Cross-Project Testing

  • Different GCP projects
  • Project-specific configurations
  • Resource isolation

Service Account Validation

  • Environment-specific service accounts
  • Permission validation
  • Security compliance

Cluster Configuration Testing

  • Environment-appropriate machine types
  • Resource allocation validation
  • Cost optimization verification

Performance Thresholds

Response Time Targets

Operation Target (ms) Threshold (ms)
Schema Validation 2 5
Parameter Injection 1 2
Credential Validation 25 50
MCP Tool Call 50 100
Concurrent Operations 75 150

Throughput Targets

Operation Target (ops/sec) Minimum (ops/sec)
Schema Validation 2000 1000
Parameter Injection 5000 2000
Credential Validation 200 100
MCP Tool Call 100 50

Memory Usage Limits

Operation Target (MB) Maximum (MB)
Schema Validation 0.5 1
Parameter Injection 0.25 0.5
Credential Validation 1 2
MCP Tool Call 2.5 5

Test Configuration

Environment Variables

# Test environment configuration
NODE_ENV=test
GOOGLE_APPLICATION_CREDENTIALS=test-credentials.json
DATAPROC_PROJECT_ID=test-project
DATAPROC_REGION=us-central1

# Performance testing
PERFORMANCE_ITERATIONS=1000
PERFORMANCE_TIMEOUT=30000

# Chaos testing
CHAOS_FAILURE_RATE=0.1
CHAOS_RECOVERY_TIMEOUT=5000

Test Data and Fixtures

Mock Service Accounts

{
  "type": "service_account",
  "project_id": "test-project",
  "private_key_id": "test-key-id",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMOCK_KEY\n-----END PRIVATE KEY-----\n",
  "client_email": "test@test-project.iam.gserviceaccount.com",
  "client_id": "123456789"
}

Test Cluster Configurations

development:
  masterConfig:
    numInstances: 1
    machineTypeUri: e2-medium
  workerConfig:
    numInstances: 2
    machineTypeUri: e2-medium

production:
  masterConfig:
    numInstances: 3
    machineTypeUri: n1-standard-4
  workerConfig:
    numInstances: 10
    machineTypeUri: n1-standard-4

Continuous Integration Integration

GitHub Actions Integration

The test suite integrates with the CI/CD pipeline:

# .github/workflows/ci.yml
- name: Run Enhanced Tests
  run: |
    npm run test:auth
    npm run test:e2e
    npm run test:performance
    
- name: Performance Regression Check
  run: npm run test:performance
  continue-on-error: false
  
- name: Multi-Environment Validation
  run: npm run test:multi-env
  env:
    TEST_ENVIRONMENTS: "dev,staging"

Coverage Requirements

  • Unit Tests: 90% coverage minimum
  • Integration Tests: Critical path coverage
  • Performance Tests: Baseline establishment
  • Chaos Tests: Error path coverage

Troubleshooting

Common Issues

Test Timeouts

# Increase timeout for slow operations
npm run test:integration -- --timeout 60000

Memory Issues

# Run with increased memory
node --max-old-space-size=4096 tests/performance/benchmark.test.ts

Authentication Failures

# Check credentials
npm run validate
export GOOGLE_APPLICATION_CREDENTIALS=path/to/key.json

Performance Degradation

# Run performance tests in isolation
npm run test:performance
# Check system resources
top -p $(pgrep node)

Debugging Tests

Verbose Output

# Enable debug logging
DEBUG=* npm run test:auth

# Performance debugging
NODE_ENV=development npm run test:performance

Test Isolation

# Run single test file
node tests/integration/auth-methods.test.ts

# Run specific test category
npm run test:auth

Best Practices

Writing Tests

  1. Use Descriptive Names

    await runner.runTest('Service Account Key File Authentication - Valid', async () => {
  // Test implementation
});

  2. Proper Setup and Cleanup

    beforeEach(() => {
  // Save original state
});

afterEach(() => {
  // Restore original state
});

  3. Meaningful Assertions

    assert(result.isValid, 'Service account key should be valid');
assertEqual(result.projectId, 'expected-project', 'Project ID should match');

  4. Error Testing

    try {
  await functionThatShouldFail();
  throw new Error('Should have failed');
} catch (error) {
  assert(error.message.includes('expected error'));
}

Performance Testing

  1. Establish Baselines

    • Run performance tests on clean system
    • Document baseline metrics
    • Set realistic thresholds

  2. Monitor Trends

    • Track performance over time
    • Identify regressions early
    • Optimize critical paths

  3. Resource Monitoring

    • Monitor memory usage
    • Track CPU utilization
    • Watch for memory leaks

Chaos Testing

  1. Gradual Failure Introduction

    • Start with simple failures
    • Increase complexity gradually
    • Test recovery mechanisms

  2. Real-World Scenarios

    • Simulate actual failure modes
    • Test with realistic data volumes
    • Include network conditions

  3. Recovery Validation

    • Verify graceful degradation
    • Test automatic recovery
    • Validate error reporting

Reporting and Metrics

Test Reports

Tests generate comprehensive reports:

  • Authentication Test Summary: Pass/fail rates, error details
  • Performance Benchmark Report: Metrics, thresholds, trends
  • Chaos Test Results: Failure scenarios, recovery times
  • Multi-Environment Validation: Cross-environment compatibility

Metrics Collection

Key metrics tracked:

  • Test Execution Time: Individual and suite timing
  • Performance Metrics: Response times, throughput, memory usage
  • Error Rates: Failure frequencies, error types
  • Coverage Metrics: Code coverage, test coverage

Integration with Monitoring

Test results integrate with:

  • CI/CD Pipelines: Automated test execution
  • Performance Monitoring: Baseline establishment
  • Error Tracking: Failure pattern analysis
  • Quality Gates: Release criteria validation

This comprehensive testing infrastructure ensures the Dataproc MCP Server maintains high quality, performance, and reliability across all deployment scenarios.