tsuite

YAML-driven integration test framework with container isolation and real-time monitoring.

Installation

npm install -g @mcpmesh/tsuite

Quick Start

# View the quickstart guide
tsuite man quickstart

# Start the dashboard (includes API server)
tsuite api --port 9999

# Open http://localhost:9999 in your browser

Commands

Run Tests

# Run all tests in Docker mode
tsuite run --suite ./my-suite --all --docker

# Run specific use case
tsuite run --suite ./my-suite --uc uc01_registry --docker

# Run specific test case
tsuite run --suite ./my-suite --tc uc01_registry/tc01_agent_registration --docker

# Run tests matching tags
tsuite run --suite ./my-suite --tag smoke --docker

# Dry run (list tests without executing)
tsuite run --suite ./my-suite --dry-run --all

Dashboard & API Server

# Start on default port (9999)
tsuite api

# Start on custom port
tsuite api --port 8080

# Start in background (detached)
tsuite api --detach

# Stop background server
tsuite stop

Scaffold Test Cases

Generate test cases from agent directories:

# Interactive mode
tsuite scaffold --suite ./my-suite ./path/to/agent1 ./path/to/agent2

# Non-interactive mode
tsuite scaffold --suite ./my-suite --uc uc01_tags --tc tc01_test ./agent1 ./agent2

# Preview without creating files
tsuite scaffold --suite ./my-suite --uc uc01_tags --tc tc01_test --dry-run ./agent1

Documentation

# List available topics
tsuite man --list

# View specific topic
tsuite man quickstart
tsuite man handlers
tsuite man assertions
tsuite man routines

Clear Data

# Clear all test data
tsuite clear --all

# Clear specific run
tsuite clear --run-id <run_id>

Features

• YAML-based test definitions - Tests as configuration, not code
• Container isolation - Each test runs in a fresh Docker container
• Parallel execution - Worker pool for concurrent test runs
• Web dashboard - Real-time monitoring, history, and test editor
• Pluggable handlers - shell, http, file, wait, pip-install, npm-install
• Expression language - Flexible assertions with jq, JSONPath support
• Reusable routines - Define once, use across tests
• Scaffold command - Auto-generate test cases from agent directories

Test Suite Structure

my-suite/
├── config.yaml              # Suite configuration
├── global/
│   └── routines.yaml        # Global reusable routines
└── suites/
    └── uc01_example/        # Use case folder
        ├── routines.yaml    # UC-level routines (optional)
        └── tc01_test/       # Test case folder
            ├── test.yaml    # Test definition
            └── artifacts/   # Test artifacts (agents, fixtures)

Example Test

name: "Agent Registration Test"
description: "Verify agent registers with mesh"
tags: [smoke, registry]
timeout: 300

pre_run:
  - routine: global.setup_for_python_agent
    params:
      meshctl_version: "${config.packages.cli_version}"

test:
  - name: "Copy agent to workspace"
    handler: shell
    command: "cp -r /artifacts/my-agent /workspace/"

  - name: "Start agent"
    handler: shell
    command: "meshctl start my-agent/main.py -d"
    workdir: /workspace

  - name: "Wait for registration"
    handler: wait
    seconds: 5

  - name: "Verify agent registered"
    handler: shell
    command: "meshctl list"
    capture: agent_list

assertions:
  - expr: "${captured.agent_list} contains 'my-agent'"
    message: "Agent should be registered"

post_run:
  - handler: shell
    command: "meshctl stop || true"
    workdir: /workspace

Documentation

Run tsuite man <topic> for detailed documentation:

Topic	Description
quickstart	Getting started guide
suites	Suite structure and config.yaml
testcases	Test case structure and test.yaml
handlers	Built-in handlers (shell, http, file, etc.)
routines	Reusable test routines
assertions	Assertion syntax and expressions
variables	Variable interpolation syntax
docker	Docker mode and container isolation
api	API server and dashboard

See docs/USER_GUIDE.md for comprehensive documentation.

Development

# Build from source
make build

# Build with dashboard embedded
make build-with-dashboard

# Run tests
make test

License

MIT