Skip to content

englishm/moq-interop-runner

Repository files navigation

MoQ Interop Runner

A framework for testing interoperability between MoQT (Media over QUIC Transport) implementations.

Why Interoperability Testing?

The IETF standards process requires "multiple, independent, and interoperable implementations" before a specification can advance (RFC 2026). Interop testing catches ambiguities in specs that only surface when different teams interpret the same text differently, and builds confidence that implementations will work together in real deployments.

This project is modeled on the QUIC Interop Runner, which was instrumental during QUIC standardization.

Non-Goals

Reference compliance testing. This project tests whether different MoQT implementations can successfully communicate with each other - interoperability, not compliance. We're not building a reference implementation that serves as the definitive arbiter of spec compliance. Interop testing happens between implementations - if two implementations agree and work together, that's what we care about.

Adversarial/negative testing. Our test cases follow scenarios that occur between valid MoQT implementations. We don't require test clients to produce malformed messages or violate the protocol to test error handling. That kind of negative testing requires purpose-built tools capable of representing illegal protocol states - not something we expect from production MoQT implementations.

Browser-based implementations (for now). The current Docker-based flow assumes CLI execution. Browser-only implementations (TypeScript/WebTransport) can't plug in directly today. Playwright-based automation is a potential future direction.

Getting Started

Start with Getting Started for a guided walkthrough; run make help to discover commands.

Prerequisites

  • Docker with buildx support (for building multi-platform images)
  • jq (for parsing JSON configuration)
  • openssl (for generating TLS certificates)
  • bash (works with macOS default bash 3.2+)

Overview

This project provides:

  • Implementation Registry (implementations.json) - A catalog of MoQT implementations with their capabilities, Docker images, and public relay endpoints
  • Test Orchestration - Scripts to run interop tests across implementation combinations
  • Test Specifications (docs/TEST-SPECIFICATIONS.md) - Standardized test cases that any implementation can implement

Quick Start

# List registered implementations and their endpoints
make interop-list

# Build the moq-rs test client (one-time, takes a few minutes)
make build-moq-rs BUILD_ARGS="--target client"

# Run tests against all public relays
make interop-remote

Filter a single relay: make interop-remote RELAY=moxygen. See Getting Started for more, or make help for all commands.

Registered Implementations

Implementation Organization Draft Versions Roles Public Endpoints
moq-rs Cloudflare draft-14 relay, client https://draft-14.cloudflare.mediaoverquic.com:443/moq
moxygen Meta draft-12-14 relay https://fb.mvfst.net:9448/moq-relay
moqtransport TUM draft-13 relay (no persistent relay)
quiche-moq Google draft-14 relay https://quichemoq.dev:443
moqtail OzU draft-14 relay https://relay.moqtail.dev
libquicr Cisco draft-14 relay https://us-west-2.relay.quicr.org:33437/relay
imquic Meetecho draft-13-14 relay https://lminiero.it:9000
moq (moq-dev) Luke Curley draft-14 relay https://cdn.moq.dev/anon

This table is a snapshot — run make interop-list or see implementations.json for the current state. See IMPLEMENTATIONS.md for how to add your implementation.

Test Cases

Tests are organized by functional category:

Test Category Description
setup-only Session Connect, complete SETUP exchange, close gracefully
announce-only Namespace Announce namespace, receive OK, close
publish-namespace-done Namespace Announce, then send PUBLISH_NAMESPACE_DONE
subscribe-error Subscription Subscribe to non-existent track, expect error
announce-subscribe Subscription Publisher announces, subscriber subscribes
subscribe-before-announce Subscription Subscribe before publisher announces

See docs/tests/TEST-CASES.md for detailed specifications with protocol references.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      Test Orchestrator                          │
│  (run-interop-tests.sh / Makefile)                              │
│  - Reads implementations.json                                    │
│  - Manages Docker containers or tests remote endpoints           │
│  - Collects results                                              │
└─────────────────────────────────────────────────────────────────┘
         │                                    │
         ▼                                    ▼
┌─────────────────────┐            ┌─────────────────────┐
│   Test Client       │            │   Relay Under Test  │
│   (Docker image)    │───────────▶│   (Docker or remote)│
│                     │   MoQT     │                     │
│   Runs test cases,  │            │   Implementation    │
│   reports results   │            │   being tested      │
└─────────────────────┘            └─────────────────────┘

Key Files

Path Purpose
implementations.json Implementation registry (the central config file)
run-interop-tests.sh Main test orchestration script
Makefile All commands — run make help to see them
docs/ Getting started, test specs, interface docs
adapters/ Thin Docker wrappers for implementations (README)
builds/ Source-based Docker builds (README)

Related Projects

MoQT Implementations

Inspiration

Contributing

Contributions welcome! Key areas:

  1. Add your implementation - See IMPLEMENTATIONS.md
  2. Propose new test cases - Open an issue or PR to docs/tests/TEST-CASES.md
  3. Build a test client - See docs/IMPLEMENTING-A-TEST-CLIENT.md
  4. Improve tooling - Better reporting, CI integration, etc.

Acknowledgments

The initial implementation of this framework was generously supported by Cloudflare.

License

Licensed under either of:

at your option.

About

MoQ Interop Runner

Topics

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors 10