Skip to content

refactor(engine): extract test runner to standalone rivet-engine-runner package#4089

Draft
NathanFlurry wants to merge 1 commit intomainfrom
engine-runner-package
Draft

refactor(engine): extract test runner to standalone rivet-engine-runner package#4089
NathanFlurry wants to merge 1 commit intomainfrom
engine-runner-package

Conversation

@NathanFlurry
Copy link
Member

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

…er package

- Move runner code from engine/packages/engine/tests/common/test_runner/ to engine/sdks/rust/engine-runner/
- Add RunnerConfig builder pattern matching TypeScript engine-runner API
- Update test wrapper to use the new package via re-exports
- Add workspace dependency for rivet-engine-runner
- Enables reuse of runner logic outside of tests (e.g., openworkers integration)
@railway-app
Copy link

railway-app bot commented Feb 1, 2026

🚅 Deployed to the rivet-pr-4089 environment in rivet-frontend

Service Status Web Updated (UTC)
frontend-inspector 😴 Sleeping (View Logs) Web Feb 1, 2026 at 5:34 am
website 😴 Sleeping (View Logs) Web Feb 1, 2026 at 5:33 am
frontend-cloud 😴 Sleeping (View Logs) Web Feb 1, 2026 at 5:32 am
mcp-hub ✅ Success (View Logs) Web Feb 1, 2026 at 5:26 am

Copy link
Member Author


How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@pkg-pr-new
Copy link

pkg-pr-new bot commented Feb 1, 2026

More templates

@rivetkit/cloudflare-workers

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@4089

@rivetkit/db

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/db@4089

@rivetkit/framework-base

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/framework-base@4089

@rivetkit/next-js

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/next-js@4089

@rivetkit/react

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/react@4089

rivetkit

pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@4089

@rivetkit/sql-loader

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sql-loader@4089

@rivetkit/virtual-websocket

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/virtual-websocket@4089

@rivetkit/engine-runner

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner@4089

@rivetkit/engine-runner-protocol

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner-protocol@4089

commit: 91131e8

@claude
Copy link

claude bot commented Feb 1, 2026

Pull Request Review: Extract Test Runner to Standalone Package

Summary

This PR successfully refactors the test runner from engine/packages/engine/tests/common/test_runner/ into a standalone, reusable package at engine/sdks/rust/engine-runner/. The refactoring maintains backward compatibility while making the runner available for use outside of tests.

✅ Strengths

  1. Clean Architecture: The extraction is well-structured with clear separation between the standalone package and test-specific wrapper
  2. Backward Compatibility: The test code maintains the same API through re-exports and type aliases (TestRunner = Runner)
  3. Proper Dependency Management: All workspace dependencies are correctly referenced
  4. Good Documentation: The new lib.rs provides clear module-level documentation with examples
  5. Consistent Error Handling: Properly replaced anyhow\! macro with anyhow::anyhow\! to avoid glob imports

🔍 Issues Found

Critical Issues

1. Incorrect Example in Documentation (engine/sdks/rust/engine-runner/src/lib.rs:10-25)

The example code will not compile:

let mut runner = Runner::new(config)?;  // ❌ Runner::new() doesn't exist
runner.register_actor("echo", |_| Box::new(EchoActor::new()));  // ❌ No register_actor method

Correct usage should be:

let config = RunnerConfig::builder()
    .endpoint("http://127.0.0.1:8080")
    .token("dev")
    .namespace("my-namespace")
    .runner_name("my-runner")
    .runner_key("unique-key")
    .build()?;

let runner = RunnerBuilder::new(config)
    .with_actor_behavior("echo", |_| Box::new(EchoActor::new()))
    .build()?;
runner.start().await?;

2. Unused metadata Field (engine/sdks/rust/engine-runner/src/runner.rs:49, 505-506)

The RunnerConfig.metadata field is defined but never used:

  • Line 49: Field is defined
  • Line 505-506: Hardcoded to None in build_init_message()

Either use it or remove it from the public API.

Minor Issues

3. Dead Code Warnings (engine/sdks/rust/engine-runner/src/runner.rs:175-178)

struct ActorState {
    #[allow(dead_code)]
    actor_id: String,
    #[allow(dead_code)]
    generation: u32,
    actor: Box<dyn TestActor>,
}

These fields should either be used (e.g., for debug logging) or the struct should be simplified. The #[allow(dead_code)] attributes suggest the fields were kept for future use but aren't currently necessary.

4. Missing Default Implementations (engine/sdks/rust/engine-runner/src/behaviors.rs)

Good additions of Default implementations for test actors (lines 413-418, 426-431, etc.), but consider whether all test actors should implement Default for consistency. Currently only some do.

5. Removed Helper Function (engine/packages/engine/tests/common/test_helpers.rs:82-103)

The assert_actor_in_runner helper was removed. Verify that:

  • No tests were using this function
  • If tests need this functionality, they should have an alternative

6. Type Inconsistency (engine/sdks/rust/engine-runner/src/runner.rs:889)

pub async fn get_actor_ids(&self) -> Vec<String> {
    let actors = self.actors.lock().await;
    actors.keys().cloned().collect()
}

Previously returned Vec<Id>, now returns Vec<String>. This is a breaking change for external users (though likely only tests). Document this in the PR or provide both methods.

🔒 Security Considerations

No security issues identified. The refactoring:

  • Maintains the same authentication/authorization patterns
  • Properly validates inputs through the builder pattern
  • No new untrusted input paths introduced

⚡ Performance Considerations

No performance regressions expected:

  • Same WebSocket handling logic
  • Same actor lifecycle management
  • No additional allocations or copies introduced

🧪 Test Coverage

Missing: No unit tests for the new standalone package itself. The existing tests in engine/packages/engine/tests/ will exercise it, but consider adding:

  • Unit tests for RunnerConfig::builder() validation
  • Tests for the builder pattern edge cases
  • Documentation tests (fix the example and make it a real #[test])

📝 Code Quality & Style

Adherence to CLAUDE.md Guidelines

Good:

  • Hard tabs used correctly
  • Imports at the top of files
  • Lowercase log messages (e.g., line 865: "starting engine runner")
  • Workspace dependencies properly used
  • Error handling follows patterns

⚠️ Needs Attention:

  • The example in documentation should be a proper doc test or clearly marked as pseudocode
  • Consider if the metadata field needs implementation

🎯 Recommendations

Must Fix

  1. Fix the documentation example in lib.rs to show correct API usage
  2. Decide on metadata field: Either implement it or remove it from public API

Should Fix

  1. Remove or use the dead code in ActorState
  2. Add tests for the new package's public API
  3. Verify removal of assert_actor_in_runner helper doesn't break tests

Nice to Have

  1. Add a CHANGELOG or migration guide if this package will be used externally
  2. Consider making all test actors implement Default for consistency
  3. Add more detailed rustdoc comments for public types

📊 Overall Assessment

Quality: ⭐⭐⭐⭐ (4/5)

This is a solid refactoring that successfully extracts test infrastructure into a reusable package. The code is well-structured and maintains backward compatibility. The main issues are documentation accuracy and some unused fields. After addressing the critical issues (documentation example and metadata field), this will be production-ready.

Recommendation: Approve with requested changes to documentation example and metadata field handling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant