Skip to content

feat: add scene-explorer-tests locally and improve coverage workflow#951

Draft
kuruk-mm wants to merge 6 commits intomainfrom
feat/scene-explorer-tests-local
Draft

feat: add scene-explorer-tests locally and improve coverage workflow#951
kuruk-mm wants to merge 6 commits intomainfrom
feat/scene-explorer-tests-local

Conversation

@kuruk-mm
Copy link
Member

Summary

  • Add scene-explorer-tests source code directly to tests/scene-explorer-tests/ instead of downloading from npm
  • Create index.json to control which scenes are tested (easy enable/disable)
  • Move coverage logic to separate src/coverage.rs module
  • Integrate test server management into cargo run -- coverage command
  • Simplify CI workflow by removing npm package download steps
  • Disable livekit during coverage runs (use_deno only)

Changes

  • New file: src/coverage.rs - Coverage-related logic extracted from main.rs
  • New folder: tests/scene-explorer-tests/ - Scene test source code from decentraland/scene-explorer-tests
  • Modified: .github/workflows/coverage.yml - Simplified workflow, removed npm download
  • Modified: src/main.rs - Cleaned up, delegates to coverage module

How it works

When running cargo run -- coverage:

  1. Runs cargo unit tests (without livekit)
  2. Runs integration tests
  3. Builds scene-explorer-tests (npm install, npm run build, npm run export-static-local)
  4. Starts http-server on port 7666
  5. Runs scene tests against http://localhost:7666/scene-explorer-tests
  6. Stops http-server
  7. Runs client tests
  8. Generates coverage report

Test plan

  • Run cargo run -- coverage locally
  • Verify CI workflow passes

@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

📊 Benchmark Report

📊 Baseline: Comparing against main branch at commit 4f6bc2f (⚠️ used fallback: 1 commits back)

Click to expand full benchmark report

Decentraland Godot Explorer - Benchmark Report

Generated: 2025-12-29_17-56-54

Total Tests: 5

📊 Comparison: vs main branch baseline

  • 🟢 = Improvement (better performance)
  • 🔴 = Regression (worse performance)
  • ⚪ = No significant change (<0.5%)

Table of Contents

  1. 1_Terms_and_Conditions
  2. 2_Lobby
  3. 3_Menu
  4. 4_Explorer_Goerli_Plaza
  5. 4_Explorer_Genesis_Plaza

Summary Overview

Memory Metrics

Test Process RSS (MiB) Godot Static (MiB) GPU VRAM (MiB) Rust Heap (MiB) Deno Total (MiB)
1_Terms_and_Conditions 389.80 82.36 24.55 0.99 🔴 (+3.1%) -0.00
2_Lobby 491.05 🔴 (+0.5%) 112.99 45.51 2.16 🔴 (+3.8%) -0.00
3_Menu 1011.16 🟢 (-1.1%) 262.25 126.37 🔴 (+0.5%) 3.26 🟢 (-7.4%) -0.00
4_Explorer_Goerli_Plaza 1461.41 🟢 (-1.1%) 328.09 170.32 6.14 🟢 (-6.4%) 21.17 🟢 (-5.1%)
4_Explorer_Genesis_Plaza 3614.14 🟢 (-11.1%) 976.44 🟢 (-11.4%) 672.61 🟢 (-16.3%) 13.03 🔴 (+8.4%) 12.19 🟢 (-47.0%)

Object Counts

Test Total Objects Nodes Resources Orphan Nodes
1_Terms_and_Conditions 2366 127 583 12
2_Lobby 4574 804 768 44
3_Menu 12844 3873 1183 44
4_Explorer_Goerli_Plaza 19274 6315 1461 136
4_Explorer_Genesis_Plaza 46242 🔴 (+5.7%) 19785 🟢 (-2.9%) 3432 🔴 (+133.5%) 5030 🔴 (+117.5%)

Rendering Metrics

Test FPS Draw Calls Primitives Objects in Frame
1_Terms_and_Conditions 30.00 14 376 139
2_Lobby 30.00 15 644 218
3_Menu 18.00 🔴 (-5.3%) 95 4278 254
4_Explorer_Goerli_Plaza 3.00 428 160777 🔴 (+1.6%) 557
4_Explorer_Genesis_Plaza 2.00 🔴 (-33.3%) 1232 947251 🟢 (-15.9%) 1446

Resource Analysis

Test Meshes Materials Mesh RIDs Material RIDs Dedup Potential
1_Terms_and_Conditions 0 0 0 0 0
2_Lobby 0 0 0 0 0
3_Menu 0 0 0 0 0
4_Explorer_Goerli_Plaza 274 276 274 13 0
4_Explorer_Genesis_Plaza 1468 1931 591 476 0

Detailed Test Results

Test 1: 1_Terms_and_Conditions

Benchmark Report: 1_Terms_and_Conditions

Timestamp: 2025-12-29_17-54-25
Location: UI Scene


Memory Metrics

Metric Value
Process Memory Usage (RSS) 389.80 MiB
Godot Static Memory 82.36 MiB
Godot Peak Memory 87.70 MiB
GPU Video RAM 24.55 MiB
GPU Texture Memory 17.27 MiB
GPU Buffer Memory 7.28 MiB
Rust Heap Usage 0.99 🔴 (+3.1%) MiB
Rust Total Allocated 2.27 🟢 (-0.9%) MiB

Object Counts

Metric Count
Total Objects 2366
Resources 583
Nodes 127
Orphan Nodes 12

Rendering Metrics

Metric Value
FPS 30.00
Draw Calls per Frame 14
Primitives per Frame 376
Objects per Frame 139

Test 2: 2_Lobby

Benchmark Report: 2_Lobby

Timestamp: 2025-12-29_17-54-31
Location: UI Scene


Memory Metrics

Metric Value
Process Memory Usage (RSS) 491.05 🔴 (+0.5%) MiB
Godot Static Memory 112.99 MiB
Godot Peak Memory 161.89 🔴 (+11.9%) MiB
GPU Video RAM 45.51 MiB
GPU Texture Memory 37.45 MiB
GPU Buffer Memory 8.06 MiB
Rust Heap Usage 2.16 🔴 (+3.8%) MiB
Rust Total Allocated 19.97 🟢 (-2.9%) MiB

Object Counts

Metric Count
Total Objects 4574
Resources 768
Nodes 804
Orphan Nodes 44

Rendering Metrics

Metric Value
FPS 30.00
Draw Calls per Frame 15
Primitives per Frame 644
Objects per Frame 218

Test 3: 3_Menu

Benchmark Report: 3_Menu

Timestamp: 2025-12-29_17-54-38
Location: UI Scene


Memory Metrics

Metric Value
Process Memory Usage (RSS) 1011.16 🟢 (-1.1%) MiB
Godot Static Memory 262.25 MiB
Godot Peak Memory 266.76 🟢 (-0.8%) MiB
GPU Video RAM 126.37 🔴 (+0.5%) MiB
GPU Texture Memory 115.96 🔴 (+0.6%) MiB
GPU Buffer Memory 10.41 MiB
Rust Heap Usage 3.26 🟢 (-7.4%) MiB
Rust Total Allocated 261.34 🔴 (+5.4%) MiB

Object Counts

Metric Count
Total Objects 12844
Resources 1183
Nodes 3873
Orphan Nodes 44

Rendering Metrics

Metric Value
FPS 18.00 🔴 (-5.3%)
Draw Calls per Frame 95
Primitives per Frame 4278
Objects per Frame 254

Test 4: 4_Explorer_Goerli_Plaza

Benchmark Report: 4_Explorer_Goerli_Plaza

Timestamp: 2025-12-29_17-55-10
Location: (72, -10)
Realm: https://sdk-team-cdn.decentraland.org/ipfs/goerli-plaza-main-latest


Memory Metrics

Metric Value
Process Memory Usage (RSS) 1461.41 🟢 (-1.1%) MiB
Godot Static Memory 328.09 MiB
Godot Peak Memory 330.09 MiB
GPU Video RAM 170.32 MiB
GPU Texture Memory 154.12 MiB
GPU Buffer Memory 16.21 MiB
Rust Heap Usage 6.14 🟢 (-6.4%) MiB
Rust Total Allocated 333.07 🟢 (-1.3%) MiB
Deno/V8 Total Memory 21.17 🟢 (-5.1%) MiB
Deno Active Scenes 1
Deno Avg per Scene 21.17 🟢 (-5.1%) MiB

Object Counts

Metric Count
Total Objects 19274
Resources 1461
Nodes 6315
Orphan Nodes 136

Rendering Metrics

Metric Value
FPS 3.00
Draw Calls per Frame 428
Primitives per Frame 160777 🔴 (+1.6%)
Objects per Frame 557

Resource Analysis

Metric Value
Total Mesh References 274
Total Material References 276
Unique Mesh RIDs 274
Unique Material RIDs 13
Hashed Mesh Count 0
Potential Deduplication 0 (0.0% savings)

Test 5: 4_Explorer_Genesis_Plaza

Benchmark Report: 4_Explorer_Genesis_Plaza

Timestamp: 2025-12-29_17-56-39
Location: (-6, 0)
Realm: https://realm-provider-ea.decentraland.org/main


Memory Metrics

Metric Value
Process Memory Usage (RSS) 3614.14 🟢 (-11.1%) MiB
Godot Static Memory 976.44 🟢 (-11.4%) MiB
Godot Peak Memory 1181.43 🟢 (-3.5%) MiB
GPU Video RAM 672.61 🟢 (-16.3%) MiB
GPU Texture Memory 553.25 🟢 (-21.3%) MiB
GPU Buffer Memory 119.36 🔴 (+19.3%) MiB
Rust Heap Usage 13.03 🔴 (+8.4%) MiB
Rust Total Allocated 765.73 🟢 (-43.6%) MiB
Deno/V8 Total Memory 12.19 🟢 (-47.0%) MiB
Deno Active Scenes 1
Deno Avg per Scene 12.19 🟢 (-47.0%) MiB

Object Counts

Metric Count
Total Objects 46242 🔴 (+5.7%)
Resources 3432 🔴 (+133.5%)
Nodes 19785 🟢 (-2.9%)
Orphan Nodes 5030 🔴 (+117.5%)

Rendering Metrics

Metric Value
FPS 2.00 🔴 (-33.3%)
Draw Calls per Frame 1232
Primitives per Frame 947251 🟢 (-15.9%)
Objects per Frame 1446

Resource Analysis

Metric Value
Total Mesh References 1468
Total Material References 1931
Unique Mesh RIDs 591
Unique Material RIDs 476
Hashed Mesh Count 0
Potential Deduplication 0 (0.0% savings)


📋 Logs & Artifacts

  • 📊 CSV Data: benchmark_report.csv - Raw benchmark data in S3
  • 🔧 Full Logs: benchmark_run.log - Complete benchmark run output (build + execution)
  • 🌐 Workflow Run: View full logs
  • 📦 Download All: Get the benchmark-report artifact from the workflow run

🔄 Updated: 2025-12-29 17:56:58 UTC

@kuruk-mm kuruk-mm force-pushed the feat/scene-explorer-tests-local branch from 43c64e7 to 659f8c1 Compare December 15, 2025 21:18
- Add scene-explorer-tests source code to tests/scene-explorer-tests/
- Create index.json to control which scenes are tested (easy enable/disable)
- Move coverage logic to separate src/coverage.rs module
- Integrate test server management into xtask coverage command
- Simplify CI workflow by removing npm package download
- Disable livekit during coverage runs (use_deno only)
@kuruk-mm kuruk-mm force-pushed the feat/scene-explorer-tests-local branch from 659f8c1 to 2c8550a Compare December 15, 2025 21:19
@github-actions
Copy link
Contributor

📸 Snapshot Test Differences

Found 70 snapshot(s) with differences.

Download the coverage-snapshots artifact to review the full images.

Category Test Name Status
scenes screenshot_godot_snapshot_avatar_attach_1 ⚠️ Changed
scenes screenshot_godot_snapshot_avatar_attach_2 ⚠️ Changed
scenes screenshot_godot_snapshot_avatar_attach_3 ⚠️ Changed
scenes screenshot_godot_snapshot_avatar_attach_4 ⚠️ Changed
scenes screenshot_godot_snapshot_avatar_shape ⚠️ Changed
scenes screenshot_godot_snapshot_gltfcontainer_avocado ⚠️ Changed
scenes screenshot_godot_snapshot_gltfcontainer_hev ⚠️ Changed
scenes screenshot_godot_snapshot_material_01 ⚠️ Changed
scenes screenshot_godot_snapshot_material_02 ⚠️ Changed
scenes screenshot_godot_snapshot_material_03 ⚠️ Changed
scenes screenshot_godot_snapshot_material_04 ⚠️ Changed
scenes screenshot_godot_snapshot_material_05 ⚠️ Changed
scenes screenshot_godot_snapshot_material_06 ⚠️ Changed
scenes screenshot_godot_snapshot_material_07 ⚠️ Changed
scenes screenshot_godot_snapshot_material_08 ⚠️ Changed
scenes screenshot_godot_snapshot_material_09 ⚠️ Changed
scenes screenshot_godot_snapshot_material_10 ⚠️ Changed
scenes screenshot_godot_snapshot_material_11 ⚠️ Changed
scenes screenshot_godot_snapshot_material_12 ⚠️ Changed
scenes screenshot_godot_snapshot_material_13 ⚠️ Changed
scenes screenshot_godot_snapshot_material_14 ⚠️ Changed
scenes screenshot_godot_snapshot_mesh_renderer_box ⚠️ Changed
scenes screenshot_godot_snapshot_mesh_renderer_cylinder ⚠️ Changed
scenes screenshot_godot_snapshot_mesh_renderer_plane ⚠️ Changed
scenes screenshot_godot_snapshot_mesh_renderer_sphere ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_baroque_ornament ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_blocky ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_canvas ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_classic ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_diamond_ornament ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_gold_carved ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_gold_edges ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_gold_rounded ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_gold_wide ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_metal_medium ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_metal_rounded ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_metal_slim ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_metal_wide ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_minimal_black ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_minimal_grey ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_minimal_white ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_minimal_wide ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_none ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_pins ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_tape ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_wood_slim ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_wood_twigs ⚠️ Changed
scenes screenshot_godot_snapshot_nft_shape_nft_wood_wide ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_1 ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_2 ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_3 ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_4 ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_5 ⚠️ Changed
scenes screenshot_godot_snapshot_text_shape_6 ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_all_screen_green ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_all_screen_red ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_all_screen_rocks ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_nineslices_with_default_textureslices ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_nineslices_with_right_values ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_smaller_nineslices_with_right_values ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_stretch ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_stretch_with_colored ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_stretch_with_uvs ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_test_flexbox ⚠️ Changed
scenes screenshot_godot_snapshot_ui_background_using_avatar_texture ⚠️ Changed
scenes screenshot_godot_snapshot_ui_dropdown ⚠️ Changed
scenes screenshot_godot_snapshot_ui_text_1 ⚠️ Changed
scenes screenshot_godot_snapshot_ui_text_2 ⚠️ Changed
scenes screenshot_godot_snapshot_visibility_false ⚠️ Changed
scenes screenshot_godot_snapshot_visibility_true ⚠️ Changed

How to review

  1. Download the coverage-snapshots artifact from this workflow run
  2. The comparison folder contains:
    • <name>.png - Generated snapshot from this run
    • <name>.diff.png - Visual difference highlighting changes
  3. Original snapshots are in the parent folder with the same name

To update snapshots

If the changes are expected, copy the generated snapshots to replace the originals.

@github-actions
Copy link
Contributor

📦 Build Report

🤖 Android

Artifact Status
APK 📱 Download APK
AAB 📦 Download AAB
Debug Symbols 🔧 Debug Symbols

Build Status: ✅ Success

🍏 iOS

iOS builds are triggered manually. Add the build-ios-internal label to trigger an iOS build.


🔗 Workflow Run: View logs

🔄 Updated: 2025-12-29 18:05:18 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant