[CI Failure Doctor] CI Failure Investigation - Run #33691

[github-actions]

🏥 CI Failure Investigation - Run #33691

Summary

CI run 33691 failed because the coverage gate counted 137 tests that exist in the codebase but were not executed by the canary jobs.

Failure Details

• Run: 21662949681
• Commit: bb7ce55
• Trigger: push

Root Cause Analysis

The compare-test-coverage.sh step in the failing jobs enumerated 3,977 tests from all-tests.txt but only observed 3,840 entries in executed-tests.txt. The gate therefore failed with ❌ FAILURE: Found 137 tests that are NOT being executed in CI, even though the referenced integration suites completed successfully. The gap exists because the CI job did not run those 137 tests (such as TestAllCommandsExist, TestCompilationStats, TestCompileWorkflowWithValidation_InvalidFile, etc.) that are still tracked in the codebase.

Failed Jobs and Errors

• Integration: CLI Completion & Other — aborted once compare-test-coverage.sh reported missing tests.
• canary_go — failed at the same coverage gate step, logging the 137 tests that are NOT being executed in CI message.

Investigation Findings

• ./scripts/compare-test-coverage.sh all-tests.txt executed-tests.txt pulled the discrepancy between defined and executed tests.
• The missing list grows when new tests are added without ensuring the CI job runs them or when executed-tests.txt isn’t generated from the suites that include the new names.
• This job-level coverage gate is stricter than the individual test suites and therefore fails fast whenever the two files drift apart.

Recommended Actions

• Update the integration jobs so that their go test invocations cover the missing test names before running compare-test-coverage.sh.
• Run ./scripts/compare-test-coverage.sh all-tests.txt executed-tests.txt locally (after regenerating all-tests.txt) to catch coverage gate failures before pushing.
• If certain tests should remain excluded intentionally, document them and have the coverage script skip those names explicitly.

Prevention Strategies

• Automate regeneration of all-tests.txt and validation of executed-tests.txt as part of pre-commit hooks or CI prechecks whenever new tests are added.
• Introduce a faster preflight step that runs the coverage comparison in a lightweight job so developers discover mismatches before reaching the final canary job.

AI Team Self-Improvement

Before adding or renaming tests, rerun ./scripts/compare-test-coverage.sh and ensure the changes are reflected in the CI jobs that build executed-tests.txt; document any intentional omissions in instructions.md so future agents know how to keep the coverage files aligned.

Historical Context

No prior investigation matching this coverage-gate failure pattern was found in the cached /tmp/gh-aw/cache-memory/investigations directory.

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

• expires on Feb 6, 2026, 8:08 AM UTC