Skip to content

Run Assertoor tests in CI #6882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mergify merged 6 commits into from
Jun 2, 2025
Merged

Run Assertoor tests in CI #6882

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9366b5f
Run Assertoor tests in CI
ackintosh Jan 29, 2025
1ec41d2
Return Assertoor test result
ackintosh Jan 30, 2025
f589f6d
Add tests
ackintosh Feb 14, 2025
294e454
Add a flag to run Assertoor tests
ackintosh Feb 15, 2025
ffdf572
Merge branch 'unstable' into run-assertoor-tests-in-ci
ackintosh May 30, 2025
50ff402
Remove the local_testnet runs that don't perform any verification
ackintosh May 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 31 additions & 10 deletions .github/workflows/local-testnet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,20 +52,18 @@ jobs:
- name: Load Docker image
run: docker load -i lighthouse-docker.tar

- name: Start local testnet
run: ./start_local_testnet.sh -e local -c -b false && sleep 60
- name: Start local testnet with Assertoor
run: ./start_local_testnet.sh -e local-assertoor -c -a -b false && sleep 60
working-directory: scripts/local_testnet
Copy link
Member

@jimmygchen jimmygchen May 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can probably have this replace the Start local testnet and Start local testnet with blinded block production tasks, as they don't currently do any verification.

Copy link
Member Author

@ackintosh ackintosh May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, thanks!

Copy link
Member

@jimmygchen jimmygchen Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, let's merge this - I think we can update the network_params to have some nodes do blinded block production too.


- name: Stop local testnet and dump logs
run: ./stop_local_testnet.sh local
working-directory: scripts/local_testnet

- name: Start local testnet with blinded block production
run: ./start_local_testnet.sh -e local-blinded -c -p -b false && sleep 60
working-directory: scripts/local_testnet
- name: Await Assertoor test result
id: assertoor_test_result
uses: ethpandaops/assertoor-github-action@v1
with:
kurtosis_enclave_name: local-assertoor

- name: Stop local testnet and dump logs
run: ./stop_local_testnet.sh local-blinded
run: ./stop_local_testnet.sh local-assertoor
working-directory: scripts/local_testnet

- name: Upload logs artifact
Expand All @@ -76,6 +74,29 @@ jobs:
scripts/local_testnet/logs
retention-days: 3

- name: Return Assertoor test result
shell: bash
run: |
test_result="${{ steps.assertoor_test_result.outputs.result }}"
test_status=$(
cat <<"EOF"
${{ steps.assertoor_test_result.outputs.test_overview }}
EOF
)
failed_test_status=$(
cat <<"EOF"
${{ steps.assertoor_test_result.outputs.failed_test_details }}
EOF
)

echo "Test Result: $test_result"
echo "$test_status"
if ! [ "$test_result" == "success" ]; then
echo "Failed Test Task Status:"
echo "$failed_test_status"
exit 1
fi

doppelganger-protection-success-test:
needs: dockerfile-ubuntu
runs-on: ubuntu-22.04
Expand Down
14 changes: 12 additions & 2 deletions scripts/local_testnet/start_local_testnet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@ BUILD_IMAGE=true
BUILDER_PROPOSALS=false
CI=false
KEEP_ENCLAVE=false
RUN_ASSERTOOR_TESTS=false

# Get options
while getopts "e:b:n:phck" flag; do
while getopts "e:b:n:phcak" flag; do
case "${flag}" in
a) RUN_ASSERTOOR_TESTS=true;;
e) ENCLAVE_NAME=${OPTARG};;
b) BUILD_IMAGE=${OPTARG};;
n) NETWORK_PARAMS_FILE=${OPTARG};;
Expand All @@ -34,6 +36,7 @@ while getopts "e:b:n:phck" flag; do
echo " -n: kurtosis network params file path default: $NETWORK_PARAMS_FILE"
echo " -p: enable builder proposals"
echo " -c: CI mode, run without other additional services like Grafana and Dora explorer"
echo " -a: run Assertoor tests"
echo " -k: keeping enclave to allow starting the testnet without destroying the existing one"
echo " -h: this help"
exit
Expand Down Expand Up @@ -63,11 +66,18 @@ if [ "$BUILDER_PROPOSALS" = true ]; then
fi

if [ "$CI" = true ]; then
# TODO: run assertoor tests
yq eval '.additional_services = []' -i $NETWORK_PARAMS_FILE
echo "Running without additional services (CI mode)."
fi

if [ "$RUN_ASSERTOOR_TESTS" = true ]; then
yq eval '.additional_services += ["assertoor"] | .additional_services |= unique' -i $NETWORK_PARAMS_FILE
# The available tests can be found in the `assertoor_params` section:
# https://github.com/ethpandaops/ethereum-package?tab=readme-ov-file#configuration
yq eval '.assertoor_params = {"run_stability_check": true, "run_block_proposal_check": true, "run_transaction_test": true, "run_blob_transaction_test": true}' -i $NETWORK_PARAMS_FILE
echo "Assertoor has been added to $NETWORK_PARAMS_FILE."
fi

if [ "$BUILD_IMAGE" = true ]; then
echo "Building Lighthouse Docker image."
ROOT_DIR="$SCRIPT_DIR/../.."
Expand Down