Have nightly tests against nightly generated artifacts#7366
Have nightly tests against nightly generated artifacts#7366GaAstDev wants to merge 10 commits intosigp:unstablefrom
Conversation
|
Hi @GabrielAstieres, if you rebase on |
335154f to
06699e7
Compare
Hey @michaelsproul! |
|
Thanks! |
|
Hey Michael @michaelsproul, did you get a chance to look at this? :) |
There was a problem hiding this comment.
It's up to the team, but personally I think we should remove this workflow.
There was a problem hiding this comment.
@jtraglia Are you concerned about noisy breakages, or accidentally revealing some bug on public CI? Both?
There was a problem hiding this comment.
Just noise. Lots of the time, these tests will fail due to unimplemented spec changes. And I'm planning on aggregating the status of these in a new website (and eventually on Hive).
|
Closing to prevent noisy breakages. LH generally won't be able to keep up with nightly test vectors |
|
Hey @michaelsproul & @GabrielAstieres, sounds good. Let's open a new PR which just adds the ability to run the nightly reference tests, eg the Makefile stuff from here. |
|
Sounds good. Maybe even a Github action that we could trigger on demand? |
Issue Addressed
Hello !
This PR aims to have a new nightly action that pulls the nightly generated artifacts from the CL specs repo and have the unstable branch run against them.
See: https://github.com/ethereum/consensus-specs/actions/workflows/nightly-tests.yml
Proposed Changes
To achieve this goal, this PR:
.github/workflows/nightly-tests.ymlMakefile run-ef-nightly-teststesting/ef_tests/MakefilePlease note, this required a Github Authentication Token, I reused the one existing in the Github secret vault that has repo access:
secrets.LIGHTHOUSE_GITHUB_TOKEN