Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue Addressed
#39
Proposed Changes
Introductory rough draft to spec tests. This is extremely WIP and any feedback is welcome.
I have attempted to setup a general structure that provides a clean interface for parsing, setting up, and running the spec tests. A key observation here is that spec tests are divided into categories (can discern which category based on .json file name) and each category has a different json structure. We need to be able to read these into an internal format, setup the test, and then run the test.
Test are identified via the
SpecTestTypeenum, which is further subdivided into specifics areas where each inner enum has a variant for each test category.The core behind this is the
SpecTesttrait. Each different test category must implement this trait on its respective struct. This provides functionality for setting up the test, running the test, and identifying which test category it is.All test categories must register itself with the
register_test_loadersmacro. This allows for a very clean way of parsing and running all of the test categories in a generic way. Without this, we would have to match against all variants and optimistically decode each one in succession.The whole flow is roughly the following
run_testswith the variant of test that you would like to runThere is still a lot of work to be done for setting up the tests and mapping the types into our
ssv_typesrepresentation, but I think this provides a very extensible and clean framework for implmenting all of the spec tests.Note: Also included only one testfile for each category for now. Otherwise this pr is 200,000 lines and impossible to review.