Skip to content

Added new ERROR event to EndpointTestsTraitValidator when builtin params are inconsistent#2955

Merged
sugmanue merged 7 commits intosmithy-lang:mainfrom
rchache:main
Feb 10, 2026
Merged

Added new ERROR event to EndpointTestsTraitValidator when builtin params are inconsistent#2955
sugmanue merged 7 commits intosmithy-lang:mainfrom
rchache:main

Conversation

@rchache
Copy link
Contributor

@rchache rchache commented Feb 4, 2026

Background

  • Added an ERROR level validator that checks for invalid EP tests where the param value is inconsistent with the corresponding builtInParam value.

Testing

  • Added test cases

Links


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

This pull request does not contain a staged changelog entry. To create one, use the ./.changes/new-change command. For example:

./.changes/new-change --pull-requests "#2955" --type feature --description "Added EndpointTestCasesParamsValidator"

Make sure that the description is appropriate for a changelog entry and that the proper feature type is used. See ./.changes/README or run ./.changes/new-change -h for more information.

// All required params from a ruleset must be present in all test cases.
if (parameter.isRequired() && !parameter.getDefault().isPresent()
&& (!testSuiteHasParam || testSuiteParams.get(name).size() != trait.getTestCases().size())) {
&& (!testSuiteHasParam || testSuiteParams.get(name).size() < trait.getTestCases().size())) {
Copy link
Contributor Author

@rchache rchache Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is only a partial/bandaid fix. there is still a bug in extractTestSuiteParameters that double counts and causes testSuiteParams to be larger than it should be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a plan to get the root cause fixed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and it seemed fairly complicated at least to me. I didn't want to accidentally create a regression, so I did this minimal fix for now

@rchache rchache marked this pull request as ready for review February 4, 2026 15:56
@rchache rchache requested a review from a team as a code owner February 4, 2026 15:56
@rchache rchache requested a review from sugmanue February 4, 2026 15:56
@rchache
Copy link
Contributor Author

rchache commented Feb 4, 2026

type Bump = Literal["minor", "patch"]
     ^

SyntaxError: invalid syntax

when running ./.changes/new-change --pull-requests "#2955" --type feature --description "Added EndpointTestCasesParamsValidator"

// All required params from a ruleset must be present in all test cases.
if (parameter.isRequired() && !parameter.getDefault().isPresent()
&& (!testSuiteHasParam || testSuiteParams.get(name).size() != trait.getTestCases().size())) {
&& (!testSuiteHasParam || testSuiteParams.get(name).size() < trait.getTestCases().size())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a plan to get the root cause fixed?

rchache and others added 2 commits February 9, 2026 13:29
Co-authored-by: Manuel Sugawara <sugmanue@amazon.com>
@sugmanue
Copy link
Contributor

sugmanue commented Feb 9, 2026

Ok, I look a bit more into this, the problem you’re to avoid is not AWS-specific, so I think it would be more appropriate to just do it in the class that validates the tests, EndpointTestsTraitValidator, there we already do similar validations but not this case, also there is already some of the information you need already computed.

@rchache rchache changed the title Added EndpointTestCasesParamsValidator Added new ERROR event to EndpointTestsTraitValidator when builtin params are inconsistent Feb 10, 2026
@github-actions
Copy link
Contributor

This pull request does not contain a staged changelog entry. To create one, use the ./.changes/new-change command. For example:

./.changes/new-change --pull-requests "#2955" --type feature --description "Added new ERROR event to EndpointTestsTraitValidator when builtin params are inconsistent"

Make sure that the description is appropriate for a changelog entry and that the proper feature type is used. See ./.changes/README or run ./.changes/new-change -h for more information.

@sugmanue sugmanue merged commit c9c2439 into smithy-lang:main Feb 10, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants