Skip to content

Conversation

@hermya
Copy link
Contributor

@hermya hermya commented Nov 5, 2024

Fixed flaky test-case

Fixes Issue

PR Branch
fork

Motivation and Context

Test Case
createRequestBuilder_spaceInKeyValue
createRequestBuilder
createRequestBuilder_jsonValue
createRequestBuilder_frontSlash

POINT OF FAILURE

The above mentioned test-case are flaky in nature as they assert the string obtained from requestBuilder.getEntity(). The root cause of the issue lies in: FileUploadUtils:getFileFieldNameValue where the json of reqBody (in BasicHttpClientTest) is converted to a HashMap and its content is unordered. This causes the requestBuilder.getEntity() to return a non-deterministic string whose content is similar but differs in ordered.

Example:

String entity = "Company=Amazon&Title=CEO&state%2Fregion=singapore+north";
String entity2 = "Title=CEO&Company=Amazon&state%2Fregion=singapore+north"; # different string result due to non-deterministic order in HashMap entries

The fix converts the entity string to a Set<String> of name-value pairs (splitting at &), and asserts equality of sets, where order of content doesn't make a difference as opposed to string-representation.

Fixed using NonDex pugin

For a particular test-suite, run:

mvn -pl ./core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=org.jsmart.zerocode.core.httpclient.BasicHttpClientTest -DnondexRuns=10 -DnondexMode=ONE

For more information : https://github.com/TestingResearchIllinois/NonDex

Checklist:

  • New Unit tests were added

    • Covered in existing Unit tests
  • Integration tests were added

    • Covered in existing Integration tests
  • Test names are meaningful

  • Feature manually tested and outcome is successful

  • PR doesn't break any of the earlier features for end users

    • WARNING! This might break one or more earlier earlier features, hence left a comment tagging all reviewrs
  • Branch build passed in CI

  • No 'package.*' in the imports

  • Relevant DOcumentation page added or updated with clear instructions and examples for the end user

    • Not applicable. This was only a code refactor change, no functional or behaviourial changes were introduced
  • Http test added to http-testing module(if applicable) ?

    • Not applicable. The changes did not affect HTTP automation flow
  • Kafka test added to kafka-testing module(if applicable) ?

    • Not applicable. The changes did not affect Kafka automation flow

Failing test cases:
createRequestBuilder_spaceInKeyValue
createRequestBuilder
createRequestBuilder_jsonValue
createRequestBuilder_frontSlash
@hermya
Copy link
Contributor Author

hermya commented Sep 30, 2025

Closing this PR, fix available in #732

@hermya hermya closed this Sep 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant