Skip to content

Add out of the box support for a11y tests#557

Merged
yannbf merged 2 commits intonextfrom
yann/automated-a11y-tests
Apr 30, 2025
Merged

Add out of the box support for a11y tests#557
yannbf merged 2 commits intonextfrom
yann/automated-a11y-tests

Conversation

@yannbf
Copy link
Member

@yannbf yannbf commented Apr 24, 2025

Closes #

What I did

Before:

  • Users had to install axe-playwright and write code from a recipe with test-runner hooks
  • It was easy to miss passing the story context to the a11y report and therefore there would be a mismatch between the addon a11y settings and the a11y test in the test-runner
  • Logs weren't attached to the test result
  • Logs took a lot of space and were hard to parse
    image

After:

  • No need for extra dependencies, works out of the box
  • Respects the a11y.test parameter as per the docs
  • Error logs are attached to the test result
  • Logs are formatted and easy to parse
  • There is a link to debug directly in the a11y panel
    image

Additionally, there is the a11y.test: 'todo' mode, where the tests pass but there are shorter logs in the report.
We should discuss how this should work. Unfortunately the logs aren't attached to the report in this mode.
image

Checklist for Contributors

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes in this repository
  • Request documentation updates in the test-runner docs website

Checklist for Maintainers

  • Make sure this PR contains one of the labels below:

    Available labels
    • skip-release: Skip any releases, e.g., documentation only changes, CI config etc.
    • patch: Upgrade patch version (e.g. 0.0.x)
    • minor: Upgrade patch version (e.g. 0.x.0)
    • major: Upgrade patch version (e.g. x.0.0)

@yannbf yannbf added the minor Increment the minor version when merged label Apr 24, 2025
@yannbf yannbf self-assigned this Apr 24, 2025
@yannbf yannbf requested a review from kasperpeulen as a code owner April 24, 2025 10:07
@yannbf yannbf force-pushed the yann/automated-a11y-tests branch from 9b00a76 to 3c0ceb2 Compare April 24, 2025 10:15
Copy link
Member

@ghengeveld ghengeveld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yannbf
Copy link
Member Author

yannbf commented Apr 29, 2025

smoke test -> render test
add prefix to message with story title > story name

@yannbf yannbf merged commit 2569a2e into next Apr 30, 2025
9 checks passed
@yannbf yannbf mentioned this pull request Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minor Increment the minor version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants