Skip to content

fix testonly with js_binary so it's not a test#1486

Open
Aghassi wants to merge 3 commits intoaspect-build:2.xfrom
Aghassi:patch-3
Open

fix testonly with js_binary so it's not a test#1486
Aghassi wants to merge 3 commits intoaspect-build:2.xfrom
Aghassi:patch-3

Conversation

@Aghassi
Copy link
Collaborator

@Aghassi Aghassi commented Feb 21, 2024

the current logic is wrong because it tries to test js_binary when testonly is passed. We actually want the inverse which is we don't want to run js_binary as a test if it is marked as test_only.

Bug discovered by myself and @joeljeske


Type of change

  • Bug fix (change which fixes an issue)

For changes visible to end-users

  • Breaking change (this change will force users to change their own code or config)
  • Relevant documentation has been updated
  • Suggested release notes are provided below:

Test plan

  • Covered by existing test cases
  • New test cases added
  • Manual testing; please provide instructions so we can reproduce:

@gregmagolan
Copy link
Member

Interesting one. Suppose there is no value of tagging a js_binary as testonly. I'm still confused how you get this to fail since neither bazel test or bazel coverage should run a js_binary target as test right?

@jbedard
Copy link
Member

jbedard commented Feb 21, 2024

Can you show an example that fails without the change?

@joeljeske
Copy link
Contributor

joeljeske commented Feb 21, 2024

We have a js_binary target that is not a test itself, but a dependency of a test. The testonly here is a guard against accidental usage outside of a test dag.

Presumably you could fix this more properly by having js_test also have a _is_test attr that is set to true. This would allow you to ensure all js_tests have a lcov_merger but allow a js_binary to be used with testonly = True.

@jbedard
Copy link
Member

jbedard commented Feb 21, 2024

Can that be shown with a test that fails without this change though?

@Aghassi
Copy link
Collaborator Author

Aghassi commented Feb 21, 2024

@jbedard here is a screenshot from me reproducing the failure with bazel coverage using vscode codespaces off of main
image

Here is the code

# A simple program that runs the Acorn JS parser to produce an AST
js_binary(
    name = "my_bin",
    # Reference the location where the acorn npm module was linked in the root Bazel package
    data = ["//:node_modules/acorn"],
    entry_point = "require_acorn.js",
    testonly = True
)

js_test(name = "test_testonly_bin", data = ["my_bin"], entry_point = "npm_version_test.js")

@Aghassi
Copy link
Collaborator Author

Aghassi commented Feb 21, 2024

I can confirm also that applying this fix with that code fixes the bug

@thesayyn
Copy link
Member

i believe this is still wrong as _lcov_merger should be present if the coverage is enabled but since we are sharing implementation between js_test and js_binary we need a sentinel attribute for determining where the impl is being invoked so we can still fail if it's js_test.

@Aghassi
Copy link
Collaborator Author

Aghassi commented Feb 21, 2024

@thesayyn so are you suggesting that @joeljeske's fix would be preferred?

Presumably you could fix this more properly by having js_test also have a _is_test attr that is set to true. This would allow you to ensure all js_tests have a lcov_merger but allow a js_binary to be used with testonly = True.

#1486 (comment)

@thesayyn
Copy link
Member

Exactly.

@Aghassi
Copy link
Collaborator Author

Aghassi commented Feb 21, 2024

@thesayyn Ok I believe I have added the fix as interpreted from @joeljeske's comment. I confirmed with the same test as before that this fix works too

@CLAassistant
Copy link

CLAassistant commented Feb 22, 2024

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@Aghassi
Copy link
Collaborator Author

Aghassi commented Feb 22, 2024

Looks like this breaks some CI stuff. I'll have to spin up an ubuntu container and debug why bazel coverage //... is failing

Aghassi added 3 commits March 1, 2024 04:15
the current logic is wrong because it tries to test `js_binary` when `testonly` is passed. We actually want the inverse which is we don't want to run js_binary as a test if it is marked as test_only
This addresses the problem in a different way that allows us to not change the existing logic, but instead differentiate one step further with attributes
@thesayyn thesayyn removed their request for review September 12, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants