Skip to content

Conversation

@max-leuthaeuser
Copy link
Contributor

@max-leuthaeuser max-leuthaeuser commented Jan 12, 2026

I have seen a lot of 403s and timeouts while downloading stuff during tests for frontends (especially during wacky Cloudflare things in the last weeks). Some frontend tests rely on downloading dependencies (e.g., kotlin2cpg for resolving types, csharpsrc2cpg) during tests and fail hard if some resource is affected.

We could gradually fix/adjust the tests affected in these frontends over time.

…ential backoff support

I have seen a lot of 403s and timeouts while downloading stuff during tests for frontends (especially during wacky Cloudflare things in the last weeks). Some frontend tests rely on downloading dependencies (e.g., kotlin2cpg) and fail hard if some resource is affected.
Copy link
Contributor

@maltek maltek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do these tests fail with exceptions that point to a network failure? Or does the download code internally ignore the failure, and then just fail later with random assertion failures or match errors?

I'd like this a lot more if it only retried for a (configurable) exception type that is associated with the flakyness. Otherwise, non-flaky failing tests can take a loong time to show up if there are enough of them. But that only works if the flaky network code (can be configured to) fatally throw a recognizable exception type.

This can also only work for test code that doesn't work on any mutable state. We could end up throwing exceptions just from repetition alone for tests that are not idempotent. But I guess that's just a gotcha to note when using these wrappers.

@max-leuthaeuser
Copy link
Contributor Author

do these tests fail with exceptions that point to a network failure? Or does the download code internally ignore the failure, and then just fail later with random assertion failures or match errors?

I have seen both. The first type during kotlin tests, the second in csharp.

@maltek
Copy link
Contributor

maltek commented Jan 13, 2026

I think I'd add a generic type parameter then to constrain the exception types which trigger a retry. For better-behaved tests we can be more specific then. For the others, we can still just use Exception or Throwable. (Though ideally we'd fix them to not swallow the network exceptions.)

With that, 👍 from me.

@max-leuthaeuser max-leuthaeuser marked this pull request as ready for review February 5, 2026 09:07
@max-leuthaeuser max-leuthaeuser changed the title RFC: [x2cpg] Add retryable assertion scalatest util with timeout and exponential backoff support [x2cpg] Add retryable assertion scalatest util with timeout and exponential backoff support Feb 5, 2026
@max-leuthaeuser max-leuthaeuser merged commit 2f3d248 into master Feb 10, 2026
8 checks passed
@max-leuthaeuser max-leuthaeuser deleted the max/scalatestRetry branch February 10, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants