Skip to content

fix: correct setDeletionBillingTimeStamp trigger to set deletionTime#4577

Merged
mociarain merged 12 commits intomasterfrom
gniranjan/CosmosbugFix
Feb 3, 2026
Merged

fix: correct setDeletionBillingTimeStamp trigger to set deletionTime#4577
mociarain merged 12 commits intomasterfrom
gniranjan/CosmosbugFix

Conversation

@gouthamMN
Copy link
Contributor

Which issue this PR addresses:

Fixes: https://issues.redhat.com/browse/ARO-23757

What this PR does / why we need it:

The setDeletionBillingTimeStamp CosmosDB trigger was incorrectly setting the creationTime field instead of deletionTime when marking billing documents for deletion. This fix updates the trigger to correctly populate the deletionTime field when MarkForDeletion is called.

This was introduced in the PR #3584

The trigger is executed server-side in CosmosDB as a pre-trigger on Replace operations, ensuring atomic timestamp updates.

Files updated:

  • pkg/deploy/generator/const.go: Fixed trigger JavaScript function
  • Generated deployment templates will be updated via make generate

Test plan for issue:

  • existing e2e
  • existing unit tests

Is there any documentation that needs to be updated for this PR?

  • N/A

The setDeletionBillingTimeStamp CosmosDB trigger was incorrectly setting
the creationTime field instead of deletionTime when marking billing
documents for deletion. This fix updates the trigger to correctly populate
the deletionTime field when MarkForDeletion is called.

This was introduced in the PR [#3584](#3584)

The trigger is executed server-side in CosmosDB as a pre-trigger on
Replace operations, ensuring atomic timestamp updates.

Files updated:
- pkg/deploy/generator/const.go: Fixed trigger JavaScript function
- Generated deployment templates will be updated via make generate

Signed-off-by: Goutham Muguluvalli Niranjan <gniranjan@microsoft.com>
hlipsig
hlipsig previously approved these changes Jan 30, 2026
Copy link
Collaborator

@hlipsig hlipsig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Refactored CosmosDB trigger functions in const.go to use raw string
literals (backticks) instead of escaped strings for improved readability
and maintainability.

Signed-off-by: Goutham Muguluvalli Niranjan <gniranjan@microsoft.com>
mociarain and others added 9 commits February 1, 2026 13:30
The original code used r.instance which was fetched once at reconciliation
start. If another process modified the Cluster resource during the
reconciliation loop, the patch would fail with a conflict error (HTTP 409)
due to stale resourceVersion.

This change wraps the annotation update in retry.RetryOnConflict, which is
the standard Kubernetes pattern for optimistic concurrency control. Each
retry iteration fetches a fresh copy of the Cluster resource, ensuring we
have the latest resourceVersion before patching.

Trade-offs:
- Extra API call: Each retry does a Get before Patch, adding latency but
  ensuring correctness
- Retry limits: retry.DefaultRetry uses exponential backoff with ~5 retries;
  in extreme contention scenarios this could still fail
- No r.instance sync: The in-memory r.instance is not updated, which is fine
  since it's only used for reading during reconciliation and the annotation
  update is terminal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Collaborator

@mociarain mociarain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I'd like to see a review from some of the folks who worked on the local CosmosDB recently and got deep into the triggers. I've asked them so 🤞 . Not blocking

Also the E2E test failure is a know flake. This PR should address is: #4572

@aasserzo
Copy link
Collaborator

aasserzo commented Feb 2, 2026

LGTM
Once thing I'm curious about is what kind of effect this bug had on billing

@gouthamMN
Copy link
Contributor Author

Once thing I'm curious about is what kind of effect this bug had on billing

We had many orphaned billing docs as the billing wasn't deleting them due to deletion time not set. No impacts on the pricing or charging the customers.

@gouthamMN
Copy link
Contributor Author

@mociarain - I merged the changes from PR #4572. Will you or @hlipsig could approve and merge this PR if everything looks good on CI?

Copy link
Collaborator

@mociarain mociarain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tests have actually passed. It was just the cluster cleanup and we had a recent PR merged that will help with this

@mociarain mociarain merged commit a13d45a into master Feb 3, 2026
28 of 30 checks passed
@mociarain mociarain deleted the gniranjan/CosmosbugFix branch February 3, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants