Skip to content

S3 tree optimizations#2913

Merged
sugmanue merged 10 commits intomainfrom
s3-tree-optimizations
Jan 13, 2026
Merged

S3 tree optimizations#2913
sugmanue merged 10 commits intomainfrom
s3-tree-optimizations

Conversation

@mtdowling
Copy link
Member

Background

  • What do these changes do?
  • Why are they important?

Testing

  • How did you test these changes?

Links

  • Links to additional context, if necessary
  • Issue #, if applicable (see here for a list of keywords to use for linking issues)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

mtdowling and others added 7 commits December 18, 2025 08:56
- Add AdaptiveEffort to dynamically adjust optimization parameters based
  on observed improvement rates (increase effort when making progress,
  decrease when plateauing)
- Add block moves optimization that moves groups of dependent conditions
  together to escape local minima
- Add cost-based tie-breaking using BddCostEstimator when multiple
  positions have the same node count
This commit adds a new function to the rules engine, ite, that performs
an if-then-else check on a boolean expression without branching. By not
needing to branch in the decision tree, we avoid SSA transforms on
divergent branches which would create syntactically different but
semantically identical expressions that the BDD cannot deduplicate.

This commit also adds an S3-specific decision tree transform that
canonicalizes S3Express rules for better BDD compilation:

1. AZ extraction: Rewrites position-dependent substring operations to
   use a single split(Bucket, "--")[1] expression across all branches
2. URL canonicalization: Uses ITE to compute FIPS/DualStack URL segments,
   collapsing 4 URL variants into a single template with {_s3e_fips} and
   {_s3e_ds} placeholders
3. Auth scheme canonicalization: Uses ITE to select sigv4 vs
   sigv4-s3express based on DisableS3ExpressSessionAuth

The transform makes the rules tree ~30% larger but enables dramatic BDD
compression by making URL templates identical across FIPS/DualStack/auth
variants. Endpoints that previously appeared distinct now collapse into
single BDD results, reducing nodes and results by ~43%.
Add nullable boolean transform.
Improve variable and expression sharing.
Add S3 BDD integration test.
Add TreeMapper to simplify transforms.
@mtdowling mtdowling requested a review from a team as a code owner December 31, 2025 18:10
@mtdowling mtdowling requested a review from yasmewad December 31, 2025 18:10
@mtdowling mtdowling force-pushed the s3-tree-optimizations branch 4 times, most recently from 785281e to 6184e14 Compare January 6, 2026 22:41
Split the monolithic S3TreeRewriter into three independent transforms:
- S3AzCanonicalizerTransform: substring → split for AZ extraction
- S3RegionUnifierTransform: unify Region/bucketArn#region/us-east-1
- S3ExpressEndpointTransform: canonicalize FIPS/DualStack/auth

Added region unification pass that rewrites all region references to
_effective_std_region (for aws-global → us-east-1 mapping) or
_effective_arn_region (for UseArnRegion logic). This enables additional
BDD sharing across endpoints with different region sources.

S3 BDD stats: 77 conditions, 97 results, 484 nodes after sifting.
@mtdowling mtdowling force-pushed the s3-tree-optimizations branch from 6184e14 to 21cce54 Compare January 6, 2026 23:13
@mtdowling mtdowling force-pushed the s3-tree-optimizations branch from ac9c854 to 99e402a Compare January 9, 2026 23:40
@sugmanue sugmanue merged commit 4c97b7c into main Jan 13, 2026
16 checks passed
@sugmanue sugmanue deleted the s3-tree-optimizations branch January 13, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants