Merge main into feature/http-1.x#4391
Merged
ysaito1001 merged 19 commits intofeature/http-1.xfrom Nov 12, 2025
Merged
Conversation
## Motivation and Context
Bumps MSRV to 1.88.0
## Description
Although the number of files changed is large, the changes themselves
are straightforward. During MSRV upgrade to 1.88.0, one of the clippy
warnings enforced is `clippy::uninlined_format_args`, i.e. suggesting to
update
```
format!("{}", a);
```
to
```
format!("{a}");
```
The PR swept through the codebase and updated almost all the
occurrences, except for:
- codegen-client (suppressed the following)
-
[RequestBindingGenerator](https://github.com/smithy-lang/smithy-rs/pull/4367/files#diff-fea7e7ebc2f668be75ee8e15dd463bd38064b416eea4c0d272e17ce52e2e7598R138-R142)
-
[EndpointTraitBindingGenerator](https://github.com/smithy-lang/smithy-rs/pull/4367/files#diff-bf24c9ba358f747e62b6603c3ddfae91340a9971c41b268a148f705dfface9deR92-R96)
- codegen-server
- #4366
## Testing
- CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context The SDK for that service has been removed in awslabs/aws-sdk-rust#1372. This PR finalizes the cleanup by removing relevant model/tests from this repository. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
## Motivation and Context To be filled, if we decide to proceed with this ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ Co-authored-by: Landon James <lnj@amazon.com>
# SigV4 Event Stream Support for Server SDK
## Problem
Clients wrap event stream messages in SigV4 envelopes with signature
headers (`:chunk-signature`, `:date`), but servers couldn't parse these
signed messages because they expected the raw event shape, not the
envelope.
## Solution
Added server-side SigV4 event stream unsigning support that
automatically extracts inner messages from signed envelopes while
maintaining compatibility with unsigned messages.
## Implementation
### Type System Changes
Event stream types are wrapped to handle both signed and unsigned
messages:
- `Receiver<Events, Error>` → `Receiver<SignedEvent<Events>,
SignedEventError<Error>>`
- `SignedEvent<T>` provides access to both the inner message and
signature information
- `SignedEventError<E>` wraps both extraction errors and underlying
event errors
### Runtime Components
**SigV4Unmarshaller**: Wraps the base event stream unmarshaller to
handle SigV4 extraction:
```rust
impl<T: UnmarshallMessage> UnmarshallMessage for SigV4Unmarshaller<T> {
type Output = SignedEvent<T::Output>;
type Error = SignedEventError<T::Error>;
fn unmarshall(&self, message: &Message) -> Result<...> {
match extract_signed_message(message) {
Ok(Signed { message: inner, signature }) => {
// Process inner message with base unmarshaller
self.inner.unmarshall(&inner).map(|event| SignedEvent {
message: event,
signature: Some(signature),
})
}
Ok(Unsigned) => {
// Process unsigned message directly
self.inner.unmarshall(message).map(|event| SignedEvent {
message: event,
signature: None,
})
}
Err(err) => Ok(SignedEventError::InvalidSignedEvent(err))
}
}
}
```
### Code Generation Integration
**SigV4EventStreamDecorator**:
- Detects services with `@sigv4` trait and event streams
- Wraps event stream types using `SigV4EventStreamSymbolProvider`
- Generates support structures (`SignedEvent`, `SigV4Unmarshaller`,
etc.)
- Injects unmarshaller wrapping via HTTP binding customizations
**HTTP Binding Customization**:
- Added `BeforeCreatingEventStreamReceiver` section to
`HttpBindingGenerator`
- Allows decorators to wrap the unmarshaller before `Receiver` creation
- Generates: `let unmarshaller = SigV4Unmarshaller::new(unmarshaller);`
### Usage
Server handlers receive `SignedEvent<T>` and extract the inner message:
```rust
async fn streaming_operation_handler(input: StreamingOperationInput) -> Result<...> {
let event = input.events.recv().await?;
if let Some(signed_event) = event {
let actual_event = &signed_event.message; // Extract inner message
let signature_info = &signed_event.signature; // Access signature if present
// Process actual_event...
}
}
```
## Testing
- Added `test_sigv4_signed_event_stream` that sends SigV4-wrapped events
- Verifies both signed and unsigned messages work correctly
- All existing event stream tests continue to pass
## Architecture Benefits
- **Backward Compatible**: Unsigned messages work unchanged
- **Type Safe**: Compile-time guarantees about message structure
- **Extensible**: Pattern can be applied to other authentication schemes
- **Minimal Impact**: Only affects services with `@sigv4` trait and
event streams
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [ ] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context Track S3 Express bucket credential usage in User-Agent metrics to provide visibility into S3 Express adoption and usage patterns. ## Description This PR adds tracking for S3 Express bucket credentials by embedding the S3ExpressBucket feature in credentials resolved through the S3 Express identity provider. The feature is reported as metric code J in the User-Agent header. ## Testing - Unit tests verify the feature is correctly embedded in S3 Express credentials - Integration tests verify metric J appears in User-Agent for S3 Express bucket operations - Integration tests verify metric J does NOT appear when S3 Express auth is disabled or for regular buckets ## Checklist <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [ ] I have updated CHANGELOG.next.toml if I made changes to the smithy-rs codegen or runtime crates - [ ] I have updated CHANGELOG.next.toml if I made changes to the AWS SDK, generated SDK code, or SDK runtime crates ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
## Motivation and Context <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> ## Description <!--- Describe your changes in detail --> ## Testing <!--- Please describe in detail how you tested your changes --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> ## Checklist <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [ ] For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "client," "server," or both in the `applies_to` key. - [ ] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
If CI fails, commit the necessary fixes to this PR until all checks pass. If changes are required to [crateNameToLastKnownWorkingVersions](https://github.com/smithy-lang/smithy-rs/blob/92916b5484cdfef9ff58540ebf5e845eeeccf860/aws/sdk/build.gradle.kts#L504), revert the first commit in the PR, run `./gradlew aws:sdk:cargoUpdateAllLockfiles`, and commit the updated lockfiles. Co-authored-by: Landon James <lnj@amazon.com>
If CI fails, commit the necessary fixes to this PR until all checks pass. If changes are required to [crateNameToLastKnownWorkingVersions](https://github.com/smithy-lang/smithy-rs/blob/92916b5484cdfef9ff58540ebf5e845eeeccf860/aws/sdk/build.gradle.kts#L504), revert the first commit in the PR, run `./gradlew aws:sdk:cargoUpdateAllLockfiles`, and commit the updated lockfiles.
## Motivation and Context <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> Bumping crc-fast to the newest version. Keeping it pinned until we do something about #3981 and #4380 ## Testing <!--- Please describe in detail how you tested your changes --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> Made this change in aws-sdk-rust and ran our more comprehensive canaries there against it: https://github.com/awslabs/aws-sdk-rust/actions/runs/19080044066 ## Checklist <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [x] For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "client," "server," or both in the `applies_to` key. - [x] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
## Motivation and Context `V1988105516` ## Description * Adds validation for edge case where region is not a valid host label. * Introduces new `EndpointCustomization::endpointParamsBuilderValidator` method that customizations can use to add further validation to `crate::config::endpoint::ParamsBuilder::build()`. We could have done this in a few different places but this seemed most appropriate. ## Testing New codegen integration test. ## Checklist <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [x] For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "client," "server," or both in the `applies_to` key. - [x] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
## Motivation and Context This PR breaks up `RequestChecksumInterceptor` into two interceptors: `RequestChecksumInterceptor` and `AwsChunkedContentEncodingInterceptor` with additional code restructuring to support the upcoming aws-chunked content encoding enhancement. ## Description `RequestChecksumInterceptor` previously handled two distinct responsibilities: 1. Adding checksums (body or trailer) 2. Applying aws-chunked content encoding This refactor extracts the second responsibility (aws-chunked logic) into `AwsChunkedContentEncodingInterceptor`, aiming to enable it via the dedicated trait when available in Smithy. In addition, both interceptors now wrap bodies in `modify_before_transmit` instead of `modify_before_signing` per design requirements. ## Tips for merging to `feature/http-1.x` This refactor reluctantly adds `http_0x` constructs that will conflict with `feature/http-1.x`. Update these files when merging: - `aws-inlineable/src/http_request_checksum.rs` - Apply same 1.x updates as done [in branch](https://github.com/smithy-lang/smithy-rs/compare/feature/http-1.x?expand=1#diff-7c4006e142da69a7a442cd1d152bf5d0ae05a18c93a02c9754776ef29cef5dcd) (note: `test_checksum_body_is_retryable` was removed as it tested a utility only used in streaming cases against non-streaming bodies) - `aws-inlineable/src/aws_chunked.rs` - Use 1.x for header names, values, and body trait methods in unit tests, as in `http_request_checksum.rs` - `aws/rust-runtime/aws-runtime/src/content_encoding.rs` - Needs 1.x updates ``` fn size_hint(&self) -> http_body_1x::SizeHint { http_body_1x::SizeHint::with_exact(self.encoded_length()) } ``` to ``` fn size_hint(&self) -> http_body_1x::SizeHint { http_body_1x::SizeHint::with_exact(self.options.encoded_length()) } ``` because the `.encoded_length()` method has been moved from `AwsChunkedBody` to `AwsChunkedBodyOptions` (due to the `content-length` header needing encoded length during signing but body creation deferred to `modify_before_transmit`) ## Testing - CI ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
aajtodd
approved these changes
Nov 12, 2025
landonxjames
added a commit
that referenced
this pull request
Dec 2, 2025
**Note:** This PR will need to be released and have at least one daily release run to get the changes into the `aws-sdk-s3` crate before the `feature/http-1.x` branch can be successfully merged to main. ## Motivation and Context <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> When merging `main` back to the `feature/http-1.x` branch in #4422 we started to see a new failure in the [semver hazards test](https://github.com/smithy-lang/smithy-rs/actions/runs/19772608981/job/56659623913). ``` failures: ---- aws_chunked::tests::test_aws_chunked_body_is_retryable stdout ---- thread 'aws_chunked::tests::test_aws_chunked_body_is_retryable' panicked at sdk/s3/src/aws_chunked.rs:195:47: called `Result::unwrap()` on an `Err` value: "Trailers polled while body still has data" ``` The semver hazards test works by checking out the published versions of the SDK crates in the `aws-sdk-rust` repo, patching those crates to use the PRed versions of the runtime crates, and running the tests in the SDK crates with those patches in place. The currently published version of `test_aws_chunked_body_is_retryable` [creates a temp file](https://github.com/awslabs/aws-sdk-rust/blob/d8ccbf91973613be3c4c618fef7f452b347749af/sdk/s3/src/aws_chunked.rs#L168-L171) with size `10000 x line.len()`. The `AwsChunkedBodyOptions` are created with `::default()` which sets `stream_length = 0` . That means we hit [this check](https://github.com/smithy-lang/smithy-rs/blob/d38979ea7f6a3e2f12f5430689f2502a8c00ad38/aws/rust-runtime/aws-runtime/src/content_encoding.rs#L188-L192) in the `http_body_04x::Body` impl for `AwsChunkedBody` which sets the state to `AwsChunkedBodyState::WritingTrailers` without ever polling the inner body. The next time we poll we call `poll_trailers` on the [inner body](https://github.com/smithy-lang/smithy-rs/blob/d38979ea7f6a3e2f12f5430689f2502a8c00ad38/aws/rust-runtime/aws-runtime/src/content_encoding.rs#L228-L229) it ends up at the ["Trailers polled while body still has data" error](https://github.com/smithy-lang/smithy-rs/blob/d38979ea7f6a3e2f12f5430689f2502a8c00ad38/rust-runtime/aws-smithy-types/src/body.rs#L301-L302). This error is correct, we are polling trailers while the body still has data. It turns out Yuki already did a fix for this in his [merge from `main` to the feature branch](#4391) by setting the `stream_length` and changing the [asserts to properly look for the body data](https://github.com/smithy-lang/smithy-rs/blob/d38979ea7f6a3e2f12f5430689f2502a8c00ad38/aws/rust-runtime/aws-inlineable/src/aws_chunked.rs#L204-L231). That change is in an inlineable and still in the feature branch, so the published version of the S3 crate used by the semver hazards check doesn't have the fixed test. The minimum fix here is to port the changes for the tests in `aws_chunked` to main since the current version of those tests is kind of wrong anyway, not polling through the body data at all. We got (un)lucky this showed up in the feature branch because the http-1x `SdkBody` checks for body data when polling trailers since they can't be polled separately. But all of this debugging brings us to the underlying bug, when the inner body of an `SdkBody` is `BoxBody::HttpBody04` our implementation will happily let you poll trailers while the body still has data. As far as I am aware there is no use case that would make this a reasonable thing to do. To fix the actual bug I see two possible solutions: * We could fix the underlying bug by updating the `BoxBody::HttpBody04` branch in `SdkBody`'s `poll_next_trailer` method to return an error by calling `poll_data` on the inner body in `poll_next_trailers` and failing if it returns anything other than `Poll::Ready(none)`. Unfortunately it seems possible that this could cause tricky runtime breakages for existing users, so probably isn't a tenable option. * Update the `BoxBody::HttpBody1` branch in `SdkBody`'s `poll_next_trailer` method to act more like the `BoxBody::HttpBody04` branch. In this case if it encountered a data frame rather than a trailer frame it would keep polling the frames until the body data is exhausted (either buffering or just throwing away the data frames) and we get a trailer frame or run out of frames. This feels incorrect as polling the trailers before the body is exhausted shouldn't ever be correct, and polling the trailer in `HttpBody04` doesn't actually throw away the body bytes, they could still be polled later. ## Testing <!--- Please describe in detail how you tested your changes --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> I used `runtime-versioner patch-runtime` locally to patch the `feature/http-1.x` branch versions of the runtime crates into a local checkout of the`aws-sdk-s3` crate. This reproduced the failure locally. Manually updating the inlineable `aws_chunked.rs` in the S3 crate with the changes from this PR allowed the tests to pass. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
drganjoo
pushed a commit
that referenced
this pull request
Dec 3, 2025
Merge the latest main to the `feature/http-1.x` branch, plus replace http-0.x constructs with http-1.x constructs as described in [Tips for merging to feature/http-1.x](#4384) - CI: ignore server related failures and semver hazard failure (the type being complained about `Uri` isn't exposed since its enclosing module `request` isn't exposed as pub, which confuses the semver hazard check) ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ --------- Co-authored-by: Landon James <lnj@amazon.com> Co-authored-by: Russell Cohen <rcoh@amazon.com> Co-authored-by: AWS SDK Rust Bot <aws-sdk-rust-primary@amazon.com> Co-authored-by: AWS SDK Rust Bot <97246200+aws-sdk-rust-ci@users.noreply.github.com> Co-authored-by: vcjana <vcjana@amazon.com> Co-authored-by: Jason Gin <67525213+jasgin@users.noreply.github.com> Co-authored-by: Aaron Todd <aajtodd@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Merge the latest main to the
feature/http-1.xbranch, plus replace http-0.x constructs with http-1.x constructs as described in Tips for merging to feature/http-1.xTesting
Uriisn't exposed since its enclosing modulerequestisn't exposed as pub, which confuses the semver hazard check)By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.