Rotation metadata support

[mollyxu]

Add rotation metadata support

Context: #1084

Videos recorded on mobile devices often store rotation metadata indicating how the video should be displayed. TorchCodec now extracts this metadata and applies the rotation during decoding so frames are returned in the correct orientation.

This PR adds support for extracting and applying rotation metadata from video streams. The rotation angle (in degrees counter clockwise) is extracted from the display matrix side data and:

1. Exposed via VideoStreamMetadata.rotation
2. Automatically applied during decoding — frames are rotated so they display correctly

When there are user transforms, the rotation is applied before the transforms as a part of filtergraph. In the future, we should benchmark this against alternatives and see if there should be optimizations.

from torchcodec.decoders import VideoDecoder

decoder = VideoDecoder("rotated_video.mp4")
print(decoder.metadata.rotation)  # e.g., 90

The rotation property

• None: The video has no rotation metadata (most videos)
• float: The rotation angle in degrees (counter-clockwise rounded to the nearest 90 degrees) needed for correct display. Common values are 0, 90, 180, -90, or -180.

Implementation

Metadata extraction (FFMPEGCommon.cpp):

• Added getRotationFromStream() which extracts the rotation from AV_PKT_DATA_DISPLAYMATRIX side data
• Handles FFmpeg version differences:
  • FFmpeg ≥ 6.1: Uses av_packet_side_data_get() with codecpar->coded_side_data
  • FFmpeg 5/6: Uses av_stream_get_side_data() with deprecation warning suppression for FFmpeg 6
  • FFmpeg 4: Uses av_stream_get_side_data() with int* size parameter
• Uses av_display_rotation_get() to compute the angle from the display matrix

Metadata propagation:

• StreamMetadata struct now includes an std::optional<double> rotation field (Metadata.h)
• SingleStreamDecoder populates this during stream scanning
• Python VideoStreamMetadata dataclass exposes rotation: float | None (_metadata.py)
• JSON metadata serialization includes the rotation value (custom_ops.cpp)

Test coverage

• Added nasa_13013_rotated.mp4 test resource with 90° rotation metadata
• generated video using

ffmpeg -y -display_rotation 90 -i test/resources/nasa_13013.mp4 \
  -c copy \
  test/resources/nasa_13013_rotated.mp4

• test_rotation_metadata (in test_metadata.py): Verifies rotation is correctly extracted for rotated videos and is None for non-rotated videos
• test_rotation_applied_to_frames (in test_decoders.py): Performs pixel-exact comparison with ffmpeg CLI output
• test_rotation_with_transform (in test_decoders.py): Verifies that user-specified transforms (Resize, CenterCrop, RandomCrop) work correctly with rotated videos, ensuring output dimensions match the requested (H, W) regardless of rotation metadata
• test_rotation_with_transform_pipeline (in test_decoders.py): Verifies that chained transforms work correctly with rotated videos, with each transform operating in post-rotation coordinate space

[Dan-Flores]

Thanks for working on this! I left one small comment, but overall LGTM!

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1173

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2fb9f32 with merge base 06d4538 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

Thank you @mollyxu ! Left a bunch of comments but I'll approve now to unblock.