Skip to content

Fix GzipDecompressor to handle concatenated gzip members#3577

Open
veeceey wants to merge 1 commit intotornadoweb:masterfrom
veeceey:fix/issue-3560-gzip-concatenated-members
Open

Fix GzipDecompressor to handle concatenated gzip members#3577
veeceey wants to merge 1 commit intotornadoweb:masterfrom
veeceey:fix/issue-3560-gzip-concatenated-members

Conversation

@veeceey
Copy link

@veeceey veeceey commented Feb 8, 2026

Summary

Fixes #3560

The GzipDecompressor class now properly handles concatenated gzip members by checking for unused_data after each decompression and creating a new decompressor to continue processing subsequent members. This prevents silent data loss when decompressing multi-member gzip streams, which is a common pattern in HTTP streaming and other applications.

Changes

  • Modified GzipDecompressor.decompress() to loop through concatenated gzip members using unused_data
  • Added _flushed flag to prevent decompress() calls after flush() has been called
  • Added comprehensive test cases covering:
    • Single gzip member (regression test)
    • Two concatenated gzip members (issue reproduction)
    • Three concatenated gzip members (edge case)
    • RuntimeError after flush() (API contract)

Test Results

All existing tests pass, and new tests verify the fix:

$ python3 -m pytest tornado/test/util_test.py::GzipDecompressorTest -v
============================== test session starts ==============================
tornado/test/util_test.py::GzipDecompressorTest::test_concatenated_gzip_members PASSED
tornado/test/util_test.py::GzipDecompressorTest::test_decompress_after_flush_raises PASSED
tornado/test/util_test.py::GzipDecompressorTest::test_multiple_concatenated_members PASSED
tornado/test/util_test.py::GzipDecompressorTest::test_single_gzip_member PASSED
============================== 4 passed in 0.02s

The reproduction script from the issue now works correctly:

import gzip
from tornado.util import GzipDecompressor

data1 = b"This is some example data that will be compressed using gzip."
data2 = b"Here is some more example data to demonstrate gzip compression."

member1 = gzip.compress(data1)
member2 = gzip.compress(data2)
concatenated = member1 + member2

decompressor = GzipDecompressor()
decompressed_data = decompressor.decompress(concatenated)
expected_data = data1 + data2

assert decompressed_data == expected_data  # ✓ Passes with fix

The GzipDecompressor class now properly handles concatenated gzip members
by checking for unused_data after each decompression and creating a new
decompressor to continue processing subsequent members. This prevents silent
data loss when decompressing multi-member gzip streams.

Changes:
- Added loop in decompress() to process all concatenated gzip members
- Added _flushed flag to prevent decompress() calls after flush()
- Added comprehensive test cases for single, double, and triple member streams
- Added test to verify RuntimeError after flush()

Fixes tornadoweb#3560
@veeceey veeceey force-pushed the fix/issue-3560-gzip-concatenated-members branch from e74c3f8 to c2af718 Compare February 8, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GzipDecompressor silently loses data from concatenated gzip members

1 participant