Use streaming download for large bulk-data files

[bburda]

Summary

download_bulk_data in client.py currently loads the entire HTTP response into memory via response.content. For rosbag files near or exceeding the current 100 MB cap (max_bag_size_mb), this can cause high memory usage.

Current behavior

response = await client.get(bulk_data_uri, timeout=httpx.Timeout(300.0))
# entire file loaded into memory
return response.content, filename

Proposed solution

Use httpx streaming to write directly to disk in chunks:

async with client.stream("GET", bulk_data_uri, timeout=httpx.Timeout(300.0)) as response:
    async for chunk in response.aiter_bytes(chunk_size=8192):
        file.write(chunk)

This would also require refactoring download_bulk_data to accept an output path (or return a streaming iterator), and updating save_bulk_data_file and download_rosbags_for_fault accordingly.

Context

• Current limit is 100 MB (max_bag_size_mb in gateway config) — manageable in memory but not ideal
• If rosbag sizes grow larger (e.g., longer recording windows), OOM becomes a real risk
• Raised in PR Add bulk-data MCP tools and structured fault response models #7 review (comment)

Location

• src/ros2_medkit_mcp/client.py, line ~1052