CAMEL-22721 Use CachedOutputStream to enable streaming of large zip f…#21249
CAMEL-22721 Use CachedOutputStream to enable streaming of large zip f…#21249lantzen wants to merge 1 commit intoapache:mainfrom
Conversation
|
🌟 Thank you for your contribution to the Apache Camel project! 🌟 🤖 CI automation will test this PR automatically. 🐫 Apache Camel Committers, please review the following items:
|
| private int testSplitterLargeFile(File testFile) throws IOException, FileNotFoundException, InterruptedException { | ||
| String expectedBody = null; | ||
|
|
||
| System.out.println("Generating 50MB test file..."); |
There was a problem hiding this comment.
remove or change to log.info
|
|
||
| if ("testSplitterLargeFileWithSpoolEnabled()".equals(contextManagerExtension.getCurrentTestName())) { | ||
| StreamCachingStrategy streamCachingStrategy = context.getStreamCachingStrategy(); | ||
| streamCachingStrategy.setSpoolEnabled(true); |
There was a problem hiding this comment.
the default spool threshold is 128kb
|
|
||
| public class ZipSplitterRouteTest extends CamelTestSupport { | ||
| private static final Logger LOG = LoggerFactory.getLogger(ZipIterator.class); | ||
|
|
||
| private static final File testDirectory = new File("test/in"); |
| IOHelper.copy(zipInputStream, baos); | ||
| byte[] data = baos.toByteArray(); | ||
| answer.setBody(new ByteArrayInputStream(data)); | ||
| CachedOutputStream cos = new CachedOutputStream(exchange); |
There was a problem hiding this comment.
who is supposed to close this stream?
There was a problem hiding this comment.
the zip iterator should close it when its done, I would assume when itself is closed - the COS may spool to disk and therefore need to be closed so it can delete the temp file
| } | ||
|
|
||
| private void captureMemory(AtomicInteger storage, String logPrefix) { |
There was a problem hiding this comment.
System.gc() is not reliable, I'd remove this captureMemory()
4 participants
…iles
Description
Use CachedOutputStream to enable streaming of large zip files and added tests that verify that the ZipSplitter doesn't read the whole files in the zip file into memory
Target
mainbranch)Tracking
Apache Camel coding standards and style
mvn clean install -DskipTestslocally from root folder and I have committed all auto-generated changes.