-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Problem
When starting data extraction on a large block range or on a job handling large amounts of data, the program will consume all the available memory, possibly rendering the host PC irresponsive if left for too long.
This hasn't been tested extensively but is visible with any sizable workload sent to the tool.
Proposition
Several safeguards could be put in place to prevent this (ranging from most amount of work to least):
- Monitor resources used, warn and possibly take actions (pause
asynciotasks, send processed blocks to post-process, save state, etc.) when certain threshold are attained. - Just warn in case of high memory usage detected (so still include monitoring from the first option).
- Warn preemptively by judging the size of the workload at the beginning.
- Only warn in the documentation.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request