Improve live replay of failures from other workers #151
Description
If two workers are working on the same nodeid, and one worker finds a failure, the other worker currently continues to fuzz that nodeid. Instead, we would like the second worker to:
- Stop working on the failing test for n minutes, to give the other worker time to shrink the failure.
- If we don't get a database update that the failure has been fully shrunk after n minutes (refresh on every successful shrink?), replay and start shrinking that failure ourselves, under the assumption the other worker has dropped out..
- Also: be careful around multiple interesting origins, and around failure size, in case we already have this failure (at a different shrunk state).
via #113 (comment)
(to check: if a worker starts up and replays an unshrunk failure, it should start shrinking it again before stopping; we might not do this currently)