capture Debug.log and present it with a failing test

[BrianHicks]

When I'm trying to figure out why a test is failing, I'll often put Debug.log statements into the things I'm modifying to see what's up. If I run my whole test suite after that, I get way more log messages than I want, with no indication of which ones are actually relevant to the test failure I'm trying to debug.

I'd really like it if, instead, elm-test would capture stdout and associate it with a given test run. A few other test frameworks I've used (py.test maybe? but it's been a while) do this and it's super nice.

If I could have a pony, I'd also like this for fuzz tests. I'd like to only see the debug messages from the final shrunk failure case!

[BrianHicks]

current workaround: slapping an only on the relevant test. I don't like this as much because it makes me choose between useful debug output and running the whole suite to make sure I didn't break something else with a change.

[harrysarson]

See rtfeldman/node-test-runner#400 and (slightly less relevant) rtfeldman/node-test-runner#327. Ava does this really nicely and we should too!

[drathier]

I've suggested this in the past too, specifically for fuzz tests, where shrinking duplicates the debug.log output massively. I definitely think it's doable, by passing a context record to the test/fuzz lambdas, where one field could be a log function, which is a no-op until a test fails, or a failing fuzz test has been shrunk enough. This should also allow sorting output by test.

[drathier]

I did a proof of concept at #156