Reconsider default runtime? (mediapipe vs. tf.js)

[shiffman]

I've noticed that almost all end-of-semester final projects, where students typically use just one model, greatly benefit from { runtime: 'mediapipe' }. I wonder if we should reconsider making this the default, paired with clear documentation and instructions for switching to other runtimes when using multiple models. (see #69 #134)

Additionally, there's #225, which could be worth exploring further in this context.

@ziyuan-linn, is there anything I might be overlooking here?

[ziyuan-linn]

I think mediapipe runtime's memory leak issue is still the main concern. If we do switch over, we would want clear documentation that any long-running projects (in a gallery or show setting for example) should use the tfjs runtime. I am happy to take a look again and investigate whether mediapipe's memory leak has been addressed not.

Additionally, this would be a breaking change. For some models, the options are different depending on which runtime is selected.

[shiffman]

Ah right, I forgot about the memory leak! Let's continue to leave tf.js as the default then and revisit if/when that is resolved! Can you elaborate on how this is "breaking", I'm having trouble imagining a scenario where that would apply.

[ziyuan-linn]

Ah I looked through the source code and it would be as big of a problem as I thought. The "breaking" probably only applies to one case: when loading a offline model, a tfjs models require detectorModelUrl and landmarkModelUrl in the options object, but mediapipe models require solutionPath instead.