Coco dataset explorer

[AyushExel]

A while back I created ultralytics explorer, but its not open licensed. It'd be nice to create the same feature in this repo

[hardikdava]

@AyushExel Why don't we create a separate project for it?

[AyushExel]

yeah we could..but I imagined this workflow that would make sense if its end-to-end:

• Train a model -> realise its not working well -> use explorer to check dataset quality and export a new version of the dataset -> train it again -> set a custom metric (like similarity, num samples etc ) to filter out or merge data further -> train again -> automate this process programmatically (similar to continuous learning)..

This kind of workflow will only be possible if both features (dataset exploration/curation and training) work well together...

[hardikdava]

Make sense @AyushExel But in general it depends on your vision of how you want to see usage of trolo in future. I think you should think about it and make a list of tasks/feature then some of them will be picked up by external contributors.

[AyushExel]

Yeah I'll try to do that.. I'm kind of tilting towards building anything that the users would want to use. Rather than having rigid idea myself I'd rather take the user-request driven development. But still I'll add some ideas in issues in a day or two

[hardikdava]

@AyushExel yeah make sense. Take your time to think about this. This will help you to develop whole thing in one direction. Most of the users will be ml devs like us. So using multiple package will not be the issue. But maintaining one huge project will bite us in the future. Though using data exploration tool as sub-module will fix our issue. Just think on this.

[AyushExel]

Yeah I was thinking of submodule approach..Roughly like this:

• Dataset.create() (Save current dataset locally incrementing version if no name is provided)
• Dataset.log(provide=) (Lag the dataset to platforms like RF, wandb etc.)

Now this is what a deep integration/submodule would allow seamlessly:

• Trainer.fit(data="trolo://my_dataset_ver1") - Automatically look for the dataset in this order - locally, or any cloud dashboard if found in user config.