About the distillation process #36
Description
Hi, thanks for your great work on AIMv2!
I'm currently working on training a smaller model and am very interested in how to distill the aimv2-huge model into a lighter version.
Could you please share more details or demonstrate the distillation method used in [AIMv2 distilled ViT-Large]?
Any guidance or suggestions would be greatly appreciated. Thanks again for open-sourcing this wonderful work!