Have you tried 90 epochs training with mixup on ImageNet or iNaturalist ? #10
Description
Hi @zs-zhong ,
Have you tried 90 epochs training with mixup on ImageNet or iNaturalist ?
I have made some improvements based on your work, but due to the lack of computing resources, training a model for 180/200 epochs is too time-consuming for me, especially for iNaturalist.
In my reproduction, under the condition of training 90 epochs with mixup (alpha 0.2) on ImageNet-LT, epochs of stage-2 is 10, the accuracy of methods with ResNet-50 are as follows:
| Stage-1 | mixup | Stage-2 | cRT | LWS | |
|---|---|---|---|---|---|
| Reported in Decouple | 90 epochs | 10 epochs | 47.3 | 47.7 | |
| My Reproduce | 90 epochs | 10 epochs | 48.7 | 49.3 | |
| My Reproduce | 90 epochs | ✅ | 10 epochs | 47.6 | 47.4 |
| My Reproduce | 180 epochs | 10 epochs | 51.0 | 51.8 | |
| Reported in MiSLAS | 180 epochs | 10 epochs | 50.3 | 51.2 | |
| Reported in MiSLAS | 180 epochs | ✅ | 10 epochs | 51.7 | 52.0 |
They look much worse than the model trained for 180 epochs with mixup, and it does not even have improvement compared to normal training.
I guess this is because mixup could be regarded as a regularization method, which requires longer training epochs, 90 epochs cannot make the network converge.
However, I cannot get the result of using mixup to train 90 epochs on the iNaturalist data set, because the iNaturalist data set is too large and I can't put it in the memory, which makes it take about a week for me to train R50 once.
If possible, could you please provide the pre-trained ResNet-50 model for training 90 epochs with mixup on iNaturalist? I believe this will also be beneficial for fair comparison of future work.
Thank you again for your contribution and look forward to your reply.