What's the learning rate of FLatten-DeiT? #35
Description
Is it 5e-4 or 1e-3?
It is mentioned in paper "The basic learning rate for a batch size of 1024 is set to 1 × 10^(-3)", so if it follow DeiT recipe, than true lr of batch size 1024 is 2× 10^(-3)?