Loss function problems

[jacky4323]

Hi ,

thanks for your great work
I have some questions.
Why in the details implementation, just use square than mean,not using L2-norm in the paper you described?

def at(x):
    return F.normalize(x.pow(2).mean(1).view(x.size(0), -1))


def at_loss(x, y):
    return (at(x) - at(y)).pow(2).mean()

[lndip]

Hi, I also wonder why mean(1) is in line F.normalize(x.pow(2).mean(1).view(x.size(0), -1)) instead of sum(1) as described in the paper. Have you got any different performance in this 2 versions?
And the normalization, should it be sample-wise normalization as F.normalize(x.pow(2).mean(1).view(x.size(0), -1), dim=-1) ? Because the code is currently using batch statistics to normalize the features.