Reproducing results

[itsyoavshalev]

Hi, I'm having problems while trying to reproduce the numeric results using your code.
For CLIP I'm getting R1:29.66 R5:54.14 R10:65.24, which is different but very close to the results presented in the paper.
But for CLIP-clip I'm getting R1:23.82 R5:46.5 R10:56.56, which is much worse.

Thanks.

[Faldict]

Hi, can you show which CLIP model you are using? For different architectures, the result might be slightly different.

[itsyoavshalev]

Hi, ViT-B/32 - I'm just running clip_coco.py.
I'm also getting better bias results without clipping.
Can you please share all input files for Flickr30k and MS-COCO that reproduce the numeric results appearing in the paper?

Thanks.

[Faldict]

Hi, I just take a look. The recall results are evaluated with the original captions. This should have been mentioned in the experimental details. Sorry for the confusion.

[itsyoavshalev]

Hi, as I mentioned, I can't reproduce the bias results as well.
Can you please share all input files for Flickr30k and MS-COCO that reproduce the numeric results appearing in the paper?

Thank you very much for replying.

[nayeon7lee]

Hi @Faldict ! thanks for the great work. I would also appreciate if you could provide all these input files so that it is easier for us to replicate! thanks 👍