More evaluation methods including CIDEr, METEOR and ROUGE besides BLEU1-4

[yolo615]

I use the COCO.api and change the eval.py a little to achieve evaluation using CIDEr, METEOR, ROUGE and BLEU.

The packages of all the evaluation methods are in the folder: evalfunc.

[kmario23]

@ruizhao1997 thanks! I'm reviewing this PR. In the meantime, would it also be possible for you to incorporate SPICE as an additional eval metric? :)

[yolo615]

@sgrvinod Yes, of course. But I have been busy with other things recently. I would add the SPICE a few days later.

[kmario23]

@ruizhao1997 that'd be great! you could also have a look at coco-caption. Specifically, pycocoevalcap where a host of common eval metrics are available.

[zyan97]

@ruizhao1997 Hi, I try to do evaluation using metrics CIDEr, METEOR, and ROUGE. But there is something wrong with METEOR.

Traceback (most recent call last):
File "eval.py", line 200, in
score_dict = evaluate(beam_size)
File "eval.py", line 190, in evaluate
score_i, scores_i = scorer.compute_score(ref, hypo)
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 41, in compute_score
stat = self._stat(res[i][0], gts[i])
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 65, in _stat
self.meteor_p.stdin.write(score_line + '\n')
BrokenPipeError: [Errno 32] Broken pipe

How can I fix this problem? Thanks!

[qin79]

@ ruizhao1997您好，我尝试使用指标CIDEr，METEOR和ROUGE进行评估。但是，METEOR出了点问题。

回溯（最后最近一次调用）：
文件“eval.py”，线200，在
score_dict =评估（beam_size）
文件“eval.py”，线190，在评估
score_i，scores_i = scorer.compute_score（REF，海波）
文件“ /Users/Ryan/a-PyTorch-Tutorial-to-Image-captioning-master/evalfunc/meteor/meteor.py”，第41行，位于compute_score
stat = self._stat（res [i] [0]，gts [ i]）
在“ _stat
self.meteor_p.stdin.write（score_line +'\ n'）
BrokenPipeError：[Errno 32]管道损坏

我该如何解决这个问题？谢谢！

@ruizhao1997 Hi, I try to do evaluation using metrics CIDEr, METEOR, and ROUGE. But there is something wrong with METEOR.

Traceback (most recent call last):
File "eval.py", line 200, in
score_dict = evaluate(beam_size)
File "eval.py", line 190, in evaluate
score_i, scores_i = scorer.compute_score(ref, hypo)
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 41, in compute_score
stat = self._stat(res[i][0], gts[i])
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 65, in _stat
self.meteor_p.stdin.write(score_line + '\n')
BrokenPipeError: [Errno 32] Broken pipe

How can I fix this problem? Thanks!

i meet this error too.have you fixed?

[ydyrx-ldm]

@ ruizhao1997您好，我尝试使用指标CIDEr，METEOR和ROUGE进行评估。但是，METEOR出了点问题。
回溯（最后最近一次调用）：
文件“eval.py”，线200，在
score_dict =评估（beam_size）
文件“eval.py”，线190，在评估
score_i，scores_i = scorer.compute_score（REF，海波）
文件“ /Users/Ryan/a-PyTorch-Tutorial-to-Image-captioning-master/evalfunc/meteor/meteor.py”，第41行，位于compute_score
stat = self._stat（res [i] [0]，gts [ i]）
在“ _stat
self.meteor_p.stdin.write（score_line +'\ n'）
BrokenPipeError：[Errno 32]管道损坏
我该如何解决这个问题？谢谢！

@ruizhao1997 Hi, I try to do evaluation using metrics CIDEr, METEOR, and ROUGE. But there is something wrong with METEOR.
Traceback (most recent call last):
File "eval.py", line 200, in
score_dict = evaluate(beam_size)
File "eval.py", line 190, in evaluate
score_i, scores_i = scorer.compute_score(ref, hypo)
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 41, in compute_score
stat = self._stat(res[i][0], gts[i])
File "/Users/Ryan/a-PyTorch-Tutorial-to-Image-Captioning-master/evalfunc/meteor/meteor.py", line 65, in _stat
self.meteor_p.stdin.write(score_line + '\n')
BrokenPipeError: [Errno 32] Broken pipe
How can I fix this problem? Thanks!

i meet this error too.have you fixed?

i meet this error tootoo.have you fixed?