Questions about the results

[VongolaWu]

May I ask some questions about the test results?
Could you explain the meanings of the names:
Time, intp_testPSNR_prime, deblur_testPSNR_prime, intp_testPSNR, deblur_testPSNR.

Does this 'Time' represent the whole time for deblurring and interpolating 8 different images or for 1 image?
What is the difference between 'prime' and not prime?
Also, why are the results always followed by an (avg)? What is it?

For example:

[JihyongOh]

@VongolaWu
Thank you for your interest.

1. 'prime' means the performances for the architecture of DeMFI-Net_bs(first stage)'s outputs, w/o 'prime' means the performances for those of DeMFI-Net_rb(second stage)'s.

2. 'Time' represents an inference time (runtime) for one image.

3. Performance results are shown based on the total average for each time index (1~7: intermediate time index for joint deblurring and interpolation [MFI x8], 8: time index for only deblurring).

[VongolaWu]

I still feel a little confused. Could you further explain it:

The part below "------x8 MFI results----" should be the final result, right? Using the above image as an example, I have already split it as 5 parts: "Test after Epoch", "x8 MFI results[PSNR Stage 1]", "x8 MFI results[SSIM Stage 1]", "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]"

1. What is the relationship between "Test after Epoch" and the other 4 parts? Is the "Test after Epoch" whole report for all the test images? But I'm still confused about the (avg:) after each item. For example, "intp_testPSNR: 2.6699e+01 (avg:2.3329e+01)", what does the avg mean?
2. Also I still feel confused about the (avg) in "x8 MFI results[PSNR Stage 1]", "x8 MFI results[SSIM Stage 1]", "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]".
3. Does "Time" mean that generating 8 sharp images for 1 blur image needs 1.136 seconds?

[JihyongOh]

@VongolaWu

1. Yes, first block is average of all results for all time indices. "avg:~" means that it shows literally average on that time index (in your example, "avg:2.3329e+01" is a final performance for intp_testPSNR).
2. The final DeMFI-Net_{rb} (Stage II) performance is based on "avg" values of "x8 MFI results[PSNR Stage 2]" and "x8 MFI results[SSIM Stage 2]" for each time index (as in Fig. 1). Therefore, the "Average" performance in Table is calculated by averaging all "avg" values for all time indices of "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]" for each metric.
3. No, "Time" means an average runtime for running DeMFI-Net once, i.e. synthesizing each pair of three frames (two deblurred frame - S0, S1 and one interpolated image - St)

[VongolaWu]

Thank you for your explanation.
Let me ask a further question:

I make a table for this result.

Therefore, for the first table, the intp_testPSNR(avg) should be the final results for 7 interpolated frames? and deblur_testPSNR(avg) should the final results for the 1 deblurred frames?
But they are not consistent with the second table.
So I would like to ask if I would like to represent the average PSNR for the whole test dataset containing both interpolated frames and deblurred frames, which number should I use?

[JihyongOh]

@VongolaWu
First table is for an average of "whole" samples, regardless for each scene's numbers of frames.
Second table shows an average of "average for each scene", as done in BIN paper's evaluation. It considers that the numbers of frames are different for each scene. Therefore, it is recommended to use the second table format to be robust to imbalance of numbers of frames per each scene. (in your case, average 30.8135 dB to evaluate joint performance)

[VongolaWu]

@JihyongOh
Thank you for your reply. One more thing:
For PSNR_1 to PSNR_7, are they the generated frame at timestamp t*1/8 (eg. PSNR_1 is at time 1/8)?
And the PSNR_blur is the sharp frame of B0?

[JihyongOh]

@VongolaWu
Yes, your understanding for PSNR_1 (1/8) to PSNR_7 (7/8) is right (multiple of 1/8), and PSNR_blur is measured for the sharp frame of S1 and S1 without overlapped evaluation by considering a temporal window. Please refer these two lines; line1 and line2.