Forward timing benchmark meaning

[dalishi]

Hi this is not a issue I am just not sure where to ask questions. If there is another place to put questions, please let me know. Thanks.

Does the "forward" timing in the benchmark mean the time the network takes to inference a SINGLE image (image/sec) or the time to process a batch (e.g. 16 images)? Take the AlexNet on Maxwell Titan X as an example which is 7.09ms. How can I deduce the images/sec from it? Is it 1/7.09ms=141image/sec or 1/(7.09ms/16)=2256image/sec? According to what NVIDIA announced, the specs for AlexNet on Maxwell Titan X is 450images/sec. However, neither of these two values (141 and 2256) is close which is quite confused.

http://cdn.wccftech.com/wp-content/uploads/2016/01/NVIDIA-Drive-PX-2-Specifications.jpg

Thanks and look forward to reply.

[caldweln]

From looking at the code, its forward time for the batch.
So for AlexNet on Maxwell Titan X, it took 7.09ms (average over 10 runs) to run a forward pass over the batch of 16 images.

Similar figures appear here: https://devblogs.nvidia.com/inference-next-step-gpu-accelerated-deep-learning/
And a similar query you raised is here: https://devtalk.nvidia.com/default/topic/1020769/how-to-understand-the-specs-450image-sec-using-alexnet-on-titan-x-/

It's all in the batching!