Framework to Benchmark CNNs (FaBCNN) for Processing Real-Time HD Video Streams on FPGAs

Abstract

The deployment of Convolutional Neural Networks (CNNs) on resource-constrained edge devices for inference is challenging due to its computation, memory, energy, and bandwidth requirements. To address these issues, FPGAs are commonly used to implement CNNs because of their high flexibility and low power consumption. This paper proposes a methodology that provides a technique to benchmark CNNs using HDMI input and output in real-time with 720p high definition (HD) resolution. This methodology can be utilized in a classroom set up to teach CNN and computer vision fundamentals. To illustrate the effectiveness of the proposed methodology, several object detection and image classification CNNs were deployed on the Xilinx ZCU104 FPGA board. Video is provided to the FPGA in real-time from an HDMI input source. The output of a given CNN is converted to an HDMI stream and displayed on a separate monitor at 720p HD resolution. The experimental results show that this methodology can perform object detection and image classification on real-time video at speeds of around 10 FPS and 30 FPS, respectively.