Abstract
In this paper, based on our previous work, we present a multi-scale super-resolution (SR) hardware (HW) architecture using a convolutional neural network (CNN), where the up-scaling factors of 2, 3 and 4 are supported. In our dedicated multi-scale CNN-based SR HW, low-resolution (LR) input frames are processed line-by-line, and the number of convolutional filter parameters is significantly reduced by incorporating depth-wise separable convolutions with residual connections. As for 3× and 4× up-scaling, the number of channels for point-wise convolution layer before a pixel-shuffle layer is set to 9 and 16, respectively. Additionally, we propose an integrated timing generator that supports 3× and 4× up-scaling. For efficient HW implementation, we use a simple and effective quantization method with a minimal peak signal-to-noise ratio (PSNR) degradation. Also, we propose a compression method to efficiently store intermediate feature map data to reduce the number of line memories used in HW. Our CNN-based SR HW implementation on the FPGA can generate 4K ultra high-definition (UHD) frames of higher PSNR at 60 fps, which have higher visual quality compared to conventional CNN-based SR methods that were trained and tested in software. The resources in our CNN-based SR HW can be shared for multi-scale upscaling factors of 2, 3 and 4 so that can be implemented to generate 8K UHD frames from 2K FHD input frames.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.