Technology Strategy for Meticulous Governance of Smart Cities: Memory Bus Width Aware Pruning for Efficient Image Super-Resolution Implementation in FPGA

Abstract

The mega-city governance is based on the aggregation, collation, development and application of multi-scene data, and efficient access to scene data is a key link to promote the meticulous governance of smart cities. Super-resolution technique is the process of upscaling and improving the details within an image. In this paper, we implement a 16-layer residual neural network (ResNet) for the efficient image super-resolution in FPGA. We discover that the memory access is the major performance bottleneck of this implementation. To reduce the memory access overhead, we design a pruning algorithm with the consideration of the memory bus width. Since the memory used in our design generates 256 bits for each access, the proposed pruning algorithm drops the kernel by aligning this bit width. That is, all kernels in one layer are ranked by its L1-norm and we drop the kernels out of the 256 bits. The experimental results show that the proposed method reduces the number of weights by 50% compared with the baseline. As a result, the inference speed can be enhanced by 3 times.