SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm2 CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution

Abstract

This paper proposes a convolutional neural network (CNN)-based super-resolution accelerator for up-scaling to ultra-HD (UHD) resolution in real-time in edge devices. A novel error-compensated bit quantization is adopted to reduce bit depth in the SR task. Spatially independent layer fusion is exploited to satisfy high throughput requirements at UHD resolution by increasing parallelism. Burst operation with write mask in the dual-port SRAM increases the process element utilization by allowing the concurrent multi-access without exploiting additional memory. The accelerator is implemented in the 28nm technology and shows at least 4.3 times higher <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{FoM}(\text{TOPS}/\text{mm}^{2}\times \text{TOPS/W)}$</tex> of 0.87 than the state-of-art CNN accelerators. The implemented accelerator supports up-scaling up to 96 frames-per-seconds in UHD resolution.