A 250-mW 5.4G-Rendered-Pixel/s Realistic Refocusing Processor for High-Performance Five-Camera Mobile Devices

Abstract

Digital refocusing in multi-camera mobile devices is becoming crucial. Realistic refocusing, which is a subset of digital refocusing, provides physically-correct quality; however, its intense computational complexity results in low processing speed and restricts its applicability. Moreover, its complex computation flow requires substantial DRAM bandwidth and a large SRAM area, making it more challenging to implement in hardware. In this paper, we present a high-performance refocusing processor based on a hardware-oriented realistic refocusing algorithm. The proposed compact computation flow saves 92% of the DRAM bandwidth and 32% of the SRAM area without noticeable quality degradation. To support high-performance refocusing, we develop highly paralleled engines for view rendering. They deliver 5.4G rendered-pixel/sec throughput. The hardware accelerator improves the processing speed by 100× to 350× that of the original refocusing algorithm running on a general purpose processor. The chip is fabricated with 40nm CMOS technology and comprises 271KB of SRAM and 2.3M logic gates. The chip processes Full-HD light fields up to 40 frames per second under 250mW power consumption.