A Multi-Task Hardwired Accelerator for Face Detection and Alignment

Abstract

Face detection and alignment are two fundamental tasks for facial applications and the corresponding accelerators have been designed to enable energy-efficient acceleration. However, these dedicated accelerators are always designed separately, thereby ignoring the inherent correlation between face detection and alignment and causing additional communication and area overhead. Based on this motivation, a multi-task cascaded convolutional networks (MTCNN) algorithm-based accelerator is presented in this work to support both face detection and alignment for multiple faces. First, multiply-accumulate (MAC) operations and memory access of the magnification process in the resize module are reduced by 22.8&#x0025; and 24.8&#x0025; on average, respectively, when compared with those of similar methods. Second, clustering non-maximum suppression (C-NMS) is proposed to significantly reduce the intersection over union computation and eliminate the hardware-inference sorting process in NMS, yielding a 16.0&#x0025; speedup in the overall process. Third, an efficient pipeline architecture is proposed to implement a complexity- and memory-intensive proposal network of MTCNN in a more computationally efficient manner, with 38.3&#x0025; less memory capacity than a similar solution. Meanwhile, only approximately half of the multipliers are needed to achieve the same throughput with high pipeline utilization. Fourth, considering the variable number of faces in each input, a batch schedule mechanism is proposed to improve the fully-connected layer hardware utilization by 16.7&#x0025; on average in the batch process. Based on a simulation with the TSMC 28 nm CMOS process, this accelerator consumes only 10.9ms at 400 MHz to simultaneously process 5 faces. The power efficiency reaches 4.80 TOPS/W, which is <inline-formula> <tex-math notation="LaTeX">$227.4\times $ </tex-math></inline-formula> higher than that of the state-of-the-art solution.