Double-inputs Illumination Pattern Recognizing Model with Automatic Shadow Detection Network in a Single Face Image

Abstract

Illumination pattern recognition of face image has always been a hot research topic in the field of human-computer interaction, and has been widely used in lighting recovery, virtual scene construction and other multimedia fields. Most of the traditional methods achieve this task by analyzing the illumination components from the image texture and structure information, which is often considered to be indirect, complex and time-consuming. This paper introduces a Double-inputs Illumination Pattern Recognizing Model (DIPRM) with Automatic Shadow Detection Network (ASDN) based on Convolution Neural Networks (CNNs). In the proposed coherent system framework, we first annotate the shadow regions of face images with uneven illumination to obtain the face shadow detection dataset. Second, an ASDN which has the encoder-decoder structure is designed. The main architecture of the ASDN is based on the nested U-Net, and for this nesting, the attention mechanism is applied to fuse the output of each sublayer. Third, a double-inputs Facial Illumination Pattern Recognizing Network (FIPRN) following the ASDN is organized, which consists of AlexNet and the attention module. As the double-inputs, the binary image after the shadow segmentation from the ASDN and the original image are input into the FIPRN to make the whole network converge to a good collaboration state eventually. For shadow detection task, the ASDN was evaluated in comparisons with U-Net and UNet++. Experimental results demonstrated that the ASDN achieved an average IoU and Dice gains of 1.9 and 1.2 points over the base-line model with best results. Moreover, the FIPRN was tested in comparison with some baseline models in illumination pattern recognizing task, where the results demonstrated that it achieved an accuracy rate gain of 1.0 points than the AlexNet with a signal-input.