Abstract
In this paper, we present a novel single shot face-related task analysis method, called Face-SSD, for detecting faces and for performing various face-related (classification/regression) tasks including smile recognition, face attribute prediction and valence-arousal estimation in the wild. Face-SSD uses a Fully Convolutional Neural Network (FCNN) to detect multiple faces of different sizes and recognise/regress one or more face-related classes. Face-SSD has two parallel branches that share the same low-level filters, one branch dealing with face detection and the other one with face analysis tasks. The outputs of both branches are spatially aligned heatmaps that are produced in parallel—therefore Face-SSD does not require that face detection, facial region extraction, size normalisation, and facial region processing are performed in subsequent steps. Our contributions are threefold: 1) Face-SSD is the first network to perform face analysis without relying on pre-processing such as face detection and registration in advance–Face-SSD is a simple and a single FCNN architecture simultaneously performing face detection and face-related task analysis—those are conventionally treated as separate consecutive tasks; 2) Face-SSD is a generalised architecture that is applicable for various face analysis tasks without modifying the network structure—this is in contrast to designing task-specific architectures; and 3) Face-SSD achieves real-time performance (21 FPS) even when detecting multiple faces and recognising multiple classes in a given image (300 × 300). Experimental results show that Face-SSD achieves state-of-the-art performance in various face analysis tasks by reaching a recognition accuracy of 95.76% for smile detection, 90.29% for attribute prediction, and Root Mean Square (RMS) error of 0.44 and 0.39 for valence and arousal estimation.
Highlights
Face analysis is one of the most studied areas in various research communities including Computer Vision (CV) and Affective Computing (AC)
We show the performance of the proposed FaceSSD on three representative face analysis applications such as smile recognition, facial attribute prediction, and valence-arousal estimation
For the valence-arousal estimation experiment we used the AffectNet [32] dataset consisting of continuous level labels and face images captured in the wild
Summary
Face analysis is one of the most studied areas in various research communities including Computer Vision (CV) and Affective Computing (AC). (a) Smile Recognition (b) Facial Attribute Prediction (c) Valence-Arousal Estimation consuming preprocessing steps such as face detection and registration before performing face analysis. In order to address the above mentioned challenges, we propose Face-SSD, a network that performs simultaneously face detection and one or more face analysis tasks (see Fig. 1) in a single architecture. Face-SSD aims to detect faces in a given colour image (upper part in Fig. 2 (a)), and to perform several other face analysis tasks (lower part in Fig. 2 (a)) associated with the detected faces. Multi-scaled convolution layers are added after the convolutional layers of the VGG16 to perform both classification (face classification and face analysis task) and regression (bounding box localisation) tasks (see Fig. 2 (a) [G6:G10]). To the best of our knowledge, Face-SSD is the first single face network that can handle several face analysis tasks without a pre-normalisation step
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have