A survey on facial soft biometrics for video surveillance and forensic applications

Abstract

The face is one of the most reliable and easy-to-acquire biometric features, widely used for the recognition of individuals. In controlled environments facial recognition systems are highly effective, however, in real world scenarios and under varying lighting conditions, pose changes, facial expressions, occlusions and low resolution of captured images/videos, the task of recognizing faces becomes significantly complex. In this context it has been shown that certain attributes can be retrieved with a relative probability of success, being useful to complement a non conclusive result of a biometric system. In this paper we present an overview on face describable visual attributes and in particular of the so-called soft biometrics (e.g., facial marks, gender, age, skin color, and other physical characteristics). We review core issues regarding this topic, for instance what are the soft biometrics, which of them are the most robust in video surveillance and other uncontrolled scenarios, how different approaches have been addressed in the literature for their representation and classification, which datasets can be used for evaluation, which related problems remain unresolved and which are the possible ways to approach them.