Touchscreen gestures as images. A transfer learning approach for soft biometric traits recognition

Abstract

Mobile devices are nowadays ubiquitous. They are equipped with a variety of sensors, each designed for capturing specific signals which can be exploited to acquire discriminating user’s traits, thus allowing to recognize, for instance, the authorized users. In this regard, we focus on capturing soft biometric traits from smartphones. Soft biometric information extracted from a human body (e.g., gender and age) is ancillary information proved to improve the performance of biometric authentication systems, and it has drawn a great deal of attention for its applications in healthcare, smart spaces and digital world as a whole. This paper presents an approach to gender and age-group recognition, namely TGSB, leveraging a transfer learning strategy applied to state-of-the-art Convolutional Neural Networks (CNNs) fed with image-based representations of touch gestures performed by users on mobile devices. We perform experiments considering, one at a time, touch gestures of the same kind, and combinations thereof, with intermediate and late fusion learning strategies. Experiments prove that TGSB is a promising approach, with up to 94% accuracy for the gender recognition and up to 99% for the age-group recognition. We highlight the most useful touch gesture for gender and age-group recognition, that is Scroll with 81% and 96% accuracy, respectively. We show that combining multiple touch gestures (intermediate fusion) with a joint latent subspace learning mechanism in CNNs improves the TGSB performance, up to 99% accuracy when considering a combination of two Scroll. Compared to previous works, TGSB exhibits much better performance in both gender and age-group recognition.