Improving user verification in human-robot interaction from audio or image inputs through sample quality assessment

Abstract

• Novel approach to identity verification in Human-Robot interaction. • Identity verification produced from both sound (voice) and image (face) inputs. • Proposed approaches focus on identifying the parts of the signal that contains the high value information. • Through analysis of regions of interest substantial improvements are obtained over the state of the art. • Evaluation of performance is obtained in a complex dataset that portrays real life scenarios. In this paper, we tackle the task of improving biometric verification in the context of Human-Robot Interaction (HRI). A robot that wants to identify a specific person to provide a service can do so by either image verification or, if light conditions are not favourable, through voice verification. In our approach, we will take advantage of the possibility a robot has of recovering further data until it is sure of the identity of the person. The key contribution is that we select from both image and audio signals the parts that are of higher confidence. For images we use a system that looks at the face of each person and selects frames in which the confidence is high while keeping those frames separate in time to avoid using very similar facial appearance. For audio our approach tries to find the parts of the signal that contain a person talking, avoiding those in which noise is present by segmenting the signal. Once the parts of interest are found, each input is described with an independent deep learning architecture that obtains a descriptor for each kind of input (face/voice). We also present in this paper fusion methods that improve performance by combining the features from both face and voice, results to validate this are shown for each independent input and for the fusion methods.