Mobile face detection and tracking for media streaming applications

Abstract

This paper presents a method for mobile face detection and tracking in media streaming applications. To account specifics of mobile image processing and pattern recognition, the experimental benchmarking of Intel x86 and Nokia S60 platforms is done. Various aspects on mobile phone use are considered in order to record a profile on a mobile media streaming user. The profile is generated by analysing pictures of mobile phone users and screenshots of their devices. The mobile face detection model is based on the optimised version of robust real-time object detection algorithm. Our optimisation enables to reduce the computational complexity of the original version in 4 times. We also describe the tracking mode in which the system achieves a near real-time frame rate (45 12 fps depending on the video complexity). The training of the face detector is done on the databases of more than 400 hr of video and the major parameters such as FRR and FAR are evaluated on a database of 30 min of video (including more than 5000 facial images) and 200 hr of video, respectively. Face tracking is benchmarked using 20 mobile video sequences, which cover a broad range of human activities and environmental changes. We describe the construction of the low bit-rate video coder that uses face detection and tracking to segment foreground/background object and encode them with a different bit-rate. We also describe a framework for smart videoconferences and experimental results on low bit-rate image coding for this application.