SPACE: Finding Key-Speaker in Complex Multi-Person Scenes

Abstract

Intelligent image processing based on deep learning is a recently emerging artificial Intelligence method which is critical in the next generation computing systems. Key-speaker detection using image processing aims to find the speaker who plays the most important role in complex multi-person scenes. However, existing speaker detection methods just judge whether there is somebody talking. To overcome this problem, this paper proposes a novel key-speaker detection approach named Speaker Pose-Attention deteCtion modEl (SPACE). This approach extracts the space information and analyze people's attention gathering to find the key-speaker. The method consists of two main parts. The first part is our proposed Importance Detection Model (IDM) using human pose estimation and an effective evaluation strategy to find candidates of important speaker. The second part consists of Pose-Attention Graph (PAG) and link analysis algorithm. The PAG represents people's attention graph. The link analysis algorithm analyzes the connectivity of this graph to find the key-speaker. Due to the lack of available key-speaker detection datasets, a self-collected dataset containing speech and meeting scenes has been built to estimate SPACE model. The experimental results show that the proposed method can obtain the best detection accuracy compared with other existing methods.