Global-local combined features to detect pain intensity from facial expression images with attention mechanism

Abstract

The estimation of pain intensity is critical for medical diagnosis and treatment of patients. With the development of image monitoring technology and artificial intelligence, automatic pain assessment based on facial expression and behavioral analysis shows a potential value in clinical applications. This paper reports a framework of convolutional neural network with global and local attention mechanism (GLA-CNN) for the effective detection of pain intensity at four-level thresholds using facial expression images. GLA-CNN includes two modules, namely global attention network (GANet) and local attention network (LANet). LANet is responsible for extracting representative local patch features of faces, while GANet extracts whole facial features to compensate for the ignored correlative features between patches. In the end, the global correlational and local subtle features are fused for the final estimation of pain intensity. Experiments under the UNBC-McMaster Shoulder Pain database demonstrate that GLA-CNN outperforms other state-of-the-art methods. Additionally, a visualization analysis is conducted to present the feature map of GLA-CNN, intuitively showing that it can extract not only local pain features but also global correlative facial ones. Our study demonstrates that pain assessment based on facial expression is a non-invasive and feasible method, and can be employed as an auxiliary pain assessment tool in clinical practice.