Automatic Coding of Facial Expressions of Pain: Are We There Yet?

Carlos Suso-Ribera,Ute Schmid,Dominik Seuss,Stefan Lautenbacher,Frederik W Loy,Teena Hassan,Miriam Kunz,Jens-Uwe Garbas

doi:10.1155/2022/6635496

Abstract

Introduction The experience of pain is regularly accompanied by facial expressions. The gold standard for analyzing these facial expressions is the Facial Action Coding System (FACS), which provides so-called action units (AUs) as parametrical indicators of facial muscular activity. Particular combinations of AUs have appeared to be pain-indicative. The manual coding of AUs is, however, too time- and labor-intensive in clinical practice. New developments in automatic facial expression analysis have promised to enable automatic detection of AUs, which might be used for pain detection. Objective Our aim is to compare manual with automatic AU coding of facial expressions of pain. Methods FaceReader7 was used for automatic AU detection. We compared the performance of FaceReader7 using videos of 40 participants (20 younger with a mean age of 25.7 years and 20 older with a mean age of 52.1 years) undergoing experimentally induced heat pain to manually coded AUs as gold standard labeling. Percentages of correctly and falsely classified AUs were calculated, and we computed as indicators of congruency, “sensitivity/recall,” “precision,” and “overall agreement (F1).” Results The automatic coding of AUs only showed poor to moderate outcomes regarding sensitivity/recall, precision, and F1. The congruency was better for younger compared to older faces and was better for pain-indicative AUs compared to other AUs. Conclusion At the moment, automatic analyses of genuine facial expressions of pain may qualify at best as semiautomatic systems, which require further validation by human observers before they can be used to validly assess facial expressions of pain.

Highlights

For clinical pain assessment, facial responses to pain are of great diagnostic relevance beside the subjective responses, especially for verbally undeveloped or impaired individuals, like young children, children with intellectual disability, or aged individuals with dementia or aphasia [1, 2]
Prior research concerning the influence of age on facial communication of pain has shown that age does not severely impact how pain is facially expressed [18]; age influences how observers rate the facial responses [19,20,21]. us, we investigated the automatic detection of facial responses to pain by FaceReader in a naturalistic age range relevant for pain patients [17] and compared the automatic detection between younger and older individuals. e present study is one of the first that challenges a state-of-the-art AU detection algorithms (AUDA) with a collection of real pain expressions in middle adult-aged individuals
Highest recall values were found for AU4 and AU43, with approximately 2/3 of these facial responses being detected by FaceReader

Summary

Introduction

Facial responses to pain are of great diagnostic relevance beside the subjective responses, especially for verbally undeveloped or impaired individuals, like young children, children with intellectual disability, or aged individuals with dementia or aphasia [1, 2]. The standard assessment of facial expressions of pain is performed by human observers, either by the use of behavioral observation scales (see Herr et al [8] for a review on pain observation scales) or by the use of finegrained analyses using the Facial Action Coding System (FACS [9]), which provides so-called action units (AUs) as parametrical indicators of facial muscle activity Manual FACS coding, on the other hand, is very time- and labor-intensive given that coding of one minute of video material can take up to two hours and approximately 100 hours of training is needed to achieve FACS certification as a coder [9] Both approaches mainly target acute pain (postsurgical, procedural, and acute injuries and diseases) but not chronic pain and do not allow for a constant monitoring of pain as they are based on relatively short observational time windows. Since pain often occurs in episodes or fluctuates in intensity, only constant monitoring would prevent missing critical pain events. ese shortcomings limit the clinical usefulness of manual approaches via human observers. us, the question arises whether automatic pain detection systems might be a good alternative and whether they are performing better than human observers

Objectives

Results

Conclusion