Automatic Pain Assessment Research Articles

Physiological Time-Series Fusion With Hybrid Attention for Adaptive Recognition of Pain.

Automatic pain assessment is an application in healthcare serving personalized pain care, and patients cannot self-report pain. Pain at the present is inferred from physiological dynamics at the present and in the near past. However, heterogeneous pain responses cross-subject and cross-type hinder accurate recognition of pain. This work solves the adaptive pain recognition problem across pain types. We concrete the adaptivity problem into recognizing both phasic/short and tonic/long pain from the physiological sequences of the same length. The adaptivity of the proposed solution (TCAtt-PainNet) was ensured by hybrid temporal-channel attention when fusing multivariate time-series of electrocardiogram (ECG) and galvanic skin response (GSR) features. The attention was obtained by learning the dependencies between the point at present and the sequence in the near past, where sequence point temporal attention was constructed via modified self-attention, and the following feature channel attention was constructed by squeeze-and-excitation temporal attention weighted deep feature sequence. The proposed solution successfully enhanced recognition adaptivity by addressing relevant information only from long input sequences when testing with tonic and phasic pain databases, making progress towards automatic pain assessment for real application scenarios with attributes unknown pain.

Advancing the integration of biosignal-based automated pain assessment methods into a comprehensive model for addressing cancer pain

BackgroundTailoring effective strategies for cancer pain management requires a careful analysis of multiple factors that influence pain phenomena and, ultimately, guide the therapy. While there is a wealth of research on automatic pain assessment (APA), its integration with clinical data remains inadequately explored. This study aimed to address the potential correlations between subjective and APA-derived objectives variables in a cohort of cancer patients.MethodsA multidimensional statistical approach was employed. Demographic, clinical, and pain-related variables were examined. Objective measures included electrodermal activity (EDA) and electrocardiogram (ECG) signals. Sensitivity analysis, multiple factorial analysis (MFA), hierarchical clustering on principal components (HCPC), and multivariable regression were used for data analysis.ResultsThe study analyzed data from 64 cancer patients. MFA revealed correlations between pain intensity, type, Eastern Cooperative Oncology Group Performance status (ECOG), opioids, and metastases. Clustering identified three distinct patient groups based on pain characteristics, treatments, and ECOG. Multivariable regression analysis showed associations between pain intensity, ECOG, type of breakthrough cancer pain, and opioid dosages. The analyses failed to find a correlation between subjective and objective pain variables.ConclusionsThe reported pain perception is unrelated to the objective variables of APA. An in-depth investigation of APA is required to understand the variables to be studied, the operational modalities, and above all, strategies for appropriate integration with data obtained from self-reporting.Trial registrationThis study is registered with ClinicalTrials.gov, number (NCT04726228), registered 27 January 2021, https://classic.clinicaltrials.gov/ct2/show/NCT04726228?term=nct04726228&draw=2&rank=1

A Review of Automatic Pain Assessment from Facial Information Using Machine Learning

A Review of Automatic Pain Assessment from Facial Information Using Machine Learning

Global-local combined features to detect pain intensity from facial expression images with attention mechanism

The estimation of pain intensity is critical for medical diagnosis and treatment of patients. With the development of image monitoring technology and artificial intelligence, automatic pain assessment based on facial expression and behavioral analysis shows a potential value in clinical applications. This paper reports a framework of convolutional neural network with global and local attention mechanism (GLA-CNN) for the effective detection of pain intensity at four-level thresholds using facial expression images. GLA-CNN includes two modules, namely global attention network (GANet) and local attention network (LANet). LANet is responsible for extracting representative local patch features of faces, while GANet extracts whole facial features to compensate for the ignored correlative features between patches. In the end, the global correlational and local subtle features are fused for the final estimation of pain intensity. Experiments under the UNBC-McMaster Shoulder Pain database demonstrate that GLA-CNN outperforms other state-of-the-art methods. Additionally, a visualization analysis is conducted to present the feature map of GLA-CNN, intuitively showing that it can extract not only local pain features but also global correlative facial ones. Our study demonstrates that pain assessment based on facial expression is a non-invasive and feasible method, and can be employed as an auxiliary pain assessment tool in clinical practice.

Multimodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures.

Accurate and objective pain evaluation is crucial in developing effective pain management protocols, aiming to alleviate distress and prevent patients from experiencing decreased functionality. A multimodal automatic assessment framework for acute pain utilizing video and heart rate signals is introduced in this study. The proposed framework comprises four pivotal modules: the Spatial Module, responsible for extracting embeddings from videos; the Heart Rate Encoder, tasked with mapping heart rate signals into a higher dimensional space; the AugmNet, designed to create learning-based augmentations in the latent space; and the Temporal Module, which utilizes the extracted video and heart rate embeddings for the final assessment. The Spatial-Module undergoes pre-training on a two-stage strategy: first, with a face recognition objective learning universal facial features, and second, with an emotion recognition objective in a multitask learning approach, enabling the extraction of high-quality embeddings for the automatic pain assessment. Experiments with the facial videos and heart rate extracted from electrocardiograms of the BioVid database, along with a direct comparison to 29 studies, demonstrate state-of-the-art performances in unimodal and multimodal settings, maintaining high efficiency. Within the multimodal context, 82.74% and 39.77% accuracy were achieved for the binary and multi-level pain classification task, respectively, utilizing million parameters for the entire framework.

Calibration trial of an innovative medical device (NEVVA© ) for the evaluation of pain in non-communicating patients in the intensive care unit.

Pain management is an essential and complex issue for non-communicative patients undergoing sedation in the intensive care unit (ICU). The Behavioral Pain Scale (BPS), although not perfect for assessing behavioral pain, is the gold standard based partly on clinical facial expression. NEVVA© , an automatic pain assessment tool based on facial expressions in critically ill patients, is a much-needed innovative medical device. In this prospective pilot study, we recorded the facial expressions of critically ill patients in the medical ICU of Caen University Hospital using the iPhone and Smart Motion Tracking System (SMTS) software with the Facial Action Coding System (FACS) to measure human facial expressions metrically during sedation weaning. Analyses were recorded continuously, and BPS scores were collected hourly over two 8 h periods per day for 3 consecutive days. For this first stage, calibration of the innovative NEVVA© medical device algorithm was obtained by comparison with the reference pain scale (BPS). Thirty participants were enrolled between March and July 2022. To assess the acute severity of illness, the Sequential Organ Failure Assessment (SOFA) and the Simplified Acute Physiology Score (SAPS II) were recorded on ICU admission and were 9 and 47, respectively. All participants had deep sedation, assessed by a Richmond Agitation and Sedation scale (RASS) score of less than or equal to -4 at the time of inclusion. One thousand and six BPS recordings were obtained, and 130 recordings were retained for final calibration: 108 BPS recordings corresponding to the absence of pain and 22 BPS recordings corresponding to the presence of pain. Due to the small size of the dataset, a leave-one-subject-out cross-validation (LOSO-CV) strategy was performed, and the training results obtained the receiver operating characteristic (ROC) curve with an area under the curve (AUC) of 0.792. This model has a sensitivity of 81.8% and a specificity of 72.2%. This pilot study calibrated the NEVVA© medical device and showed the feasibility of continuous facial expression analysis for pain monitoring in ICU patients. The next step will be to correlate this device with the BPS scale.

Human vs machine towards neonatal pain assessment: A comprehensive analysis of the facial features extracted by health professionals, parents, and convolutional neural networks

Neonates are not able to verbally communicate pain, hindering the correct identification of this phenomenon. Several clinical scales have been proposed to assess pain, mainly using the facial features of the neonate, but a better comprehension of these features is yet required, since several related works have shown the subjectivity of these scales. Meanwhile, computational methods have been implemented to automate neonatal pain assessment and, although performing accurately, these methods still lack the interpretability of the corresponding decision-making processes. To address this issue, we propose in this work a facial feature extraction framework to gather information and investigate the human and machine neonatal pain assessments, comparing the visual attention of the facial features perceived by health-professionals and parents of neonates with the most relevant ones extracted by eXplainable Artificial Intelligence (XAI) methods, considering the VGG-Face and N-CNN deep learning architectures. Our experimental results show that the information extracted by the computational methods are clinically relevant to neonatal pain assessment, but yet do not agree with the facial visual attention of health-professionals and parents, suggesting that humans and machines can learn from each other to improve their decision-making processes. We believe that these findings might advance our understanding of how humans and machines code and decode neonatal facial responses to pain, enabling further improvements in clinical scales widely used in practical situations and in face-based automatic pain assessment tools as well.

Development of a binary classifier model from extended facial codes toward video-based pain recognition in cancer patients.

The Automatic Pain Assessment (APA) relies on the exploitation of objective methods to evaluate the severity of pain and other pain-related characteristics. Facial expressions are the most investigated pain behavior features for APA. We constructed a binary classifier model for discriminating between the absence and presence of pain through video analysis. A brief interview lasting approximately two-minute was conducted with cancer patients, and video recordings were taken during the session. The Delaware Pain Database and UNBC-McMaster Shoulder Pain dataset were used for training. A set of 17 Action Units (AUs) was adopted. For each image, the OpenFace toolkit was used to extract the considered AUs. The collected data were grouped and split into train and test sets: 80 % of the data was used as a training set and the remaining 20 % as the validation set. For continuous estimation, the entire patient video with frame prediction values of 0 (no pain) or 1 (pain), was imported into an annotator (ELAN 6.4). The developed Neural Network classifier consists of two dense layers. The first layer contains 17 nodes associated with the facial AUs extracted by OpenFace for each image. The output layer is a classification label of "pain" (1) or "no pain" (0). The classifier obtained an accuracy of ∼94 % after about 400 training epochs. The Area Under the ROC curve (AUROC) value was approximately 0.98. This study demonstrated that the use of a binary classifier model developed from selected AUs can be an effective tool for evaluating cancer pain. The implementation of an APA classifier can be useful for detecting potential pain fluctuations. In the context of APA research, further investigations are necessary to refine the process and particularly to combine this data with multi-parameter analyses such as speech analysis, text analysis, and data obtained from physiological parameters.

Exploring Biosignals for Quantitative Pain Assessment in Cancer Patients: A Proof of Concept

Perception and expression of pain in cancer patients are influenced by distress levels, tumor type and progression, and the underlying pathophysiology of pain. Relying on traditional pain assessment tools can present limitations due to the highly subjective and multifaceted nature of the symptoms. In this scenario, objective pain assessment is an open research challenge. This work introduces a framework for automatic pain assessment. The proposed method is based on a wearable biosignal platform to extract quantitative indicators of the patient pain experience, evaluated through a self-assessment report. Two preliminary case studies focused on the simultaneous acquisition of electrocardiography (ECG), electrodermal activity (EDA), and accelerometer signals are illustrated and discussed. The results demonstrate the feasibility of the approach, highlighting the potential of EDA in capturing skin conductance responses (SCR) related to pain events in chronic cancer pain. A weak correlation (R = 0.2) is found between SCR parameters and the standard deviation of the interbeat interval series (SDRR), selected as the Heart Rate Variability index. A statistically significant (p < 0.001) increase in both EDA signal and SDRR is detected in movement with respect to rest conditions (assessed by means of the accelerometer signals) in the case of motion-associated cancer pain, thus reflecting the relationship between motor dynamics, which trigger painful responses, and the subsequent activation of the autonomous nervous system. With the objective of integrating parameters obtained from biosignals to establish pain signatures within different clinical scenarios, the proposed framework proves to be a promising research approach to define pain signatures in different clinical contexts.

Artificial Intelligence for Automatic Pain Assessment: Research Methods and Perspectives.

Although proper pain evaluation is mandatory for establishing the appropriate therapy, self-reported pain level assessment has several limitations. Data-driven artificial intelligence (AI) methods can be employed for research on automatic pain assessment (APA). The goal is the development of objective, standardized, and generalizable instruments useful for pain assessment in different clinical contexts. The purpose of this article is to discuss the state of the art of research and perspectives on APA applications in both research and clinical scenarios. Principles of AI functioning will be addressed. For narrative purposes, AI-based methods are grouped into behavioral-based approaches and neurophysiology-based pain detection methods. Since pain is generally accompanied by spontaneous facial behaviors, several approaches for APA are based on image classification and feature extraction. Language features through natural language strategies, body postures, and respiratory-derived elements are other investigated behavioral-based approaches. Neurophysiology-based pain detection is obtained through electroencephalography, electromyography, electrodermal activity, and other biosignals. Recent approaches involve multimode strategies by combining behaviors with neurophysiological findings. Concerning methods, early studies were conducted by machine learning algorithms such as support vector machine, decision tree, and random forest classifiers. More recently, artificial neural networks such as convolutional and recurrent neural network algorithms are implemented, even in combination. Collaboration programs involving clinicians and computer scientists must be aimed at structuring and processing robust datasets that can be used in various settings, from acute to different chronic pain conditions. Finally, it is crucial to apply the concepts of explainability and ethics when examining AI applications for pain research and management.

ATLAS: An Adaptive Transfer Learning Based Pain Assessment System: A Real Life Unsupervised Pain Assessment Solution.

Undertreatment or overtreatment of pain will cause severe consequences physiologically and psychologically. Thus, researchers have made great efforts to develop automatic pain assessment approaches based on physiological signals using machine learning techniques. However, state-of-art research mainly focuses on verifying the hypothesis that physiological signals can be used to assess pain. The critical assumption of these studies is that training data and testing data have the same distribution. However, this assumption may not hold in reallife scenarios, for instance, the adoption of machine learning model by a new patient. Such real-life scenarios in which user's data is unlabeled is largely neglected in literature. This study compensates for the rift by proposing an adaptive transfer learning based pain assessment system (ATLAS), a novel adaptive learning system based on the transfer learning algorithm Transfer Components Analysis (TCA) to minimize the distance between training data and unlabeled testing data. Experiments were conducted on BioVid database, and the results showed our approach outperforms three existing traditional machine learning-based approaches and achieves an accuracy just 2.0% below the accuracy with labeled data.

An Automatic System for Continuous Pain Intensity Monitoring Based on Analyzing Data from Uni-, Bi-, and Multi-Modality.

Pain is a reliable indicator of health issues; it affects patients’ quality of life when not well managed. The current methods in the clinical application undergo biases and errors; moreover, such methods do not facilitate continuous pain monitoring. For this purpose, the recent methodologies in automatic pain assessment were introduced, which demonstrated the possibility for objectively and robustly measuring and monitoring pain when using behavioral cues and physiological signals. This paper focuses on introducing a reliable automatic system for continuous monitoring of pain intensity by analyzing behavioral cues, such as facial expressions and audio, and physiological signals, such as electrocardiogram (ECG), electromyogram (EMG), and electrodermal activity (EDA) from the X-ITE Pain Dataset. Several experiments were conducted with 11 datasets regarding classification and regression; these datasets were obtained from the database to reduce the impact of the imbalanced database problem. With each single modality (Uni-modality) experiment, we used a Random Forest [RF] baseline method, a Long Short-Term Memory (LSTM) method, and a LSTM using a sample weighting method (called LSTM-SW). Further, LSTM and LSTM-SW were used with fused modalities (two modalities = Bi-modality and all modalities = Multi-modality) experiments. Sample weighting was used to downweight misclassified samples during training to improve the performance. The experiments’ results confirmed that regression is better than classification with imbalanced datasets, EDA is the best single modality, and fused modalities improved the performance significantly over the single modality in 10 out of 11 datasets.

Facial Expressions Based Automatic Pain Assessment System

Pain assessment is used to improve patients’ treatment outcomes. Human observers may be influenced by personal factors, such as inexperience and medical organizations are facing a shortage of experts. In this study, we developed a facial expressions-based automatic pain assessment system (FEAPAS) to notify medical staff when a patient suffers pain by activating an alarm and recording the incident and pain level with the date and time. The model consists of two identical concurrent subsystems, each of which takes one of the two inputs of the model, i.e., “full face” and “the upper half of the same face”. The subsystems extract the relevant input features via two pre-trained convolutional neural networks (CNNs), using either VGG16, InceptionV3, ResNet50, or ResNeXt50, while freezing all convolutional blocks and replacing the classifier layer with a shallow CNN. The concatenated outputs in this stage is then sent to the model’s classifier. This approach mimics the human observer method and gives more importance to the upper part of the face, which is similar to the Prkachin and Soloman pain intensity (PSPI). Additionally, we further optimized our models by applying four optimizers (SGD/ADAM/RMSprop/RAdam) to each model and testing them on the UNBC-McMaster shoulder pain expression archive dataset to find the optimal combination, InceptionV3-SGD. The optimal model showed an accuracy of 99.10% on 10-fold cross-validation, thus outperforming the state-of-the-art model on the UNBC-McMaster database. It also scored 90.56% on unseen subject data. To speed up the system response time and reduce unnecessary alarms associated with temporary facial expressions, a select but effective subset of frames was inspected and classified. Two frame-selection criteria were reported. Classifying only two frames at the middle of 30-frame sequence was optimal, with an average reaction time of at most 6.49 s and the ability to avoid unnecessary alarms.

Evaluación del dolor neonatal y opinión de los profesionales sobre el uso de un sistema automático para su valoración

Objective: to understand the characteristics of the evaluation of neonatal pain in Neonatal Intensive Care Units (NICUs) and the opinion of professionals about the use of an automatic assessment system. Method: a cross-sectional study (2017). Population: NICU nurses and doctors from Spanish hospitals. Data collection through online questionnaire which included variables of the professional profile, frequency of pain assessment, evaluation scale used, training received, difficulties perceived to assess pain, opinion about pain evaluation and about the use of an automatic assessment system. Descriptive statistical analysis; Jamovi 1.2.7® program.Results: there were 133 answers: 83.5% were women, with a mean age of 40 (9.43) years. 88% from tertiary level hospitals; 44.1% were Nursing professionals, and 72% with >6 years’ experience at the NICU. Of these respondents, 66.9% assessed pain according to the status of the newborn, 29.3% with a scale specific to the unit, 72.6% considered that there was a subjective component in the assessment, 48.1% did not receive any training to assess pain. An automatic system was regarded as useful by 93.2%, but for 36.1% it would not be reliable, and 23.3% reckoned that an automatic system could not replace human perception. Conclusions: no protocol by consensus was applied for pain assessment in newborns, there was variability, sometimes it was not measured due to lack of time, and when it was measured, data could be subjective. The majority considered that the use of an automatic pain assessment system could be useful, although some doubted its reliability due to lack of knowledge of the technology currently applied.

Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective.

Traditional pain assessment approaches ranging from self-reporting methods, to observational scales, rely on the ability of an individual to accurately assess and successfully report observed or experienced pain episodes. Automatic pain assessment tools are therefore more than desirable in cases where this specific ability is negatively affected by various psycho-physiological dispositions, as well as distinct physical traits such as in the case of professional athletes, who usually have a higher pain tolerance as regular individuals. Hence, several approaches have been proposed during the past decades for the implementation of an autonomous and effective pain assessment system. These approaches range from more conventional supervised and semi-supervised learning techniques applied on a set of carefully hand-designed feature representations, to deep neural networks applied on preprocessed signals. Some of the most prominent advantages of deep neural networks are the ability to automatically learn relevant features, as well as the inherent adaptability of trained deep neural networks to related inference tasks. Yet, some significant drawbacks such as requiring large amounts of data to train deep models and over-fitting remain. Both of these problems are especially relevant in pain intensity assessment, where labeled data is scarce and generalization is of utmost importance. In the following work we address these shortcomings by introducing several novel multi-modal deep learning approaches (characterized by specific supervised, as well as self-supervised learning techniques) for the assessment of pain intensity based on measurable bio-physiological data. While the proposed supervised deep learning approach is able to attain state-of-the-art inference performances, our self-supervised approach is able to significantly improve the data efficiency of the proposed architecture by automatically generating physiological data and simultaneously performing a fine-tuning of the architecture, which has been previously trained on a significantly smaller amount of data.

Pain expression assessment based on a locality and identity aware network

In clinical medicine, the pain feeling is a significant indicator for the medical condition of patients. Of late, automatic pain assessment methods have received more and more interests. Many researchers proposed corresponding methods and achieved impressive results. However, they always ignore the locality and individual differences of painful expression. Therefore, a locality and identity aware network (LIAN) for pain assessment is presented here. Concretely, for the locality characteristic, a locality aware module consisting of a two-branch structure, feature and attention branches, is presented. The former learns pain features by a deep network, while the latter guides the pain features to focus on the discriminational regions of pain. As for the individual differences, an identity aware module with a multi-task method is proposed to represent identity-related information to achieve identity-invariant pain assessment. Extensive experiments on public databases show the superiority of LIAN in pain assessment.

Pain Assessment Tool With Electrodermal Activity for Postoperative Patients: Method Validation Study.

BackgroundAccurate, objective pain assessment is required in the health care domain and clinical settings for appropriate pain management. Automated, objective pain detection from physiological data in patients provides valuable information to hospital staff and caregivers to better manage pain, particularly for patients who are unable to self-report. Galvanic skin response (GSR) is one of the physiologic signals that refers to the changes in sweat gland activity, which can identify features of emotional states and anxiety induced by varying pain levels. This study used different statistical features extracted from GSR data collected from postoperative patients to detect their pain intensity. To the best of our knowledge, this is the first work building pain models using postoperative adult patients instead of healthy subjects.ObjectiveThe goal of this study was to present an automatic pain assessment tool using GSR signals to predict different pain intensities in noncommunicative, postoperative patients.MethodsThe study was designed to collect biomedical data from postoperative patients reporting moderate to high pain levels. We recruited 25 participants aged 23-89 years. First, a transcutaneous electrical nerve stimulation (TENS) unit was employed to obtain patients' baseline data. In the second part, the Empatica E4 wristband was worn by patients while they were performing low-intensity activities. Patient self-report based on the numeric rating scale (NRS) was used to record pain intensities that were correlated with objectively measured data. The labels were down-sampled from 11 pain levels to 5 different pain intensities, including the baseline. We used 2 different machine learning algorithms to construct the models. The mean decrease impurity method was used to find the top important features for pain prediction and improve the accuracy. We compared our results with a previously published research study to estimate the true performance of our models.ResultsFour different binary classification models were constructed using each machine learning algorithm to classify the baseline and other pain intensities (Baseline [BL] vs Pain Level [PL] 1, BL vs PL2, BL vs PL3, and BL vs PL4). Our models achieved higher accuracy for the first 3 pain models than the BioVid paper approach despite the challenges in analyzing real patient data. For BL vs PL1, BL vs PL2, and BL vs PL4, the highest prediction accuracies were achieved when using a random forest classifier (86.0, 70.0, and 61.5, respectively). For BL vs PL3, we achieved an accuracy of 72.1 using a k-nearest-neighbor classifier.ConclusionsWe are the first to propose and validate a pain assessment tool to predict different pain levels in real postoperative adult patients using GSR signals. We also exploited feature selection algorithms to find the top important features related to different pain intensities.International Registered Report Identifier (IRRID)RR2-10.2196/17783

PLAAN: Pain Level Assessment with Anomaly-detection based Network

Automatic chronic pain assessment and pain intensity estimation has been attracting growing attention due to its widespread applications. One of the prevalent issues in automatic pain analysis is inadequate balanced expert-labelled data for pain estimation. This work proposes an anomaly detection based network addressing one of the existing limitations of automatic pain assessment. The evaluation of the network is performed on pain intensity estimation and protective behaviour estimation tasks from body movements in the EmoPain Challenge dataset. The EmoPain dataset consists of body part based sensor data for both the tasks. The proposed network, PLAAN (Pain Level Assessment with Anomaly-detection based Network), is a lightweight LSTM-DNN network which considers features based on sensor data as the input and predicts intensity level of pain and presence or absence of protective behaviour in chronic low back pain patients. Joint training considering body movement patterns, such as exercise type, corresponding to pain exhibition as a label improves the performance of the network. However, contrary to perception, protective behaviour rather exists sporadically alongside pain in the EmoPain dataset. This induces yet another complication in accurate estimation of protective behaviour. This problem is resolved by incorporating anomaly detection in the network. A detailed comparison of different networks with varied features is outlined in the paper, presenting a significant improvement with the final proposed anomaly detection based network.

Metrological Characterization of a Pain Detection System Based on Transfer Entropy of Facial Landmarks

Pain is an alert state of the human body that can be conveyed to the external world through different modalities. A possible communication channel for human pain is represented by facial expressions, whose role in social interactions has been well established. In this work, the link between pain and transfer entropy (TE), passing through facial expressions, is investigated. A new approach to the vision-based measurement (VBM) of pain is presented, which is based on TE among the time-series of facial landmarks positions. The system is composed of three main blocks: A VBM block for the automatic landmarking and the generation of the time-series from the video-sequences; a second block for the evaluation of TE; and finally a classification model based on machine learning algorithms for pain assessment. A public database of video sequences of patients experiencing pain in a controlled scenario was used for the characterization of the system in terms of accuracy and precision. Different uncertainty contributions related to realistic signals interruptions and fluctuations were modeled and propagated to provide a comprehensive evaluation of the proposed measurement system. The obtained results indicate that TE-based approaches can provide great benefits in automatic pain assessment, opening new perspectives for remote management of patients.

Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation

• We propose a self-supervised representation learning for pain intensity estimation from the face. • We use a novel similarity function in a Siamese network to learn representations from unlabeled data as the pretext task. • Statistical Spatiotemporal Distillation is presented to encode dynamic and appearance information of video into an image map. • Our proposed method shows promising performance in cross-dataset evaluation setting using two publicly available datasets. Recently, automatic pain assessment technology, in particular automatically detecting pain from facial expressions, has been developed to improve the quality of pain management, and has attracted increasing attention. In this paper, we propose self-supervised learning for automatic yet efficient pain assessment, in order to reduce the cost of collecting large amount of labeled data. To achieve this, we introduce a novel similarity function to learn generalized representations using a Siamese network in the pretext task. The learned representations are finetuned in the downstream task of pain intensity estimation. To make the method computationally efficient, we propose Statistical Spatiotemporal Distillation (SSD) to encode the spatiotemporal variations underlying the facial video into a single RGB image, enabling the use of less complex 2D deep models for video representation. Experiments on two publicly available pain datasets and cross-dataset evaluation demonstrate promising results, showing the good generalization ability of the learned representations.