Facial Expressions Of Pain Research Articles

Natural human postural oscillations enhance the empathic response to a facial pain expression in a virtual character

Virtual reality platforms producing interactive and highly realistic characters are being used more and more as a research tool in social and affective neuroscience to better capture both the dynamics of emotion communication and the unintentional and automatic nature of emotional processes. While idle motion (i.e., non-communicative movements) is commonly used to create behavioural realism, its use to enhance the perception of emotion expressed by a virtual character is critically lacking. This study examined the influence of naturalistic (i.e., based on human motion capture) idle motion on two aspects (the perception of other’s pain and affective reaction) of an empathic response towards pain expressed by a virtual character. In two experiments, 32 and 34 healthy young adults were presented video clips of a virtual character displaying a facial expression of pain while its body was either static (still condition) or animated with natural postural oscillations (idle condition). The participants in Experiment 1 rated the facial pain expression of the virtual human as more intense, and those in Experiment 2 reported being more touched by its pain expression in the idle condition compared to the still condition, indicating a greater empathic response towards the virtual human’s pain in the presence of natural postural oscillations. These findings are discussed in relation to the models of empathy and biological motion processing. Future investigations will help determine to what extent such naturalistic idle motion could be a key ingredient in enhancing the anthropomorphism of a virtual human and making its emotion appear more genuine.

“Emoji, I can feel your pain” – Neural responses to facial and emoji expressions of pain

The facial expression of pain is a universal social language that transcends cultures and societies. Gaining increasing popularity, emoji is now widely used to express emotional states during digital communication. Yet, it remains unclear whether emoji expression of pain can be processed as effectively as facial expression. This event-related brain potential (ERP) study investigates whether perception of pain expressions in emoji shares similarity with faces. Participants judged pain or neutral expressions in emoji and faces in a classic empathy paradigm. A dynamic convergent-divergent-convergent process was found during the observation of pain expressions. While the recognition for pain is better for faces than for emoji (both in accuracy and reaction time), we found similar P2 amplitudes during the early stage of processing and similar LPP amplitudes during later stage. Interestingly, the neural response diverges during P3 and N2 between faces and emoji: P3 respond more strongly to pain expression (relative to neutral) in faces, whereas N2 respond more strongly to pain expression (relative to neutral) in emoji. Our findings point to the commonality of perceiving pain in faces and in emoji, as well as the distinct salience of detecting pain in faces.

Effect of transportation and social isolation on facial expressions of healthy horses.

Horses have the ability to generate a remarkable repertoire of facial expressions, some of which have been linked to the affective component of pain. This study describes the facial expressions in healthy horses free of pain before and during transportation and social isolation, which are putatively stressful but ordinary management procedures. Transportation was performed in 28 horses by subjecting them to short-term road transport in a horse trailer. A subgroup (n = 10) of these horses was also subjected to short-term social isolation. During all procedures, a body-mounted, remote-controlled heart rate monitor provided continuous heart rate measurements. The horses’ heads were video-recorded during the interventions. An exhaustive dataset was generated from the selected video clips of all possible facial action units and action descriptors, time of emergency, duration, and frequency according to the Equine Facial Action Coding System (EquiFACS). Heart rate increased during both interventions (p<0.01), confirming that they caused disruption in sympato-vagal balance. Using the current method for ascribing certain action units (AUs) to specific emotional states in humans and a novel data-driven co-occurrence method, the following facial traits were observed during both interventions: eye white increase (p<0.001), nostril dilator (p<0.001), upper eyelid raiser (p<0.001), inner brow raiser (p = 0.042), tongue show (p<0.001). Increases in ‘ear flicker’ (p<0.001) and blink frequency (p<0.001) were also seen. These facial actions were used to train a machine-learning classifier to discriminate between the high-arousal interventions and calm horses, which achieved at most 79% accuracy. Most facial features identified correspond well with previous findings on behaviors of stressed horses, for example flared nostrils, repetitive mouth behaviors, increased eye white, tongue show, and ear movements. Several features identified in this study of pain-free horses, such as dilated nostrils, eye white increase, and inner brow raiser, are used as indicators of pain in some face-based pain assessment tools. In order to increase performance parameters in pain assessment tools, the relations between facial expressions of stress and pain should be studied further.

Towards Machine Recognition of Facial Expressions of Pain in Horses.

Simple SummaryFacial activity can convey valid information about the experience of pain in a horse. However, scoring of pain in horses based on facial activity is still in its infancy and accurate scoring can only be performed by trained assessors. Pain in humans can now be recognized reliably from video footage of faces, using computer vision and machine learning. We examine the hurdles in applying these technologies to horses and suggest two general approaches to automatic horse pain recognition. The first approach involves automatically detecting objectively defined facial expression aspects that do not involve any human judgment of what the expression “means”. Automated classification of pain expressions can then be done according to a rule-based system since the facial expression aspects are defined with this information in mind. The other involves training very flexible machine learning methods with raw videos of horses with known true pain status. The upside of this approach is that the system has access to all the information in the video without engineered intermediate methods that have filtered out most of the variation. However, a large challenge is that large datasets with reliable pain annotation are required. We have obtained promising results from both approaches.Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning. However, the application of such methods to horses has proven difficult. Major barriers are the lack of sufficiently large, annotated databases for horses and difficulties in obtaining correct classifications of pain because horses are non-verbal. This review describes our work to overcome these barriers, using two different approaches. One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine learning principles. We have devised tools that can aid manual labeling by identifying the faces and facial keypoints of horses. This approach provides promising results in the automated recognition of facial action units from images. The second approach, recurrent neural network end-to-end learning, requires less extraction of features and representations from the video but instead depends on large volumes of video data with ground truth. Our preliminary results suggest clearly that dynamics are important for pain recognition and show that combinations of recurrent neural networks can classify experimental pain in a small number of horses better than human raters.

Mast Cell-Specific Receptor/Corticotropin-Releasing Factor Axis Regulates Alcohol Withdrawal-Associated Headache

Rehabilitation from alcohol addiction or abuse is challenging due to alcohol withdrawal symptoms. Headache is a severe alcohol withdrawal symptom that frequently contributes to rehabilitation failure. Despite the need for treating alcohol withdrawal-induced headache, there is no appropriate therapeutic option available. Development of improved therapeutics will depend on obtaining a clearer understanding of alcohol withdrawal-induced headache pain mechanisms. Here, we show that the mast cell-specific receptor MrgprB2 controls development of alcohol withdrawal-induced headache. Withdrawing alcohol from alcohol-acclimated mice induces strong headache behaviors, including facial allodynia, facial pain expressions, and reduced walking movement, symptoms often observed in humans suffering from headache. Observed pain behaviors were abolished in MrgprB2-deficient mice. We observed in vivo spontaneous activation and hypersensitization of trigeminal ganglia neurons in alcohol withdrawal mice, but not in MrgprB2-deficient mice. Corticotropin-releasing factor (CRF) was increased in dura mater after alcohol withdrawal. Injection of CRF into dura mater resulted in activation of trigeminal ganglia neurons and vasodilation, which was accompanied by headache behavior. In cells, CRF evoked Ca2+ transients via MrgprB2 or human MrgprX2. The results indicate that alcohol withdrawal causes headache via mast cell degranulation in dura mater. The process is under control of MrgprB2/MrgprX2, which would appear to represent a potential target for treating alcohol withdrawal-related headache.

Visceromotor roots of aesthetic evaluation of pain in art: an fMRI study.

Empathy for pain involves sensory and visceromotor brain regions relevant also in the first-person pain experience. Focusing on brain activations associated with vicarious experiences of pain triggered by artistic or non-artistic images, the present study aims to investigate common and distinct brain activation patterns associated with these two vicarious experiences of pain and to assess whether empathy for pain brain regions contributes to the formation of an aesthetic judgement (AJ) in non-art expert observers. Artistic and non-artistic facial expressions (painful and neutral) were shown to participants inside the scanner and then aesthetically rated in a subsequent behavioural session. Results showed that empathy for pain brain regions (i.e. bilateral insular cortex, posterior sector of the anterior cingulate cortex and the anterior portion of the middle cingulate cortex) and bilateral inferior frontal gyrus are commonly activated by artistic and non-artistic painful facial expressions. For the artistic representation of pain, the activity recorded in these regions directly correlated with participants’ AJ. Results also showed the distinct activation of a large cluster located in the posterior cingulate cortex/precuneus for non-artistic stimuli. This study suggests that non-beauty-specific mechanisms such as empathy for pain are crucial components of the aesthetic experience of artworks.

Facial Muscles Reactions to Other Person’s Facial Expressions of Pain

The aim of this study was to record facial electromiograms (EMG) while subjects were viewing facial expressions of different pain levels (no-pain, medium pain and very painful) and to find objective criteria for measuring pain expressed in human’s face. The study involved 18 students with age 21 years. The magnitude of the EMG response of m. corrugator supercilii depended on voluntary performed facial pain expression in the subjects. EMG responses of voluntary performed facial pain expressions to mirrored pain reactions were detected at two time span intervals: 200–300 ms after stimulation in m. zygomaticus major, and 400–500 ms after stimulation in m. corrugator supercilii. These differences disappear after 1300 ms. In the second time interval, differences in EMG responses of both muscle groups occur 1600 ms after stimulus presentation, but disappear differently: 3100 ms after stimulation in m. zygomaticus major and 4000 ms in m. corrugator supercilii. Constant responding with “medium pain” expression when recognizing faces of different pain expressions have an effect on the voluntary EMG responses of individual subjects. Images with emotional expression “no pain” reduce m. corrugator supercilii activity and increase m. zygomaticus major activity for those observers.

3D Facial Pain Expression for a Care Training Assistant Robot in an Elderly Care Education Environment.

As the elderly population increases, the importance of the caregiver’s role in the quality of life of the elderly has increased. To achieve effective feedback in terms of care and nursing education, it is important to design a robot that can express emotions or feel pain like an actual human through visual-based feedback. This study proposes a care training assistant robot (CaTARo) system with 3D facial pain expression that simulates an elderly person for improving the skills of workers in elderly care. First, in order to develop an accurate and efficient system for elderly care training, this study introduces a fuzzy logic–based care training evaluation method that can calculate the pain level of a robot for giving the feedback. Elderly caregivers and trainees performed the range of motion exercise using the proposed CaTARo. We obtained quantitative data from CaTARo, and the pain level was calculated by combining four key parameters using the fuzzy logic method. Second, we developed a 3D facial avatar for use in CaTARo that is capable of expressing pain based on the UNBC-McMaster Pain Shoulder Archive, and we then generated four pain groups with respect to the pain level. To mimic the conditions for care training with actual humans, we designed the system to provide pain feedback based on the opinions of experts. The pain feedback was expressed in real time by using a projector and a 3D facial mask during care training. The results of the study confirmed the feasibility of utilizing a care training robot with pain expression for elderly care training, and it is concluded that the proposed approach may be used to improve caregiving and nursing skills upon further research.

The effect of optimism on the facial expression of pain: Implications for pain communication.

There is a broad range of evidence on optimism dampening the pain experience, as assessed by subjective self-report. Facial expression of pain conveys supplementary information about the pain experience, is an integral part of pain communication and assists psychosocial pain coping. Nevertheless, the effect of induced optimism on facial activity during pain has to our knowledge not been examined. In our experiment, 40 healthy participants underwent two blocks of thermal stimulation containing phasic non-painful and painful stimuli. Between the two blocks, the Best Possible Self imagery and writing task was performed to induce situational optimism, while a control group wrote about their typical day. Facial activity and self-report ratings of intensity and unpleasantness were recorded. Facial activity was analysed using the Facial Action Coding System. The optimism manipulation was successful in increasing state optimism. It did not affect self-report ratings, but resulted in a stronger facial expression of pain, caused especially by increases in Action Units 4 (furrowed brows) and 6_7 (narrowed eyes). All Action Units, which were affected by the optimism induction, are known to be prevalent during pain stimulation. The increase in facial expression might reflect reduced inhibition of pain communication in temporarily optimistic participants. Optimism might lead to expecting positive and helpful reactions from others and, by that, to great readiness to elicit these reactions by non-verbal social behaviour. This study is the first to indicate that state optimism increases the facial expression of pain as a social signal for help and empathy without concomitant changes in the subjective pain experience.

Conditioned Pain Modulation (CPM) Effects Captured in Facial Expressions.

PurposeConditioned pain modulation (CPM) is most often assessed using self-report of pain. However, self-report of pain is not always available (eg in individuals with cognitive impairment) and is susceptible to report bias. In comparison, the facial expression of pain is more reflex-like and represents one of the most sensitive and specific non-verbal signals of pain. The aim of the present study was to investigate whether the facial expression of pain is sensitive enough to capture endogenous pain inhibition as elicited during CPM paradigms.Patients and MethodsIn total, 26 female participants took part in this study. Facial and verbal responses to phasic heat pain were assessed once while participants immersed their hand in a hot water bath and once without additional stimulation. Facial responses were analyzed using the Facial Action Coding System (FACS). Verbal responses were assessed using a Numerical Rating Scale (NRS).ResultsPain-relevant facial responses as well as pain ratings to phasic heat pain were significantly reduced when participants simultaneously immersed their hand in a hot water bath compared to baseline. Thus, CPM effects could be demonstrated both on subjective as well as on facial responses. Moreover, CPM-induced changes in pain-relevant facial responses and in NRS ratings were significantly correlated.ConclusionThe present study shows that facial expressions of pain are sensitive enough to capture CPM effects. Given the proven clinical usefulness of assessing CPM, the parallel assessment of verbal and facial CPM effects might be a promising approach with wider scope of applications. Further research in other demographic healthy participant and clinical cohorts is warranted.

Classification of Facial Expression of Post-Surgical Pain in Children

There are certain difficulties in differentiating between children's facial expression related to pain and other stimuli. In addition, the limited communication ability of children in the preverbal stage leads to misdiagnosis when the child feels pain, for example, post-surgical conditions. In this article, a classification approach of facial expression of child pain is presented based on models of pre-trained convolutional neuronal networks from the study carried out in a Colombian hospital of level 4 (Hospital Universitario San Vicente Fundación), in the recovery areas of child surgery services. AlexNet and VGG (16, 19 and Face) networks are evaluated in the own dataset using the FLACC scale and their performances are compared in three experiments. The results show that the VGG-19 model achieves the best performance (92.9%) compared to the other networks. The effectiveness of the model and transfer learning for the classification of facial expression of child pain shows a promising solution for the assessment of post-surgical pain.

Facial expressions of pain in daily clinical practice to assess postoperative pain in children: Reliability and validity of the facial action summary score

Behavioural pain scales are recommended to assess postoperative pain for children who are too young to use self-report tools. Their main limitation is underestimation of pain in the days following an intervention. Although relevant, facial expression is not used in daily clinical practice. This prospective study aimed to assess the validity and reliability of the Facial Action Summary Score (FASS), a five-item scale, to assess postoperative pain until hospital discharge in children <7years. Assessments of pain and anxiety of 123 children using FASS and validated scales were used to study the psychometric validity of the FASS in clinical practice. The content validity was previously investigated in a development study. The internal validity of the FASS was high with excellent reliability (intraclass coefficient=0.94) and a high Cronbach α (0.89). Convergent validity with pain scales (FLACC [Face, Legs, Activity, Cry, Consoling] and FPS-R [Faces Pain Scale - Revised]) was high (r>0.8). Sensitivity to change was verified by a significant decrease in the score after rescue analgesia. For a threshold of 2/5, the FASS shows excellent specificity (97%) and sensitivity (82%). The low number of false negatives is the main strength of this tool. This work highlights the interest in using facial expression in daily clinical practice to manage postoperative pain. The FASS is easy to use with excellent psychometric properties and is particularly sensitive to measure pain in the days following surgery. The aim of this study was to prove that facial expression of pain can be used in clinical practice to measure postoperative pain in children. The reduced number of false negatives is the main strength of this tool.

Sorting pain out of salience: assessment of pain facial expressions in the human fetus.

The question of whether the human fetus experiences pain has received substantial attention in recent times. With the advent of high-definition 4-dimensional ultrasound (4D-US), it is possible to record fetal body and facial expressions. To determine whether human fetuses demonstrate discriminative acute behavioral responses to nociceptive input. This cross-sectional study included 5 fetuses with diaphragmatic hernia with indication of intrauterine surgery (fetoscopic endoluminal tracheal occlusion) and 8 healthy fetuses, who were scanned with 4D-US in 1 of 3 conditions: (1) acute pain group: Fetuses undergoing intrauterine surgery were assessed in the preoperative period during the anesthetic injection into the thigh; (2) control group at rest: Facial expressions at rest were recorded during scheduled ultrasound examinations; and (3) control group acoustic startle: Fetal facial expressions were recorded during acoustic stimulus (500-4000 Hz; 60-115 dB). Raters blinded to the fetuses' groups scored 65 pictures of fetal facial expressions based on the presence of 12 items (facial movements). Analyses of redundancy and usefulness excluded 5 items for being of low discrimination capacity (P>0.2). The final version of the pain assessment tool consisted of a total of 7 items: brow lowering/eyes squeezed shut/deepening of the nasolabial furrow/open lips/horizontal mouth stretch/vertical mouth stretch/neck deflection. Odd ratios for a facial expression to be detected in acute pain compared with control conditions ranged from 11 (neck deflection) to 1,400 (horizontal mouth stretch). Using the seven-item final tool, we showed that 5 is the cutoff value discriminating pain from nonpainful startle and rest. This study inaugurates the possibility to study pain responses during the intrauterine life, which may have implications for the postoperative management of pain after intrauterine surgical interventions.

Unobtrusive Pain Monitoring in Older Adults With Dementia Using Pairwise and Contrastive Training

Although pain is frequent in old age, older adults are often undertreated for pain. This is especially the case for long-term care residents with moderate to severe dementia who cannot report their pain because of cognitive impairments that accompany dementia. Nursing staff acknowledge the challenges of effectively recognizing and managing pain in long-term care facilities due to lack of human resources and, sometimes, expertise to use validated pain assessment approaches on a regular basis. Vision-based ambient monitoring will allow for frequent automated assessments so care staff could be automatically notified when signs of pain are displayed. However, existing computer vision techniques for pain detection are not validated on faces of older adults or people with dementia, and this population is not represented in existing facial expression datasets of pain. We present the first fully automated vision-based technique validated on a dementia cohort. Our contributions are threefold. First, we develop a deep learning-based computer vision system for detecting painful facial expressions on a video dataset that is collected unobtrusively from older adult participants with and without dementia. Second, we introduce a pairwise comparative inference method that calibrates to each person and is sensitive to changes in facial expression while using training data more efficiently than sequence models. Third, we introduce a fast contrastive training method that improves cross-dataset performance. Our pain estimation model outperforms baselines by a wide margin, especially when evaluated on faces of people with dementia. Pre-trained model and demo code available at https://github.com/TaatiTeam/pain_detection_demo.

Clinical pointers from developing self-awareness.

Seven typical developmental steps are described as cardinal changes leading children from implicit embodied self-awareness at birth, self-consciousness by 2 years, and the adoption of an ethical stance toward others by the preschool years (3-5y). This development may be a useful benchmark for clinicians. In this review, some clinical pointers are outlined in relation to each developmental step, but with a particular focus on the testing of self-awareness in children with developmental disorders, including autism spectrum disorder, cerebral palsy, and intellectual disability. WHAT THIS PAPER ADDS: There are seven major steps in early self and social awareness. Clinical pointers related to each of the seven steps are provided.

Decoding of facial expressions of pain in avatars: does sex matter?

The decoding of facial expressions of pain plays a crucial role in pain diagnostic and clinical decision making. For decoding studies, it is necessary to present facial expressions of pain in a flexible and controllable fashion. Computer models (avatars) of human facial expressions of pain allow for systematically manipulating specific facial features. The aim of the present study was to investigate whether avatars can show realistic facial expressions of pain and how the sex of the avatars influence the decoding of pain by human observers. For that purpose, 40 female (mean age: 23.9 years) and 40 male (mean age: 24.6 years) observers watched 80 short videos showing computer-generated avatars, who presented the five clusters of facial expressions of pain (four active and one stoic cluster) identified by Kunz and Lautenbacher (2014). After each clip, observers were asked to provide ratings for the intensity of pain the avatars seem to experience and the certainty of judgement, i.e. if the shown expression truly represents pain. Results show that three of the four active facial clusters were similarly accepted as valid expressions of pain by the observers whereas only one cluster ("raised eyebrows") was disregarded. The sex of the observed avatars influenced the decoding of pain as indicated by increased intensity and elevated certainty ratings for femaleavatars. The assumption of different valid facial expressions of pain could be corroborated in avatars, which contradicts the idea of only one uniform pain face. The observers' rating of the avatars' pain was influenced by the avatars' sex, which resembles known observer biases for humans. The use of avatars appeared to be a suitable method in research on the decoding of the facial expression of pain, mirroring closely the known forms of human facial expressions.

Freeze-like responses to pain in humans and its modulation by social context

BackgroundMaladaptive defensive responses such as excessive avoidance behavior have received increasing attention as a main mechanism for the development and maintenance of chronic pain complaints. However, another defensive response which is commonly studied in animals as a proxy for fear is freezing behavior. No research to date has investigated human freezing behavior in the context of pain. In addition, there is an increasing realization that social context can affect pain-relevant processes such as pain experience and pain behavior but less is known about the effects of social context on defensive responses to pain. Hence, this study investigated freezing behavior and facial pain expression in the context of pain, and their modulation by social context.MethodsHealthy, pain-free participants (N = 39) stood on a stabilometric force platform in a threatening or safe social context, which was manipulated using angry or happy facial stimuli. In some trials, an auditory cue (conditioned stimulus; CS) predicted the occurrence of painful electrocutaneous stimulus (unconditioned stimulus; pain-US). We assessed body sway (an index of freezing), heart rate, facial pain expression, self-reported pain intensity, unpleasantness, and pain-US expectancy during the CS and the context alone (no CS).ResultsThe results were mixed. Neither the anticipation of pain, nor social context affected body sway. Heart rate and painful facial expression were reduced in the threatening social context at high anxiety levels. A threatening social context also elicited higher pain-US expectancy ratings. In sum, a threatening social context increases the expectation of pain, but reduces the facial expression of pain and lowers heart rate in highly anxious individuals.

Pharmacological fMRI provides evidence for opioidergic modulation of discrimination of facial pain expressions.

The endogenous opioid system is strongly involved in the modulation of pain. However, the potential role of this system in perceiving painful facial expressions from others has not been sufficiently explored as of yet. To elucidate the contribution of the opioid system to the perception of painful facial expressions, we conducted a double‐blind, within‐subjects pharmacological functional magnetic resonance imaging (fMRI) study, in which 42 participants engaged in an emotion discrimination task (pain vs. disgust expressions) in two experimental sessions, receiving either the opioid receptor antagonist naltrexone or an inert substance (placebo). On the behavioral level, participants less frequently judged an expression as pain under naltrexone as compared to placebo. On the neural level, parametric modulation of activation in the (putative) right fusiform face area (FFA), which was correlated with increased pain intensity, was higher under naltrexone than placebo. Regression analyses revealed that brain activity in the right FFA significantly predicted behavioral performance in disambiguating pain from disgust, both under naltrexone and placebo. These findings suggest that reducing opioid system activity decreased participants' sensitivity for facial expressions of pain, and that this was linked to possibly compensatory engagement of processes related to visual perception, rather than to higher level affective processes, and pain regulation.

The Delaware Pain Database: a set of painful expressions and corresponding norming data.

Facial expressions of pain serve an essential social function by communicating suffering and soliciting aid. Accurate visual perception of painful expressions is critical because the misperception of pain signals can have serious clinical and social consequences. Therefore, it is essential that researchers have access to high-quality, diverse databases of painful expressions to better understand accuracy and bias in pain perception. This article describes the development of a large-scale face stimulus database focusing on expressions of pain. We collected and normed a database of images of models posing painful facial expressions. We also characterized these stimuli in terms of the presence of a series of pain-relevant facial action units. In addition to our primary database of posed expressions, we provide a separate database of computer-rendered expressions of pain that may be applied to any neutral face photograph. The resulting database comprises 229 unique (and now publicly available) painful expressions. To the best of our knowledge, there are no existing databases of this size, quality, or diversity in terms of race, gender, and expression intensity. We provide evidence for the reliability of expressions and evaluations of pain within these stimuli, as well as a full characterization of this set along dimensions relevant to pain such as perceived status, strength, and dominance. Moreover, our second database complements the primary set in terms of experimental control and precision. These stimuli will facilitate reproducible research in both experimental and clinical domains into the mechanisms supporting accuracy and bias in pain perception and care.

FENP: A Database of Neonatal Facial Expression for Pain Analysis

In this article, we introduce a new neonatal facial expression database for pain analysis. This database, called facial expression of neonatal pain (FENP), contains 11,000 neonatal facial expression images associated with 106 Chinese neonates from two children's hospitals, i.e., the Children's Hospital Affiliated to Nanjing Medical University and Second Affiliated Hospital Affiliated to Nanjing Medical University in China. The facial expression images cover four categories of facial expressions, i.e., severe pain expression, mild pain expression, crying expression and calmness expression, where each category contains 2750 neonatal facial expression images. Based on this database, we also investigate the pain facial expression recognition problem using several state-of-the-art facial expression features and expression recognition methods, such as Gabor+SVM, LBP+SVM, HOG+SVM, LBP+HOG+SVM, and several Convolutional Neural Network (CNN) methods (including AlexNet, VGGNet, GoogLeNet, ResNet and DenseNet). The experimental results indicate that the proposed neonatal pain facial expression database is very suitable for the study of both neonatal pain and facial expression recognition. Moreover, the FENP database is publicly available after signing a license agreement (the users can contact Jingjie Yan (yanjingjie@njupt.edu.cn), Guanming Lu (lugm@njupt.edu.cn)) or Xiaonan Li (xnli@njmu.edu.cn).