Smiling Response Research Articles

Considering Cuteness Enhances Smiling Responses to Infant Faces1,2

AbstractLaboratory and field research has reported that the appearance of infants causes observers to smile. The current study examined whether this smiling response is modulated by the observer's task and evaluative dimension. Thirty‐nine young nulliparous women were asked to rate the cuteness or beauty levels of 6‐month‐old infants' faces using a 7‐point scale in different blocks. Facial electromyograms (EMGs) were recorded of participants' facial muscles related to both smiling (the zygomaticus major [ZM] and orbicularis oculi [OO]) and frowning (the corrugator supercilii [CS]). The results showed that cuteness and beauty ratings were highly correlated with each other (r = .90), indicating that these evaluations were based on similar attractiveness‐related physical features. Facial EMG responses on the smiling muscle sites, ZM and OO, increased significantly from the baseline when participants rated the faces' cuteness, and the responses were larger than when participants rated the faces' beauty. CS activity was not found to have any effect. The perceived cuteness level of the infant faces did not affect the facial EMG responses. Moreover, the magnitude of the smiling response was shown to be much smaller than that associated with voluntary smiling. These findings suggest that facial expressions while viewing infant faces do not exhibit a fixed pattern but are modulated by observers' tasks and that considering cuteness, which is based on more affective evaluations than beauty, can enhance smiling responses.

The other person's smiling amount affects one's smiling response during face-to-face conversations.

Smiling during conversation occurs interactively between people and is known to build good interpersonal relationships. However, whether and how much the amount that an individual smiles is influenced by the other person's smile has remained unclear. This study aimed to quantify the amount of two individuals' smiles during conversations and investigate the dependency of one's smile amount (i.e., intensity and frequency) on that of the other. Forty participants (20 females) engaged in three-minute face-to-face conversations as speakers with a listener (male or female), under three conditions, where the amount of smiling response by listeners was controlled as "less," "moderate," and "greater." The amount of the smiles was quantified based on their facial movements through automated facial expression analysis. The results showed that the amount of smiling by the speaker changed significantly depending on the listener's smile amount; when the listeners smiled to a greater extent, the speakers tended to smile more, especially when they were of the same gender (i.e., male-male and female-female pairs). Further analysis revealed that the smiling intensities of the two individuals changed in a temporally synchronized manner. These results provide quantitative evidence for the dependence of one's smile on the other's smile, and the differential effect between gender pairs.

Motive-Specific Affective Contingencies and Their Relevance for Personality and Motivated Behavior

Individuals differ in the tendency to derive pleasure out of motive-specific incentives, such as being socially included or attaining power. Multiple theoretical approaches have proposed that such motive-specific positive affective contingencies (PACs) are central building blocks of motive dispositions and personality more broadly. In the current research, we put this claim to test and investigated individual differences with regard to motive-specific PACs in the affiliation and power domains. We measured PACs via spontaneous emotional reactions to motive-specific cues, as assessed by affect ratings and electromyographic (EMG) recordings of smile responses. Both of these PAC operationalizations were highly internally consistent and moderately to highly stable across time. Furthermore, motive-specific PACs were linked in a manner consistent with theory to measures of motive dispositions and to personality traits with motivational underpinnings (i.e., extraversion, agreeableness, and narcissism). Finally, in the affiliation domain, motive-specific PACs were linked to objectively assessed, key motivational outcomes (i.e., attentional orientation, behavior in daily life, and in the laboratory). Taken together, the findings underscore the relevance of affective contingencies for the understanding of personality and motivated behavior.

Deliberate control over facial expressions in motherhood. Evidence from a Stroop-like task

The deliberate control of facial expressions is an important ability in human interactions, in particular for mothers with prelinguistic infants. Because research on this topic is still scarce, we investigated the control over facial expressions in a Stroop-like paradigm. Mothers of 2–6 months old infants and nullipara women produced smiles and frowns in response to verbal commands written on distractor faces of adults or infants showing expressions of happiness or anger/distress. Analyses of video recordings with a machine classifier for facial expression revealed pronounced effects of congruency between the expressions required by the participants and those displayed by the face stimuli on the onset latencies of the deliberate facial expressions. With adult distractor faces this Stroop effect was similar whether participants smiled or frowned. With infant distractor faces mothers and non-mothers showed indistinguishable Stroop effects on smile responses; however, for frown responses, the Stroop effect in mothers was smaller than in non-mothers. We suggest that for frown responses in mothers when facing infants, the effect of mimicry or stimulus response compatibility, leading to the Stroop effect, is offset by a caregiving response or empathy.

Effects of Social Context on Deliberate Facial Expressions: Evidence from a Stroop-like Task

Facial expressions contribute to nonverbal communication, social coordination, and interaction. Facial expressions may reflect the emotional state of the expressor, but they may be modulated by the presence of others, for example, by facial mimicry or through social display rules. We examined how deliberate facial expressions of happiness and anger (smiles and frowns), prompted by written commands, are modulated by the congruency with the facial expression of background faces and how this effect depends on the age of the background face (infants vs. adults). Our main interest was whether the quality of the required expression could be influenced by a task-irrelevant background face and its emotional display. Background faces from adults and infants displayed happy, angry, or neutral expressions. To assess the activation pattern of different action units, we used a machine classifier software; the same classifier was used to assess the chronometry of the expression responses. Results indicated slower and less correct performance when an incongruent facial expression was in the background, especially when distractor stimuli showed adult faces. Interestingly, smile responses were more intense in congruent than incongruent conditions. Depending on stimulus age, frown responses were affected in their quality by incongruent (smile) expressions in terms of the additional activation or deactivation of the outer brow raiser (AU2), resulting in a blended expression, somewhat different from the prototypical expression for anger. Together, the present results show qualitative effects on deliberate facial expressions, beyond typical chronometric effects, confirming machine classification of facial expressions as a promising tool for emotion research.

The eye contact smile: The effects of sending and receiving a direct gaze

ABSTRACT Eye contact often elicits a smiling response. We investigated whether an individual’s awareness that the recipient perceives their direct gaze during eye contact has an influence on this smiling response. Participants wore glasses with either clear or dark lenses (preventing the other person from seeing their eyes). Measurements of electromyographic activity from the zygomatic and periocular muscle regions showed that the smiling responses were greater to seeing the other’s direct versus averted gaze. The participants’ own gaze direction had also an effect, and this effect was modified by the visibility of their eyes. Zygomatic responses were greater when the participants were aware that their eyes were visible. Thus, awareness of sending a direct gaze to another plays a role in the smiling response. In addition, participants’ self-evaluated level of social anxiety was positively correlated with the magnitude of the zygomatic responses to the other person’s direct versus averted gaze.

I Can See It in Your Face. Affective Valuation of Exercise in More or Less Physically Active Individuals.

The purpose of this study was to illustrate that people’s affective valuation of exercise can be identified in their faces. The study was conducted with a software for automatic facial expression analysis and it involved testing the hypothesis that positive or negative affective valuation occurs spontaneously when people are reminded of exercise. We created a task similar to an emotional Stroop task, in which participants responded to exercise-related and control stimuli with a positive or negative facial expression (smile or frown) depending on whether the photo was presented upright or tilted. We further asked participants how much time they would normally spend for physical exercise, because we assumed that the affective valuation of those who exercise more would be more positive. Based on the data of 86 participants, regression analysis revealed that those who reported less exercise and a more negative reflective evaluation of exercise initiated negative facial expressions on exercise-related stimuli significantly faster than those who reported exercising more often. No significant effect was observed for smile responses. We suspect that responding with a smile to exercise-related stimuli was the congruent response for the majority of our participants, so that for them no Stroop interference occurred in the exercise-related condition. This study suggests that immediate negative affective reactions to exercise-related stimuli result from a postconscious automatic process and can be detected in the study participants’ faces. It furthermore illustrates how methodological paradigms from social–cognition research (here: the emotional Stroop paradigm) can be adapted to collect and analyze biometric data for the investigation of exercisers’ and non-exercisers’ automatic valuations of exercise.

Interactive video-player to improve social smile in individuals with autism spectrum disorder

PurposeThe purpose of this paper is to propose and develop a live interaction-based video player system named LIV4Smile for the improvement of the social smile in individuals with autism spectrum disorder (ASD).Design/methodology/approachThe proposed LIV4Smile intervention was a video player that operated by detecting smile using a convolutional neural network (CNN)-based algorithm. To maintain a live interaction, a CNN-based smile detector was configured and used in this system. The statistical test was also conducted to validate the performance of the system.FindingsThe significant improvement was observed in smile responses of individuals with ASD with the utilization of the proposed LIV4Smile system in a real-time environment.Research limitations/implicationsA small sample size and clinical utilizing for validation and initial training of ASD individuals for LIV4Smile could be considered under implications.Originality/valueThe main aim of this study was to address the inclusive practices for children with autism. The proposed CNN algorithm-based LIV4Smile intervention resulted in high accuracy in facial smile detection.

EP 24. Contralateral smiles induced by deep brain stimulation of the nucleus accumbens region in a patient with major depressive disorder

Introduction Deep brain stimulation (DBS) is a well-established procedure for the treatment of therapy-refractory movement disorders. Limited evidence suggests that DBS of the nucleus accumbens region (NACC) may be effective for the treatment of psychiatric disorders, such as obsessive compulsive disorder and major depressive disorder (MDD). Hitherto, little is known concerning behavioural responses upon acute high-frequency stimulation of the NACC region in patients suffering from MDD. Objective To further characterize behavioural responses and clinical effects elicited upon acute unilateral DBS of the NACC area. Intra- and postoperative observations (including video-sequences) as well as a two-month outcome are presented. Material and Methods We performed bilateral NACC-DBS in a female patient (age: 47) with MDD refractory to psycho-, pharmaco- and electroconvulsive (>50 sessions) therapy on a compassionate-use basis. The stereotactic procedure was performed under local anaesthesia. The NACC was targeted (x) 7 mm lateral to the midline, (y) 3 mm anterior to the anterior commissure, (z) 4 mm below the anterior commissure-posterior commissure plane. A commercially available DBS lead with 4 contacts (model 3391, Medtronic Co., MN, USA; contact length: 3 mm; interelectrode distance: 4 mm) was inserted bilaterally for permanent stimulation. Results Upon test-stimulation in the operating room, acute unilateral NACC-DBS on both hemispheres stereotypically induced a visible facial reaction with a contralateral smile (bipolar stimulation of the ventralmost contacts (0− cathodal vs. 1+ anodal); pulse width: 90 μs; frequency: 130 Hz). This smile could repeatedly be elicited either when stimulation amplitude was slowly ramped up to 7 mA, or abruptly switched on to 4–5 mA. The smile was invariably associated with a significantly improved mood (as assessed by an analog scale). This effect was documented by different examiners and clearly reproducible in the subsequent clinical course. A dramatic improvement of the depression scores, patient’s mood and social behaviour was documented at a first follow-up (6 weeks postoperatively). Conclusion Albeit preliminary, the patient’s beneficial response tentatively suggests that NACC-DBS may be a viable option for selected patients with MDD. In line with previous reports on stimulation-induced smile responses in patients with obsessive compulsive disorder, our intraoperative observations support and expand the notion of the NACC area constituting or containing an important interface between limbic and motor networks, respectively, that may be involved in the pathophysiology of MDD. It is suggested that the results of long-term NACC-DBS in patients with MDD should be evaluated by larger patient samples and randomized, controlled trials.

Zapping prediction for online advertisement based on cumulative smile sparse representation

As the amount of online multimedia keeps growing rapidly, study of online user behavior is highly demanded in many applications to provide better user experience and online services. Due to the surging online user population in recent years, more and more online contents are now associated with advertisements. For marketers, online advertising can reach broader audience with reduced cost. Meanwhile, the revenue for many IT companies such as Google mainly comes from advertisement hosting. Therefore, for both marketers and advertisement hosts, effective online advertising is the primary goal. Thus, it is of great interest to predict and prevent the users from zapping, i.e., skipping the advertisement. Reduced zapping would increase the possibility for the user to become potential consumer. Despite its importance, zapping prediction for online advertising has received very limited attention. Since the zapping behavior is related to the user׳s emotional states, in this paper, we propose to predict zapping from user׳s facial expressions. Our prediction is non-intrusive, meaning that the user׳s experience during advertisement watching is not disturbed. Specifically, a robust encoding of the smile responses termed as Cumulative Smile Sparse Representation (CSSR) is extracted from the user׳s facial expressions. Then, this representation is incorporated in a semi-supervised hypergraph learning framework to predict the moment-to-moment zapping behavior. The hypergraph is able to discover both pairwise and higher order data relationships, leading to more accurate prediction as compared to conventional classifiers. Experiments are conducted on a recently published zapping behavior analysis dataset and state-of-the-art performance is achieved as compared to other competing methods.

Why, When, and How Much to Entertain Consumers in Advertisements? A Web-Based Facial Tracking Field Study

The presence of positive entertainment (e.g., visual imagery, upbeat music, humor) in TV advertisements can make them more attractive and persuasive. However, little is known about the downside of too much entertainment. This research focuses on why, when, and how much to entertain consumers in TV advertisements. We collected data in a large scale field study using 82 ads with various levels of entertainment shown to 178 consumers in their homes and workplaces. Using a novel web-based face tracking system, we continuously measure consumers' smile responses, viewing interest, and purchase intent. A simultaneous Bayesian hierarchical model is estimated to assess how different levels of entertainment affect purchases by endogenizing viewing interest. We find that entertainment has an inverted U-shape relationship to purchase intent. Importantly, we separate entertainment into that which comes before the brand versus that which comes after, and find that the latter is positively associated with purchase intent while the former is not.

Zapping Index:Using Smile to Measure Advertisement Zapping Likelihood

In marketing and advertising research, “zapping” is defined as the action when a viewer stops watching a commercial. Researchers analyze users’ behavior in order to prevent zapping which helps advertisers to design effective commercials. Since emotions can be used to engage consumers, in this paper, we leverage automated facial expression analysis to understand consumers’ zapping behavior. Firstly, we provide an accurate moment-to-moment smile detection algorithm. Secondly, we formulate a binary classification problem (zapping/non-zapping) based on real-world scenarios, and adopt smile response as the feature to predict zapping. Thirdly, to cope with the lack of a metric in advertising evaluation, we propose a new metric called Zapping Index (ZI). ZI is a moment-to-moment measurement of a user’s zapping probability. It gauges not only the reaction of a user, but also the preference of a user to commercials. Finally, extensive experiments are performed to provide insights and we make recommendations that will be useful to both advertisers and advertisement publishers.

Technology-aided programs to enable persons with multiple disabilities to choose among environmental stimuli using a smile or a tongue response

Persons with multiple disabilities, including pervasive motor impairment, may have problems controlling even small responses (e.g., vocal emissions, finger movements, or prolonged eyelid closures) within time-sensitive situations, such as those involved in choice programs. Recent research has indicated that smile expressions can be used as functional choice responses for some of these persons. The present two studies were aimed at assessing the smile response for a child with congenital multiple disabilities and a tongue response for a post-coma man who had recovered his consciousness but presented with pervasive multiple disabilities. The first of the two studies represented a research extension (i.e., a new case with a slightly adapted microswitch technology) concerning the smile response, which had recently been evaluated with few other cases. The second study represented a new effort to assess the tongue response within a choice program and for a post-coma man with multiple disabilities. The results showed that the participants used the smile and the tongue responses successfully while they were apparently unsuccessful in using a slight head/chin movement response. Their choice behavior focused reliably on preferred stimuli and avoided non-preferred stimuli. Implications of the results are discussed.

Persons with Multiple Disabilities Choose Among Environmental Stimuli Using a Smile Response and a Technology–Aided Program

Recent studies with three persons with multiple disabilities have indicated that smile expressions can be used as functional choice responses (i.e., occurring in relation to samples of preferred stimuli to be selected and absent in relation to samples of non-preferred stimuli to be bypassed). The purpose of this study was to extend the evaluation of the smile response with two new participants (a girl and a woman) with multiple disabilities within a technology-aided choice program. The smile response was monitored through an optic microswitch pointing to the girl’s lower lip or across the area before the woman’s right cheekbone. The study started with a baseline phase during which the participants were to use a vocalization response to choose among the stimuli presented to them. The intervention phase with the smile response and optic microswitch was introduced according to a non-concurrent multiple baseline design across participants. The results showed that both participants used the smile response and optic microswitch to choose the preferred stimuli and abstained from using such response (i.e., from making choices) in relation to non-preferred stimuli. Implications of the results are discussed.

Persons with multiple disabilities use forehead and smile responses to access or choose among technology-aided stimulation events

A variety of technology-aided programs have been developed to help persons with congenital or acquired multiple disabilities access preferred stimuli or choose among stimulus options. The application of those programs may pose problems when the participants have very limited behavior repertoires and are unable to use conventional responses and microswitches. The present two studies assessed non-conventional response-microswitch solutions for three of those participants. Study I included two participants who were exposed to a program in which forehead skin movement was the response required to access preferred stimulation. The microswitch was an optic sensor combined with a small black sticker on the forehead. Study II included one participant who was exposed to a program in which a smile response was required to choose among stimuli. The microswitch for monitoring the smile was a new camera-based technology. The results of the two studies showed that the response-microswitch solutions were suitable for the participants and enabled them to perform successfully. Implications of the studies for people with limited motor behavior and issues for future research were discussed.

Crowdsourcing Facial Responses to Online Videos

We present results validating a novel framework for collecting and analyzing facial responses to media content over the Internet. This system allowed 3,268 trackable face videos to be collected and analyzed in under two months. We characterize the data and present analysis of the smile responses of viewers to three commercials. We compare statistics from this corpus to those from the Cohn-Kanade+ (CK+) and MMI databases and show that distributions of position, scale, pose, movement, and luminance of the facial region are significantly different from those represented in these traditionally used datasets. Next, we analyze the intensity and dynamics of smile responses, and show that there are significantly different facial responses from subgroups who report liking the commercials compared to those that report not liking the commercials. Similarly, we unveil significant differences between groups who were previously familiar with a commercial and those that were not and propose a link to virality. Finally, we present relationships between head movement and facial behavior that were observed within the data. The framework, data collected, and analysis demonstrate an ecologically valid method for unobtrusive evaluation of facial responses to media content that is robust to challenging real-world conditions and requires no explicit recruitment or compensation of participants.

Persons with multiple disabilities select environmental stimuli through a smile response monitored via camera-based technology

Objective: To assess whether two persons with multiple disabilities could use smile expressions and new camera-based microswitch technology to select environmental stimuli.Method: Within each session, a computer system provided samples/reminders of preferred and non-preferred stimuli. The camera-based microswitch determined whether the participants had smile expressions in relation to those samples. If they did, stimuli matching the specific samples to which they responded were presented for 20 seconds.Results: The smile expression could be profitably used by the participants who managed to select means of ∼70% or 75% of the preferred stimulus opportunities made available by the environment while avoiding almost all the non-preferred stimulus opportunities.Conclusion: Smile expressions (a) might be an effective and rapid means for selecting preferred stimulation and (b) might develop into cognitively more elaborate forms of responding through the learning experience (i.e. their consistent association with positive/reinforcing consequences).

Infant and Maternal Sensitivity to Interpersonal Timing

A perturbation paradigm was employed to assess 3- and 6-month-old infants' and their mothers' sensitivity to a 3-s temporal delay implemented in an ongoing televised interaction. At both ages, the temporal delay affected infant but not maternal behavior and only when implementing the temporal delay in maternal (Experiment 1, N=64) but not infant (Experiment 2, N=60) behavior. In addition, the experimental manipulation influenced promptness of maternal smiling responses reliably more than promptness of infant smiling responses. The findings suggest that the timing of maternal behavior plays an important role in infants' perception of maternal responsiveness, whereas mothers seem to monitor general aspects of infant behavior such as overall level of engagement.

Baby smile response circuits of the parental brain

AbstractThe parent-infant dyad, characterized by contingent social interactions that develop over the first three months postpartum, may depend heavily on parental brain responses to the infant, including the capacity to smile. A range of brain regions may subserve this social key function in parents and contribute to similar capacities in normal infants, capacities that may go awry in circumstances of reduced care.

Outcomes of Microneurovascular Facial Reanimation Using Masseteric Innervation in Patients With Long-Standing Facial Palsy Resulting From Cured Brainstem Lesions

The functions of the human face are not only of esthetic significance but also extend into metaphoric nuances of psychology. The loss of function of one or both facial nerves has a remarkable impact on patients' lives. To retrospectively analyze the functional outcomes of microneurovascular facial reanimation using masseteric innervation. Seventeen patients with irreparable facial paralysis resulting from benign lesions involving the facial nuclei (n = 14) or Möbius syndrome (n = 3) were treated with free muscle flaps for oral commissural reanimation using ipsilateral masseteric innervation and using temporalis muscle transfer for eyelid reanimation. Results were analyzed by the absolute commissural excursion and commissural excursion index and by a patient self-evaluation score. Presence of synkinesis was documented. Follow-up ranged from 8 to 48 months (mean, 26.4 months). Normalization of the commissural excursion index was observed in 8 of 17 patients (47%), an improvement was seen in 7 of 17 (41%), and failure was observed in 2 of 17 (12%). The individual dynamics of absolute commissural excursion and commissural excursion index changes are presented. A natural smiling response was observed in 10 of 17 patients (59%) but not in the remaining 7 (41%). This response reflected the patient's ability to relay the natural emotion of smiling through the masseteric nerve. Patients' self-evaluation scores were a level higher than objective indices. Innervation of free muscle flaps with the masseteric nerve for oral commissure reanimation might play an important role in patients with lesions of the facial nuclei (as in Möbius syndrome). Synkinesis persists for long periods after surgery. However, most of the patients learned to express their emotions by overcoming this phenomenon. Despite hypercorrection or inadequate correction, patients evaluated themselves favorably.