Terms Of Arousal Research Articles

End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning

To train machine learning algorithms to predict emotional expressions in terms of arousal and valence, annotated datasets are needed. However, as different people perceive others' emotional expressions differently, their annotations are subjective. To account for this, annotations are typically collected from multiple annotators and averaged to obtain ground-truth labels. However, when exclusively trained on this averaged ground-truth, the model is agnostic to the inherent subjectivity in emotional expressions. In this work, we therefore propose an end-to-end Bayesian neural network capable of being trained on a distribution of annotations to also capture the subjectivity-based label uncertainty. Instead of a Gaussian, we model the annotation distribution using Student's t-distribution, which also accounts for the number of annotations available. We derive the corresponding Kullback-Leibler divergence loss and use it to train an estimator for the annotation distribution, from which the mean and uncertainty can be inferred. We validate the proposed method using two in-the-wild datasets. We show that the proposed t-distribution based approach achieves state-of-the-art uncertainty modeling results in speech emotion recognition, and also consistent results in cross-corpora evaluations. Furthermore, analyses reveal that the advantage of a t-distribution over a Gaussian grows with increasing inter-annotator correlation and a decreasing number of annotations available.

Evening prolonged relatively low melanopic equivalent daylight illuminance light exposure increases arousal before and during sleep without altering sleep structure

SummaryLight can influence many psychophysiological functions beyond vision, including alertness, circadian rhythm, and sleep, namely the non‐image forming (NIF) effects of light. Melanopic equivalent daylight illuminance (mel‐EDI) is currently recommended as the predictor of the NIF effects of light. Although light dose is also critical for entraining and regulating circadian cycle, it is still unknown whether relatively low mel‐EDI light exposure for prolonged duration in the evening would affect pre‐sleep arousal and subsequent sleep. In all, 18 healthy college students (10 females, mean [standard deviation] age 21.67 [2.03] years) underwent 2 experimental nights with a 1 week interval in a simulated bedroom environment. During experimental nights, participants were either exposed to high or low mel‐EDI light (73 versus 38 lx mel‐EDI, 90 versus 87 photopic lx at eye level, 150 photopic lx at table level) for 3.5 h before regular bedtime, and their sleep was monitored by polysomnography. Subjective sleepiness, mood, and resting‐state electroencephalography during light exposure were also investigated. Results showed no significant differences in sleep structure and sleep quality between the two light conditions, whereas 3.5 h of exposure to high versus low mel‐EDI light induced marginally higher physiological arousal in terms of a lower delta but higher beta power density before sleep, as well as a lower delta power density during sleep. Moreover, participants felt happier before sleep under exposure to high versus low mel‐EDI light. These findings together with the current literature suggest that evening prolonged relatively low mel‐EDI light exposure may mildly increase arousal before and during sleep but affected sleep structure less.

Noradrenaline tracks emotional modulation of attention in human amygdala

The noradrenaline (NA) system is one of the brain's major neuromodulatory systems; it originates in a small midbrain nucleus, the locus coeruleus (LC), and projects widely throughout the brain.1,2 The LC-NA system is believed to regulate arousal and attention3,4 and is a pharmacological target in multiple clinical conditions.5,6,7 Yet our understanding of its role in health and disease has been impeded by a lack of direct recordings in humans. Here, we address this problem by showing that electrochemical estimates of sub-second NA dynamics can be obtained using clinical depth electrodes implanted for epilepsy monitoring. We made these recordings in the amygdala, an evolutionarily ancient structure that supports emotional processing8,9 and receives dense LC-NA projections,10 while patients (n= 3) performed a visual affective oddball task. The task was designed to induce different cognitive states, with the oddball stimuli involving emotionally evocative images,11 which varied in terms of arousal (low versus high) and valence (negative versus positive). Consistent with theory, the NA estimates tracked the emotional modulation of attention, with a stronger oddball response in a high-arousal state. Parallel estimates of pupil dilation, a common behavioral proxy for LC-NA activity,12 supported a hypothesis that pupil-NA coupling changes with cognitive state,13,14 with the pupil and NA estimates being positively correlated for oddball stimuli in a high-arousal but not a low-arousal state. Our study provides proof of concept that neuromodulator monitoring is now possible using depth electrodes in standard clinical use.

Encountering an emotional landmark on the route for a better spatial memory: What matters, valence or arousal?

This study explores the impact of the interplay between the valence and arousal dimensions of landmarks on spatial memory. Participants were asked to watch a movie of a walk in a large urban virtual environment with landmarks that differed in terms of arousal and valence across conditions (high-arousal-positive; low-arousal-positive; high-arousal-negative; low-arousal-negative). Their route knowledge was assessed through a direction recall task, and their survey knowledge by asking them to plot the position of each landmark encountered during the route on a map. Results showed that the presence of positive landmarks benefited route knowledge, whereas high arousing ones impaired it. However, no clear effect of the landmarks’ valence and arousal was observed on survey knowledge. In addition to supporting some of the previous evidence while extending it to more complex and real-like environments, these new results call into question the relevance of the dimensional approach to emotions classically adopted in this type of studies.

An embodiment of the cinematographer: emotional and perceptual responses to different camera movement techniques.

We investigate the relationship between camera movement techniques and cognitive responses in audiences, reporting on an experiment exploring the effects of different camera movement methods on viewers' degree of immersion and emotional response. This follows directly from preceding experimental literature and is further motivated by accounts and experiences of practicing cinematographers (authors included), which indicates a correspondence between the two. We designed three different cinematic scenes with indifferent moods, and shot each one time with Steadicam, dolly, handheld, and static camera, resulting in 12 different clips. A total of 44 non-professional participants watched the clips and rated their reactions in terms of arousal and degree of involvement. Experimental results are mixed: movement affects the sense of involvement but not necessarily emotional response. We present and discuss some further explorative results and possible future directions to improve the design. We argue in this contribution that there is value in experimental approaches to cinematography, enabling the systematic study of creative intuitions and audience responses in controlled settings.

Affective state estimation based on Russell’s model and physiological measurements

Affective states are psycho-physiological constructs connecting mental and physiological processes. They can be represented in terms of arousal and valence according to the Russel’s model and can be extracted from physiological changes in human body. However, a well-established optimal feature set and a classification method effective in terms of accuracy and estimation time are not present in the literature. This paper aims at defining a reliable and efficient approach for real-time affective state estimation. To obtain this, the optimal physiological feature set and the most effective machine learning algorithm, to cope with binary as well as multi-class classification problems, were identified. ReliefF feature selection algorithm was implemented to define a reduced optimal feature set. Supervised learning algorithms, such as K-Nearest Neighbors (KNN), cubic and gaussian Support Vector Machine, and Linear Discriminant Analysis, were implemented to compare their effectiveness in affective state estimation. The developed approach was tested on physiological signals acquired on 20 healthy volunteers during the administration of images, belonging to the International Affective Picture System, conceived for inducing different affective states. ReliefF algorithm reduced the number of physiological features from 23 to 13. The performances of machine learning algorithms were compared and the experimental results showed that both accuracy and estimation time benefited from the optimal feature set use. Furthermore, the KNN algorithm resulted to be the most suitable for affective state estimation. The results of the assessment of arousal and valence states on 20 participants indicate that KNN classifier, adopted with the 13 identified optimal features, is the most effective approach for real-time affective state estimation.

Affect Intensity and Subjective Ratings of Emotional Pictures and Sounds

Affect intensity refers to the intensity with which people experience their emotional response. Individual differences in affect intensity are supposed to be related to the strength of the response to emotional stimuli. Previous studies showed that participants with high affect intensity responded to emotional stimuli with stronger or more intense affective reactions than participants scoring low in affect intensity However, previous studies are mainly limited to the impact of affect intensity on consumer responses to advertising appeals or are limited to the use of life events descriptions as emotional stimuli. No previous studies used behavioural measures of the emotional response to standardized stimuli, varying in terms of arousal. In the present study the predictive value of affect intensity, measured by a self-report questionnaire, the Affect Intensity Measure (AIM), on the emotional response to standardized pictures and sounds has been investigated. In particular, the predictive value of affective intensity measured by the AIM, using both the total AIM total score and the four subscales scores, on subjective arousal ratings of different categories of standardized emotional pictures and sounds was assessed on a nonclinical sample. The total AIM score has been found to be predictive for subjective arousal scores for low unpleasant pictures while, using the AIM subscales scores, results showed that the Negative Reactivity subscale was predictive for arousal scores to high negative pictures and sounds. These findings seem to show that the use of the total AIM score can obscure the relationships between specific features of affect intensity and other variables. Moreover, the present results didn’t show a general effect of affect intensity on behavioural responses to emotional standardized stimuli but an emotion specific effect for high negative stimuli.

How Do Looming and Receding Emotional Faces Modulate Duration Perception?

The direction of visual motion has been shown to affect the perception of interval duration; objects moving towards an observer (i.e., looming) are perceived to last longer than objects moving away (i.e., receding), and this has been explained in terms of arousal- or attention-based modulation. To dissociate the two competing accounts, we investigated how the influence of motion direction on duration perception is modulated by the emotional content of stimuli. Participants were given the temporal bisection task with images of emotional faces (angry, happy, and neutral) presented in a static (Experiment 1) or dynamic (Experiment 2) display. In Experiment 1, we found no influence of facial emotion on perceived duration. In Experiment 2, however, looming (i.e., expanding) stimuli were perceived as lasting longer than receding (contracting) ones. More importantly, we found an interaction between participant-rated arousal to faces and motion direction: The looming/receding asymmetry was pronounced when the stimulus arousal was rated low, but this asymmetry diminished with increasing arousal ratings. Thus, looming/receding temporal asymmetry seems to be reduced when arousing facial expressions enhance attentional engagement.

What if becoming information literate were an adventure?

What if becoming information literate were an adventure? This question was posed in a keynote for the Creating Knowledge Conference 2021. It was answered in a thought piece by examining adventure-based ways to prepare students to be information literate adults through the principles and mechanisms that people find arousing and pleasurable and that are not classically a part of university pedagogy. How might these mechanisms be used to engage students more in the IL learning process and to encourage them to pursue being an information literate person as a lifelong endeavor? Adventure is presented as an experience that is situated, soft or hard, emotionally charged, challenging and rewarding. The risk aspect of adventure, often linked to danger, is also examined. Some of the dangers that exist in the management of information are explored (e.g., traps in sharing practices, seductive novelty and bypassing reason) as well as the personal costs of not managing information literacy well. How we nevertheless find danger alluring is explained in terms of arousal, as well as how we navigate zones of danger and delight with the help of protective frames. In order to arouse and sustain student engagement in becoming information literate adults, the value of interest is also introduced with the four-phase model of interest development. Suggestions for where to get started in translating the mechanisms of adventure, danger and interest into theoretically motivated and enjoyable teaching in order to support student growth as lifelong information literate adults are woven into the text for reflection.

Multi-Task Conformer with Multi-Feature Combination for Speech Emotion Recognition

Along with automatic speech recognition, many researchers have been actively studying speech emotion recognition, since emotion information is as crucial as the textual information for effective interactions. Emotion can be divided into categorical emotion and dimensional emotion. Although categorical emotion is widely used, dimensional emotion, typically represented as arousal and valence, can provide more detailed information on the emotional states. Therefore, in this paper, we propose a Conformer-based model for arousal and valence recognition. Our model uses Conformer as an encoder, a fully connected layer as a decoder, and statistical pooling layers as a connector. In addition, we adopted multi-task learning and multi-feature combination, which showed a remarkable performance for speech emotion recognition and time-series analysis, respectively. The proposed model achieves a state-of-the-art recognition accuracy of 70.0 ± 1.5% for arousal in terms of unweighted accuracy on the IEMOCAP dataset.

Affect recognition from scalp-EEG using channel-wise encoder networks coupled with geometric deep learning and multi-channel feature fusion

The expression of human emotions is a complex process that often manifests through physiological and psychological traits and results in spatio-temporal brain activity. The brain activity can be captured with an electroencephalogram (EEG) and can be used for emotion recognition. In this paper, we present a novel approach to EEG-based emotion recognition (in terms of arousal, valence, and dominance) using unprocessed EEG signals. Input EEG samples are passed through channel-specific encoders consisting of SincNet based convolution blocks (filters are fine-tuned for the emotion recognition during learning) to learn high-level features related to the objectives. The resultant feature embeddings are then passed through a set of graph convolution networks to model the spatial propagation of brain activity under the assumption that the brain activity captured through an electrode is impacted by the brain activity captured by neighbouring electrodes. The channels are represented as nodes in a graph following the relative positioning of the electrodes during dataset acquisition. Multi-head attention is applied together with the graph convolutions to jointly attend to features from different representation sub-spaces, which leads to improved learning. The resultant features are then passed through a deep neural network-based multi-task classifier to identify the dimensional emotional states (low/high). Our proposed model achieves an accuracy of 88.24%, 88.80% and 88.22% for arousal, valence and dominance respectively using a 10-fold cross-validation; and 63.71%, 64.98% and 61.81% with Leave-One-Subject-Out cross-validation (LOSO) on the Dreamer dataset, and 69.72%, 69.43% and 70.72% for a LOSO evaluation on the DEAP dataset, surpassing state-of-the-art methods.

How did you feel during the navigation? Influence of emotions on browsing time and interaction frequency in immersive virtual environments

ABSTRACT Immersive virtual environments (IVEs) represent virtual settings that simulate the physical world. Users interacting in such virtual venues commonly display behaviours like the ones that would occur in the physical world. However, little is known about how the affective states experienced while browsing IVEs may in turn, influence user’s interactive behaviour. The present research investigates how to affect in terms of arousal and valence generated by IVEs influences browsing time and interaction frequency. Three studies analyse various facets of affect in IVEs. Study 1 investigates the cognitive responses and shows that browsing time is positively affected by self-reported arousal. Study 2 analyses neurophysiological responses and demonstrates consistent results with Study 1. It further shows that physiological correlates of arousal and valence positively influence interaction frequency. Study 3 delves into personal interest, a recurring factor emerged in the two previous studies, and investigates its interaction with arousal. Findings show that in a high arousal IVE, highly interested users are more likely to browse longer than low-interested users. Overall, the results show that behavioural realism evoked in IVEs involves both users’ actions and affective states. Implications in terms of design guidelines to foster positive affect in IVEs are drawn.

Influence of Sales Promotion on Impulse Buying: A Dual Process Approach

This study investigates how sales promotions influence the psychological processing that precedes impulse buying. Applying a dual process theory, we distinguish between Reflective and Impulsive processing pathways. Through a survey study on 470 consumers, we tested four sales promotions characterized by different rewards (monetary versus nonmonetary) and gratification typologies (immediate-reward versus delayed-reward). The study further analyses the effects of promotion-induced affect, in terms of arousal and valence, as well as the influence of individual impulse buying tendency and sales proneness. Results show that impulsive responses are influenced by promotion-induced affect and individual differences, while reflective responses differ depending on the reward typology. The results contribute to the extant literature by discriminating the effects of sales promotions on the Reflective and Impulsive systems. Implications for retailers and promotion managers concerning the formulation of effective promotion strategies are discussed.

Cognitive Dissonance: Affecting Party Orientation and Selective Recall of Political Information

Cognitive dissonance theory posits that inconsistencies between attitudes and behaviors cause an uncomfortable arousal state, and people are motivated to reduce this discomfort by changing attitudes or behaviors to increase consistency. This principle applies to research on political affiliation. Due to dissonance processes, individuals focus less on political information that opposes their views and pay greater attention when it is congruent with their views. This study adds to this research by examining whether political orientation causes a similar pattern of selective attention bias during the recall stage. Participants (117) studied a political article on a social issue representing a viewpoint that was favorable to Democrats. Next, participants recalled as much information as possible by typing the information in a textbox. Using a sliding scale, they also rated how they felt about the article in terms of arousal and affect, and indicated whether the article was neutral, positive, or negative. Democrats were predicted to recall more positive information and more positive affect after reading the article than Republicans. Surprisingly, more Republicans, rather than Democrats, recalled more positive information about the article. Finally, those who scored more conservatively on the political slider also reported more positive affect toward the article. Although contrary to the study hypotheses, these results have implications for our current understanding of selective attention in a political context by showing the bias also occurs at the recall stage. It takes a special effort to be able to think outside the bubble. The purpose of this study is to find what it takes to pop the bubble and change the mindset of political engaged people. Keywords: cognitive dissonance, attentional bias, political orientation, selective recall

Relationships of attention and arousal are responsible for action in sports

Abstract Study aim: The most important psychological mechanisms that are responsible for sports activity are arousal and attention. Study aim the relationships between the level of arousal and the level of attention, an attempt has been made to explain the mechanisms responsible for sportsman activity. Material and method: The study was conducted in a group of sportsman-students (68 individuals) using the Vienna Test System: FLIM, a test of flicker/fusion frequency, which is a measurement procedure allowing to determine the functional readiness of the central nervous system in terms of arousal and COG (Cognitron), which is a test measuring the level of attention. Results: The following in statistical analysis of the data were observed: inversely proportional relationships of image fusion frequency (FLIM1) during the recording of the level of arousal with: the correct acceptance of stimuli (COG1, r = –0.287), the correct rejection of stimuli (COG2, r = –0.320), the time of correctly accepted stimuli (COG3, r = –0.299), and with the time of correctly rejected stimuli (COG4, r = –0.317) in the attentional activity. Conclusion: Fusion frequency indicates the level of fatigue and is inversely proportional to the correctly accepted stimuli in attentional activity, the correctly rejected stimuli in attentional activities, and the duration of the attentional actions.

How children experience virtual reality travel: a psycho-physiological study based on flow theory

PurposeDrawn from flow theory, this study aims to explore children’s flow experiences in virtual reality (VR) by examining the relationships between VR experience length, arousal and flow experience (time estimation and enjoyment).Design/methodolog/approachA within-subject laboratory experiment was conducted using 9D VR and iMEC 12. A total of 47 children participated in this study. Both survey data and physiological data were collected.FindingsThe results revealed that longer VR videos evoked lower arousal, higher respiratory rates and less enjoyment among child participants than shorter videos. This study also verified the mediating effects of respiratory rates measured arousal on time estimations and self-reported arousal in terms of enjoyment.Originality/valueThis study fills a research gap related to children’s flow experiences in VR, as children’s voices remain limited in tourism research. The study findings offer meaningful insights for destination marketers to leverage the growth of parent-child tours and the increasing effects of VR on the tourism industry.

An integrated emotional and physiological assessment for VR-based active shooter incident experiments

Unfortunately, active shooter incidents are on the rise in the United States. With the recent technological advancements, virtual reality (VR) experiments could serve as an effective method to prepare civilians and law enforcement personnel for such scenarios. However, for VR experiments to be effective for active shooter training and research, such experiments must be able to evoke emotional and physiological responses as live active shooter drills and events do. The objective of this study is thus to test the effectiveness of an active shooter VR experiment on emotional and physiological responses. Additionally, we consider different locomotion techniques (i.e., walk-in-place and controller) and explore their impact on users’ sense of presence. The results suggest that the VR active shooter experiment in this study can induce emotional arousal and increase heart rate of the participants immersed in the virtual environment. Furthermore, compared to the controller, the walk-in-place technique resulted in a higher emotional arousal in terms of negative emotions and a stronger sense of presence. The study presents a foundation for future active shooter experiments as it supports the ecological validity using VR for active shooter incident related work for the purposes of training or research.

A Bottom-Up Validation of the IAPS, GAPED, and NAPS Affective Picture Databases: Differential Effects on Behavioral Performance.

The concept of emotion is a complex neural and psychological phenomenon, central to the organization of human social behavior. As the result of subjective experience, emotions involve bottom-up cognitive styles responsible for efficient adaptation of human behavior to the environment based on salient goals. Indeed, bottom-up cognitive processes are mandatory for clarifying emotion-cognition interactions. Accordingly, a huge number of studies and standardized affective stimuli databases have been developed (i.e., International Affective Picture System (IAPS), Geneva Affective Picture Database (GAPED), and Nencki Affective Picture System (NAPS)). However, these neither accurately reflect the complex neural system underlying emotional responses nor do they offer a comprehensive framework for researchers. The present article aims to provide an additional bottom-up validation of affective stimuli that are independent from cognitive processing and control mechanisms, related to the implicit relevance and evolutionistic significance of stimuli. A subset of 360 images from the original NAPS, GAPED, and IAPS datasets was selected in order to proportionally cover the whole dimensional affective space. Among these, using a two-step analysis strategy, we identified three clusters (“good performance”, “poor performance”, and “false alarm”) of stimuli with similar cognitive response profiles. Results showed that the three clusters differed in terms of arousal and database membership, but not in terms of valence. The new database, with accompanying ratings and image parameters, allows researchers to select visual stimuli independent from dimensional/discrete-categories, and provides information on the implicit effects triggered by such stimuli.

Differences between young and older adults in physiological and subjective responses to emotion induction using films

Emotional response in aging is typically studied using the dimensional or the discrete models of emotion. Moreover, it is typically studied using subjective or physiological variables but not using both perspectives simultaneously. Additionally, tenderness is neglected in emotion induction procedures with older adults, with the present work being the first to include the study of physiological tenderness using film clips. This study integrated two separate approaches to emotion research, comparing 68 younger and 39 older adults and using a popular set of film clips to induce tenderness, amusement, anger, fear, sadness and disgust emotions. The direction of subjective emotional patterns was evaluated with self-reports and that of physiological emotional patterns was evaluated with a wearable emotion detection system. The findings suggest a dual-process framework between subjective and physiological responses, manifested differently in young and older adults. In terms of arousal, the older adults exhibited higher levels of subjective arousal in negative emotions and tenderness while young adults showed higher levels of physiological arousal in these emotions. These findings yield information on the multidirectionality of positive and negative emotions, corroborating that emotional changes in the adult lifespan appear to be subject to the relevance of the emotion elicitor to each age group.

Touching the audience: musical haptic wearables for augmented and participatory live music performances

This paper introduces the musical haptic wearables for audiences (MHWAs), a class of wearable devices for musical applications targeting audiences of live music performances. MHWAs are characterized by embedded intelligence, wireless connectivity to local and remote networks, a system to deliver haptic stimuli, and tracking of gestures and/or physiological parameters. They aim to enrich musical experiences by leveraging the sense of touch as well as providing new capabilities for creative participation. The embedded intelligence enables the communication with other external devices, processes input data, and generates music-related haptic stimuli. We validate our vision with two concert-experiments. The first experiment involved a duo of electronic music performers and twenty audience members. Half of the audience used an armband-based prototype of MHWA delivering vibro-tactile feedback in response to performers’ actions on their digital musical instruments, and the other half was used as a control group. In the second experiment, a smart mandolin performer played live for twelve audience members wearing a gilet-based MHWA, which provided vibro-tactile sensations in response to the performed music. Overall, results from both experiments suggest that MHWAs have the potential to enrich the experience of listening to live music in terms of arousal, valence, enjoyment, and engagement. Nevertheless, results showed that the audio-haptic experience was not homogeneous across participants, who could be grouped as those appreciative of the vibrations and those less appreciative of them. The causes for a lack of appreciation of the haptic experience were mainly identified as the sensation of unpleasantness caused by the vibrations in certain parts of the body and the lack of the comprehension of the relation between what was felt and what was heard. Based on the reported results, we offer suggestions for practitioners interested in designing wearables for enriching the musical experience of audiences of live music via the sense of touch. Such suggestions point towards the need of mechanisms of personalization, systems able to minimize the latency between the sound and the vibrations, and a time of adaptation to the vibrations.