Vocal Emotion Research Articles

Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users.

Objectives:Research on cochlear implants (CIs) has focused on speech comprehension, with little research on perception of vocal emotions. We compared emotion perception in CI users and normal-hearing (NH) individuals, using parameter-specific voice morphing.Design:Twenty-five CI users and 25 NH individuals (matched for age and gender) performed fearful-angry discriminations on bisyllabic pseudoword stimuli from morph continua across all acoustic parameters (Full), or across selected parameters (F0, Timbre, or Time information), with other parameters set to a noninformative intermediate level.Results:Unsurprisingly, CI users as a group showed lower performance in vocal emotion perception overall. Importantly, while NH individuals used timbre and fundamental frequency (F0) information to equivalent degrees, CI users were far more efficient in using timbre (compared to F0) information for this task. Thus, under the conditions of this task, CIs were inefficient in conveying emotion based on F0 alone. There was enormous variability between CI users, with low performers responding close to guessing level. Echoing previous research, we found that better vocal emotion perception was associated with better quality of life ratings.Conclusions:Some CI users can utilize timbre cues remarkably well when perceiving vocal emotions.

Functional patterns of neural activation during vocal emotion recognition in youth with and without refractory epilepsy

Epilepsy has been associated with deficits in the social cognitive ability to decode others’ nonverbal cues to infer their emotional intent (emotion recognition). Studies have begun to identify potential neural correlates of these deficits, but have focused primarily on one type of nonverbal cue (facial expressions) to the detriment of other crucial social signals that inform the tenor of social interactions (e.g., tone of voice). Less is known about how individuals with epilepsy process these forms of social stimuli, with a particular gap in knowledge about representation of vocal cues in the developing brain. The current study compared vocal emotion recognition skills and functional patterns of neural activation to emotional voices in youth with and without refractory focal epilepsy. We made novel use of inter-subject pattern analysis to determine brain areas in which activation to emotional voices was predictive of epilepsy status. Results indicated that youth with epilepsy were comparatively less able to infer emotional intent in vocal expressions than their typically developing peers. Activation to vocal emotional expressions in regions of the mentalizing and/or default mode network (e.g., right temporo-parietal junction, right hippocampus, right medial prefrontal cortex, among others) differentiated youth with and without epilepsy. These results are consistent with emerging evidence that pediatric epilepsy is associated with altered function in neural networks subserving social cognitive abilities. Our results contribute to ongoing efforts to understand the neural markers of social cognitive deficits in pediatric epilepsy, in order to better tailor and funnel interventions to this group of youth at risk for poor social outcomes.

Mexican Emotional Speech Database Based on Semantic, Frequency, Familiarity, Concreteness, and Cultural Shaping of Affective Prosody

In this paper, the Mexican Emotional Speech Database (MESD) that contains single-word emotional utterances for anger, disgust, fear, happiness, neutral and sadness with adult (male and female) and child voices is described. To validate the emotional prosody of the uttered words, a cubic Support Vector Machines classifier was trained on the basis of prosodic, spectral and voice quality features for each case study: (1) male adult, (2) female adult and (3) child. In addition, cultural, semantic, and linguistic shaping of emotional expression was assessed by statistical analysis. This study was registered at BioMed Central and is part of the implementation of a published study protocol. Mean emotional classification accuracies yielded 93.3%, 89.4% and 83.3% for male, female and child utterances respectively. Statistical analysis emphasized the shaping of emotional prosodies by semantic and linguistic features. A cultural variation in emotional expression was highlighted by comparing the MESD with the INTERFACE for Castilian Spanish database. The MESD provides reliable content for linguistic emotional prosody shaped by the Mexican cultural environment. In order to facilitate further investigations, a corpus controlled for linguistic features and emotional semantics, as well as one containing words repeated across voices and emotions are provided. The MESD is made freely available.

Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

While the human perceptual system constantly adapts to the environment, some of the underlying mechanisms are still poorly understood. For instance, although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional voices and ambiguous voices were used as reference conditions. All adaptor stimuli were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of subsequently presented target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptation were much larger than following F0 adaptation and were only marginally smaller than those following full adaptation. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-correspondence between adaptor and target stimuli. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

Extraction and Utilization of Excitation Information of Speech: A Review

Speech production can be regarded as a process where a time-varying vocal tract system (filter) is excited by a time-varying excitation. In addition to its linguistic message, the speech signal also carries information about, for example, the gender and age of the speaker. Moreover, the speech signal includes acoustical cues about several speaker traits, such as the emotional state and the state of health of the speaker. In order to understand the production of these acoustical cues by the human speech production mechanism and utilize this information in speech technology, it is necessary to extract features describing both the excitation and the filter of the human speech production mechanism. While the methods to estimate and parameterize the vocal tract system are well established, the excitation appears less studied. This article provides a review of signal processing approaches used for the extraction of excitation information from speech. This article highlights the importance of excitation information in the analysis and classification of phonation type and vocal emotions, in the analysis of nonverbal laughter sounds, and in studying pathological voices. Furthermore, recent developments of deep learning techniques in the context of extraction and utilization of the excitation information are discussed.

Vocal communication across cultures: theoretical and methodological issues.

The study of human vocal communication has been conducted primarily in Western, educated, industrialized, rich, democratic (WEIRD) societies. Recently, cross-cultural investigations in several domains of voice research have been expanding into more diverse populations. Theoretically, it is important to understand how universals and cultural variations interact in vocal production and perception, but cross-cultural voice research presents many methodological challenges. Experimental methods typically used in WEIRD societies are often not possible to implement in many populations such as rural, small-scale societies. Moreover, theoretical and methodological issues are often unnecessarily intertwined. Here, I focus on three areas of cross-cultural voice modulation research: (i) vocal signalling of formidability and dominance, (ii) vocal emotions, and (iii) production and perception of infant-directed speech. Research in these specific areas illustrates challenges that apply more generally across the human behavioural sciences but also reveals promise as we develop our understanding of the evolution of human communication.This article is part of the theme issue ‘Voice modulation: from origin and mechanism to social impact (Part II)’.

Emotion Recognition of Foreign Language Teachers in College English Classroom Teaching.

In order to improve students’ learning effect, more and more universities favor foreign language teachers who are native speakers of English. Based on the analysis and summary of the research status of emotion recognition, this paper proposes that, in college English classroom teaching, foreign language teachers can reduce the communication barriers with Chinese students through emotion recognition. Based on literature review and actual situation investigation, this study identified four influencing factors on emotion recognition of foreign language teachers, namely, interactive action, facial expression, vocal emotion, and body posture. In our opinion, in the teaching process, teachers can adjust the four factors of emotion recognition to achieve better teaching effect. Further, improve students’ learning efficiency. Analytic Hierarchy Process (AHP) is chosen as the research method in this study. After building the analysis model, we collected the questionnaire using the Questionnaire Star, and finally got 12 valid data. After determining the importance of different factors by pairwise comparison, we draw the following conclusions: the influence degree of emotion recognition factors of foreign language teachers is in descending order, interactive action (43%), facial expression (28%), vocal emotion (21%), and body posture (9%). Our research adds to the body of knowledge on emotion recognition among college English teachers. Furthermore, this research assists students in improving their grasp of course content based on the emotions of foreign English lecturers. Based on the findings, we recommend that foreign language teachers in college English classrooms alter their interactive behaviors, facial expressions, and vocal emotions in response to various instructional materials and emphases.

Associations between vocal emotion recognition and socio-emotional adjustment in children.

The human voice is a primary channel for emotional communication. It is often presumed that being able to recognize vocal emotions is important for everyday socio-emotional functioning, but evidence for this assumption remains scarce. Here, we examined relationships between vocal emotion recognition and socio-emotional adjustment in children. The sample included 141 6- to 8-year-old children, and the emotion tasks required them to categorize five emotions (anger, disgust, fear, happiness, sadness, plus neutrality), as conveyed by two types of vocal emotional cues: speech prosody and non-verbal vocalizations such as laughter. Socio-emotional adjustment was evaluated by the children's teachers using a multidimensional questionnaire of self-regulation and social behaviour. Based on frequentist and Bayesian analyses, we found that, for speech prosody, higher emotion recognition related to better general socio-emotional adjustment. This association remained significant even when the children's cognitive ability, age, sex and parental education were held constant. Follow-up analyses indicated that higher emotional prosody recognition was more robustly related to the socio-emotional dimensions of prosocial behaviour and cognitive and behavioural self-regulation. For emotion recognition in non-verbal vocalizations, no associations with socio-emotional adjustment were found. A similar null result was obtained for an additional task focused on facial emotion recognition. Overall, these results support the close link between children's emotional prosody recognition skills and their everyday social behaviour.

Impaired emotion perception and categorization in semantic aphasia

According to a constructionist model of emotion, conceptual knowledge plays a foundational role in emotion perception; reduced availability of relevant conceptual knowledge should therefore impair emotion perception. Conceptual deficits can follow both degradation of semantic knowledge (e.g., semantic ‘storage’ deficits in semantic dementia) and deregulation of retrieval (e.g., semantic ‘access’ deficits in semantic aphasia). While emotion recognition deficits are known to accompany degraded conceptual knowledge, less is known about the impact of semantic access deficits. Here, we examined emotion perception and categorization tasks in patients with semantic aphasia, who have difficulty accessing semantic information in a flexible and controlled fashion following left hemisphere stroke. In Study 1, participants were asked to sort faces according to the emotion they portrayed – with numbers, written labels and picture examples each provided as category anchors across tasks. Semantic aphasia patients made more errors and showed a larger benefit from word anchors that reduced the need to internally constrain categorization than comparison participants. They successfully sorted portrayals that differed in valence (positive vs. negative) but had difficulty categorizing different negative emotions. They were unimpaired on a control task that involved sorting faces by identity. In Study 2, participants matched facial emotion portrayals to written labels following vocal emotion prosody cues, miscues, or no cues. Patients presented with overall poorer performance and benefited from cue trials relative to within-valence miscue trials. This same effect was seen in comparison participants, who also showed deleterious effects of within-valence miscue relative to no cue trials. Overall, we found that patients with deregulated semantic retrieval have deficits in emotional perception but that word anchors and cue conditions can facilitate emotion perception by increasing access to relevant emotion concepts and reducing reliance on semantic control. Semantic control may be of particular importance in emotion perception when it is necessary to interpret ambiguous inputs, or when there is interference between conceptually similar emotional states. These findings extend constructionist accounts of emotion to encompass difficulties in controlled semantic retrieval.

Age and sex effects in emotional prosody processing revealed in infants' mismatch responses but not in preferential looking time

Emotional prosody is crucial in early communication, but the developmental trend and the emergence of sex differences in infants’ emotional processing remain unclear. This report compared 46 infants’ behavioral and neurophysiological responses to four vocal emotions (happy, angry, sad, and neutral). We used a central fixation paradigm to measure infants’ listening times to different emotions and a multi-feature oddball paradigm with electroencephalography (EEG) recordings to measure infants’ mismatch responses (MMRs) to vocal emotional changes. We modified both paradigms by including non-repeating words within each emotion, enforcing infant listeners to respond to the overarching emotional prosodic category. We only observed age and sex effects in infants’ MMRs. Older infants showed more negative MMRs to happy and sad voices than younger infants, but more positive MMRs to angry voices. Female infants showed more positive MMRs at a later time window than male infants across all emotional changes, indicating an emotion-general sex effect. The behavioral study showed longer listening times for happy and sad prosodies across age and sex. The findings indicate that neurophysiological measurement may be more sensitive in capturing age and sex effects in early emotional prosody perception when complex speech materials are used.

Attention to voices is increased in non-clinical auditory verbal hallucinations irrespective of salience

Alterations in the processing of vocal emotions have been associated with both clinical and non-clinical auditory verbal hallucinations (AVH), suggesting that changes in the mechanisms underpinning voice perception contribute to AVH. These alterations seem to be more pronounced in psychotic patients with AVH when attention demands increase. However, it remains to be clarified how attention modulates the processing of vocal emotions in individuals without clinical diagnoses who report hearing voices but no related distress. Using an active auditory oddball task, the current study clarified how emotion and attention interact during voice processing as a function of AVH proneness, and examined the contributions of stimulus valence and intensity. Participants with vs. without non-clinical AVH were presented with target vocalizations differing in valence (neutral; positive; negative) and intensity (55 decibels (dB); 75 dB). The P3b amplitude was larger in response to louder (vs. softer) vocal targets irrespective of valence, and in response to negative (vs. neutral) vocal targets irrespective of intensity. Of note, the P3b amplitude was globally increased in response to vocal targets in participants reporting AVH, and failed to be modulated by valence and intensity in these participants. These findings suggest enhanced voluntary attention to changes in vocal expressions but reduced discrimination of salient and non-salient cues. A decreased sensitivity to salience cues of vocalizations could contribute to increased cognitive control demands, setting the stage for an AVH.

Carrots or sticks in debt collection services? A voice metrics and text analysis of debt collection calls

PurposeAlthough phone calls are widely used by debt collection services to persuade delinquent customers to repay, few financial services studies have analyzed the unstructured voice and text data to investigate how debt collection call strategies drive customers to repay. Moreover, extant research opens the “black box” mainly through psychological theories without hard behavioral data of customers. The purpose of our study is to address this research gap.Design/methodology/approachThe authors randomly sampled 3,204 debt collection calls from a large consumer finance company in East Asia. To rule out alternative explanations for the findings, such as consumers' previous experience of being persuaded by debt collectors or repeated calls, the authors selected calls made to delinquent customers who had not been delinquent before and were being called by the company for the first time. The authors transformed the unstructured voice and textual data into structured data through automatic speech recognition (ASR), voice mining, natural language processing (NLP) and machine learning analyses.FindingsThe findings revealed that (1) both moral appeal (carrot) and social warning (stick) strategies decrease repayment time because they arouse mainly happy emotion and fear emotion, respectively; (2) the legal warning (stick) strategy backfires because of decreasing the happy emotion and triggering the anger emotion, which impedes customers' compliance; and (3) in contrast to traditional wisdom, the combination of carrot and stick fails to decrease the repayment time.Originality/valueThe findings provide a valuable and systematic understanding of the effect of carrot strategies, stick strategies and the combinations of them on repayment time. This study is among the first to empirically analyze the effectiveness of carrot strategies, stick strategies and their joint strategies on repayment time through unstructured vocal and textual data analysis. What's more, the previous studies open the “black box” through psychological mechanism. The authors firstly elucidate a behavioral mechanism for why consumers behave differently under varying debt collection strategies by utilizing ASR, NLP and vocal emotion analyses.

Auditory deviance detection and involuntary attention allocation in occupational burnout-A follow-up study.

Here, we investigated the central auditory processing and attentional control associated with both recovery and prolongation of occupational burnout. We recorded the event-related brain potentials N1, P2, mismatch negativity (MMN) and P3a to nine changes in speech sounds and to three rarely presented emotional (happy, angry and sad) utterances from individuals with burnout (N = 16) and their matched controls (N = 12). After the 5 years follow-up, one control had acquired burnout, half (N = 8) of the burnout group had recovered, and the other half (prolonged burnout) still had burnout. The processing of acoustical changes in speech sounds was mainly intact. Prolongation of the burnout was associated with a decrease in MMN amplitude and an increase in P3a amplitude for the happy stimulus. The results suggest that, in the absence of interventions, burnout is a persistent condition, associated with alterations of attentional control, that may be amplified with the prolongation of the condition.

Analysis of acoustic and voice quality features for the classification of infant and mother vocalizations

Classification of infant and parent vocalizations, particularly emotional vocalizations, is critical to understanding how infants learn to regulate emotions in social dyadic processes. This work is an experimental study of classifiers, features, and data augmentation strategies applied to the task of classifying infant and parent vocalization types. Our data were recorded both in the home and in the laboratory. Infant vocalizations were manually labeled as cry, fus (fuss), lau (laugh), bab (babble) or scr (screech), while parent (mostly mother) vocalizations were labeled as ids (infant-directed speech), ads (adult-directed speech), pla (playful), rhy (rhythmic speech or singing), lau (laugh) or whi (whisper). Linear discriminant analysis (LDA) was selected as a baseline classifier, because it gave the highest accuracy in a previously published study covering part of this corpus. LDA was compared to two neural network architectures: a two-layer fully-connected network (FCN), and a convolutional neural network with self-attention (CNSA). Baseline features extracted using the OpenSMILE toolkit were augmented by extra voice quality, phonetic, and prosodic features, each targeting perceptual features of one or more of the vocalization types. Three web data augmentation and transfer learning methods were tested: pre-training of network weights for a related task (adult emotion classification), augmentation of under-represented classes using data uniformly sampled from other corpora, and augmentation of under-represented classes using data selected by a minimum cross-corpus information difference criterion. Feature selection using Fisher scores and experiments of using weighted and unweighted samplers were also tested. Two datasets were evaluated: a benchmark dataset (CRIED) and our own corpus. In terms of unweighted-average recall of CRIED dataset, the CNSA achieved the best UAR compared with previous studies. In terms of classification accuracy, weighted F1, and macro F1 of our own dataset, the neural networks both significantly outperformed LDA; the FCN slightly (but not significantly) outperformed the CNSA. Cross-examining features selected by different feature selection algorithms permits a type of post-hoc feature analysis, in which the most important acoustic features for each binary type discrimination are listed. Examples of each vocalization type of overlapped features were selected, and their spectrograms are presented, and discussed with respect to the type-discriminative acoustic features selected by various algorithms. MFCC, log Mel Frequency Band Energy, LSP frequency, and F1 are found to be the most important spectral envelope features; F0 is found to be the most important prosodic feature.

Age-Related Changes in Voice Emotion Recognition by Postlingually Deafened Listeners With Cochlear Implants.

Identification of emotional prosody in speech declines with age in normally hearing (NH) adults. Cochlear implant (CI) users have deficits in the perception of prosody, but the effects of age on vocal emotion recognition by adult postlingually deaf CI users are not known. The objective of the present study was to examine age-related changes in CI users' and NH listeners' emotion recognition. Participants included 18 CI users (29.6 to 74.5 years) and 43 NH adults (25.8 to 74.8 years). Participants listened to emotion-neutral sentences spoken by a male and female talker in five emotions (happy, sad, scared, angry, neutral). NH adults heard them in four conditions: unprocessed (full spectrum) speech, 16-channel, 8-channel, and 4-channel noise-band vocoded speech. The adult CI users only listened to unprocessed (full spectrum) speech. Sensitivity (d') to emotions and Reaction Times were obtained using a single-interval, five-alternative, forced-choice paradigm. For NH participants, results indicated age-related declines in Accuracy and d', and age-related increases in Reaction Time in all conditions. Results indicated an overall deficit, as well as age-related declines in overall d' for CI users, but Reaction Times were elevated compared with NH listeners and did not show age-related changes. Analysis of Accuracy scores (hit rates) were generally consistent with d' data. Both CI users and NH listeners showed age-related deficits in emotion identification. The CI users' overall deficit in emotion perception, and their slower response times, suggest impaired social communication which may in turn impact overall well-being, particularly so for older CI users, as lower vocal emotion recognition scores have been associated with poorer subjective quality of life in CI patients.

The Neural Processing of Vocal Emotion After Hearing Reconstruction in Prelingual Deaf Children: A Functional Near-Infrared Spectroscopy Brain Imaging Study.

As elucidated by prior research, children with hearing loss have impaired vocal emotion recognition compared with their normal-hearing peers. Cochlear implants (CIs) have achieved significant success in facilitating hearing and speech abilities for people with severe-to-profound sensorineural hearing loss. However, due to the current limitations in neuroimaging tools, existing research has been unable to detail the neural processing for perception and the recognition of vocal emotions during early stage CI use in infant and toddler CI users (ITCI). In the present study, functional near-infrared spectroscopy (fNIRS) imaging was employed during preoperative and postoperative tests to describe the early neural processing of perception in prelingual deaf ITCIs and their recognition of four vocal emotions (fear, anger, happiness, and neutral). The results revealed that the cortical response elicited by vocal emotional stimulation on the left pre-motor and supplementary motor area (pre-SMA), right middle temporal gyrus (MTG), and right superior temporal gyrus (STG) were significantly different between preoperative and postoperative tests. These findings indicate differences between the preoperative and postoperative neural processing associated with vocal emotional stimulation. Further results revealed that the recognition of vocal emotional stimuli appeared in the right supramarginal gyrus (SMG) after CI implantation, and the response elicited by fear was significantly greater than the response elicited by anger, indicating a negative bias. These findings indicate that the development of emotional bias and the development of emotional perception and recognition capabilities in ITCIs occur on a different timeline and involve different neural processing from those in normal-hearing peers. To assess the speech perception and production abilities, the Infant-Toddler Meaningful Auditory Integration Scale (IT-MAIS) and Speech Intelligibility Rating (SIR) were used. The results revealed no significant differences between preoperative and postoperative tests. Finally, the correlates of the neurobehavioral results were investigated, and the results demonstrated that the preoperative response of the right SMG to anger stimuli was significantly and positively correlated with the evaluation of postoperative behavioral outcomes. And the postoperative response of the right SMG to anger stimuli was significantly and negatively correlated with the evaluation of postoperative behavioral outcomes.

Autism, music and Alexithymia: A musical intervention to enhance emotion recognition in adolescents with ASD

BackgroundDifficulties identifying and describing emotions in Autism Spectrum Disorder (ASD) have been linked with an increased prevalence of Type 2 Alexithymia. Alexithymia is associated with difficulties in interpreting and verbally labelling physiological arousal. Children and adults with ASD show typical patterns of physiological arousal to music and can attribute verbal labels to musical emotions. AimThis pilot study aimed to develop a music-based intervention to improve facial and vocal emotion recognition (ER) and Alexithymia in adolescents with ASD. Methods and proceduresAdolescents with ASD completed 5 music sessions and pre and post-tests of Alexithymia, ER and language. Each intervention began with a researcher-led group analysis of the emotions expressed in a series of musical excerpts, followed by a group-led discussion of the participants’ experiences of these emotions and the ways they may be communicated. Finally, the likely causes and outward expression of these emotions were discussed. Outcome and resultsResults showed that at pre-test, chronological age (CA) and receptive vocabulary were significantly associated with recognition of facial and verbal emotions and Not hiding emotions. At post-test, older children showed a greater increase in recognition of voices and in emotional bodily awareness. Correlations suggested a trend towards increased ER in voices and faces in children with lower language scores. Conclusions and implicationsMusic-based interventions may enhance ER in adolescents with ASD and Alexithymia. Limitations and recommendations for future investigations are discussed.

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants.

Normally-hearing (NH) listeners rely more on prosodic cues than on lexical-semantic cues for emotion perception in speech. In everyday spoken communication, the ability to decipher conflicting information between prosodic and lexical-semantic cues to emotion can be important: for example, in identifying sarcasm or irony. Speech degradation in cochlear implants (CIs) can be sufficiently overcome to identify lexical-semantic cues, but the distortion of voice pitch cues makes it particularly challenging to hear prosody with CIs. The purpose of this study was to examine changes in relative reliance on prosodic and lexical-semantic cues in NH adults listening to spectrally degraded speech and adult CI users. We hypothesized that, compared with NH counterparts, CI users would show increased reliance on lexical-semantic cues and reduced reliance on prosodic cues for emotion perception. We predicted that NH listeners would show a similar pattern when listening to CI-simulated versions of emotional speech. Sixteen NH adults and 8 postlingually deafened adult CI users participated in the study. Sentences were created to convey five lexical-semantic emotions (angry, happy, neutral, sad, and scared), with five sentences expressing each category of emotion. Each of these 25 sentences was then recorded with the 5 (angry, happy, neutral, sad, and scared) prosodic emotions by 2 adult female talkers. The resulting stimulus set included 125 recordings (25 Sentences × 5 Prosodic Emotions) per talker, of which 25 were congruent (consistent lexical-semantic and prosodic cues to emotion) and the remaining 100 were incongruent (conflicting lexical-semantic and prosodic cues to emotion). The recordings were processed to have 3 levels of spectral degradation: full-spectrum, CI-simulated (noise-vocoded) to have 8 channels and 16 channels of spectral information, respectively. Twenty-five recordings (one sentence per lexical-semantic emotion recorded in all five prosodies) were used for a practice run in the full-spectrum condition. The remaining 100 recordings were used as test stimuli. For each talker and condition of spectral degradation, listeners indicated the emotion associated with each recording in a single-interval, five-alternative forced-choice task. The responses were scored as proportion correct, where "correct" responses corresponded to the lexical-semantic emotion. CI users heard only the full-spectrum condition. The results showed a significant interaction between hearing status (NH, CI) and congruency in identifying the lexical-semantic emotion associated with the stimuli. This interaction was as predicted, that is, CI users showed increased reliance on lexical-semantic cues in the incongruent conditions, while NH listeners showed increased reliance on the prosodic cues in the incongruent conditions. As predicted, NH listeners showed increased reliance on lexical-semantic cues to emotion when the stimuli were spectrally degraded. The present study confirmed previous findings of prosodic dominance for emotion perception by NH listeners in the full-spectrum condition. Further, novel findings with CI patients and NH listeners in the CI-simulated conditions showed reduced reliance on prosodic cues and increased reliance on lexical-semantic cues to emotion. These results have implications for CI listeners' ability to perceive conflicts between prosodic and lexical-semantic cues, with repercussions for their identification of sarcasm and humor. Understanding instances of sarcasm or humor can impact a person's ability to develop relationships, follow conversation, understand vocal emotion and intended message of a speaker, following jokes, and everyday communication in general.

The perceived salience of vocal emotions is dampened in non-clinical auditory verbal hallucinations

ABSTRACT Introduction: Auditory verbal hallucinations (AVH) are a cardinal symptom of schizophrenia but are also reported in the general population without need for psychiatric care. Previous evidence suggests that AVH may reflect an imbalance of prior expectation and sensory information, and that altered salience processing is characteristic of both psychotic and non-clinical voice hearers. However, it remains to be shown how such an imbalance affects the categorisation of vocal emotions in perceptual ambiguity. Methods: Neutral and emotional nonverbal vocalisations were morphed along two continua differing in valence (anger; pleasure), each including 11 morphing steps at intervals of 10%. College students (N = 234) differing in AVH proneness (measured with the Launay-Slade Hallucination Scale) evaluated the emotional quality of the vocalisations. Results: Increased AVH proneness was associated with more frequent categorisation of ambiguous vocalisations as ‘neutral’, irrespective of valence. Similarly, the perceptual boundary for emotional classification was shifted by AVH proneness: participants needed more emotional information to categorise a voice as emotional. Conclusions: These findings suggest that emotional salience in vocalisations is dampened as a function of increased AVH proneness. This could be related to changes in the acoustic representations of emotions or reflect top-down expectations of less salient information in the social environment.

Mothers' discourse during shared reading of books relating to ‘positive’ and ‘negative’ emotions in different genres

BackgroundThe current study provides information about differences in parent–child discourse during shared book reading (SBR) surrounding narrative and didactic book genres about ‘positive’ (e.g. love, happiness) and ‘negative’ (e.g. anger and sadness) emotions. Research shows that characteristics of different children's book genres impact discourse during SBR, but narrative and didactic books have not been examined. Children's books often relate to the characters' inner mental and emotional states. Research has not explored parent–child conversations during SBR regarding different types of emotions, yet studies on other discourse contexts reveal that parents discuss ‘positive’ emotions differently than ‘negative’ emotions.MethodsParticipants were 26 mothers of 5‐ to 6‐year‐olds (M = 64.30 SD = 6.27). There were 18 boys and 8 girls. Eleven dyads received a narrative and didactic book on ‘positive’ emotions, and 15 dyads received a narrative and didactic book on ‘negative’ emotions. Interactions were video‐recorded and analysed. We examined the number of turns, initiations, general utterances and mental–emotional utterances.ResultsPrimary results revealed that mothers used more general utterances when reading both narrative and didactic books surrounding ‘negative’ emotions compared with books surrounding ‘positive’ emotions. Mothers used more mental–emotional utterances with a didactic book on ‘negative’ emotions compared with a narrative on ‘negative’ emotions and with a didactic book on ‘positive’ emotions compared to a narrative on ‘positive’ emotions.ConclusionsThis study highlights the importance of text selection for SBR and how it may impact the discourse. Mothers used didactic books for detailed conversations with their children. The simpler structure of the book may have facilitated the discourse about emotions. Mothers related more to anger and sadness compared with love and happiness. This may indicate that they feel they have to explain and teach their children about ‘negative’ emotions and how to cope with them, whereas positive emotions may be more implicitly understood.