Pitch Patterns Research Articles

Deep Learning-Based Feature Extraction for Speech Emotion Recognition

Emotion recognition from speech signals is an important and challenging component of Human-Computer Interaction. In the field of speech emotion recognition (SER), many techniques have been utilized to extract emotions from speech signals, including many well-established speech analysis and classification techniques. This model can be built by using various methods such as RNN, SVM, deep learning, cepstral coefficients, and various other methods, out of which SVM normally gives us the highest accuracy. We propose a model that can identify emotions present in the speech, which can be identified by various parameters such as pitch, speaking rate, speech time, and frequency patterns. Emotion detection in digitized speech contains 3 components: Signal processing, Feature extraction, and Classification. The model first tries to remove the background noises then extract the features present in the speech and classify it into a single emotion. This model is capable of identifying seven different emotions that can be found in human speech. We can use different classifiers like GMM and HMM to classify features such as Spectral Subtraction, Wiener Filtering, Adaptive Filtering, and Deep Learning Techniques. This model can be used in various fields such as healthcare, security, psychology, medicine, education, and entertainment.

Voice and Temporal Auditory Processing in Elderly People: A Correlation Study.

Introduction The voice and hearing can be affected to different degrees by aging, which can cause communication difficulties for elderly people. Vocal production requires effective temporal auditory processing at central levels within the nervous system, which can be compromised by the aging process. Objective To analyze the correlation between voice and temporal auditory processing in older adults. Materials and Methods A total of 40 elderly people aged 60 years or older were subdivided into 2 groups according to the presence or absence of vocal symptoms measured by the Voice Symptom Scale. All of the participants were submitted to auditory temporal tests, vocal self-assessment, and acoustic and perceptual auditory analyses of voice. Results Most of the subjects assessed had decreased voice intensity and normal variability in terms of vocal quality. The performance was normal in the Pitch Pattern Sequence test and altered in the Random Gap Detection test. In the Masking Period Pattern test, the detection thresholds for the target signal were increased in the presence of masking in different temporal target signal positions. Only pitch differed between the two groups. There were differences between the genders regarding frequency, shimmer, the overall severity of the alteration, and roughness. There was a correlation regarding temporal resolution ability and the overall severity of the alteration and roughness of the voice. Conclusion There is a central auditory impairment in temporal resolution which is correlated with vocal alterations in the elderly.

Metaphorical Expression of Emotions in Modern Music Composition Based on Emotional Calculations

Abstract In this paper, audio and lyrics features of music are fused from the audio and lyrics attributes of modern music tracks to obtain a joint representation of music. The features such as tempo, directionality, average value of intensity, beat, and change of rhythm of each musical segment are extracted from the main audio track to form the music feature vector. Subsequently, the music was separated into multiple segments with distinct emotions, and the emotional expression of modern music was evaluated using the Hevner emotion model. Finally, the note extraction performance of this paper’s model is analyzed, and the spectral contrast features of different music pieces and the emotional expression of music pieces under different features are explored. The results show that the pitch patterns of sad emotions are mostly distributed in the range of 0.5-0.55, and the values of the pitch patterns of angry music are basically larger than 0.55. The mean values of the spectral entropy of joy and calmness are mostly distributed in the range of 0.5-0.7, and the sad emotions are mostly in the vicinity of 0.7, while the mean values of the spectral entropy of angry emotions are larger than 0.8.

PORÓWNANIE TESTÓW SEKWENCJI CZĘSTOTLIWOŚCI OPARTYCH NA SAMOGŁOSKACH I TONACH

IntroductionThe pitch pattern test (PPT) is a test of temporal ordering in which a subject needs to correctly repeat a sequence of three auditory stimuli comprised of high and low frequencies. Traditionally, tones are used, but here the aim was to develop a PPT using vowels. The tests were done in young adults and a comparison was made between the two methods.Material and methodsThe study involved 40 young adults with normal hearing sensitivity and was done in two phases. Phase I involved the development of a PPT using the vowel /a/, and in Phase II the test was administered. A comparison was made with the PPT based on tones. In both tests, stimuli were delivered monaurally.ResultsAll participants were able to do the PPT using either vowels or tones. No ear or gender effects were observed. However, the mean scores for PPT using tones were notably better than those obtained using vowels.ConclusionsThe present study has developed a PPT using the vowel /a/ and has compared scores with the tonal pitch pattern test. The absence of a ceiling effect in the PPT using a vowel gives it an advantage in terms of potential clinical utility. The strong positive correlation between the PPT using a vowel and the PPT using a tone shows a significant relationship between the two tests in the specific domain of temporal ordering.

THE SUPRASEGMENTAL CHARACTERISTICS OF PUNCTUATION MARKS IN LINGUISTIC ANALYSIS

There is a consistent and growing interest in the ongoing process of converting spoken language into written form. The use of punctuation marks in written communication during the interactive discourse between the writer and the reader allows for the assessment of the human factor, that is, the communicative impact between the writer and the reader. The scientific integration of spoken and written language is currently gaining significant importance.The prosodic aspects of punctuation marks play a main role in linguistics, influencing the rhythm, intonation, and overall expression of spoken language. Prosody refers to the patterns of stress, pitch, and rhythm in speech, and punctuation marks contribute to these prosodic elements in various ways.

The auditory phenotype of behavioural variant frontotemporal dementia

AbstractBackgroundPatients with behavioural variant frontotemporal dementia commonly exhibit abnormal hedonic and other behavioural responses to sounds, however hearing dysfunction in this disorder is poorly characterised. Here we addressed this issue using the Queen Square Tests of Auditory Cognition (QSTAC) – a neuropsychological battery for the systematic assessment of central auditory functions (including pitch pattern perception, environmental sound recognition, sound localisation and emotion processing) in cognitively impaired people.MethodThe QSTAC was administered to 15 patients with behavioural variant frontotempoal dementia, 24 patients with comparator dementia syndromes (primary progressive aphasia and typical Alzheimer’s disease) and 15 healthy age‐matched individuals. Participants also had pure tone audiometry to assess peripheral hearing function and a comprehensive general neuropsychological assessment.ResultAfter accounting for nonverbal executive and peripheral hearing performance, patients with behavioural variant frontotemporal dementia showed deficits of environmental sound and auditory emotion recognition and sound localisation, compared both with healthy controls and patients with Alzheimer’s disease. Patients with behavioural variant frontotemporal dementia were further differentiated from primary progressive aphasia patients on their performance across QSTAC (sound recognition and localisation) subtests.ConclusionBehavioural variant frontotemporal dementia has a distinct phenotype of auditory cognitive dysfunction which likely contributes to the hearing alterations that many patients with this diagnosis experience in daily life. Our findings call attention to an under‐recognised issue in frontotemporal dementia that warrants further clinical interpretation and the development of management strategies tailored to real‐world acoustic environments.

The roles of prosody in Chinese-English reading comprehension

BackgroundDespite being an essential component of children's oral reading fluency, prosodic reading, which involves expressive changes in pitch patterns and pause durations, has not been explored in Cantonese-English bilingual children, whose first language (L1) is tonal, non-alphabetic, and whose second language (L2) is non-tonal, alphabetic. AimsThis study examined the development of prosodic reading and its within- and cross-language associations with reading comprehension among Cantonese-English bilingual children from second to third grade. SampleOne hundred and twenty-one 7-to 8-year-old Cantonese-English bilingual children completed initial testing in grade 2, with 52 tested in grade 3. MethodsProsodic reading was assessed using one Chinese and one English passage, each comprising six types of syntactic structures: declaratives, clause-final commas, yes-no questions, wh-questions, complex adjectival phrases, and quotatives. Word-reading efficiency, oral passage-reading fluency, and reading comprehension in Chinese and English were also measured. ResultsSpectrographic analyses revealed that these children were aware of language-independent functions and language-specific manifestations of pitch and pause cues within and across their L1 Chinese and L2 English. Wh question pitch contours emerged as the most robust link to reading comprehension across both languages, while a crossover effect occurred from Cantonese pitch to English reading comprehension. Shorter pauses for English declarative quotative sentences and phrase-final commas were concurrently associated with greater English reading comprehension. ConclusionsThese findings are interpreted within a new reading framework, the Prosodic Catalysing Hypothesis (PCH), which proposes that pitch and pause production can bridge prosody and syntax to facilitate reading comprehension.

The effect of musical training and language background on vocal imitation of pitch in speech and song.

Vocal imitation plays a critical function in the development and use of both language and music. Previous studies have reported more accurate imitation for sung pitch than spoken pitch, which might be attributed to the structural differences in acoustic signals and/or the distinct mental representations of pitch patterns across speech and music. The current study investigates the interaction between bottom-up (i.e., acoustic structure) and top-down (i.e., participants' language and musical background) factors on pitch imitation by comparing speech and song imitation accuracy across four groups: English and Mandarin speakers with or without musical training. Participants imitated pitch sequences that were characteristic of either song or speech, derived from pitch patterns in English and Mandarin spoken sentences. Overall, song imitation was more accurate than speech imitation, and this advantage was larger for English than Mandarin pitch sequences, regardless of participants' musical and language experiences. This effect likely reflects the perceptual salience of linguistic tones in Mandarin relative to English speech. Music and language knowledge were associated with optimal imitation of different acoustic features. Musicians were more accurate in matching absolute pitch across syllables and musical notes compared to nonmusicians. By contrast, Mandarin speakers were more accurate at imitating fine-grained changes within and across pitch events compared to English speakers. These results suggest that different top-down factors (i.e., language and musical background) influence pitch imitation ability for different dimensions of bottom-up features (i.e., absolute pitch and relative pitch patterns). (PsycInfo Database Record (c) 2023 APA, all rights reserved).

An investigation of ongoing tonal changes in Punjabi

Punjabi developed the falling and the rising tones due to the loss of the voiced aspirated consonants and the word-final glottal fricative /ɦ/ and the default tone appeared elsewhere. A falling pitch has recently been observed to develop in words with an initial glottal fricative and both falling and rising pitch patterns have been observed in words without any apparent conditioning factor (‘novel’ words). These developments within some traditionally default tone words have been only briefly mentioned in the literature (Shackle, 2003; Schniske, 2015; and Bhardwaj, 2016) and have not been investigated experimentally. The aim of this study was to document the f0 patterns observed in the glottal-initial and novel words and determine how they compare to the f0 of the traditionally default tone words. 24 native speakers of Indian Punjabi (9F, 15M) were recorded online. The stimuli contained 35 monosyllabic words (glottal-initial = 7, novel falling = 4, novel rising = 6, default = 18) with either /a/ or /ə/. The acoustic analysis showed falling f0 for the glottal-initial and novel falling words and a rising f0 for the novel rising words. Welch’s t-tests were conducted to compare the default tone to each of the three newer pitch patterns. The results showed that nearly half of the speakers produced a non-default pitch on the glottal-initial and novel words. Considerable within- and between-speaker variation was observed.

A Survey on Screening and Diagnostic Criteria of Auditory Processing Disorders in India.

The current study aimed to determine the criteria used for screening and diagnosing cases with central auditory processing disorders (CAPD) in India. A cross-sectional questionnaire-based survey design was used in the present study. A questionnaire was developed to determine the criteria used for screening and diagnosing CAPD across clinics in India. Responses were obtained from 83 participants from all over India. Results indicated that 78% of respondents were currently doing CAPD evaluation. In that, the majority of respondents (63%) had a predetermined minimum battery that was relatively adaptable depending on the case history and age of the patient. In screening, most respondents used a screening questionnaire (SCAP, 75%) and a screening test (STAP, 60%). In the diagnostic protocol, the most used tests by the respondents were masking level difference (MLD), repetition of words (RW), gap detection test (GDT), pitch pattern test (PPT), speech perception in noise (SPIN), digit span test (DST), dichotic digit test (DDT), binaural fusion test (BFT), auditory brainstem response (ABR), dichotic CV test (DCVT), and duration pattern test (DPT). The current study's result will help professionals choose the minimum test battery for diagnosing CAPD.

Legislators' Emotional Engagement with Women's Issues: Gendered Patterns of Vocal Pitch in the German Bundestag

Abstract Through an innovative analysis of audio recordings of plenary speeches, Dietrich, Hayes, and O'Brien (2019) find that women in the U.S. House of Representatives speak with greater emotional intensity about women than other issues. With vocal pitch as a new measure of personal issue commitment, the finding suggests that women legislators' efforts to work on behalf of women result from intrinsic motivation. I ask whether the same is true in an alternative parliamentary setting, the German Bundestag, where personal preferences play a very different role due to strict party discipline. The answer is yes. Analyzing audio and text data from more than 30,000 speeches in the Bundestag between 2011 and 2020, I find that women in the Bundestag address women more frequently and with greater emotional intensity than men. The results suggest that women legislators are more emotionally invested in women-related issues than men.

Prosodic linking in apology sequences in Finnish elementary school mediations

In this article, we analyze pupils' prosodic realizations of the apology sequence anteeksi–saat anteeksi “I apologize–apology accepted”, focusing on pitch, rhythm, and loudness. Drawing on teacher-led mediations in which teachers settle pupils’ peer conflicts in Finnish elementary schools as data, and conversation analysis as a method, we examine how the participants' orientation to the interaction of the mediation episode as unproblematic (i.e. contributing to the smooth progress of interaction) vs. problematic (i.e. long delays in turn-taking and explicit disagreements) is prosodically reflected in the apology sequence. We show that prosodic linking of pitch, rhythm, and loudness play a key role in this regard. The apology sequence was produced using complementary ascending–descending pitch patterns in the unproblematic cases, but the pupils deviated from this pattern in the problematic cases. The pupils also aligned their turns to co-construct an isochronous rhythmic pattern throughout the unproblematic apology sequences, while this did not happen in the problematic sequences. Moreover, the loudness levels were matched in the unproblematic cases but not in the problematic ones. We argue that a mutual apology sequence is highly conventionalized, and that prosody plays a major role in how it is received and treated as legitimate, sincere, and ritually correct.

Quantified Uniformity and Selectivity of TiO2 Films in 45‐nm Half Pitch Patterns Using Area‐Selective Deposition Supercycles

AbstractArea‐selective deposition (ASD) shows great promise for sub‐10 nm manufacturing in nanoelectronics, but significant challenges remain in scaling to ultrasmall dimensions and understanding feature‐dependent nonuniformity and selectivity loss. This work addresses these problems by simultaneously quantifying uniformity and selectivity for passivation/deposition/etch supercycles in 45 nm half‐pitch TiN/SiO2 line/space patterns. This work employs three selective processes that are uniquely suited for supercycle processing: dimethylamino‐trimethylsilane (DMA‐TMS) inhibition, TiO2 atomic layer deposition (ALD), and HBr/BCl3 plasma etch. The DMA‐TMS inhibition selectively passivates the SiO2 nongrowth surface without affecting deposition on the TiN and TiO2 growth surfaces. The plasma etch removes TiO2 defect particles at a faster rate than the conformal TiO2 film or SiO2 lines. Using three supercycles of this process, this work demonstrates 8 nm of TiO2 with 88% uniformity and ≈100% selectivity according to transmission electron microscopy (TEM), a 2×improvement in film thickness from previous reports in similar nanoscale patterns. Integrated consideration of uniformity and selectivity at specific feature scales will facilitate the effective design of selective deposition processes for nanoscale electronic devices.

A Phonological Study of Rongpa Choyul

This paper presents a detailed description of the phonology of the Rongpa variety of Choyul, an understudied Tibeto-Burman language spoken in Lithang (理塘) County, Dkarmdzes (甘孜) Tibetan Autonomous Prefecture of Sichuan Province, China. Based on firsthand fieldwork data, this paper lays out Rongpa phonology with details, examining its syllable canon, initial and rhyme systems, and word prosody. Peculiar characteristics of this phonological system are as follows: First, Rongpa has a substantial phonemic inventory, which comprises 43 consonants, 13 vowels, and 2 tones. 84 consonant clusters are observed to serve as the initial of a syllable. Secondly, the phonemic contrast between plain and uvularized vowels is attested. In addition, regressive vowel harmony on uvularization, height, and lip-roundedness can be clearly observed in various constructions including prefixed verb stems. Finally, regarding word prosody, two tones in monosyllabic words, /H/ and /L/, are observed to distinguish lexical meanings, and disyllabic words exhibit four surface pitch patterns. Pitch patterns in verb morphology are also examined. The findings and analyses as presented in this paper could form a foundation for future research on Rongpa Choyul.

Projection, Call‐Response and the Improvisational Moment

Projection, Call‐Response and the Improvisational Moment

A dynamic neural field model of leaky prosody: proof of concept

Recent work has shown that lexical items come to take on the phonetic characteristics of the prosodic environments in which they are typically produced, a phenomenon referred to as "leaky prosody". Focusing on pitch patterns in Mandarin, we show that leaky prosody can be derived from a flat (i.e., non-transformational, non-optimizing) model of speech production. Formalized using Dynamic Field Theory, in our model, lexical, phonological, and prosodic inputs each exert forces on a Dynamic Neural Field representing pitch. Notably, the forces exerted by these inputs reflect surface distributions in a large corpus of spontaneous speech. Our simulations showed that the flat model derives the short timescale effect of prosodic prominence on pitch production as well as the longer timescale effect of leaky prosody. By updating lexical items based on surface phonetic form, words that are consistently produced in high/low prosodic prominence positions take on the phonetic characteristics of those environments.

An Acoustic Analysis and Comparison between Remotely Collected and In-Person Laboratory Collected Data in Vocal Imitation of Pitch

ABSTRACT Restrictions on face-to-face interactions due to the outbreak of the coronavirus pandemic (COVID-19) in early 2020 have impacted experimental behavioral research. The rapid change from in-person to online data collections has been challenging in many behavioral studies, especially those that require vocal production, and the quality of the remotely collected data needs to be investigated. The current study examines the recording quality and corresponding measures of vocal production accuracy in online and in-person settings using two measurements: harmonic-to-noise ratio (HNR) and fundamental frequency, f 0. Participants imitated pitch patterns extracted from recordings of song or speech, either in a laboratory or via an online platform. The results showed that the recordings from the online setting had higher HNR than those from the in-person setting, whereas the pitch imitation accuracy in both settings did not differ. We also report an experiment that simulated differences between the online and in-person settings within participants, focusing on software used, type of microphone, and presence of ambient noise. Pitch accuracy did not differ according to these variables, except ambient noise, whereas HNR again varied across conditions. Based on these results we conclude that measures of pitch accuracy are reliable across these different types of data collection, whereas finer-grained spectral measures like HNR might be affected by various factors.

Successful lexical tone production of Mandarin Chinese autistic children with intellectual impairment.

Atypical speech prosody has been commonly found among autistic children. Yet it remains unknown whether prosody impairment originates from poor pitch ability in general or whether it is the result of the difficulty in understanding and using prosody for communicative purposes. To investigate whether native Mandarin Chinese-speaking autistic children with intellectual impairment were able to accurately produce native lexical tones, which are pitch patterns that distinguish word meaning lexically and serve little social purpose. Using a picture-naming task, thirteen 8-13-year-old Mandarin Chinese-speaking autistic children with intellectual impairment were tested on their production of Chinese lexical tones. Chronical age-matched typically developing (TD) children were included as the control group. Perceptual assessment and phonetic analyses were conducted with the produced lexical tones. The majority of the lexical tones produced by the autistic children were perceived as accurate by adult judges. Phonetic analysis of the pitch contours found no significant difference between the two groups, and the autistic children and TD children used the phonetic features in comparable ways when differentiating the lexical tones. However, the lexical tone accuracy rate was lower among the autistic children than among the TDs, and the larger individual difference was observed among the autistic children than the TD children. These results indicate that autistic children are able to produce the global contours of the lexical tones, and pitch deficits do not seem to qualify as a core feature of autism. What is already known on the subject Atypical prosody has been considered a maker of the speech of autistic children, and meta-analysis found a significant difference in mean pitch and pitch range between TD children and autistic children. Yet it remains unknown whether the pitch deficits are the result of impaired perceptual-motoric ability or if they reflect failure in learning sentential prosody, which requires an understanding of the interlocutors' mind. In addition, research on pitch ability of autistic children with intellectual disabilities has been scarce, and whether these children are able to produce pitch variation is largely unknown. What this paper adds to existing knowledge We tested native Mandarin Chinese autistic children with intellectual impairment on their production of native lexical tones. The lexical tones in Chinese are pitch variations realized on individual syllables that distinguish lexical meaning, but they do not serve social pragmatic purposes. We found that although these autistic children had only developed limited spoken language, the majority of their lexical tones were perceived as accurate. They were able to use the phonetic features in comparable ways with the TD children when distinguishing the lexical tones. What are the potential or actual clinical implications of this work? It seems unlikely that pitch processing at the lexical level is fundamentally impaired in autistic children, and pitch deficits do not seem to qualify for a core feature of their speech. Practitioners should be cautious when using pitch production as a clinical marker for autistic children.

Musicality in Infancy

The present review summarizes the available evidence on musicality, or music-related abilities, in infants (birth to 3 years). In the early months of life, infants are responsive to the pitch and temporal patterns of music. Their perceptual skills are similar, in many respects, to those of adults, presumably because of the nature of the human auditory system. Adult-infant differences, where evident, are attributable to infants’ unfamiliarity with the musical conventions of their culture. Musical enculturation proceeds more rapidly for temporal than for pitch processing. Musical exposure, especially the singing of caregivers, is prevalent in infancy. Caregivers’ music-making for infants has consequences for their emotional and social regulation and for their subsequent self-regulation abilities. By the end of their first year, most infants have become music-makers as well as music listeners. They move spontaneously to music, and their patterns of “dancing” undergo considerable change in the subsequent months and years. Early dancing is influenced by the familiarity of the music, and later dancing may include aspects of caregivers’ dance movements. Although the onset of singing occurs considerably later than the onset of dance, early singing is remarkably accurate in terms of its pitch range, pitch contours, rhythmic patterning, and fluency, especially when infants sing in their familiar home environment.

High-performance glass filters for capturing and culturing circulating tumor cells and cancer-associated fibroblasts

Various liquid biopsy methods have been developed for the non-invasive and early detection of diseases. In particular, the detection of circulating tumor cells (CTCs) and cancer-associated fibroblasts (CAFs) in blood has been receiving a great deal of attention. We have been developing systems and materials to facilitate such liquid biopsies. In this study, we further developed glass filters (with various patterns of holes, pitches, and non-adhesive coating) that can capture CTCs, but not white blood cells. We optimized the glass filters to capture CTCs, and demonstrated that they could be used to detect CTCs from lung cancer patients. We also used the optimized glass filters for detecting CAFs. Additionally, we further developed a system for visualizing the captured cells on the glass filters. Finally, we demonstrated that we could directly culture the captured cells on the glass filters. Based on these results, our high-performance glass filters appear to be useful for capturing and culturing CTCs and CAFs for further examinations.