Conversational Speech Research Articles

Text Summarization and Conversion of Speech to Text

Abstract: This article describes the fusion of recurrent neural networks and deep learning algorithms for text summarization systems and analysis of the text learning process. Next, the text analytics learning model is summarized. In addition, applications of deep learning-based text analysis are also introduced. Language is the most important part of communication between people. Although there are many ways to express our thoughts and feelings, language is considered the most important medium of communication. Speech recognition is the process by which machines recognize different people's voices based on specific words and phrases. End-to-end deep learning techniques can be used to identify and simplify spatial representations of text data and semantic information. This study considers text analysis based on deep learning

Interpretation of Bahasa Isyarat Malaysia (BIM) Using SSD-MobileNet-V2 FPNLite and COCO mAP

This research proposes a study on two-way communication between deaf/mute and normal people using an Android application. Despite advancements in technology, there is still a lack of mobile applications that facilitate two-way communication between deaf/mute and normal people, especially by using Bahasa Isyarat Malaysia (BIM). This project consists of three parts: First, we use BIM letters, which enables the recognition of BIM letters and BIM combined letters to form a word. In this part, a MobileNet pre-trained model is implemented to train the model with a total of 87,000 images for 29 classes, with a 10% test size and a 90% training size. The second part is BIM word hand gestures, which consists of five classes that are trained with the SSD-MobileNet-V2 FPNLite 320 × 320 pre-trained model with a speed of 22 s/frame rate and COCO mAP of 22.2, with a total of 500 images for all five classes and first-time training set to 2000 steps, while the second- and third-time training are set to 2500 steps. The third part is Android application development using Android Studio, which contains the features of the BIM letters and BIM word hand gestures, with the trained models converted into TensorFlow Lite. This feature also includes the conversion of speech to text, whereby this feature allows converting speech to text through the Android application. Thus, BIM letters obtain 99.75% accuracy after training the models, while BIM word hand gestures obtain 61.60% accuracy. The suggested system is validated as a result of these simulations and tests.

English

Wireless communication has become increasingly popular in recent years due to its mobility, portability and efficient service. Quality Voice Communication is the basic need of all such devices. Due to the significant spectrum demand, the bandwidth allocated to Professional Radio devices is decreasing. This results in lowering voice/ speech quality. Due to limited bandwidth, the design of Radio is changing. The main change is shifting from Analogue signal processing to Digital signal processing of speech and radio signals. The conversion of analogue-natured speech to digital form itself creates loss and faithfulness. Such Radio includes a voice coder for analogue to digital conversion of speech. Human spoken language, speaker (male or female), and pronunciation play an essential role in direct voice and Radio communication. It is the primary object which decides the design of low bandwidth coder. This paper deals with Marathi language characteristics and the interdependence of language constructs that affects voice/speech quality. In this paper, we extracted features of the Marathi language using PRAAT software and analyzed its constructs using statistical tools. The actual testing on Radio is carried out. The speech quality observed during such testing is correlated with statistical results. It is shown that the standard deviation in pitch and formants in male speech is less (average 35%) compared to the standard deviation in pitch and formants in female speech. This results in lowering the quality of male speech compared to female speech. It is also shown that some vowels and combinations of certain vowels and consonants produce poor speech quality compared to others.

Cochlear synaptopathy impairs suprathreshold tone-in-noise coding in the cochlear nucleus.

Hearing impairment without threshold elevations can occur when there is damage to high-threshold auditory nerve fibre synapses with cochlear inner hair cells. Instead, cochlear synaptopathy produces suprathreshold deficits, especially in older patients, which affect conversational speech. Given that listening in noise at suprathreshold levels presents significant challenges to the ageing population, we examined the effects of synaptopathy on tone-in-noise coding on the central recipients of auditory nerve fibres, i.e. the cochlear nucleus neurons. To induce synaptopathy, guinea pigs received a unilateral sound overexposure to the left ears. A separate group received sham exposures. At 4weeks post-exposure, thresholds had recovered but reduced auditory brainstem response wave1 amplitudes and auditory nerve synapse loss remained on the left side. Single-unit responses were recorded from several cell types in the ventral cochlear nucleus to pure-tone and noise stimuli. Receptive fields and rate-level functions in the presence of continuous broadband noise were examined. The synaptopathy-inducing noise exposure did not affect mean unit tone-in-noise thresholds, nor the tone-in-noise thresholds in each animal, demonstrating equivalent tone-in-noise detection thresholds to sham animals. However, synaptopathy reduced single-unit responses to suprathreshold tones in the presence of background noise, particularly in the cochlear nucleus small cells. These data demonstrate that suprathreshold tone-in-noise deficits following cochlear synaptopathy are evident in the first neural station of the auditory brain, the cochlear nucleus neurons, and provide a potential target for assessment and treatment of listening-in-noise deficits in humans. KEY POINTS: Recording from multiple central auditory neurons can determine tone-in-noise deficits in animals with quantified cochlear synapse damage. Using this technique, we found that tone-in-noise thresholds are not altered by cochlear synaptopathy, whereas coding of suprathreshold tones-in-noise is disrupted. Suprathreshold deficits occur in small cells and primary-like neurons of the cochlear nucleus. These data provide important insights into the mechanisms underlying difficulties associated with hearing in noisy environments.

Contrastive Focus Reduplication in Kuwaiti Arabic

Abstract This study examines a case of total reduplication in Kuwaiti Arabic known as contrastive focus reduplication (i.e. complete copying of words or phrases). For example, tišrab čāy ḥalīb willa ČĀY–čāy? ‘Would you like to drink tea with milk, or TEA–tea’ [denoting black tea as opposed to karak chai]. The study explores the morpho-semantic properties of this construction in the dialect, shows the different meanings it allows and how it elucidates the permissible lexical units that can be reduplicated. The reduplication of a lexical item can be applied to a range of grammatical and lexical categories. Over 150 samples were collected by observing contrastive focus reduplication from participants’ (male and female native Kuwaiti speakers in their twenties and seventies) natural speech in everyday conversations. This study contributes to research on the prosody and grammar of the dialect and the theory of semantics, thereby enhancing understanding of reduplication and repetition in Semitic dialectology.1

Landmark-based analysis of speech differentiates conversational from clear speech in speakers with muscle tension dysphonia

This study evaluated the feasibility of differentiating conversational and clear speech produced by individuals with muscle tension dysphonia (MTD) using landmark-based analysis of speech (LMBAS). Thirty-four adult speakers with MTD recorded conversational and clear speech, with 27 of them able to produce clear speech. The recordings of these individuals were analyzed with the open-source LMBAS program, SpeechMark®, matlab Toolbox version 1.1.2. The results indicated that glottal landmarks, burst onset landmarks, and the duration between glottal landmarks differentiated conversational speech from clear speech. LMBAS shows potential as an approach for detecting the difference between conversational and clear speech in dysphonic individuals.

Mitigation of Respirable Aerosol Particles from Speech and Language Therapy Exercises

Phonation and speech are known sources of respirable aerosol in humans. Voice assessment and treatment manipulate all the subsystems of voice production, and previous work (Saccente-Kennedy et al., 2022) has demonstrated such activities can generate >10 times more aerosol than conversational speech and 30 times more aerosol than breathing. Aspects of voice therapy may therefore be considered aerosol generating procedures and pose a greater risk of potential airborne pathogen (eg, SARS-CoV-2) transmission than typical speech. Effective mitigation measures may be required to ensure safe service delivery for therapist and patient. To assess the effectiveness of mitigation measures in reducing detectable respirable aerosol produced by voice assessment/therapy. We recruited 15 healthy participants (8 cis-males, 7 cis-females), 9 of whom were voice-specialist speech-language pathologists. Optical Particle Sizers (OPS) (Model 3330, TSI) were used to measure the number concentration of respirable aerosol particles (0.3 µm-10 µm) generated during a selection of voice assessment/therapy tasks, both with and without mitigation measures in place. Measurements were performed in a laminar flow operating theatre, with near-zero background aerosol concentration, allowing us to quantify the number concentration of respiratory aerosol particles produced. Mitigation measures included the wearing of Type IIR fluid resistant surgical masks, wrapping the same masks around the end of straws, and the use of heat and moisture exchange microbiological filters (HMEFs) for a water resistance therapy (WRT) task. All unmitigated therapy tasks produced more aerosol than unmasked breathing or speaking. Mitigation strategies reduced detectable aerosol from all tasks to a level significantly below, or no different to, that of unmasked breathing. Pooled filtration efficiencies determined that Type IIR surgical masks reduced detectable aerosol by 90%. Surgical masks wrapped around straws reduced detectable aerosol by 96%. HMEF filters were 100% effective in mitigating the aerosol from WRT, the exercise that generated more aerosol than any other task in the unmitigated condition. Voice therapy and assessment causes the release of significant quantities of respirable aerosol. However, simple mitigation strategies can reduce emitted aerosol concentrations to levels comparable to unmasked breathing.

Clear speech promotes speaking rate normalization.

When speaking in noisy conditions or to a hearing-impaired listener, talkers often use clear speech, which is typically slower than conversational speech. In other research, changes in speaking rate affect speech perception through speaking rate normalization: Slower context sounds encourage perception of subsequent sounds as faster, and vice versa. Here, on each trial, listeners heard a context sentence before the target word (which varied from "deer" to "tier"). Clear and slowed conversational context sentences elicited more "deer" responses than conversational sentences, consistent with rate normalization. Changing speaking styles aids speech intelligibility but might also produce other outcomes that alter sound/word recognition.

E-DGAN: An Encoder-Decoder Generative Adversarial Network Based Method for Pathological to Normal Voice Conversion.

In recent years, more and more people suffer from voice-related diseases. Given the limitations of current pathological speech conversion methods, that is, a method can only convert a single kind of pathological voice. In this study, we propose a novel Encoder-Decoder Generative Adversarial Network (E-DGAN) to generate personalized speech for pathological to normal voice conversion, which is suitable for multiple kinds of pathological voices. Our proposed method can also solve the problem of improving the intelligibility and personalizing custom speech of pathological voices. Feature extraction is performed using a mel filter bank. The conversion network is an encoder-decoder structure, which is used to convert the mel spectrogram of pathological voices to the mel spectrogram of normal voices. After being converted by the residual conversion network, the personalized normal speech is synthesized by the neural vocoder. In addition, we propose a subjective evaluation metric named "content similarity" to evaluate the consistency between the converted pathological voice content and the reference content. The Saarbrücken Voice Database (SVD) is used to verify the proposed method. The intelligibility and content similarity of pathological voices are increased by 18.67% and 2.60%, respectively. Besides, an intuitive analysis based on a spectrogram was done and a significant improvement was achieved. The results show that our proposed method can improve the intelligibility of pathological voices and personalize the conversion of pathological voices into the normal voices of 20 different speakers. Our proposed method is compared with five other pathological voice conversion methods, and our proposed method has the best evaluation results.

Poeter versus skådespelare i Dagens dikt 2020–2021

“Poets versus Actors Reading Poetry on Swedish Radio 2020–2021” Following the rise of the poet as the principal performer of his or her own work, in the second half of the 20th century, actors have been criticized for performing poetry as if acting on stage, or for focusing too much on conveying the lyric I. This primarily quantitative study investigates what prosodic differences there are between poets and actors reading poetry. The material consists of 276 readings from the Swedish public radio program Dagens dikt (The Daily Poem), from 2020 and 2021. A set of prosodic measures are produced for each recording with computational aid, and the groups are compared by using descriptive statistics. The differences found in terms of tempo and pausal patterns, though small, suggest that the actors read in a conventionally poetic, expressive style, while the poets are closer to a style typical of prose reading and conversational speech; this might be connected to differences between their respective repertoire – the poets reading newer, longer, and possibly more narrative texts. When it comes to pitch, there are more substantial differences: the poets are more monotonous and their tonal patterns more predictable, indicating a formal, less expressive style. These traits are associated with the highly stylized vocal cliché Poet Voice, which in turn is connected to poetological shifts in the last few decades. In both groups, however, younger individuals (born 1970 or later) and women, respectively, are the more monotonous and predictable; this might suggest a sociolinguistic trend not limited to poetry. Qualitative, close listening to a selection of readings largely confirms the findings of the quantitative investigation. While the frequently idiosyncratic poets tend to highlight form, such as rhythm and tonal repetitions, the actors rather try to convey the content of the poem, in accordance with prosodical and performative conventions.

About the Poetics of Sergey Rafalsky

In the article the author proves that the habit of researching the poetry by Sergey Rafalsky in the traditions of the Prague group «Skete of Poets» needs revising, because the poet emphasized his disagreements with the creative practice of that group and its leader A. Bem repeatedly. The author identifies some features, that indicate how the artistic credo of the «Paris note» influenced the poetics by Rafalsky, and notices the specificity, that allows us to speak about Rafalsky’s succession to G. Ivanov and G. Adamovich. The author analyses the poems, in which Rafalsky demonstrates implicitly his solidarity with the criteria
 of the genuine poetry in G. Adamovich’s comprehension and attempts to reconstruct intonations of conversational natural speech as adequately as possible, with its fragmentariness, understatement and incomplete information. All these
 features were intrinsic to the poets of the «Paris note». The author proves that the poems of Rafalsky’s late period ought to be analysed in the context of the creative quests of the poets of Russian Montparnasse, although the influence of the avantgarde poetics is still strong in them. One of the reasons of such esemblance is an intonation dominant of the poetry of Rafalsky, who makes the most of Lermontov's and Blok's legacy in this aspect. Both of these poets were the gold standard for
 the poets of the «Paris note», as well as G. Ivanov

“Mudra” (Text to Speech Conversion with Hand Gestures)

One of the non-verbal communication techniques utilized in sign language is the hand gesture. Deaf persons who struggle with speech or hearing most frequently use it to converse with each other and hearing people. Several makers around the world have created various sign language systems, but they are neither adaptable nor cost-effective for end users. The majority of programs only provide text- based results. So, we are going to offer a system prototype in an application that solves the aforementioned issue. Hand gestures can be translated into text and then spoken by the system. Several algorithms will be used by the system, including CNN for classifying hand movements and SVM for detection of hand movement & TTS for conversion of Text To Speech.

Timing in Conversation.

Turn-taking in everyday conversation is fast, with median latencies in corpora of conversational speech often reported to be under 300 ms. This seems like magic, given that experimental research on speech planning has shown that speakers need much more time to plan and produce even the shortest of utterances. This paper reviews how language scientists have combined linguistic analyses of conversations and experimental work to understand the skill of swift turn-taking and proposes a tentative solution to the riddle of fast turn-taking.

Large-scale patterns of number use in spoken and written English.

This paper describes patterns of number use in spoken and written English and the main factors that contribute to these patterns. We analysed more than 1.7 million occurrences of numbers between 0 and a billion in the British National Corpus, including conversational speech, presentational speech (e.g., lectures, interviews), imaginative writing (e.g., fiction), and informative writing (e.g., academic books). We find that four main factors affect number frequency: (1)Magnitude- smaller numbers are more frequent than larger numbers; (2)Roundness- round numbers are more frequent than unround numbers of a comparable magnitude, and some round numbers are more frequent than others; (3) Cultural salience- culturally salient numbers (e.g., recent years) are more frequent than non-salient numbers; and (4) Register- more informational texts contain more numbers (in writing), types of numbers, decimals, and larger numbers than less informational texts. In writing, we find that the numbers 1-9 aremostly represented by number words (e.g., 'three'), 10-999,999 are mostly represented by numerals (e.g., '14'), and 1 million-1 billion are mostly represented by a mix of numerals and number words (e.g., '8 million'). Altogether, this study builds a detailed profile of number use in spoken and written English.

Cochlear Implantation and Assessment of Speech in Children

Cochlea is a hollow, spiral shaped bone in the inner ear that has sense of hearing and to produce sound. Cochlear problems, or its damage can result in loss of hearing. This study was designed to access intelligibility of speech in children with cochlear implants. A total of 30 subjects (7 to 42 months) from Fatima Memorial College of Medicine and Dentistry were considered which were implanted with cochlea from October 2012 to December 2012. A pre-designed questionnaire was used for the data collection in order to collect the views from the parents of children with cochlear implants regarding the intelligibility of conversational speech produced by their children. It was noticed a 26 (86.7%) of the 30 youngsters who had cochlear implants were able to understand what was being said by their speakers. Moreover, parents and other listeners try to understand the conversation that cochlear implant’s recipients make. It was concluded cochlear implants found useful and recommended for improving hearing of impaired individuals.

Extended high frequencies for fricative classification in conversational speech

The current study examines whether the information contained in Extended High Frequencies (EHFs) can improve random forest classification accuracy for fricatives in conversational speech. Prior phonetic research has investigated fricative categorization based on their acoustic characteristics, including spectral, temporal and amplitudinal measures. The spectral measures in these studies have largely been limited to frequency information below 8 kHZ. Only a few studies have examined the contribution of EHFs, energy exceeding 8 kHz. These studies have primarily focused on laboratory speech, so little is currently known about the role of EHFs in more spontaneous, conversational speech styles. Using a corpus of sociolinguistic interview speech from Western Canadian English sampled at 44.1kHz, we compare classification models with and without frequencies above 8 kHz. We discuss the influence of EHFs on the categorization of fricative identity, and share a cost-benefit analysis of sampling at higher frequencies for speech research corpora.

Adaptative and non-adaptive phonetic variation: Clear speech and second-language speech

Two independent lines of research have revealed a somewhat surprising convergence between clear speech and second-language (L2) speech. Specifically, relative to conversational L1 speech, both clear speech and L2 speech typically exhibit slower speaking rates (fewer syllables per second), longer and more frequent inter-word pauses, and less segment- and syllable-level reduction. These phonetic features result in a general pattern of lower information density (more phonetic material produced for a given linguistic message or text) compounded by longer utterance durations to yield substantially lower information transmission rates for both clear speech and L2 speech relative to conversational L1 speech. However, clear speech and L2 speech diverge in terms of intelligibility. While clear speech is a highly effective strategy for enhancing overall intelligibility, L2 speech is typically characterized by greater likelihood of listener errors in word recognition accuracy. This pattern of convergent phonetics and divergent intelligibility highlights the complicated relationship between phonetic form and communicative efficiency. While clear speech is a listener-oriented talker adaptation to listener limitations, L2 speech reflects talker-oriented talker-listener misalignment. The phonetic similarities between these adaptative (clear speech) and non-adaptive (L2 speech) sources of variation underscore the dynamics of human speech communication with the talker-listener relation as the fulcrum.

Acoustic correlates of perceived anger in the Ferguson clear speech database

Previous work has shown that clear speech is perceived as sounding angry significantly more often than conversational speech (Morgan and Ferguson, 2017; Young, 2021). The current study examined possible acoustic correlates of perceived anger in clear speech. A principal component analysis was completed to guide selection of acoustic variables for analysis. The remaining six acoustic measures, fundamental frequency median, fundamental frequency variability, relative mid-frequency energy from 1000 to 3150 Hz, speaking rate, and degree of amplitude modulation at 2 Hz and 16Hz envelopes, were analyzed using mixed effects regression. Results indicated that after controlling for speaking style, fundamental frequency approached significance but did not significantly predict perceived anger. However, speaking rate significantly predicted perceived anger after controlling for speaking style, with slower rates leading to an increase in perceived anger. Further, there was a significant interaction between speaking rate and speaking style, where slower speech was only perceived as angry in the clear speech style. Finally, a decrease in the degree of low-frequency amplitude modulation (i.e., equal emphasis on both content and function words) led to an increase in perceived anger after controlling for speaking style. The results have implications for counseling individuals in the use of clear speech.

Dual-task costs of clear and loud speech: Implications for talkers with Parkinson disease

Decades of research suggest that the speech production adjustments associated with adopting a clearer or louder speaking style enhance speech intelligibility, making these speech styles a common talker-oriented strategy in speech production interventions for talkers with dysarthria secondary to Parkinson disease (PD). Recent data suggest that explicit instructions to speak louder or clearer than usual may increase the processing load and effort associated with speech production. Empirical data will be presented that demonstrate these higher-effort speech styles may require greater attentional resources than habitual or conversational speech. A group of neurotypical talkers performed a sentence repetition task using Habitual, Loud, and Clear speech styles in isolation and while performing a concurrent visuomotor tracking task. Lip and jaw kinematics, speech intensity, and accuracy on the visuomotor task were collected to quantify performance on each task. Group-level data suggest that when faced with the competing task demands of the visuomotor task, neurotypical talkers prioritized the articulatory and phonatory adjustments associated with Clear and Loud speech at the cost of secondary task performance. Case examples from talkers with hypokinetic dysarthria resulting from PD will also be examined to frame and discuss clinical implications and considerations.

Breathing patterns of conversation partners

Previous research has shown that breathing patterns of conversation partners can become synchronized. For example, when one conversation partner is speaking, the listening partner's breathing patterns more closely resemble speech breathing than tidal breathing (McFarland, 2001). The present study aims to replicate and extend these findings using an unconventional instrumental setup consisting of a single inductive plethysmograph machine shared by both conversation partners. In addition to scripted and unscripted conversational speech, single-subject quiet breathing and speech breathing tasks of increasing length and complexity were also recorded using conventional inductive plethysmography, and used as baseline data for interpretating the conversational speech breathing data.