Natural Speech Corpus Research Articles

Number Attraction in Pronoun Production.

Pronoun production involves at least two processes: (i) deciding to refer to a referent with a pronoun instead of a full NP and (ii) determining the pronoun's form. In the present study, we assess whether the second of these processes occurs as a by-product of the first process-namely, does accessing the message-level representation of the referent provide access to the features required to determine pronoun form, meaning that pronouns should be robust to errors, or are pronoun features determined through an agreement operation with the antecedent, in which case they may be susceptible to agreement attraction, similar to subject-verb agreement. Prior lab experiments suggest that pronouns display number attraction at a similar rate to verbs. However, in contrast to verb attraction errors, there is no documentation of systematic pronoun attraction errors in corpora of natural speech. Our study builds upon prior lab work by eliciting pronoun sentences using a scene description paradigm that engages the pronominalization processes involved in natural speech. Across three experiments, we observed small but reliable number attraction effects for pronouns, suggesting that pronoun form is not always determined from the message-level representation of the referent. The elicited error rates were smaller than those previously observed for verbs in a similar scene-description paradigm; this smaller error rate helps to reconcile the apparent discrepancy between pronoun number attraction error rates observed in and outside the lab. The results suggest that pronoun form is determined (at least at times) through an agreement process referencing the features of the linguistic antecedent.

How Do Care Partners of People with Rare Dementia Use Language in Online Peer Support Groups? A Quantitative Text Analysis Study

We used quantitative text analysis to examine conversations in a series of online support groups attended by care partners of people living with rare dementias (PLWRD). We used transcripts of 14 sessions (>100,000 words) to explore patterns of communication in trained facilitators’ (n = 2) and participants’ (n = 11) speech and to investigate the impact of session agenda on language use. We investigated the features of their communication via Poisson regression and a clustering algorithm. We also compared their speech with a natural speech corpus. We found that differences to natural speech emerged, notably in emotional tone (d = −3.2, p < 0.001) and cognitive processes (d = 2.8, p < 0.001). We observed further differences between facilitators and participants and between sessions based on agenda. The clustering algorithm categorised participants’ contributions into three groups: sharing experience, self-reflection, and group processes. We discuss the findings in the context of Social Comparison Theory. We argue that dedicated online spaces have a positive impact on care partners in combatting isolation and stress via affiliation with peers. We then discuss the linguistic mechanisms by which social support was experienced in the group. The present paper has implications for any services seeking insight into how peer support is designed, delivered, and experienced by participants.

Code-switching input for machine translation: a case study of Vietnamese–English data

ABSTRACT Multilingualism presents both a challenge and an opportunity for Natural Language Processing, with code-switching representing a particularly interesting problem for computational models trained on monolingual datasets. In this paper, we explore how code-switched data affects the task of Machine Translation, a task which only recently has started to tackle the challenge of multilingual data. We test three Machine Translation systems on data from the Canberra Vietnamese–English Codeswitching Natural Speech Corpus (CanVEC) and evaluate translation output using both automatic and human metrics. We find that, perhaps counter-intuitively, Machine Translation performs better on code-switching input than monolingual input. In particular, comparison of human and automatic evaluation suggests that codeswitching input may boost the semantic faithfulness of the translation output, an effect we term lexico-semantic enrichment. We also report two cases where this effect is most and least clear in Vietnamese–English, namely gender-neutral 3SG pronouns and interrogative constructions respectively. Overall, we suggest that Machine Translation, and Natural Language Processing more generally, ought to view multilingualism as an opportunity rather than an obstacle. Abbreviations: 1: First person; 2: Second person; 3: Third person; CLF: Classifier; COP: Copula; DET: Determiner; PL: Plural; POSS: Possessive marker; PRT: Particle; PST: Past tense; Q: Question marker; SG: Singular

A Reanalysis of the Voicing Effect in English: With Implications for Featural Specification.

The voicing effect is among the most studied and robust of phonetic phenomena. Yet there remains a lack of consensus on why vowels preceding voiced obstruents should be longer than vowels preceding voiceless obstruents. In this paper we provide an analysis of the voicing effect in a corpus of natural speech, and using production data from a metronome-timed word repetition study. From this evidence, as well as the existing literature, we conclude that vowel duration differences follow from consonant duration differences. The characteristic voicing effect in English is largely limited to words of especially long duration, and preceding vowel duration does not reliably cue obstruent voicing under the following circumstances: when obstruent voicing or duration cues conflict; for lax or unstressed vowels; and for most conversational speech. We show that this behavior can be modeled using a competing-constraints framework, where all segments resist expanding or compressing past a preferred duration. Inherent segment elasticity determines the degree of resistance, but segment duration is ultimately determined by the interaction of these segmental constraints with constraints on the distribution of the lengthening force within the syllable, and how closely target durations are matched. This account of the voicing effect has a number of implications for phonological theory, especially the central role that the concept of prominence plays in the analysis of underlying features.

Second language acoustic–prosodic entrainment in conversation and storytelling

Prosodic entrainment is the tendency for individuals to modify their acoustic-prosodic speech behaviors to converge with the behaviors of their interlocutors. Evidence of entrainment in the native language (L1) is robust, yet research regarding its development in the second language (L2) is sparse. To examine prosodic entrainment in spoken dialogues, we developed a natural speech corpus in which spontaneous conversations of 30 young adults were collected. Data from 15 dyadic groups of participants in the conversation and the story-telling sessions in both their native language (Mandarin Chinese) and L2 (English) were collected and analyzed. Prosodic entrainment has been examined with the low-level prosodic features, a bundle of the acoustic measure (i.e., f0, speaking rate), and voice measures (i.e., jitter and NHR). Individual difference measures (i.e., attention to details, communication skills) are also examined, and their correlations with the prosodic features are reported. The research is expected to contribute to the study of L2 prosody acquisition and human–machine interaction as well as future research in turn-taking, conversational analysis, and individual differences in L1 and L2 speech behaviors.

A theory of repetition and retrieval in language production.

Repetition appears to be part of error correction and action preparation in all domains that involve producing an action sequence. The present work contends that the ubiquity of repetition is due to its role in resolving a problem inherent to planning and retrieval of action sequences: the Problem of Retrieval. Repetitions occur when the production to perform next is not activated enough to be executed. Repetitions are helpful in this situation because the repeated action sequence activates the likely continuation. We model a corpus of natural speech using a recurrent network, with words as units of production. We show that repeated material makes upcoming words more predictable, especially when more than one word is repeated. Speakers are argued to produce multiword repetitions by using backward associations to reactivate recently produced words. The existence of multiword repetitions means that speakers must decide where to reinitiate execution from. We show that production restarts from words that have seldom occurred in a predictive preceding-word context and have often occurred utterance-initially. These results are explained by competition between preceding-context and top-down cues over the course of language learning. The proposed theory improves on structural accounts of repetition disfluencies, and integrates repetition disfluencies in language production with repetitions observed in other domains of skilled action. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The predictability of implicit causes: testing frequency and topicality explanations

ABSTRACT In discourses involving implicit causality, the implicit cause of the event is referentially predictable, that is, it is likely to be rementioned. However, it is unclear how referential predictability is calculated. We test two possible explanations: (1) The frequency account suggests that people learn that implicit causes are predictable through experience with the most frequent patterns of reference in natural language, and (2) the topicality account asks whether implicit causes tend to play topical roles in the discourse, which itself may lead to the perception of discourse accessibility. With two text analyses we show that implicit causes are frequently rementioned, but only if we consider a narrow set of discourse circumstances, which would require comprehenders to track contingent frequencies. We found no evidence for the topicality account: in two experiments, implicit causality affected predictability but not topicality, and in a corpus of natural speech, implicit causes tended to not occupy topical positions.

The Italian Roots in Australian Soil (IRIAS) multilingual speech corpus. Speech variation in two generations of Italo-Australians

We present and describe the Italian Roots in Australian Soil (IRIAS) speech corpus. Following a sociophonetic approach, our aim is to extend and complement the frequently investigated macro-structures of lexical, syntactic and morphological interactions among immigrants’ languages and common sociolinguistic investigations about immigrants’ language attitudes. We first discuss and motivate the creation of the IRIAS corpus. We then focus on the specific methodological issues we addressed in compiling a corpus of natural spontaneous speech collected in Veneto or Calabrese dialects, Italian and English from first and second generation Italo-Australian speakers originating from two specific regions in Italy (Veneto and Calabria). A detailed description of the IRIAS corpus follows, including its design, collection procedure and processing. The latter focuses on novel manual and automatic solutions we implemented to overcome the challenging dearth of existing resources. These solutions help advance work on spontaneous speech data. We conclude by providing some insights on what has been achieved thus far as well as the analyses currently being carried out on subsets of the IRIAS corpus.

Borrowing from Bislama into Raga, Vanuatu

Abstract This paper reports on variation among speakers of Raga, an Oceanic language of Pentecost island, Vanuatu, in their use of borrowings from Bislama, the national language of Vanuatu, an English-lexifier contact language. The study measures the frequency of borrowings from Bislama in the speech of 50 speakers, surveys speakers’ strategies in assimilating loanwords into Raga and quantifies speakers’ rate of lexical replacement and insertion. This corpus of natural speech reveals an overall low incidence of borrowing from Bislama at 1.6 Bislama words per 100 recorded words. Women and younger speakers borrow more frequently from Bislama. Young speakers use borrowings in equal measure to add to their vocabulary and replace Raga words, while their elders tend to borrow from Bislama to add to their vocabulary, rather than replace Raga words.

The phonetic specificity of contrastive hyperarticulation in natural speech

Abstract Evidence suggests that speakers hyperarticulate phonetic cues to word identity in a way that increases phonetic distance to similar competitors. However, the degree and type of phonetic similarity between competitors which induces hyperarticulation remains unclear. Here, we compared neighborhood density (as a representative of a phonetically-general type of similarity) to the existence of a phonetic cue-specific lexical minimal pair in terms of their ability to predict hyperarticulation of two different cue-types in a corpus of natural English speech. For all phonetic cues that we investigated – word-initial voiceless stop VOT, word-initial voiced stop VOT, and vowel-vowel Euclidean formant distance – cue-specific minimal pair existence significantly predicted cue hyperarticulation, while neighborhood density did not. Further, the direction of change in a phonetic cue was found not to be consistent within a given cue-type, but instead varied as a function of the phonetic relationship, creating greater phonetic distance to the competitor. For tokens of word-initial voiceless stops, existence of a voiced-stop minimal pair predicted significantly longer VOT, while for word-initial voiced stops, existence of a voiceless-stop minimal pair predicted a shorter VOT. For tokens of vowels, existence of a minimal pair defined by a nearby vowel predicted greater Euclidean distance in formant space from that vowel, rather than greater expansion of the vowel space per se. For example, lax vowels in words with a more peripheral vowel minimal pair competitor (e.g., ship ∼ sheep) were relatively centralized, while tense vowels in words with a more interior vowel minimal pair competitor (e.g., date ∼ debt) were relatively peripheralized.

Quantitative intonation modeling of interrogative sentences for Mandarin speech synthesis

Previous intonational research on Mandarin has mainly focused on the prosody modeling of statements or the prosody analysis of interrogative sentences. To support related speech technologies, e.g., Text-to-Speech, the quantitative modeling of intonation of interrogative sentences with a large-scale corpus still deserves attention. This paper summarizes our work on the quantitative prosody modeling of interrogative sentence in Mandarin. A large-scale natural speech corpus was used in this study. By extracting the pitch contours and fitting the intonation curves, we found that F0 declination and final lowering both existed in interrogative sentences, while they were claimed to be absent in Mandarin in some previous studies. In addition, the declination function could be modeled linearly, and the bearing unit of final lowering in Mandarin was found to be the last prosodic word in the utterance, regardless of its length, rather than a fixed duration range. It was argued in this study that the difference between this finding and the commonly believed rising intonation of the interrogative sentences resulted from the nonlinear relationship between prosody production and perception. The underlying mechanism for the existence of F0 declination and final lowering in interrogative sentences is also discussed.

Assessment of disordered voices based on an optimized glottal source model

In this paper, a method for the assessment of disordered voices is proposed. A feature named mean opening quotient (MOQ) obtained from the glottal source estimation is used as an acoustic cue to summarize the degree of severity of the voice disorder. The analysis method uses the empirical mode decomposition algorithm to estimate the glottal source excitation signal from the speech signal. The logarithm of the magnitude spectrum of the speech signal is decomposed into oscillatory modes, called intrinsic mode functions, that are clustered into two classes, the spectral envelope and the harmonic component. The exploitation of the phase information jointly with the estimated harmonic component enables the estimation of the glottal source signal. An appropriate parametric model is fitted to the estimated glottal source excitation signal. The optimal parameters of the glottal source excitation model from which the MOQ is defined are obtained by using a genetic algorithm. The presented method is tested on a corpus of natural speech including the vowel [a] uttered by 22 normophonic speakers and 229 speakers with different degrees of dysphonia. Experimental results show that the proposed method is very effective for assessing the degree of severity of the voice disorder.

Modally hybrid grammar?: Celestial pointing for time-of-day reference in Nheengatú

From the study of sign languages we know that the visual modality robustly supports the encoding of conventionalized linguistic elements, yet while the same possibility exists for the visual bodily behavior of speakers of spoken languages, such practices are often referred to as ‘gestural’ and are not usually described in linguistic terms. This article describes a practice of speakers of the Brazilian indigenous language Nheengatú of pointing to positions along the east-west axis of the sun’s arc for time-of-day reference, and illustrates how it satisfies any of the common criteria for linguistic elements, as a system of standardized and productive form-meaning pairings whose contributions to propositional meaning remain stable across contexts. First, examples from a video corpus of natural speech demonstrate these conventionalized properties of Nheengatú time reference across multiple speakers. Second, a series of video-based elicitation stimuli test several dimensions of its conventionalization for nine participants. The results illustrate why modality is not an a priori reason that linguistic properties cannot develop in the visual practices that accompany spoken language. The conclusion discusses different possible morphosyntactic and pragmatic analyses for such conventionalized visual elements and asks whether they might be more crosslinguistically common than we presently know.

Intelligibility of time-compressed synthetic speech: Compression method and speaking style

We present a series of intelligibility experiments performed on natural and synthetic speech time-compressed at a range of rates and analyze the effect of speech corpus and compression method on the intelligibility scores of sighted and blind individuals. Particularly we are interested in comparing linear and non-linear compression methods applied to normal and fast speech of different speakers. We recorded English and German language voice talents reading prompts at a normal and a fast rate. To create synthetic voices we trained a statistical parametric speech synthesis system based on the normal and the fast data of each speaker. We compared three compression methods: scaling the variance of the state duration model, interpolating the duration models of the fast and the normal voices, and applying a linear compression method to the generated speech waveform. Word recognition results for the English voices show that generating speech at a normal speaking rate and then applying linear compression resulted in the most intelligible speech at all tested rates. A similar result was found when evaluating the intelligibility of the natural speech corpus. For the German voices, interpolation was found to be better at moderate speaking rates but the linear method was again more successful at very high rates, particularly when applied to the fast data. Phonemic level annotation of the normal and fast databases showed that the German speaker was able to reproduce speech at a fast rate with fewer deletion and substitution errors compared to the English speaker, supporting the intelligibility benefits observed when compressing his fast speech. This shows that the use of fast speech data to create faster synthetic voices does not necessarily lead to more intelligible voices as results are highly dependent on how successful the speaker was at speaking fast while maintaining intelligibility. Linear compression applied to normal rate speech can more reliably provide higher intelligibility, particularly at ultra fast rates.

Real Life Emotion Classification using Spectral Features and Gaussian Mixture Models

In this work, spectral features are extracted for speech emotion classification. Mel frequency cepstral coefficients (MFCCs) are used as features. Gaussian mixture models (GMMs) are explored as classifiers. The emotions considered are anger, happy, neutral, sad and surprise. Semi-natural emotional database (Graphic Era University Semi Natural Emotion Speech Corpus) is collected from the dialogues of popular Hindi movies. Average emotion recognition performance, in the case of multiple speaker database is observed to be around 55.60%. Results of male, female, multiple male and multiple female speakers are compared to study the effect of speakers and gender on expression of emotions

Recognition of Emotions from Speech using Excitation Source Features

In this paper, the speech database is introduced for characterizing the emotions present in speech. A semi natural database GEU-SNESC (GEU Semi Natural Emotion Speech Corpus) is used for obtaining emotion specific information using LP residual samples as features. The corpus is collected by recording dialogues of popular film actors/actresses from Hindi movies. The emotions which are considered in this study are sad, anger, happy and neutral. In this paper Linear Prediction (LP) residual of speech signal is used for characterizing the basic emotions present in the speech. LP residual is obtained by LP analysis, by inverse filtering of the speech signal. For capturing the emotion specific information from the higher order relations, present in the LP residual, Gaussian mixture models (GMM) are used. The emotion recognition performance is observed to be about 50-60%.

Spoken Corpora: Rationale and Application

Despite the abundance of electronic corpora now available to researchers, corpora of natural speech are still relatively rare and relatively costly. This paper suggests reasons why spoken corpora are needed, despite the formidable problems of construction. The multiple purposes of such corpora and the involvement of very different kinds of language communities in such projects mean that there is no one single blueprint for the design, markup, and distribution of spoken corpora. A number of different spoken corpora are reviewed to illustrate a range of possibilities for the construction of spoken corpora.

Conveying discourse structure in synthetic speech

A pre-processing system for assigning prosodic characteristics of speech was created in order to investigate how synthesized speech can capture the global characteristics of discourse structure. The system allows manipulation of a variety of prosodic characteristics that have been tied to the structure of discourse above the level of the sentence, in particular: pitch fluctuation, pitch range, speaking rate, and pauses. These characteristics can be used to highlight the onset of a discourse segment, defined as a group of utterances that contribute to a single discourse purpose. After highlighting the onset of the discourse segments, these speech characteristics are progressively modified to indicate the continuity of the discourse segment. The system was evaluated by comparing it to a corpus of natural speech, the Boston Directions Corpus. That corpus has been analyzed both in terms of the informational content of the discourse and the acoustic manifestations that appear in natural speech for conveying that content. Ways in which those acoustic manifestations can be realized in natural speech are discussed.

Data-driven speech synthesis

Described in this paper are the theoretical background and implementation of a speech synthesis engine that uses an index of features describing a natural speech source to provide pointers to waveform segments that can then be re-sequenced to form novel utterances. By efficiently labelling the features in speech that are minimally sufficient to describe the perceptually relevant variation in acoustic and prosodic characterisitcs, reduce the task of synthesis to ‘‘retrieval’’ rather than ‘‘replication,’’ and is reduced reuse of original waveform segments is possible without the need for (perceptually damaging) signal processing. The drawback of this system is that it requires a large corpus of natural speech from one speaker, but current improvements in data-storage devices and cpu technology have overcome this problem. The style of the corpus speech determines the style of the synthesis, but experiments with corpora of emotional speech confirm that by switching source corpora one can easily control the speaking style. By shifting ‘‘knowledge’’ out of the synthesizer into the source data an engine is produced that can work on any adequately labelled speech corpus. The interesting work for the future lies in determining which features are relevant to capture the variation in speech, and how they can be best described.

Modelamento Prosódico para Conversão de Texto-Fala do Português Falado no Brasil

This paper describes a prosodic model for a text-to-speech synthesis system developed for the portuguese language spoken in Brazil. The model provides both a duration and a fundamental frequency (pitch) control of the phones. The model is based on rules extracted from a corpus of natural speech (phrases spoken by people). These rules control the duration of the phones and define fundamental frequency profiles for neutral declarative sentences.