Phrase Break Research Articles

The final lengthening of pre-boundary syllables turns into final shortening as boundary strength levels increase

Phrase-final syllable duration and pauses are generally considered to be positively correlated: The stronger the boundary, the longer the duration of phrase-final syllables, and the more likely or longer a pause. Exploring a large sample of complex literary prose texts read aloud, we examined pause likelihood and duration, pre-boundary syllable duration, and the pitch excursion at prosodic boundaries. Comparing these features across six predicted levels of boundary strength (level 0: no break; 1: simple phrase break; 2: short comma phrase break; 3: long comma phrase break; 4: sentence boundary; 5: direct speech boundary), we find that they are not correlated in a simple monotonic fashion. Whereas pause duration monotonically increases with boundary strength, both pre-boundary syllable duration and the pitch excursion on the pre-boundary syllable are largest for level-2 breaks and decrease significantly through levels 3 to 5. Our analysis suggests that pre-boundary syllable duration is partly contingent on the tonal realization, which is subject to f0 declination as the utterance progresses. We also surmise that pre-boundary syllable duration reflects differences in planning complexity for the different prosodic and syntactic boundaries. Overall, this study shows that a simple monotonic correlation between pause duration and pre-boundary syllable duration is not valid.

DIA-TTS: Deep-Inherited Attention-Based Text-to-Speech Synthesizer

Text-to-speech (TTS) synthesizers have been widely used as a vital assistive tool in various fields. Traditional sequence-to-sequence (seq2seq) TTS such as Tacotron2 uses a single soft attention mechanism for encoder and decoder alignment tasks, which is the biggest shortcoming that incorrectly or repeatedly generates words when dealing with long sentences. It may also generate sentences with run-on and wrong breaks regardless of punctuation marks, which causes the synthesized waveform to lack emotion and sound unnatural. In this paper, we propose an end-to-end neural generative TTS model that is based on the deep-inherited attention (DIA) mechanism along with an adjustable local-sensitive factor (LSF). The inheritance mechanism allows multiple iterations of the DIA by sharing the same training parameter, which tightens the token-frame correlation, as well as fastens the alignment process. In addition, LSF is adopted to enhance the context connection by expanding the DIA concentration region. In addition, a multi-RNN block is used in the decoder for better acoustic feature extraction and generation. Hidden-state information driven from the multi-RNN layers is utilized for attention alignment. The collaborative work of the DIA and multi-RNN layers contributes to outperformance in the high-quality prediction of the phrase breaks of the synthesized speech. We used WaveGlow as a vocoder for real-time, human-like audio synthesis. Human subjective experiments show that the DIA-TTS achieved a mean opinion score (MOS) of 4.48 in terms of naturalness. Ablation studies further prove the superiority of the DIA mechanism for the enhancement of phrase breaks and attention robustness.

Switching the Backbeat

This article explores manipulations of the nearly ever-present backbeat in rock music from the “long” 1980s. The status of the backbeat as not only metrically consonant but a defining feature of rock meter is discussed within the context of dual-aspect meter. Operating from the assumption that isochronous snare drum hits in rock are heard as a backbeat, placing those snare drum hits on beats other than 2 and 4 creates a particular kind of metrical dissonance, which I refer to as a backbeat switch. I examine two ways in which a backbeat switch occurs, the quick flip and the polymetric pogo. A quick flip usually occurs at a phrase break, where the drummer apparently (but intentionally) “drops a beat” and then resumes the backbeat pattern, thus shifting it “to the left.” A polymetric pogo involves a situation where the backbeat-insistent drummer is pitted against the rest of the band playing in an odd-cardinality meter, resulting in snare hits that bounce back and forth between even-numbered beats and odd-numbered beats every other measure. Short examples by The Cars, Paul Weller, Steve Vai, and Sting are used to demonstrate the article’s fundamental concepts, and longer examples by Tesla, Jerry Goldsmith, and Extreme place these concepts into larger contexts.

Sequences of high tones across word boundaries in Tswana

The article analyses violations of the Obligatory Contour Principle (OCP) above the word level in Tswana, a Southern Bantu language, by investigating the realization of adjacent lexical high tones across word boundaries. The results show that across word boundaries downstep (i.e. a lowering of the second in a series of adjacent high tones) only takes place within a phonological phrase. A phonological phrase break blocks downstep, even when the necessary tonal configuration is met. A phrase-based account is adopted in order to account for the occurrence of downstep. Our study confirms a pattern previously reported for the closely related language Southern Sotho and provides controlled, empirical data from Tswana, based on read speech of twelve speakers which has been analysed auditorily by two annotators as well as acoustically.

Prosody prediction for arabic via the open-source boundary-annotated qur’an corpus

humans or machines. To develop phrase break classifiers, we need a boundary-annotated and part-ofspeech tagged corpus. Boundary annotations in English speech corpora are descriptive, delimiting intonation units perceived by the listener; manual annotation must be done by an expert linguist. For Arabic, there are no existing suitable resources. We take a novel approach to phrase break prediction for Arabic, deriving our prosodic annotation scheme from Tajwid (recitation) mark-up in the Qur’an which we then interpret as additional text-based data for computational analysis. This mark-up is prescriptive, and signifies a widely-used recitation style, and one of seven original styles of transmission. Here we report on version 1.0 of our Boundary-Annotated Qur’an dataset of 77430 words and 8230 sentences, where each word is tagged with prosodic and syntactic information at two coarse-grained levels. We then use this dataset to train, test, and compare two probabilistic taggers (trigram and HMM) for Arabic phrase break prediction, where the task is to predict boundary locations in an unseen test set stripped of boundary annotations by classifying words as breaks or non-breaks. The preponderance of non-breaks in the training data sets a challenging baseline success rate: 85.56%. However, we achieve significant gains in accuracy with a trigram tagger, and significant gains in performance recognition of minority class instances with both taggers via the Balanced Classification Rate metric. This is initial work on a longterm research project to produce annotation schemes, language resources, algorithms, and applications for Classical and Modern Standard Arabic.

Exploiting Morphological and Phonological Features to Improve Prosodic Phrasing for Mongolian Speech Synthesis

Prosodic phrasing is an important factor that affects naturalness and intelligibility in text-to-speech synthesis. Studies show that deep learning techniques improve prosodic phrasing when large text and speech corpus are available. However, for low-resource languages, such as Mongolian, prosodic phrasing remains a challenge for various reasons. First, the database suitable for system training is limited. Second, word composition knowledge that is prosody-informing has not been used in prosodic phrase modeling. To address these problems, in this article, we propose a feature augmentation method in conjunction with a self-attention neural classifier. We augment input text with morphological and phonological decompositions of words to enhance the text encoder. We study the use of self-attention classifier, that makes use of global context of a sentence, as a decoder for phrase break prediction. Both objective and subjective evaluations validate the effectiveness of the proposed phrase break prediction framework, that consistently improves voice quality in a Mongolian text-to-speech synthesis system.

Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS

Tacotron-based end-to-end speech synthesis has shown remarkable voice quality. However, the rendering of prosody in the synthesized speech remains to be improved, especially for long sentences, where prosodic phrasing errors can occur frequently. In this paper, we extend the Tacotron-based speech synthesis framework to explicitly model the prosodic phrase breaks. We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks. To our best knowledge, this is the first implementation of multi-task learning for Tacotron based TTS with a prosodic phrasing model. Experiments show that our proposed training scheme consistently improves the voice quality for both Chinese and Mongolian systems.

Interactive Prosodic Marking of Focus, Boundary and Newness in Mandarin

The current study investigates whether and how focus, phrase boundary and newness can be simultaneously marked in speech prosody in Mandarin Chinese. Homophones were used to construct three syntactic structures that differed only in boundary condition, focus was elicited by preceding questions, while newness of postboundary words was manipulated as whether they had occurred in the previous text. Systematic analysis of F0 and duration showed that (1) duration was a reliable correlate of boundary strength regardless of focus location, while involvement of F0 was only in terms of lowering of phrase-final F0 minima and raising of phrase-initial F0 minima at a relatively strong boundary, (2) postfocus compression (PFC) of F0 was applied across all boundaries, including those with long silent pauses (over 200 ms), and postfocus F0 was lowered to almost the same degree in all boundary conditions, and (3) newness of postfocus words had no systematic effect on F0 or duration. These results indicate that not only functionally focus is independent of prosodic structure and newness, but also phonetically its realization is separate from boundary marking. Focus is signaled mainly through pitch range adjustments, which can occur even across phrase breaks, whereas boundaries are mostly signaled by duration adjustments.

Exclusives, equatives and prosodic phrases in Samoan

This paper investigates the functions of prosodic phrasing in the Austronesian VSO language Samoan. Two types of sentences are investigated, exclusives (involving the particle na’o ‘only’) and equatives. Two complementary methodologies were used, a production study and an acceptability judgment study, to examine the prosodic realisation and relative naturalness of different word orderings of the two sentence types. The particle na’o has an unusual distribution: preceding the initial constituent, be it the verb or a fronted noun phrase; or following the verb, but only modifying the absolutive (object). It was found that post-verbal absolutives modified by na’o are usually not preceded by a phrase boundary, unlike unmodified absolutives which are consistently preceded by a high phrase tone (H-) (cf. Yu 2009). Equatives in Samoan involve clauses which are the juxtaposition of two noun phrases, one the rheme (focus) and the other the theme (topic). It was found that rhemes are usually followed by a phrase break, while for themes this is optional. Rheme-theme order was strongly preferred to theme-rheme order. These findings are argued to show a close relationship between information structure, constituent ordering and prosodic phrasing in Samoan. The preferred order of constituents in Samoan is rheme-theme, with a high phrase tone marking the end of the rheme. The absolutive argument is strongly preferred to be at the start of the theme. This article is part of the Special Collection: Prosody and constituent structure

Detection of Malay Phrase Breaks Using Energy and Duration

Detection of Malay Phrase Breaks Using Energy and Duration

Modeling Pauses for Synthesis of Storytelling Style Speech Using Unsupervised Word Features

Abstract In the storytelling style speech pauses or phrase breaks play a significant role in introducing suspense and climax in the story. More often pauses are provided by a storyteller to capture the audience's attention by emphasizing keywords, focusing emotion-salient words, and to separate key phrases in an utterance. The goal of the work presented in this paper is to predict the location of pauses, in an utterance synthesized by a Story Text-To-Speech (TTS) system using unsupervised features at word-level. Traditional methods for predicting pauses uses the foremost linguistic features like Parts-of-Speech (POS) tags, chunking information or terminal syllables, etc. These methods presuppose the availability of linguistic knowledge by an automatic tagger or manually annotated corpus. However, this information's are not readily available in case of Indian Languages. Manually annotating the text with this linguistic information is quite hectic and time consuming. Also, these pieces of information's do not capture the co-occurrence statistics of words. Hence, we propose a framework for integrating the Story TTS with proposed pause prediction module. In this module, an unlabeled text corpus is used to extract, the continuous-valued world-level features to model the pause patterns in storytelling speech. A set of story-specific (SS) features are introduced for capturing story-semantic information based on pause pattern. A various combination of pause predictions systems is- proposed such as B, POS, U, POS+SS and U+SS. These systems are evaluated objectively by F-1 Score.

Tree-based Phone Duration Modelling of the Serbian Language

Considering the importance of segmental duration from a perceptive point of view, the possibility of automatic prediction of natural duration of phones is essential for achieving the naturalness of synthesized speech. In this paper phone duration prediction model for the Serbian language using tree-based machine learning approach is presented. A large speech corpus and a feature set of 21 parameters describing phones and their contexts were used for segmental duration prediction. Phone duration modelling is based on attributes such as the current segment identity, preceding and following segment types, manner of articulation (for consonants) and voicing of neighbouring phones, lexical stress, part-of-speech, word length, the position of the segment in the syllable, the position of the syllable in a word, the position of a word in a phrase, phrase break level, etc. These features have been extracted from the large speech database for the Serbian language. The results obtained for the full phoneme set using regression tree, RMSE (root-mean-squared-error) 14.8914 ms, MAE (mean absolute error) 11.1947 ms and correlation coefficient 0.8796 are comparable with those reported in the literature for Czech, Greek, Lithuanian, Korean, Indian languages Hindi and Telugu, Turkish. DOI: http://dx.doi.org/10.5755/j01.eee.20.3.4090

Statistical parametric speech synthesis for Ibibio

Ibibio is a Nigerian tone language, spoken in the south-east coastal region of Nigeria. Like most African languages, it is resource-limited. This presents a major challenge to conventional approaches to speech synthesis, which typically require the training of numerous predictive models of linguistic features such as the phoneme sequence (i.e., a pronunciation dictionary plus a letter-to-sound model) and prosodic structure (e.g., a phrase break predictor). This training is invariably supervised, requiring a corpus of training data labelled with the linguistic feature to be predicted. In this paper, we investigate what can be achieved in the absence of many of these expensive resources, and also with a limited amount of speech recordings. We employ a statistical parametric method, because this has been found to offer good performance even on small corpora, and because it is able to directly learn the relationship between acoustics and whatever linguistic features are available, potentially mitigating the absence of explicit representations of intermediate linguistic layers such as prosody.We present an evaluation that compares systems that have access to varying degrees of linguistic structure. The simplest system only uses phonetic context (quinphones), and this is compared to systems with access to a richer set of context features, with or without tone marking. It is found that the use of tone marking contributes significantly to the quality of synthetic speech. Future work should therefore address the problem of tone assignment using a dictionary and the building of a prediction module for out-of-vocabulary words.

Non-traditional prosodic features for automated phrase break prediction

It is universally recognized that humans process speech and language in chunks, each meaningful in itself. Any two renditions or assimilations of a given sentence will exhibit similarities and discrepancies in the distribution of phrase breaks. Automated phrase break prediction assigns pauses to plain text as input, evaluated against human performance encapsulated in ‘gold standard’ boundary annotations in a speech corpus. This article advocates an enhanced feature set for phrase break prediction, incorporating non-traditional prosodic features. The authors have developed ProPOSEL, a prosody and part-of-speech English lexicon, as text annotation and text analytics tool. Application of ProPOSEL has so far uncovered a statistically significant correlation in English between certain sound patterns (i.e. the diphthongs and triphthongs of Received Pronunciation) and phrase breaks in very different genres. Thus, presence or absence of a complex vowel could easily be incorporated as an extra non-traditional classificatory feature in phrase break models. Our approach also suggests new possibilities for statistical analysis of texts, particularly authorship and genre, via favoured sound and rhythmic patterns in addition to lexis. Moreover, we suggest that our approach of text data-mining descriptive annotations of projected prosody for Text-to-Speech Synthesis and stylistic analysis is applicable to other languages.

Wh-questions in Chewa and Tumbuka: positions and prosodies

This paper presents a preliminary survey of the positions and prosodies associated with Wh-questions in two Bantu languages spoken in Malawi. The paper shows that the two languages are similar in requiring focused subjects to be clefted. Both also require 'which' questions and 'because of what' questions to be clefted or fronted. However, for other non-subjects Tumbuka rather uniformly imposes an IAV (immediately after the verb) requirement, while Chewa does not. In both languages, we found a strong tendency for there to be a prosodic phrase break following the Wh-word. In Tumbuka, this break follows from the general phrasing algorithm of the language, while in Chewa, I propose that the break can be best understood as following from the inherent prominence of Wh-words.

Automatic feature template generation for maximum entropy based intonational phrase break prediction

Automatic feature template generation for maximum entropy based intonational phrase break prediction

Holy smoke: vocalic precursors of phrase breaks in Milton's Paradise Lost

We report on a significant correlation between lexical items containing complex vowels in their present day canonical forms, and prosodic-syntactic boundaries in Milton’s Paradise Lost, where all line terminals, whether end-stopped or run-on, plus line-medials with associated punctuation, constitute boundary tokens and equate to gold-standard phrase break annotations. Real-world knowledge of present day English pronunciation is projected onto each word token in two different versions, constituting two phrasing variants, of Book 1 of the poem via ProPOSEL, a prosody and part-of-speech English lexicon developed by the authors; and pertinent differences in place of articulation of English vowels in Milton’s day and ours are also discussed. The chi-squared test for independence returns a two-tailed P-value of less than 0.0001 for the association of this vowel subset and phrase breaks in both samples. This leads us to speculate that the poet’s unpremeditated use of complex vowels — which slow down verse movement in Paradise Lost and thus generate rhythmic junctures — may represent a phrasing device habitual not just to poets but to native English speakers in general. Concurrent work on a corpus of present-day British English speech corroborates our findings. Hence, complex vowels may constitute new predictive features in phrasing models for English.

Potstro Fongers by Sietse de Vries

Potstro Fongers by Sietse de Vries

Development and evaluation of a prosodic database for Greek speech synthesis and research*

In this article the definition, construction and statistical evaluation of a prosodic database of Greek speech are presented. The main motivation for the development of such a database was its use as a research tool for Text-to-Speech synthesis and the study of prosody in general. Beginning with the task of text selection we came to a final set containing sentences with almost 95% of all Greek syllables, extracted from a widely used Greek dictionary. Then, a professional radio actress was instructed to utter these sentences in reading style and at reading rate; this was recorded at 44 kHz/16 bits, in the anechoic chamber of a professional studio. The intonational phenomena were transcribed on the corresponding speech signals by a trained phonetician using the ToBI annotation model adapted to Greek prosodic patterns. The speech data were segmented to the phoneme level employing a phoneme recognizer based on the HTK platform. All files were aligned so that possible relations among text, intonational and durational labelling could be identified. For database management, the EMU speech database system was utilized. Extensive measurements of numerous annotated events presented in histograms and tables provide detailed information on the database. Finally, we evaluate prediction models of prosodic phrase breaks and pitch accents derived from our database. Performance of these models was also compared to models derived under the same experimental conditions with a limited domain corpus of Greek speech.

An approach for detecting prosodic phrase boundaries in spoken english

Prosodic phrasing is the means by which speakers of any given language break up an utterance into meaningful chunks. The term "prosody" itself refers to the tune or intonation of an utterance, and therefore prosodic phrases literally signal the end of one tune and the beginning of another. This study uses phrase break annotations in the Aix-MARSEC corpus of spoken English as a "gold standard" for measuring the degree of correspondence between prosodic phrases and the discrete syntactic grouping of prepositional phrases, where the latter is defined via a chunk parsing rule using nltk_lite's regular expression chunk parser. A three-way comparison is also introduced between the "gold standard" chunk parsing rule and human judgment in the form of intuitive predictions about phrasing. Results show that even with a discrete syntactic grouping and a small sample of text, problems may arise for this rule-based method due to uncategorical behavior in parts of speech. Lack of correspondence between intuitive prosodic phrases and corpus annotations highlights the optional nature of certain boundary types. Finally, there are clear indications, supported by corpus annotations, that significant prosodic phrase boundaries occur within sentences and not just at full stops.