Consonant Regions Research Articles

Synthesis of Emotional Speech by Prosody Modification of Vowel Segments of Neutral Speech

Background: Emotional speech synthesis is the process of synthesising emotions in a neutral speech – potentially generated by a text-to-speech system – to make an artificial humanmachine interaction human-like. It typically involves analysis and modification of speech parameters. Existing work on speech synthesis involving modification of prosody parameters does so at sentence, word, and syllable level. However, further fine-grained modification at vowel level has not been explored yet, thereby motivating our work. Objective: To explore prosody parameters at vowel level for emotion synthesis. Methods: Our work modifies prosody features (duration, pitch, and intensity) for emotion synthesis. Specifically, it modifies the duration parameter of vowel-like and pause regions and the pitch and intensity parameters of only vowel-like regions. The modification is gender specific using emotional speech templates stored in a database and done using Pitch Synchronous Overlap and Add (PSOLA) method. Results: Comparison was done with the existing work on prosody modification at sentence, word and syllable label on IITKGP-SEHSC database. Improvements of 8.14%, 13.56%, and 2.80% for emotions angry, happy, and fear respectively were obtained for the relative mean opinion score. This was due to: (1) prosody modification at vowel-level being more fine-grained than sentence, word, or syllable level and (2) prosody patterns not being generated for consonant regions because vocal cords do not vibrate during consonant production. Conclusion: Our proposed work shows that an emotional speech generated using prosody modification at vowel-level is more convincible than prosody modification at sentence, word and syllable level.

Effect of source filter interaction on isolated vowel-consonant-vowel perception.

Source-filter interaction explains the drop in pitch in voiced consonant due to constriction in the vocal tract during vowel-consonant-vowel (VCV) production. In this work, a perceptual study is conducted where the pitch contour in the voiced consonant region is modified to four different levels and a listening test is performed to assess the naturalness of the VCVs synthesized with the modified pitch contour. The listening test with 30 listeners shows no statistically significant difference between the naturalness of the original and synthesized VCVs with modified pitch indicating that pitch drop due to source-filter interaction may not be critical for the perceived naturalness of VCVs.

Consonant-vowel unit recognition using dominant aperiodic and transition region detection

This work reports a method of Consonant-Vowel (CV) unit recognition by detecting the Dominant Aperiodic component Regions (DARs) and by predicting the Duration of Transition Regions (DTRs) in speech. DAR detection is performed using complementary information from source and vocal tract. While source information is extracted using sub-fundamental frequency filtering of speech, vocal tract information is extracted using a) Dominant Resonant Frequency (DRF) and b) High to Low Frequency component Ratio (HLFR), computed from Hilbert envelope of Numerator Group Delay (HNGD) spectrum of zero-time windowed signal. The DTR is predicted by using vocal tract constriction information. Subsequently, detected DARs and predicted DTRs are compared with manually marked regions and finally used for CV unit recognition of Indian languages. Conventionally, CV unit recognition is performed by anchoring the Vowel Onset Point (VOP) and assuming fixed durations for transition and consonant regions on either side of the VOP. However, in speech, the duration of transition and consonantal regions vary depending on the type of consonants and vowels. In the proposed method, the use of dynamic values for consonant duration and transition regions have resulted in better consonant recognition improving CV unit recognition.

Class-level spectral features for emotion recognition

The most common approaches to automatic emotion recognition rely on utterance-level prosodic features. Recent studies have shown that utterance-level statistics of segmental spectral features also contain rich information about expressivity and emotion. In our work we introduce a more fine-grained yet robust set of spectral features: statistics of Mel-Frequency Cepstral Coefficients computed over three phoneme type classes of interest – stressed vowels, unstressed vowels and consonants in the utterance. We investigate performance of our features in the task of speaker-independent emotion recognition using two publicly available datasets. Our experimental results clearly indicate that indeed both the richer set of spectral features and the differentiation between phoneme type classes are beneficial for the task. Classification accuracies are consistently higher for our features compared to prosodic or utterance-level spectral features. Combination of our phoneme class features with prosodic features leads to even further improvement. Given the large number of class-level spectral features, we expected feature selection will improve results even further, but none of several selection methods led to clear gains. Further analyses reveal that spectral features computed from consonant regions of the utterance contain more information about emotion than either stressed or unstressed vowel features. We also explore how emotion recognition accuracy depends on utterance length. We show that, while there is no significant dependence for utterance-level prosodic features, accuracy of emotion recognition using class-level spectral features increases with the utterance length.

Duration modification using glottal closure instants and vowel onset points

This paper proposes a method for duration (time scale) modification using glottal closure instants (GCI, also known as instants of significant excitation) and vowel onset points (VOP). In general, most of the time scale modification methods attempt to vary the duration of speech segments uniformly over all regions. But it is observed that consonant regions and transition regions between a consonant and the following vowel, and between two consonant regions do not vary appreciably with speaking rate. The proposed method implements the duration modification without changing the durations of the transition and consonant regions. Vowel onset points are used to identify the transition and consonant regions. A VOP is the instant at which the onset of the vowel takes place, which corresponds to the transition from a consonant to the following vowel in most cases. The VOPs are computed using the Hilbert envelope of linear prediction (LP) residual. The instants of significant excitation correspond to the instants of glottal closure (epochs) in the case of voiced speech, and to some random excitations, like the onset of burst, in the case of nonvoiced speech. Manipulation of duration is achieved by modifying the duration of the LP residual with the help of instants of significant excitation as pitch markers. The modified residual is used to excite the time-varying filter whose parameters are derived from the original speech signal. Perceptual quality of the synthesized speech is found to be natural. Performance of the proposed method is compared with the method, where the duration of speech is modified uniformly over all regions. Samples of speech signals for different modification factors is available for listening at http://sit.iitkgp.ernet.in/~ksrao/result.html.

Speakers nasalize /edh/ if it is preceded by /n/, but listeners don’t care—They still hear /edh/

Presumably as a result of coarticulation, /edh/ often assimilates to a preceding /n/ in phrases like ‘‘win those,’’ but this assimilation is not complete for all features. With respect to the feature [nasal], the assimilation is often radical. The entire consonant region in the middle of the two-word sequence is nasalized. However, acoustic evidence suggests that contextually nasalized /edh/ retains its dental place of articulation. Specifically, F2 is considerably lower at the release of a contextually nasalized /edh/ than at the release of a true /n/, as would be expected for a dental consonant. Perception tests show that listeners can generally tell the difference between natural tokens of pairs like ‘‘win nos’’ and ‘‘win those,’’ even when the /edh/ is completely nasalized. In addition, a synthetic stimulus continuum was constructed in which items differed only with respect to F2 frequency in the vicinity of the nasal consonant regions of phrases like ‘‘win nos.’’ Listeners systematically reported hearing ‘‘those’’ more often when F2 was low at the release of the nasal consonant. These results are consistent with the claim of Krakow, Fowler, and others, that listeners can at least sometimes factor out coarticulatory effects. [Work supported by NSF.]

The effects of coarticulation on Japanese voiceless consonant discrimination.

In this paper, the results of a study of automatic discrimination of Japanese voiceless fricatives in vowel (/a,i,u,e,o,n/)–consonant (/F,s,h/) –vowel (/a,i,u,e,o/) syllables are presented, and the effects of coarticulation on this discrimination are discussed. A total of 4032 speech data uttered by 16 male subjects were used for the study. A higher recognition rate was obtained when the acoustic features were extracted from the consonant and succeeding vowel transition region than when they were extracted from the consonant region or the succeeding vowel transition region alone. Significantly, the extracted features from the consonant–vowel transition region were most effective in the recognition of consonants succeeded by /a/, but less effective for those succeeded by /i/, /e/, and /o/. Furthermore, when /i/ and /e/ were preceded by /F/ and /s/ the recognition rate was low. Similarly when /o/ was preceded by /F/ and /h/ the recognition rate was low. This phenomenon can be explained from the perspective of consonant–vowel coarticulation.