Basic Speech Units Research Articles

Speech Attribute Detection to Recognize Arabic Broadcast Speech in Industrial Networks

In this work, we present an alternative model to Arabic speech recognitions to boost classification accuracy of basic speech units in Industrial Internet of Things. This approach integrates both top-down and bottom-up knowledge into the automatic speech attribute transcription (ASAT) framework. ASAT is a new family of lattice-based speech recognition systems grounded on accurate detection of speech attributes. Two state-of-the-art deep neural networks were used for attribute speech feature detection, convolutional neural networks (CNNs) and feed-forward neural network, with pretraining using a stack of restricted Boltzmann machines (DBN-DNN). These attribute detectors are trained using a data-driven approach for Arabic speech data using the Buckwalter code for Arabic transcription. The detectors work by detecting phonologically distinctive articulatory features of Arabic based on manner and place of articulation, which extract characteristics of the human speech production process from the speech dataset. The results of the work are shown in terms of average equal error rate inferring that the attribute model based on the CNN consistently outperforms the DBN-DNN model. Attribute detectors with CNN, versus DBN-DNN model, reduce the AEER producing 10.42% relative improvements for manner of articulation and producing 13.11% relative improvements for place of articulation.

Effect of voicing and articulation manner on aerosol particle emission during human speech.

Previously, we demonstrated a strong correlation between the amplitude of human speech and the emission rate of micron-scale expiratory aerosol particles, which are believed to play a role in respiratory disease transmission. To further those findings, here we systematically investigate the effect of different 'phones' (the basic sound units of speech) on the emission of particles from the human respiratory tract during speech. We measured the respiratory particle emission rates of 56 healthy human volunteers voicing specific phones, both in isolation and in the context of a standard spoken text. We found that certain phones are associated with significantly higher particle production; for example, the vowel /i/ ("need," "sea") produces more particles than /ɑ/ ("saw," "hot") or /u/ ("blue," "mood"), while disyllabic words including voiced plosive consonants (e.g., /d/, /b/, /g/) yield more particles than words with voiceless fricatives (e.g., /s/, /h/, /f/). These trends for discrete phones and words were corroborated by the time-resolved particle emission rates as volunteers read aloud from a standard text passage that incorporates a broad range of the phones present in spoken English. Our measurements showed that particle emission rates were positively correlated with the vowel content of a phrase; conversely, particle emission decreased during phrases with a high fraction of voiceless fricatives. Our particle emission data is broadly consistent with prior measurements of the egressive airflow rate associated with the vocalization of various phones that differ in voicing and articulation. These results suggest that airborne transmission of respiratory pathogens via speech aerosol particles could be modulated by specific phonetic characteristics of the language spoken by a given human population, along with other, more frequently considered epidemiological variables.

Speech Recognition for the iCub Platform.

This paper describes open source software (available at https://github.com/robotology/natural-speech) to build automatic speech recognition (ASR) systems and run them within the YARP platform. The toolkit is designed (i) to allow non-ASR experts to easily create their own ASR system and run it on iCub and (ii) to build deep learning-based models specifically addressing the main challenges an ASR system faces in the context of verbal human–iCub interactions. The toolkit mostly consists of Python, C++ code and shell scripts integrated in YARP. As additional contribution, a second codebase (written in Matlab) is provided for more expert ASR users who want to experiment with bio-inspired and developmental learning-inspired ASR systems. Specifically, we provide code for two distinct kinds of speech recognition: “articulatory” and “unsupervised” speech recognition. The first is largely inspired by influential neurobiological theories of speech perception which assume speech perception to be mediated by brain motor cortex activities. Our articulatory systems have been shown to outperform strong deep learning-based baselines. The second type of recognition systems, the “unsupervised” systems, do not use any supervised information (contrary to most ASR systems, including our articulatory systems). To some extent, they mimic an infant who has to discover the basic speech units of a language by herself. In addition, we provide resources consisting of pre-trained deep learning models for ASR, and a 2.5-h speech dataset of spoken commands, the VoCub dataset, which can be used to adapt an ASR system to the typical acoustic environments in which iCub operates.

Exploiting deep neural networks for detection-based speech recognition

In recent years deep neural networks (DNNs) – multilayer perceptrons (MLPs) with many hidden layers – have been successfully applied to several speech tasks, i.e., phoneme recognition, out of vocabulary word detection, confidence measure, etc. In this paper, we show that DNNs can be used to boost the classification accuracy of basic speech units, such as phonetic attributes (phonological features) and phonemes. This boosting leads to higher flexibility and has the potential to integrate both top-down and bottom-up knowledge into the Automatic Speech Attribute Transcription (ASAT) framework. ASAT is a new family of lattice-based speech recognition systems grounded on accurate detection of speech attributes. In this paper we compare DNNs and shallow MLPs within the ASAT framework to classify phonetic attributes and phonemes. Several DNN architectures ranging from five to seven hidden layers and up to 2048 hidden units per hidden layer will be presented and evaluated. Experimental evidence on the speaker-independent Wall Street Journal corpus clearly demonstrates that DNNs can achieve significant improvements over the shallow MLPs with a single hidden layer, producing greater than 90% frame-level attribute estimation accuracies for all 21 phonetic features tested. Similar improvement is also observed on the phoneme classification task with excellent frame-level accuracy of 86.6% by using DNNs. This improved phoneme prediction accuracy, when integrated into a standard large vocabulary continuous speech recognition (LVCSR) system through a word lattice rescoring framework, results in improved word recognition accuracy, which is better than previously reported word lattice rescoring results.

Evaluation of synthetic and natural Mandarin visual speech: Initial consonants, single vowels, and syllables

Although the auditory aspects of Mandarin speech are relatively more heavily-researched and well-known in the field, this study addresses its visual aspects by examining the perception of both Mandarin natural and synthetic visual speech. In perceptual experiments, the synthetic visual speech of a computer-animated Mandarin talking head was evaluated and subsequently improved. Also, the basic (or “minimum”) units of Mandarin visual speech were determined for initial consonants and final single-vowels. Overall, the current study achieved solid improvements of synthetic visual speech, and this was one step towards a Mandarin synthetic talking head with realistic speech.

Corpus Based Statistical Analysis of Punjabi Syllables for Preparation of Punjabi Speech Database

Syllables preserve within unit co-articulation effects, which results syllables to be a good choice as speech unit for speech database of many languages. Punjabi is a syllabic language, so syllables have been selected as the basic speech unit for the preparation of speech database for Punjabi. In order to select a minimal syllable set covering almost whole Punjabi word set to be stored in the speech database, all the available Punjabi syllables have been statistically analyzed over a large Punjabi corpus having more than 104 million words. This statistical analysis reveals very interesting and important facts about the actual use of Punjabi syllables by its speaker and writers. From the results of this analysis a relatively smaller syllable set of about first ten thousand (0.86% of total available 1156740 syllables) most frequently occurring syllables having cumulative frequency of occurrence less than 99.81% have been selected. Also, frequency of occurrence of these syllables at the three positions (starting, middle and end) in the words of corpus has been obtained, for detailed storage and better utilization of these syllables in certain applications like Text-to-Speech synthesis.

Perception of acoustically complex phonological features in vowels is reflected in the induced brain-magnetic activity

A central issue in speech recognition is which basic units of speech are extracted by the auditory system and used for lexical access. One suggestion is that complex acoustic-phonetic information is mapped onto abstract phonological representations of speech and that a finite set of phonological features is used to guide speech perception. Previous studies analyzing the N1m component of the auditory evoked field have shown that this holds for the acoustically simple feature place of articulation. Brain magnetic correlates indexing the extraction of acoustically more complex features, such as lip rounding (ROUND) in vowels, have not been unraveled yet. The present study uses magnetoencephalography (MEG) to describe the spatial-temporal neural dynamics underlying the extraction of phonological features. We examined the induced electromagnetic brain response to German vowels and found the event-related desynchronization in the upper beta-band to be prolonged for those vowels that exhibit the lip rounding feature (ROUND). It was the presence of that feature rather than circumscribed single acoustic parameters, such as their formant frequencies, which explained the differences between the experimental conditions. We conclude that the prolonged event-related desynchronization in the upper beta-band correlates with the computational effort for the extraction of acoustically complex phonological features from the speech signal. The results provide an additional biomagnetic parameter to study mechanisms of speech perception.

Design, implementation and evaluation of the Czech realistic audio-visual speech synthesis

This paper presents the whole process of creation of audio-visual speech synthesis system. Such system consists of two main parts, the acoustic synthesis emulating human speech and the facial animation emulating the human lip articulation. The acoustic subsystem is based on concatenation-based speech synthesis. The visual subsystem is designed as a realistic, fully three-dimensional parametrically controllable facial animation model. To be able to parametrically control the animation to emulate human articulation, the set of visual parameters has to be obtained for all basic speech units. To provide realistic animation, the database of lip movements of a real person need to be recorded and expressed by suitable parameterization. The set of control parameters for visual animation is then derived from this database. The 3D model of a head based on a head of a real person also makes the animation more realistic. To obtain such model, a 3D scanning of a real person has to be adopted. We present the design and implementation of above-mentioned process. The aim is to obtain realistic audio-visual speech synthesis with possibility to change the 3D head model according to particular person. The design, acquisition and processing of audio-visual speech corpus for such purpose is presented. Next, the process of both acoustic and visual speech synthesis is described. The visual speech synthesis comprises the tasks of model training, animation control, and co-articulation modelling. A facial animation can also increase intelligibility of a telephone speech to people with hearing disabilities. In such case the textual information to control the animation is not available. Solution to the problem of mapping visual parameters from speech signal either directly or through recognized text is presented. Furthermore, the 3D scanning algorithm is presented. It allows to obtain realistic 3D model based on a head of a real person and thus to personalize the talking head. In the end of this paper, evaluation of intelligibility of the presented audio-visual speech synthesis and its possible applications are presented.

Method and system for semantic speech recognition

The present invention discloses a computer-implemented method to understand queries or commands spoken by users when they use natural language utterances similar to those that people use spontaneously to communicate. More precisely, the invention discloses a method that identifies user queries or commands from the general information involved in spoken utterances directly by the speech recognition system, and not by a post-process as is conventionally used. In a phase of preparation of the system, a vocabulary of items representing data and semantic identifiers is created as well as a syntax module having valid combinations of items. When the system is in use, a user utterance is first discretized into a plurality of basic speech units which are compared to the items in the vocabulary and a combination of items is selected according to the evaluation from the syntax module in order to generate the most likely sequence of items representative of the user utterance. Finally the semantic identifiers and the data extracted from the user utterance are used to call the appropriate function that process the user request.

TRANSLATION AND BAKHTIN'S “METALINGUISTICS”

This contribution applies a Bakhtinian perspective to Translation Studies, according to which the source-text gets represented in the target-text thanks to the dialogic relations that are created by translators in the process of translating. Translation is neither a complete rewriting nor a transparent mirroring of the source text, but an understanding that involves what Bakhtin calls “outsideness”. By positioning themselves in this “outsideness”, translators achieve the understanding that is a prerequisite for their task. We examine the role that Bakhtin's “metalinguistics” plays for translators. Bakhtin's concern was with utterances, the basic speech units. We discuss how a source text becomes an utterance and hence unrepeatable for translators. Using specific examples, we then illustrate the relevance of Bakhtinian “double-voiced words” – a metalinguistic category – for translators.

Contextual vector quantization for speech recognition with discrete hidden Markov model

By using formulation of the finite mixture distribution identification, in this paper, several alternatives to the conventional LBG VQ method are investigated. A contextual VQ method based on the Markov Random Field (MRF) theory is proposed to model the speech feature vector space. Its superiority is confirmed by a series of comparative experiments in a speaker independent isolated word recognition task by using different VQ schemes as the front-end of DHMM. The motivation to use MRF to model the contextual dependence information in the underlying speech production process can be readily extended to acoustic modeling of the basic speech units in speech recognition.

Testing the speech unit hypothesis with the primed matching task: phoneme categories are perceptually basic.

The basic speech unit (phoneme or syllable) problem was investigated with the primed matching task. In primed matching, subjects have to decide whether the elements of stimulus pairs are the same or different. The prime should facilitate matching in as far as its representation is similar to the stimuli to be matched. If stimulus representations generate graded structure, with stimulus instances being more or less prototypical for the category, priming should interact with prototypicality because prototypical instances are more similar to the activated category than are low-prototypical instances. Rosch (1975a, 1975b) showed that, by varying the matching criterion (matching for physical identity or for belonging to the same category), the specific patterns of the priming x prototypicality interaction could differentiate perceptually based from abstract categories. By testing this pattern for phoneme and syllable categories, the abstraction level of these categories can be studied. After finding reliable prototypicality effects for both phoneme and syllable categories (Experiments 1 and 2), primed phoneme matching (Experiments 3 and 4) and primed syllable matching (Experiments 5 and 6) were used under both physical identity instructions and same-category instructions. The results make clear that phoneme categories are represented on the basis of perceptual information, whereas syllable representations are more abstract. The phoneme category can thus be identified as the basic speech unit. Implications for phoneme and syllable representation are discussed.

The semi-relaxed algorithm for estimating parameters of hidden Markov models

For hidden Markov model (HMM) based speech recognition where the basic speech unit is smaller than the recognizer's output unit, the standard full Baum-Welch re-estimation procedure for the HMM training is very costly in computation. This is because it requires evaluation of the HMM output densities and of the forward/backward probabilities in the entire region of the state-frame trellis. In this paper, we present an algorithm which exploits the fact that the entries of the trellis are essentially zero except near the block diagonal and hence achieves significant computational saving. The algorithm is evaluated in experiments with a large vocabulary word recognizer based on mixture-density HMM representation of phonemes. The HMM parameters trained with the new algorithm are essentially identical to those trained with the full Baum-Welch algorithm in that the resulting HMMs have nearly the same likelihood values on the same set of training data. Identical word recognition accuracies are yielded using the HMMs trained with the two algorithms. However, the new algorithm is shown to be about an order of magnitude faster than the full Baum-Welch algorithm.

Training set design for connected speech recognition

Two methods for generating training sets for a speech recognition system are studied. The first uses a nondeterministic statistical method to generate a uniform distribution of sentences from a finite state machine (FSM) represented in digraph form. The second method, a deterministic heuristic approach, takes into consideration the importance of word ordering to address the problem of coarticulation effects. The two methods are critically compared. The first algorithm, referred to as MARKOV, converts the FSM into a first-order Markov model. The digraphs are determined, transitive closure computed, transition probabilities are assigned, and stopping criteria established. An efficient algorithm for computing these parameters is described. Statistical tests are conducted to verify performance and demonstrate its utility. A second algorithm for generating training sentences, referred to as BIGRAM, uses heuristics to satisfy three requirements: adequate coverage of basic speech (subword) units; adequate coverage of words in the recognition vocabulary (intraword contextual units); and adequate coverage of word pairs bigrams (interword contextual units).< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Language models and search algorithms for real-time speech recognition

In this paper, “continuous speech recognition” problem is given a clear mathematical formulation as the search for that sequence of basic speech units that best fits the input acoustic pattern. For this purpose spoken language models in the form of hierarchical transition networks are introduced, where lower level subnetworks describe the basic units as possible sequences of spectral states. The units adopted in this paper are either whole words or smaller subword elements, called diphones. The recognition problem thus becomes that of finding the best path through the network, a task carried out by the linguistic decoder. By using this approach, knowledge sources at different levels are strongly integrated. In this way, early decision making based on partial information (in particular any segmentation operation or the speech/silence distinction) is avoided; usually this is a significant source or errors. Instead, decisions are deferred to the linguistic decoder, which possesses all the necessary pieces of information. The properties that a linguistic decoder must possess in order to operate in real-time are listed, and then a best-few algorithm with partial traceback of explored paths, satisfying the above requisites, is described. In particular, the amount of storage needed is almost constant for any sentence length, the computation is approximately linear with sentence length, and the interpretation of early words in a sentence may be possible long before the speaker has finished talking. Experimental results with two systems, one with words and the other with diphones as basic speech units, are reported. Finally, relative merits of words and diphones are discussed, taking into account aspects such as the storage and computing time requirements, their relative ability to deal with phonological variations and to discriminate between similar words, their speaker adaptation capability, and the ease with which it is possible to change the vocabulary and the language dependencies.

An evaluation test for speech synthesizers

Commercially available speech synthesizers are now being used extensively in talking machines. As their numbers continue to grow, it is essential to establish standard evaluation tests as quality control checks in their manufacture. A contribution towards this goal is described in this paper. Listeners heard the output of two commercial synthesizers. By identifying the basic units of speech (phonemes) which are wrongly perceived and comparing these with known confusions arising in listening to real speech under difficult conditions, it was possible to identify the limitations of a given synthesizer.

Continuous speech recognition without segmentation: Two ways of using diphones as basic speech units

Continuous speech recognition without segmentation: Two ways of using diphones as basic speech units

Electromyography as a Speech Research Technique with an Application to Labial Stops

Electromyographic techniques have been developed as indicators of the physiological processes of speech production. They provide a sensitive and direct means for describing the speech gesture in terms of the activity of the actuators rather than the consequent displacements of the speech mechanism. To the extent that perception is geared to articulation, this action pattern may be expected to yield simpler descriptions of the basic units of speech than have been available in acoustic or traditional phonetic terms (Franklin S. Cooper, Alvin M. Liberman, Katherine Safford Harris, and Patti Murray Grubb, “Some input-output relations observed in experiments on the perception of speech,” Proceedings of the Second International Congress on Cybernetics, Namur, 1958). This account attempts no more than a description of the experimental techniques and their application to a specific study of the English labial stops /p,b/. The electromyographic measures showed no significant differences in timing or amount of muscle activity at the lips between /p/ and /b/. In this pilot study, the electromyographic method has proved feasible and has yielded a partial description of /p,b/ that is simply structured and consistent with the acoustic-perceptual characterization of these sounds.