Problem Of Speech Recognition Research Articles

Performance of WUW and general ASR speech recognition systems in different acoustic environments

The Automatic Speech Recognition (ARS) Systems have become popular in the different areas of applications, such as education assistance, medical assistance, personal assistant, robotics and vehicles, telecommunications Industry, disabilities systems, home automation, and security access control. All these applications in life will increase the number of speech commands, which are Wake-up-words. The WUW speech recognition is similar to Key-Word spotting. However, in this paper, we propose an approach that will be used to detect the Wake-up-word (WUW) as a command with 100% accuracy and to provide a whole speech recognition system that can work in both WUW and General ASR systems with high accuracy. Also, this new ASR system will be used to solve one of the biggest problems in speech recognition, which is how to discriminate between the uses of a word or phrase in an alerting and a referential context. For example, in the use of the word “Computer'' in an alerting context is Computer, show me the chart? '' and in a referential context it is “Every computer should have a speaker. Moreover, we tested the new system with different acoustic environments, such as different background noise levels, different speaker distances to the microphone, and different speakers.

Monaural multi-talker speech recognition using factorial speech processing models

A Pascal challenge entitled monaural speech separation and recognition challenge was developed, targeting the problem of robust automatic speech recognition against speech-like noises which significantly degrade the performance of automatic speech recognition systems. In this challenge, two competing speakers say a simple command simultaneously and the objective is to recognize speech of the target speaker. Surprisingly, a team from IBM research could achieve performance better than human listeners on this task during the challenge. The IBM system consists of an intermediate speech separation and two single-talker speech recognition modules. This paper reconsiders the recognition task of this challenge based on gain adapted factorial speech processing models. It develops a joint-token passing algorithm for direct joint-decoding of target and masker speakers’ mixed-signals, simultaneously. It uses maximum uncertainty during the joint-decoding, which cannot be used in the two-phased IBM system. This paper provides a detailed derivation of inference on these models based on the general inference procedures of probabilistic graphical models. Additionally, it uses deep neural networks for joint-speaker identification and their gain estimation, which makes these two steps easier than before while producing competitive results for these steps. The proposed method of this work outperforms past super-human results and even the results recently achieved using deep neural networks by Microsoft research. It achieved 5.3% absolute task performance improvement compared to the first super-human system and 2.5% absolute task performance improvement compared to its recent competitor.

Model for Automatic Speech Recognition Using Multi-Agent Recursive Cognitive Architecture

A concept of a fundamentally new approach to the development of speech recognition systems is proposed, as applications built based on existing approaches are not effective enough when used in noisy conditions and cocktail party situations. The architecture of the speech recognition system in an environment with several speakers based on multi-agent recursive cognitive models with imitation of the attention mechanism is constructed. The speech recognition system allows to model selectivity of perception in speech peculiarities for a speaker using multi-agent self-organization. Principles for selective signature processing inside sound modality are defined. They allow to tune on a speaker. Articulatory primitives were chosen as minimal functional pattern in the speech recognition problem. Due to multi-agent nature, use of space-time characteristics and self-learning this approach allow us to separate from each other and analyze sounds of different nature. Screenshots of the cognitive architecture of the speech recognition system based on multi-agent models of semantics are presented.

Progressive Joint Modeling in Unsupervised Single-Channel Overlapped Speech Recognition

Unsupervised single-channel overlapped speech recognition is one of the hardest problems in automatic speech recognition (ASR). Permutation invariant training (PIT) is a state of the art model-based approach, which applies a single neural network to solve this single-input, multiple-output modeling problem. We propose to advance the current state of the art by imposing a modular structure on the neural network, applying a progressive pretraining regimen, and improving the objective function with transfer learning and a discriminative training criterion. The modular structure splits the problem into three sub-tasks: frame-wise interpreting, utterance-level speaker tracing, and speech recognition. The pretraining regimen uses these modules to solve progressively harder tasks. Transfer learning leverages parallel clean speech to improve the training targets for the network. Our discriminative training formulation is a modification of standard formulations, that also penalizes competing outputs of the system. Experiments are conducted on the artificial overlapped Switchboard and hub5e-swb dataset. The proposed framework achieves over 30% relative improvement of WER over both a strong jointly trained system, PIT for ASR, and a separately optimized system, PIT for speech separation with clean speech ASR model. The improvement comes from better model generalization, training efficiency and the sequence level linguistic knowledge integration.

Robust Beamforming for Speech Recognition Using DNN-Based Time-Frequency Masks Estimation

This paper addresses the robust beamforming problem for speech recognition using a novel time-frequency mask estimator. The beamformer first estimates the time-frequency mask using a deep neural network (DNN) based on which the covariance matrices of the target speech and noise are computed. Then, the beamformer coefficients are directly obtained via generalized eigenvector decomposition. To achieve accurate covariance matrix estimation for robust beamforming, we propose a DNN-based mask estimator which can exploit the spatial features of the multi-channel microphone signals. The proposed mask estimator leverages the spatial information of the microphone array by using multi-channel signals to estimate a speech-aware mask and a noise-aware mask simultaneously. Using the target-specified masks, accurate covariance matrices of the target speech and noise can be obtained from the observation independently. Experiments on CHiME4 data sets demonstrate that, compared with the baseline toolkit (BeamformIt) and the winner in the CHiME3 challenge, the proposed method achieves better results both in terms of perceptual speech quality and speech recognition error rate.

ТЕОРІЯ КОМБІНАТОРНОЇ ОПТИМІЗАЦІЇ ТА ЗАДАЧІ ШТУЧНОГО ІНТЕЛЕКТУ

Багато задач штучного інтелекту моделюються з використанням теорії комбінаторної оптимізації. Це пов’язано з тим, що переважна їхня частина при знаходженні оптимального розв’язку потребує перебору варіантів. Перебірним же задачам властива комбінаторна природа. Для задач розпізнавання та сегментації мовленнєвого сигналу, клінічної діагностики наведено математичні моделі, які побудовані в рамках теорії комбінаторної оптимізації. Показано, що в задачах цього класу комбінаторні конфігурації можуть бути як аргументом цільової функції, так і вхідними даними.

On the Derivational Entropy of Left-to-Right Probabilistic Finite-State Automata and Hidden Markov Models

Probabilistic finite-state automata are a formalism that is widely used in many problems of automatic speech recognition and natural language processing. Probabilistic finite-state automata are closely related to other finite-state models as weighted finite-state automata, word lattices, and hidden Markov models. Therefore, they share many similar properties and problems. Entropy measures of finite-state models have been investigated in the past in order to study the information capacity of these models. The derivational entropy quantifies the uncertainty that the model has about the probability distribution it represents. The derivational entropy in a finite-state automaton is computed from the probability that is accumulated in all of its individual state sequences. The computation of the entropy from a weighted finite-state automaton requires a normalized model. This article studies an efficient computation of the derivational entropy of left-to-right probabilistic finite-state automata, and it introduces an efficient algorithm for normalizing weighted finite-state automata. The efficient computation of the derivational entropy is also extended to continuous hidden Markov models.

Investigation of Combined use of MFCC and LPC Features in Speech Recognition Systems

Statement of the automatic speech recognition problem, the assignment of speech recognition and the application fields are shown in the paper. At the same time as Azerbaijan speech, the establishment principles of speech recognition system and the problems arising in the system are investigated. The computing algorithms of speech features, being the main part of speech recognition system, are analyzed. From this point of view, the determination algorithms of Mel Frequency Cepstral Coefficients (MFCC) and Linear Predictive Coding (LPC) coefficients expressing the basic speech features are developed. Combined use of cepstrals of MFCC and LPC in speech recognition system is suggested to improve the reliability of speech recognition system. To this end, the recognition system is divided into MFCC and LPC-based recognition subsystems. The training and recognition processes are realized in both subsystems separately, and recognition system gets the decision being the same results of each subsystems. This results in decrease of error rate during recognition. The training and recognition processes are realized by artificial neural networks in the automatic speech recognition system. The neural networks are trained by the conjugate gradient method. In the paper the problems observed by the number of speech features at training the neural networks of MFCC and LPC-based speech recognition subsystems are investigated. The variety of results of neural networks trained from different initial points in training process is analyzed. Methodology of combined use of neural networks trained from different initial points in speech recognition system is suggested to improve the reliability of recognition system and increase the recognition quality, and obtained practical results are shown.

Relating dynamic brain states to dynamic machine states: Human and machine solutions to the speech recognition problem.

There is widespread interest in the relationship between the neurobiological systems supporting human cognition and emerging computational systems capable of emulating these capacities. Human speech comprehension, poorly understood as a neurobiological process, is an important case in point. Automatic Speech Recognition (ASR) systems with near-human levels of performance are now available, which provide a computationally explicit solution for the recognition of words in continuous speech. This research aims to bridge the gap between speech recognition processes in humans and machines, using novel multivariate techniques to compare incremental ‘machine states’, generated as the ASR analysis progresses over time, to the incremental ‘brain states’, measured using combined electro- and magneto-encephalography (EMEG), generated as the same inputs are heard by human listeners. This direct comparison of dynamic human and machine internal states, as they respond to the same incrementally delivered sensory input, revealed a significant correspondence between neural response patterns in human superior temporal cortex and the structural properties of ASR-derived phonetic models. Spatially coherent patches in human temporal cortex responded selectively to individual phonetic features defined on the basis of machine-extracted regularities in the speech to lexicon mapping process. These results demonstrate the feasibility of relating human and ASR solutions to the problem of speech recognition, and suggest the potential for further studies relating complex neural computations in human speech comprehension to the rapidly evolving ASR systems that address the same problem domain.

Arabic stop consonants characterisation and classification using the normalized energy in frequency bands

In general, speech is made with sequences of consonants (fricatives, nasals and stops), vowels and glides. The classification of the stop consonants remains one of the most challenging problems in speech recognition. In this paper, we propose a new approach based on the normalized energy in frequency bands in the release and closure phases in order to characterize and classify the Arabic stop consonants (/b/, /d/, /t/, /k/ and /q/) and to recognize the CV syllable. Classification experiments were performed using decision algorithms on stop consonants C and CV syllables extracted from an Arabic corpus. The results yielded to an overall stop consonants classification of 90.27% and syllables CV recognition upper than 90% for all stops.

Automatic Non-native Dialect and Accent Voice Detection of South Indian English

Speech recognition has achieved enormous improvements presently. However, robustness is still one of the big tribulations, e.g. performance of recognition fluctuates penetratingly depending on the speaker, particularly as the speaker has robust accent that is not coated in the training corpus. The speaker variability, such like gender, accent, age, speaking rate, and phone realizations, are vital problems in speech recognition. The mainly South Indian accent identification is a recent challenging problem closely related to other relatively recent fields of the multilinguality area like non native speech identification and language identification. This paper explains an automatic recognition system for English accents from 5 different South Indian State. The approach is based on a corresponding set of random nets with situation independent HMM units. The random topology was in addition substituted by pronunciation transcription constraints so as to integrate accent specific automatic word recognizers.

An unsupervised deep domain adaptation approach for robust speech recognition

This paper addresses the robust speech recognition problem as a domain adaptation task. Specifically, we introduce an unsupervised deep domain adaptation (DDA) approach to acoustic modeling in order to eliminate the training–testing mismatch that is common in real-world use of speech recognition. Under a multi-task learning framework, the approach jointly learns two discriminative classifiers using one deep neural network (DNN). As the main task, a label predictor predicts phoneme labels and is used during training and at test time. As the second task, a domain classifier discriminates between the source and the target domains during training. The network is optimized by minimizing the loss of the label classifier and to maximize the loss of the domain classifier at the same time. The proposed approach is easy to implement by modifying a common feed-forward network. Moreover, this unsupervised approach only needs labeled training data from the source domain and some unlabeled raw data of the new domain. Speech recognition experiments on noise/channel distortion and domain shift confirm the effectiveness of the proposed approach. For instance, on the Aurora-4 corpus, compared with the acoustic model trained only using clean data, the DDA approach achieves relative 37.8% word error rate (WER) reduction.

Whispered speech recognition using deep denoising autoencoder

Recently Deep Denoising Autoencoders (DDAE) have shown state-of-the-art performance on various machine learning tasks. In this paper, the authors extended this approach to whispered speech recognition which is one of the most challenging problems in Automatic Speech Recognition (ASR). Namely, due to the profound differences between acoustic characteristics of neutral and whispered speech, the performance of traditional ASR systems trained on neutral speech degrades significantly when whisper is applied. This mismatch between training and testing is successfully alleviated with the new proposed system based on deep learning, where DDAE is applied for generating whisper-robust cepstral features. This system was tested and compared in terms of word recognition accuracy with conventional Hidden Markov Model (HMM) speech recognizer in an isolated word recognition task with a real database of whispered speech (WhiSpe). Three types of cepstral coefficients were used in the experiments: MFCC (Mel-Frequency Cepstral Coefficients), TECC (Teager-Energy Cepstral Coefficients) and TEMFCC (Teager-based Mel-Frequency Cepstral Coefficients). The experimental results showed that the proposed system significantly improves whisper recognition accuracy and outperforms traditional HMM-MFCC baseline, resulting in an absolute 31% improvement of whisper recognition accuracy. The highest word recognition rate of 92.81% in whispered speech was achieved with TECC feature.

Aging and the effect of target-masker alignment.

Similarity between target and competing speech messages plays a large role in how easy or difficult it is to understand messages of interest. Much research on informational masking has used highly aligned target and masking utterances that are very similar semantically and syntactically. However, listeners rarely encounter situations in real life where they must understand one sentence in the presence of another (or more than one) highly aligned, syntactically similar competing sentence(s). The purpose of the present study was to examine the effect of syntactic/semantic similarity of target and masking speech in different spatial conditions among younger, middle-aged, and older adults. The results of this experiment indicate that differences in speech recognition between older and younger participants were largest when the masker surrounded the target and was more similar to the target, especially at more adverse signal-to-noise ratios. Differences among listeners and the effect of similarity were much less robust, and all listeners were relatively resistant to masking, when maskers were located on one side of the target message. The present results suggest that previous studies using highly aligned stimuli may have overestimated age-related speech recognition problems.

Speech Recognition Algorithm in Complex Noisy Environments Based on Multi-Space Compensation

Speech recognition rate drops significantly when interfered by noise in complex environment. In order to improve the accuracy and the robustness of the speech recognition in adverse acoustical environments, this paper reviewed the main problems of noise robust speech recognition, proposed a multi-space compensation algorithm which from signal-space, feature-space and model-space based on wiener filter, histogram equalization and vector Taylor series. Theory analyses and experiment results show that the proposed method can overcome the defect of the sharp descent of speech recognition rate of existing speech recognition algorithm interfered by environmental noise and improve the accuracy and the robustness in adverse acoustical environments. The algorithm provides the theoretical support for the speech recognition in airport, station, wharf and other complex noise environment.

발화구간 검출을 위해 학습된 CNN 기반 입 모양 인식 방법

소음환경에서의 음성인식 문제점으로 인해 1990년대 중반부터 음성정보와 영양정보를 결합한 AVSR(Audio Visual Speech Recognition) 시스템이 제안되었고, Lip Reading은 AVSR 시스템에서 시각적 특징으로 사용되었다. 본 연구는 효율적인 AVSR 시스템을 구축하기 위해 입 모양만을 이용한 발화 단어 인식률을 극대화하는데 목적이 있다. 본 연구에서는 입 모양 인식을 위해 실험단어를 발화한 입력 영상으로부터 영상의 전처리 과정을 수행하고 입술 영역을 검출한다. 이후 DNN(Deep Neural Network)의 일종인 CNN(Convolution Neural Network)을 이용하여 발화구간을 검출하고, 동일한 네트워크를 사용하여 입 모양 특징 벡터를 추출하여 HMM(Hidden Markov Mode)으로 인식 실험을 진행하였다. 그 결과 발화구간 검출 결과는 91%의 인식률을 보임으로써 Threshold를 이용한 방법에 비해 높은 성능을 나타냈다. 또한 입모양 인식 실험에서 화자종속 실험은 88.5%, 화자 독립 실험은 80.2%로 이전 연구들에 비해 높은 결과를 보였다. Due to speech recognition problems in noisy environment, Audio Visual Speech Recognition (AVSR) system, which combines speech information and visual information, has been proposed since the mid-1990s,. and lip reading have played significant role in the AVSR System. This study aims to enhance recognition rate of utterance word using only lip shape detection for efficient AVSR system. After preprocessing for lip region detection, Convolution Neural Network (CNN) techniques are applied for utterance period detection and lip shape feature vector extraction, and Hidden Markov Models (HMMs) are then used for the recognition. As a result, the utterance period detection results show 91% of success rates, which are higher performance than general threshold methods. In the lip reading recognition, while user-dependent experiment records 88.5%, user-independent experiment shows 80.2% of recognition rates, which are improved results compared to the previous studies.

Two Efficient Lattice Rescoring Methods Using Recurrent Neural Network Language Models

An important part of the language modelling problem for automatic speech recognition (ASR) systems, and many other related applications, is to appropriately model long-distance context dependencies in natural languages. Hence, statistical language models (LMs) that can model longer span history contexts, for example, recurrent neural network language models (RNNLMs), have become increasingly popular for state-of-the-art ASR systems. As RNNLMs use a vector representation of complete history contexts, they are normally used to rescore N-best lists. Motivated by their intrinsic characteristics, two efficient lattice rescoring methods for RNNLMs are proposed in this paper. The first method uses an $n$ -gram style clustering of history contexts. The second approach directly exploits the distance measure between recurrent hidden history vectors. Both methods produced 1-best performance comparable to a 10 k-best rescoring baseline RNNLM system on two large vocabulary conversational telephone speech recognition tasks for US English and Mandarin Chinese. Consistent lattice size compression and recognition performance improvements after confusion network (CN) decoding were also obtained over the prefix tree structured N-best rescoring approach.

Double sparse learning model for speech emotion recognition

A double sparse learning (DSL) model with a pyramid structure-based feature extraction scheme to handle speech emotion recognition (SER) problem is proposed. The key novelty of the method is that the proposed DSL model is able to take into consideration two scales of the pyramid structure-based features for selecting the features which have great contributions to SER. Extensive experiments on eNTERFACE and AFEW emotion databases to evaluate the method are conducted. The experimental results demonstrate that, compared with some recent competitive methods, DSL with the pyramid structure-based feature extraction scheme has a more promising performance in dealing with the SER task.

Integrated search technique for parameter determination of SVM for speech recognition

Support vector machine (SVM) has a good application prospect for speech recognition problems; still optimum parameter selection is a vital issue for it. To improve the learning ability of SVM, a method for searching the optimal parameters based on integration of predator prey optimization (PPO) and Hooke-Jeeves method has been proposed. In PPO technique, population consists of prey and predator particles. The prey particles search the optimum solution and predator always attacks the global best prey particle. The solution obtained by PPO is further improved by applying Hooke-Jeeves method. Proposed method is applied to recognize isolated words in a Hindi speech database and also to recognize words in a benchmark database TI-20 in clean and noisy environment. A recognition rate of 81.5% for Hindi database and 92.2% for TI-20 database has been achieved using proposed technique.

Modeling under-resourced languages for speech recognition

One particular problem in large vocabulary continuous speech recognition for low-resourced languages is finding relevant training data for the statistical language models. Large amount of data is required, because models should estimate the probability for all possible word sequences. For Finnish, Estonian and the other fenno-ugric languages a special problem with the data is the huge amount of different word forms that are common in normal speech. The same problem exists also in other language technology applications such as machine translation, information retrieval, and in some extent also in other morphologically rich languages. In this paper we present methods and evaluations in four recent language modeling topics: selecting conversational data from the Internet, adapting models for foreign words, multi-domain and adapted neural network language modeling, and decoding with subword units. Our evaluations show that the same methods work in more than one language and that they scale down to smaller data resources.