Speech Recognition Techniques Research Articles

Using Automatic Speech Recognition to Assess Thai Speech Language Fluency in the Montreal Cognitive Assessment (MoCA).

The Montreal cognitive assessment (MoCA), a widely accepted screening tool for identifying patients with mild cognitive impairment (MCI), includes a language fluency test of verbal functioning; its scores are based on the number of unique correct words produced by the test taker. However, it is possible that unique words may be counted differently for various languages. This study focuses on Thai as a language that differs from English in terms of word combinations. We applied various automatic speech recognition (ASR) techniques to develop an assisted scoring system for the MoCA language fluency test with Thai language support. This was a challenge because Thai is a low-resource language for which domain-specific data are not publicly available, especially speech data from patients with MCIs. Furthermore, the great variety of pronunciation, intonation, tone, and accent of the patients, all of which might differ from healthy controls, bring more complexity to the model. We propose a hybrid time delay neural network hidden Markov model (TDNN-HMM) architecture for acoustic model training to create our ASR system that is robust to environmental noise and to the variation of voice quality impacted by MCI. The LOTUS Thai speech corpus was incorporated into the training set to improve the model’s generalization. A preprocessing algorithm was implemented to reduce the background noise and improve the overall data quality before feeding data into the TDNN-HMM system for automatic word detection and language fluency score calculation. The results show that the TDNN-HMM model in combination with data augmentation using lattice-free maximum mutual information (LF-MMI) objective function provides a word error rate (WER) of 30.77%. To our knowledge, this is the first study to develop an ASR with Thai language support to automate the scoring system of MoCA’s language fluency assessment.

Computational intelligence in processing of speech acoustics: a survey

Speech recognition of a language is a key area in the field of pattern recognition. This paper presents a comprehensive survey on the speech recognition techniques for non-Indian and Indian languages, and compiled some of the computational models used for processing speech acoustics. An immense number of frameworks are available for speech processing and recognition for languages persisting around the globe. However, a limited number of automatic speech recognition systems are available for commercial use. The gap between the languages being spoken around the globe and the technical support available to these languages are very few. This paper examined major challenges for speech recognition for different languages. Analysis of the literature shows that lack of standard databases availability of minority languages hinder the research recognition research across the globe. When compared with non-Indian languages, the research on speech recognition of Indian languages (except Hindi) has not achieved the expected milestone yet. Combination of MFCC and DNN–HMM classifier is most commonly used system for developing ASR minority languages, whereas in some of the majority languages, researchers are using much advance algorithms of DNN. It has also been observed that the research in this field is quite thin and still more research needs to be carried out, particularly in the case of minority languages.

Thinking out loud, an open-access EEG-based BCI dataset for inner speech recognition

Surface electroencephalography is a standard and noninvasive way to measure electrical brain activity. Recent advances in artificial intelligence led to significant improvements in the automatic detection of brain patterns, allowing increasingly faster, more reliable and accessible Brain-Computer Interfaces. Different paradigms have been used to enable the human-machine interaction and the last few years have broad a mark increase in the interest for interpreting and characterizing the “inner voice” phenomenon. This paradigm, called inner speech, raises the possibility of executing an order just by thinking about it, allowing a “natural” way of controlling external devices. Unfortunately, the lack of publicly available electroencephalography datasets, restricts the development of new techniques for inner speech recognition. A ten-participant dataset acquired under this and two others related paradigms, recorded with an acquisition system of 136 channels, is presented. The main purpose of this work is to provide the scientific community with an open-access multiclass electroencephalography database of inner speech commands that could be used for better understanding of the related brain mechanisms.

Deep learning based assistive technology on audio visual speech recognition for hearing impaired

Assistive technology would be an immense benefit for hearing impaired people by using Audio Visual Speech Recognition (AVSR). Around 466 million people worldwide suffer from hearing loss. Hearing impaired student rely on lip reading for understanding the speech. Lack of trained sign language facilitators and high cost of assistive devices are some of the major challenges faced by hearing impaired students. In this work, we have identified a visual speech recognition technique using cutting edge deep learning models. Moreover, the existing VSR techniques are erroneous. Hence to address the gaps identified, we propose a novel technique by fusion the results from audio and visual speech. This study proposes a new deep learning based audio visual speech recognition model for efficient lip reading. In this paper, an effort has been made to improve the performance of the system significantly by achieving a lowered word error rate of about 6.59% for ASR system and accuracy of about 95% using lip reading model.

Review on human speech Recognition Techniques

Review on human speech Recognition Techniques

Semantic Segmentation of Text Using Deep Learning

Given a text, can we segment it into semantically coherent sections in an automatic way? Can we detect the semantic boundaries, if we know how many they are? Can we determine how many semantically distinct sections are in the text? These are the questions we address in this paper. To respond, we use the Bidirectional Encoder Representation from Transformer (BERT) to analyze the text and evaluate a function that we call local incoherence, which we expect to show maxima at the points where a semantic boundary is detected. Our results, although preliminary, are encouraging and suggest that our approach can be successfully applied. However, they are quite sensitive with respect to the text quality, as it happens in the case in which the text is derived from an audio stream via Automatic Speech Recognition techniques.

Turkish Speech Recognition Techniques and Applications of Recurrent Units (LSTM and GRU)

A typical solution of Automatic Speech Recognition (ASR) problems is realized by feature extraction, feature classification, acoustic modeling and language modeling steps. In classification and modeling steps, Deep Learning Methods have become popular and give more successful recognition results than conventional methods. In this study, an application for solving ASR problem in Turkish Language has been developed. The data sets and studies related to Turkish Language ASR problem are examined. Language models in the ASR problems of agglutative language groups such as Turkish, Finnish and Hungarian are examined. Subword based model is chosen in order not to decrease recognition performance and prevent large vocabulary. The recogniton performance is increased by Deep Learning Methods called Long-Short Term Memory (LSTM) Neural Networks and Gated Recurrent Unit (GRU) in the classification and acoustic modeling steps. The recognition performances of systems including LSTM and GRU are compared with the the previous studies using traditional methods and Deep Neural Networks. When the results were evaluated, it is seen that LSTM and GRU based Speech Recognizers performs better than the recognizers with previous methods. Final Word Error Rate (WER) values were obtained for LSTM and GRU as 10,65% and 11,25%, respectively. GRU based systems have similar performance when compared to LSTM based systems. However, it has been observed that the training periods are short. Computation times are 73.518 and 61.020 seconds respectively. The study gave detailed information about the applicability of the latest methods to Turkish ASR research and applications.

Application of Transfer Learning Algorithm and Real Time Speech Detection in Music Education Platform

Artificial intelligence (AI), particularly machine learning (ML) and neural networks (NN), has various applications and has sparked a lot of interest in the recent years due to its superior performance in a variety of tasks. Automatic speech recognition (ASR) is a technique that is becoming more important with the passage of time and is being used in our daily lives. Speech recognition is an important application of ML and NN, which is the auditory system of machines that realize the communication between humans and machines. In general, speech recognition methods are divided into three types, i.e., based on the channel model and speech knowledge method, template matching scheme, and the use of NN method. The main problem associated with the existing speech recognition methods is the low recognition accuracy and more computation time. In order to overcome the problem of low recognition accuracy of existing speech recognition techniques, a speech recognition technology based on the combination of deep convolution neural network (DCNN) algorithm and transfer learning techniques, i.e., VGG-16, is proposed in this study. Due to the limited application range of DCNN, when the input and output parameters are changed, it is necessary to reconstruct the model that leads to a long training time of the architecture. Therefore, the migration learning method is conducive to reducing the size of the dataset. Various experiments have been performed using different dataset constructs. The simulation results show that transfer learning is not only suitable for the comparison between the source dataset and the target dataset, but also suitable for two different datasets. The application of small datasets not only reduces the time and cost of dataset generation, but also reduces the training time and the requirement of computing power. From the experimental results, it is quite obvious that the proposed system performed better than the existing speech recognition methods, and its performance is superior in terms of recognition accuracy than the other approaches.

Optimization the Accuracy of FFNN Based Speaker Recognition System Using PSO Algorithm

Speaker recognition systems use a model that learns a speaker's speech by inputting an audio recording and processing it. Time-varying signal, with frequencies that continuously change, is identified as a speech signal. There are many uncertain attributes to speech; thus traditional speech recognition techniques such as using zero crossings and the Fourier Transform are not up to the task. It aims to be accomplished with the aim of helping two causes. The first part is designed to address speaker identification technology that is resistant to noise. While most prior solutions have relied on changing mel frequency cepstrum coefficients, with a Fundamental frequency feature coefficient, this proposal integrates both of these modifications with a new cepstrum component. In order to construct the feature matrix, the system is fed with two-hundred and fifty speech imprints that are used to apply features extraction techniques. The matrix is used to teach the algorithm about features, and each one is then evaluated using incomplete data (thirty percent of total data in features matrix). Speaker recognition models with improved accuracy are developed by studying the algorithms invasively. These variables (metrics) are generated for each algorithm and applied to the algorithm for recognition accuracy and the time required to achieve that accuracy. When tested against previous research, the findings show that the Feed Forward Neural Network-based Particle Swarm Optimization method has been better. This model can accurately identify 96% of the input with less processing time. According to the findings, optimization utilizing advanced particle swarm optimization (a.k.a. Particle Swarm Optimization) is most likely responsible for the higher accuracy seen in speaker identification.

Automatic Speech Recognition for Air Traffic Control Communications

A significant fraction of communications between air traffic controllers and pilots is through speech, via radio channels. Automatic transcription of air traffic control (ATC) communications has the potential to improve system safety, operational performance, and conformance monitoring, and to enhance air traffic controller training. We present an automatic speech recognition model tailored to the ATC domain that can transcribe ATC voice to text. The transcribed text is used to extract operational information such as call-sign and runway number. The models are based on recent improvements in machine learning techniques for speech recognition and natural language processing. We evaluate the performance of the model on diverse datasets.

Proxy – An Innovative AI Monitored First Aid Bot

Basic first-aid is the need for every person, but not everyone is aware of the mechanisms in this busy world. In this paper the approach we have used is an Artificial Intelligence Bot combined with Natural Language Processing to assist in basic health care. Proxy is an AI bot designed to virtually assist in basic first aid and wound analysis. Proxy is a specialized bot that not only assists in basic first aid, but it also uses advanced machine learning techniques like image classification for wound detection and identification. It provides an easy usage by speech recognition techniques as well. Proxy is well designed to help assist people in areas of major crowd and necessity such as toll gates, public places, etc. Proxy provides a medical service which helps assist people as well as entertain people by its movable property controlled by a human.

A Synergy of Innovative Technologies towards Implementing an Autonomous DIY Electric Vehicle for Harvester-Assisting Purposes

The boom in the electronics industry has made a variety of credit card-sized computer systems and plenty of accompanying sensing and acting elements widely available, at continuously diminishing cost and size levels. The benefits of this situation for agriculture are not left unexploited and thus, more accurate, efficient and environmentally-friendly systems are making the scene. In this context, there is an increasing interest in affordable, small-scale agricultural robots. A key factor for success is the balanced selection of innovative hardware and software components, among the plethora being available. This work describes exactly the steps for designing, implementing and testing a small autonomous electric vehicle, able to follow the farmer during the harvesting activities and to carry the fruits/vegetables from the plant area to the truck location. Quite inexpensive GPS and IMU units, assisted by hardware-accelerated machine vision, speech recognition and networking techniques can assure the fluent operation of a prototype vehicle exhibiting elementary automatic control functionality. The whole approach also highlights the challenges for achieving a truly working solution and provides directions for future exploitation and improvements.

A Neural Network Based Arabic Speech Filing System.(Dept.E)

A natural man - machine communication with word processors requires the integration of speech recognition techniques. Speech recognition task is a difficult problem because of its temporal nature and the possibility of the presence of noise. Therefore, it is difficult to extract significant features by using the traditional techniques. Artificial neural networks introduce simple and fast learning techniques for this problem. This paper introduces the application of a three layer neural network for converting spoken Arabic words into Arabic text. We describe a supervised learning method which is based on the well known back propagation technique. The designed network proves itself to be speaker independent and noise immune word spotting guarantees invariance under translation in time. A user friendly word processor should nave intelligent interfaces such as the speaker interface and the hand writing interface. In this work we introduce the first interface. It is expected that neural network based speech-to-text transcription systems should have a significant effect on office

An Enhanced CNN-2D for Audio-Visual Emotion Recognition (AVER) Using ADAM Optimizer

The importance of integrating visual components into the speech recognition process for improving robustness has been identified by recent developments in audio visual emotion recognition (AVER). Visual characteristics have a strong potential to boost the accuracy of current techniques for speech recognition and have become increasingly important when modelling speech recognizers. CNN is very good to work with images. An audio file can be converted into image file like a spectrogram with good frequency to extract hidden knowledge. This paper provides a method for emotional expression recognition using Spectrograms and CNN-2D. Spectrograms formed from the signals of speech it’s a CNN-2D input. The proposed model, which consists of three layers of CNN and those are convolution layers, pooling layers and fully connected layers extract discriminatory characteristics from the representations of spectrograms and for the seven feelings, performance estimates. This article compares the output with the existing SER using audio files and CNN. The accuracy is improved by 6.5% when CNN-2D is used.

A Lightweight Application for Reading Digital Measurement and Inputting Condition Assessment in Manufacturing Industry

There is a vast need for the use of digital display instruments in the manufacturing industry due to the simple operation and high precision. In addition to the numerical data acquisition, it is usually necessary to input additional text for the condition assessment as well. However, since most of these measure instruments do not provide any interfaces for users to access the values and it often lacks proper devices to input the text during the working process, these two tasks are highly human intensive under current conditions. In order to facilitate the smooth running of the work for operators, we propose a lightweight application which can be installed on smartphones or wearable devices using multidigit recognition and speech recognition techniques without changing too much of their workflow. The experimental results demonstrate that our approach can achieve high accuracy. Thus, the proposed solution can effectively resolve data input issues in the manufacturing sites, thereby reducing human labor, increasing productivity, and automating part of the process. Taking operators’ existing workflow into consideration for design can provide an application with an easy learning curve. Moreover, with the rapid and economical approach, companies can financially benefit from the development of this low-cost application, especially for small- and medium-sized enterprises.

음성인식 및 머신러닝 기술을 적용한 모터 소음 진단법에 대한 연구

Motor noise is one of the important quality factors for motor performance. Generally, in a motor production line, motor noise is inspected by a skilled worker. Because motor noise can be caused by a variety of sources or combinations thereof, it is difficult to isolate each specific noise source. An objective/automatic noise-source detection method would be helpful for motor manufacturers. This study introduces a noise-diagnosis method using a sound recognition technique and machine learning methods. First, the raw noise data are filtered through a spectral noise-gate algorithm to reduce the background noise. Then, mel-frequency cepstral coefficient features, which are widely used in the speech-recognition technique, are extracted from the noise data. Finally, a noise-classification model is developed using a support vector machine technique for motor data whose noise sources are known. This model can be used to identify malfunctional motor noises and their sources. Finally, this method was validated by comparing its results with those of frequency analysis from the motor production line.

A Multimodal User Interface for an Assistive Robotic Shopping Cart

This paper presents the research and development of the prototype of the assistive mobile information robot (AMIR). The main features of the presented prototype are voice and gesture-based interfaces with Russian speech and sign language recognition and synthesis techniques and a high degree of robot autonomy. AMIR prototype’s aim is to be used as a robotic cart for shopping in grocery stores and/or supermarkets. Among the main topics covered in this paper are the presentation of the interface (three modalities), the single-handed gesture recognition system (based on a collected database of Russian sign language elements), as well as the technical description of the robotic platform (architecture, navigation algorithm). The use of multimodal interfaces, namely the speech and gesture modalities, make human-robot interaction natural and intuitive, as well as sign language recognition allows hearing-impaired people to use this robotic cart. AMIR prototype has promising perspectives for real usage in supermarkets, both due to its assistive capabilities and its multimodal user interface.

Emotional Speech Recognition using Deep Learning

Emotion speech recognition (SER) is to study the formation and change of speaker’s emotional state from his/her speech signal. The main purpose of this field is to produce a convenient system that is able to effortlessly communicate and interact with humans. The reliability of the current speech emotion recognition systems is far from being achieved. However, this is a challenging task due to the gap between acoustic features and human emotions, which rely strongly on the discriminative acoustic features extracted for a given recognition task. Deep Learning techniques have been recently proposed as an alternative to traditional techniques in SER. In this paper, an overview of Deep Learning techniques that could be used in Emotional Speech recognition is presented. Different extracted features like MFCC as well as feature classifications methods like HMM, GMM, LTSTM and ANN were discussion. Also, the review covers databases used, emotions extracted, contributions made toward speech emotion recognition

Performance of deer hunting optimization based deep learning algorithm for speech emotion recognition

This paper proposes a speech emotion recognition technique based on Optimized Deep Neural Network. The speech signals are denoised by presenting a novel adaptive wavelet transform with a modified galactic swarm optimization algorithm (AWT_MGSO). From the noise removed speech signals, the spectral features like LPC (Linear Prediction Coefficients), MFCC (Mel frequency cepstral coefficients), PSD (power spectral density) and prosodic features like energy, entropy, formant frequencies and pitch are extracted and certain features are selected by ASFO (Adaptive Sunflower Optimization Algorithm). The optimized DNN-DHO (Deep Neural Network with Deer Hunting Optimization Algorithm) is proposed for emotion classification. An enhanced squirrel search algorithm is proposed to update the weight in the optimized DNN_DHO classifier. In this study, all the eight emotions of the speech from RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song) and TESS (Toronto Emotional Speech Set) databases for English and IITKGP-SEHSC (Indian Institute of Technology Kharagpur Simulated Emotion Hindi Speech Corpus) database for Hindi are classified. The experimental results are obtained and compared with the classifiers such as DNN_DHO, DNN (Deep Neural Network) and DAE (Deep Auto Encoder). The experimental results show that the proposed algorithm obtains maximum accuracy as 97.85% by the TESS dataset, 97.14% by the RAVDESS dataset and 93.75% by the IITKGP-SEHSC dataset by the DNN-HHO classifier.

What Makes Online Review Videos Helpful? Evidence from Product Review Videos on Youtube

With the rapid growth and popularity of YouTube, an increasing number of consumers rely on online product review videos to obtain product-related information. Review videos, with visual and sound cues, can convey product experiences in rich and immersive ways and have a powerful impact on consumer’s perceptions and intentions to purchase the product. As the provision of online review videos grows and consumers increasingly rely on them for their purchase decisions, understanding factors that contribute to the perceived helpfulness of video reviews becomes critical for video review management. This paper examines how various video content characteristics—information accessibility, positivity, and subjectivity—and voice characteristics of reviewers—speech rate, pitch, and speech dynamics—are associated with the perceived helpfulness. We collect detailed data on 14,486 electronic product review videos posted on YouTube and employ speech recognition and natural language processing techniques to extract sentimental and speech characteristics. We find that video viewers perceive online product review videos that are more information-accessible, more positive, and less subjective as more helpful. In addition, review videos created by reviewers with faster speech rates, lower voice pitch, and more vocal dynamicity are perceived as more helpful. We additionally find that reviewers’ popularity can moderate the effect of information accessibility, positivity, and reviewers’ voice characteristics on perceived review helpfulness.