Speech Recognition Techniques Research Articles

Deep Learning Backward Recognition Model to Improve Pronunciation Accuracy in the Teaching of Spoken English

The advent of speech recognition algorithms has opened new avenues for enhancing pronunciation accuracy in the teaching of spoken English. As English continues to be a global lingua franca, proficiency in spoken communication holds paramount importance for individuals across diverse professional and academic domains. This paper introduces Forward Backward Recognition Deep Learning (FBRDL), a novel approach aimed at leveraging speech recognition algorithms to enhance pronunciation accuracy in the teaching of spoken English. FBRDL incorporates advanced deep learning techniques such as recurrent neural networks (RNNs) and transformers, which excel in modeling sequential data and capturing long-range dependencies. By leveraging these powerful architectures, FBRDL can effectively handle the inherent variability and complexity of speech signals, enabling robust and accurate recognition even in noisy or adverse environments. Moreover, FBRDL is characterized by its adaptability and scalability, making it well-suited for a wide range of applications across industries. Whether in the realm of virtual assistants, automatic transcription, or voice-controlled devices, FBRDL offers a versatile solution capable of meeting the demands of modern speech recognition tasks. FBRDL integrates principles from deep learning with advanced speech recognition techniques to provide learners with real-time feedback and guidance on their pronunciation. By analyzing spoken English inputs and identifying phonetic discrepancies, FBRDL offers targeted interventions tailored to individual learners' needs. FBRDL achieves an average increase in pronunciation accuracy of 20% compared to traditional teaching methods. Moreover, qualitative assessments underscore the effectiveness of FBRDL in facilitating more precise and efficient acquisition of spoken English skills.

Improving Speech Recognition with Convolutional Neural Networks

This project explores advanced techniques in speech recognition, focusing on emotion identification using Convolutional Neural Networks for improved accuracy and real-time processing efficiency. Emotion recognition from speech signals plays a crucial role in various applications, including human-computer interaction, customer service, mental health monitoring, and entertainment. This project proposes an innovative approach to emotion recognition using Convolutional Neural Networks (CNNs) applied to speech data. By leveraging advanced deep learning techniques, the proposed system aims to accurately identify and classify emotions conveyed through vocal expressions. The project begins with a comprehensive review of existing literature on emotion recognition and speech processing, identifying key challenges and opportunities in the field. Building upon prior research, the project introduces a novel CNN architecture optimized for emotion recognition tasks. This architecture is designed to extract relevant features from speech signals and capture subtle nuances indicative of different emotional states. One of the distinguishing features of the proposed approach is its multi-modal integration, which combines information from both audio and visual modalities to enhance emotion recognition accuracy. In addition to analysing speech signals, the system incorporates visual cues such as facial expressions and gestures, providing a more comprehensive understanding of the speaker's emotional state. Real-time processing efficiency is prioritized in the design of the system, ensuring prompt and responsive emotion recognition in interactive applications. Optimization techniques such as model quantization and lightweight architecture design are employed to minimize computational overhead while maintaining high accuracy. To address the variability and subjectivity of emotional expression, the system incorporates user-specific adaptation mechanisms. Through continuous learning and feedback integration, the system dynamically adapts to individual speakers' speech patterns and emotional characteristics, enhancing its ability to accurately recognize emotions in diverse contexts. The project also explores ensemble learning strategies to improve robustness and generalization performance. By combining predictions from multiple CNN models trained on diverse datasets, the system achieves greater resilience to variations in emotional expression and environmental factors. Ethical considerations, including privacy protection and responsible data handling, are integral aspects of the project's design and implementation. Measures are implemented to ensure the ethical collection, storage, and usage of speech data, safeguarding user privacy and maintaining trust in the system. Overall, the proposed system represents a significant advancement in emotion recognition technology, offering a sophisticated and versatile solution for accurately identifying emotions from speech signals. By leveraging deep learning techniques, multi-modal integration, real-time processing optimization, user-specific adaptation, and ensemble learning, the system demonstrates promising potential for various practical applications requiring robust and context-aware emotion recognition capabilities. Keywords: Speech Recognition, Emotion Identification, Convolutional Neural Networks (CNNs), Real-time Processing, Multi-modal Integration, User-specific Adaptation, Ensemble Learning, Deep Learning, Emotional Expression, Ethical Data Handling

Integrated Intelligent Computing Models for Cognitive-Based Neurological Disease Interpretation in Children: A Survey

INTRODUCTION: This piece of work provides the description of integrated intelligent computing models for the interpretation of cognitive-based neurological diseases in children. These diseases can have a significant impact on children's cognitive and developmental functioning. 
 OBJECTIVES: The research work review the current diagnosis and treatment methods for cognitive based neurological diseases and discusses the potential of machine learning, deep learning, Natural language processing, speech recognition, brain imaging, and signal processing techniques in interpreting the diseases.
 METHODS: A survey of recent research on integrated intelligent computing models for cognitive-based neurological disease interpretation in children is presented, highlighting the benefits and limitations of these models.
 RESULTS: The significant of this work provide important implications for healthcare practice and policy, with strengthen diagnosis and treatment of cognitive-based neurological diseases in children.
 CONCLUSION: This research paper concludes with a discussion of the ethical and legal considerations surrounding the use of intelligent computing models in healthcare, as well as future research directions in this area.

Improving the Performance of Automatic Lip-Reading Using Image Conversion Techniques

Variation in lighting conditions is a major cause of performance degradation in pattern recognition when using optical imaging. In this study, infrared (IR) and depth images were considered as possible robust alternatives against variations in illumination, particularly for improving the performance of automatic lip-reading. The variations due to lighting conditions were quantitatively analyzed for optical, IR, and depth images. Then, deep neural network (DNN)-based lip-reading rules were built for each image modality. Speech recognition techniques based on IR or depth imaging required an additional light source that emitted light in the IR range, along with a special camera. To mitigate this problem, we propose a method that does not use an IR/depth image directly, but instead estimates images based on the optical RGB image. To this end, a modified U-net was adopted to estimate the IR/depth image from an optical RGB image. The results show that the IR and depth images were rarely affected by the lighting conditions. The recognition rates for the optical, IR, and depth images were 48.29%, 95.76%, and 92.34%, respectively, under various lighting conditions. Using the estimated IR and depth images, the recognition rates were 89.35% and 80.42%, respectively. This was significantly higher than for the optical RGB images.

Speech Recognition for Smart AI Devices

This article looks at the progress, challenges and future directions in the field of speech recognition for intelligent AI devices. Speech recognition technology has made significant progress over the years, enabling seamless interactions between humans and artificial intelligence. This article provides an in-depth analysis of modern speech recognition techniques used in smart AI devices and the potential applications and limitations of the technology. It also discusses the current challenges faced by developers and researchers and suggests possible solutions to overcome them. Keywords-Natural language processing Neat vs. scruff, Soft vs. hard computing, Narrow vs general AI, Rule-based approaches, Statistical and machine learning based methods, Deep Neural Networks, Deep learning models for speech recognition, Hybrid and ensemble approaches, Voice biometrics and authentication, Speech translation and language learning, Natural language processing improvement.

An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by Speakers

The use of speakers in electronic devices has become widespread, but the security risks associated with micro-speakers, such as earphones, are often overlooked. Many assume that soundproof barriers can prevent sound leakage and protect privacy. This article presents the prototype MagEar, an eavesdropping system that exploits magnetic side-channel signals leaked by a micro-speaker to restore intelligible human speech. MagEar outperforms some high-precision magnetometers in detecting magnetic fields at the nanotesla level. Even at a distance of 60 cm, it can recover high-quality audio with a 90% similarity to the original audio. Moreover, the MagEar prototype is portable and can be concealed within a headset housing. We have implemented MagEar as a proof-of-concept system and conducted multiple case studies on the eavesdropping of various speaker-embedded devices, including earphones. The recovered speech can be transcribed using automatic speech recognition techniques, even when obstructed by soundproof walls. It is our aspiration that our work can prompt manufacturers to reconsider the security vulnerabilities of speakers.

Real-Time Subtitle Generator for Sinhala Speech

In today’s digital era, the significance of speech recognition technology cannot be overstated as it plays a pivotal role in enabling human-computer interaction and supporting various applications. This paper focuses on the development of a real-time subtitle generator for Sinhala speech using speech recognition techniques. The CMUSphinx toolkit, an open-source toolkit based on the Hidden Markov Model (HMM), is employed for the implementation of the application. Mel-frequency cepstral coefficients (MFCC) are utilized for feature extraction from the given ’wav’ format recordings. The paper places significant emphasis on the importance of a real-time subtitle generator for Sinhala speech and explores the existing literature in the field. It outlines the objectives of the research and discusses the achieved outcomes. By fine-tuning hyperparameters to enhance the recognition accuracy of the system, impressive results of 88.28% training accuracy and 11.72% Word Error Rate (WER) are attained. Thesignificance of this research is underscored by its methodological advancements, robust performance metrics, and the potential impact on facilitating seamless interactions and applications in the Sinhala speech domain.
 Keywords: Speech recognition, Real-time, Subtitle, CMUSphinx, Open source, Hidden Markov Model, Mel-frequency cepstral coefficients, ’wav’, Accuracy, Word Error Rate

Research on the Reform of the Teaching Mode of Rural English Education Assistance Based on the Technical Support of Network Technology

Abstract Under the background of the development of network technology, this paper aims to promote rural English teaching and constructs an English teaching model that combines English recognition technology and rural teaching. The main process of speech recognition is examined by analyzing different speech recognition technologies. Using a deep learning network, an English speech recognition model has been established. Combined with the English acoustic features in the network data, fluency of English speech is evaluated. Data embedding is performed on the English sequences in the network, combined with the sequence probability in the English data, so as to determine whether the English speech is correct or not. The Eval value for the English recognition model based on deep learning is 5.49%, while the test value is 5.89%, as per the results. As the English dataset increases, so does the English recognition technique proposed in this paper, and the accuracy remains above 0.6, and when the dataset is 500, the speech recognition accuracy is 0.8. The teaching model that combines speech recognition techniques with English teaching improves students’ English to a certain extent.

Predicting Transitions from Mild Cognitive Impairment to Dementia within 5‐years

AbstractBackgroundEarly detection of individuals at high risk of future progression toward dementia is in great need, particularly in the era of emerging treatments for Alzheimer’s disease. Given the easier administration of neuropsychological (NP) tests over invasive and expensive biomarker acquisition, we seek to predict the future cognitive status of an individual based on digital voice recordings of NP exams.MethodWe utilized automatic speech recognition and natural language processing techniques to create an automated tool that predicts if an individual with mild cognitive impairment (MCI) will develop dementia in a fixed follow‐up window (5 years). We characterize the predictive accuracy of this approach using data from the Framingham Heart Study. First, the automated transcription of the digital voice recordings were classified into 8 main NP sub‐tests including memory assessment, naming and language skill, verbal fluency, general questions, etc. Using a large language model, only the participants’ sentences were encoded into quantitative data. These data and the participants’ characteristics, such as age, sex, ApoE4 (+/‐) genotype, and education were employed to train and test a binary classifier, predicting dementia withing 5 years after a positive MCI diagnosis. We also provide a comparison between our approach and alternative models using demographics and ApoE genotype or the Mini‐Mental State Examination (MMSE) score.ResultWe evaluated the performance of the classification task on the data containing 111 stable MCI and 103 progressive MCI subjects. Our method achieved an average accuracy of 75.5% on the held‐out test data while a baseline model using age, sex, education, and ApoE has an average accuracy of 71.8%. The sensitivity and specificity of the proposed method versus the baseline model are 78.2% vs. 81.8% and 72.7% vs. 61.8%, respectively. Our approach also significantly outperforms the model based on the MMSE score by a margin of 14% in accuracy.ConclusionWe have analyzed digital voice recordings of NP exams for participants with MCI to predict their cognitive status in 5 years. The proposed approach offers a fully automated procedure with higher accuracy, providing an opportunity to develop an accessible and easy‐to‐administer screening tool that could be easily adapted to any language.

Evaluation of Interpretable Speech Biomarkers for Monitoring Alzheimer’s Disease and Mild Cognitive Impairment Progression

AbstractBackgroundAlzheimer’s disease (AD) is a progressive neurodegenerative disease. Like for other dementias, biomarkers may help characterize distinct aspects of the underlying pathology, predict decline, and monitor disease progression. We extracted a composite array of 24 speech‐based biomarkers using machine learning and signal processing techniques. We then determined which of the given biomarkers was predictive of MCI and AD progression from baseline by conducting a correlation analysis.MethodSpoken responses to a Cookie Theft Picture description task were recorded from 2 subjects with AD, 4 subjects with Mild Cognitive Impairment (MCI) due to AD (biomarker confirmed AD), and 4 subjects with MCI. We automatically extracted acoustic, linguistic, and cognitive biomarkers using speech recognition, acoustic, and language modeling techniques. We later computed Kendall’s tau‐b (tb) correlation between the biomarker values and Montreal Cognitive Assessment (MoCA) scores. Recordings and MoCA scores were collected at baseline, at 6 months after baseline, and at 12 months for a subgroup of the subjects. To perform the correlation analysis, we used scipy.stats.kendalltau library in Python.ResultWith respect to the acoustic biomarkers, a strong negative correlation (tb = ‐0.50) was observed between the MoCA scores and features such as pause time, pause percentage, and pause speech ratio, while a moderate positive correlation (tb = 0.2) was found for rhythm standard deviation. With respect to the linguistic biomarkers a strong positive correlation(tb = 0.50) for the features of corrected type‐token ratio, average sentence length in words, and noun count was reported, while a moderate positive correlation (tb = 0.27) was found for the features of word type count, word token count, and moving average type‐token ratio. The other acoustic, linguistic, and cognitive biomarkers showed a very weak (tb = 0.00 ‐ 0.10) or weak correlation (tb = 0.10 ‐ 0.20).ConclusionAltogether, these results suggest that speech‐based interpretable biomarkers may help clinicians to diagnose AD at earlier stages and monitor disease progression. Our preliminary data suggest that AD patients encounter more problems delivering longer and linguistically elaborated narratives as the disease progresses.

Evaluation of Interpretable Speech Biomarkers for Monitoring Alzheimer’s Disease and Mild Cognitive Impairment Progression

AbstractBackgroundAlzheimer’s disease (AD) is a progressive neurodegenerative disease. Like for other dementias, biomarkers may help characterize distinct aspects of the underlying pathology, predict decline, and monitor disease progression. We extracted a composite array of 24 speech‐based biomarkers using machine learning and signal processing techniques. We then determined which of the given biomarkers was predictive of MCI and AD progression from baseline by conducting a correlation analysis.MethodSpoken responses to a Cookie Theft Picture description task were recorded from 2 subjects with AD, 4 subjects with Mild Cognitive Impairment (MCI) due to AD (biomarker confirmed AD), and 4 subjects with MCI. We automatically extracted acoustic, linguistic, and cognitive biomarkers using speech recognition, acoustic, and language modeling techniques. We later computed Kendall’s tau‐b (τb ) correlation between the biomarker values and Montreal Cognitive Assessment (MoCA) scores. Recordings and MoCA scores were collected at baseline, at 6 months after baseline, and at 12 months for a subgroup of the subjects. To perform the correlation analysis, we used scipy.stats.kendalltau library in Python.ResultWith respect to the acoustic biomarkers, a strong negative correlation (τb= ‐0.50) was observed between the MoCA scores and features such as pause time, pause percentage, and pause speech ratio, while a moderate positive correlation (τb= 0.2) was found for rhythm standard deviation. With respect to the linguistic biomarkers a strong positive correlation(τb= 0.50) for the features of corrected type‐token ratio, average sentence length in words, and noun count was reported, while a moderate positive correlation (τb= 0.27) was found for the features of word type count, word token count, and moving average type‐token ratio. The other acoustic, linguistic, and cognitive biomarkers showed a very weak (τb= 0.00 ‐ 0.10) or weak correlation (τb= 0.10 ‐ 0.20).ConclusionAltogether, these results suggest that speech‐based interpretable biomarkers may help clinicians to diagnose AD at earlier stages and monitor disease progression. Our preliminary data suggest that AD patients encounter more problems delivering longer and linguistically elaborated narratives as the disease progresses.

A Review of Feature Extraction and Classification Techniques in Speech Recognition

A Review of Feature Extraction and Classification Techniques in Speech Recognition

Deep Neural Network-based Mixed Speech Recognition Technology for Chinese and English

In the field of human-computer interaction, the current more advanced speech recognition systems are all single speech recognition, and it is urgent to adopt new in-depth learning technology to improve the existing speech recognition system. In this context, this research is based on DNN and investigates mixed speech recognition techniques for both Chinese and English. A single speech recognition algorithm based on DNN is first investigated, and then a new hybrid Chinese and English speech recognition model is constructed by fusing the attention mechanism and CTC loss function. In the construction of the hybrid speech recognition model, the end-to-end model and Transformer framework are used to combine the monotonic alignment property of the CTC loss function, which allows complex sound units to be transformed into characters for easy extraction and recognition. The performance of the constructed models was tested on Chinese speech dataset, English speech dataset and mixed Chinese and English speech dataset to determine the recognition accuracy and speed of the models. The results show that the proposed recognition model achieves 81.2% recognition accuracy and 100 recognition speed/minute on the Chinese-English mixed speech dataset, which is much better than the other three models. This study successfully addresses the need for improved speech recognition systems by introducing a novel hybrid model for mixed Chinese-English speech recognition. The experimental results confirm the superiority of the proposed model, achieving high accuracy and rapid recognition speed. The developed model holds promising potential for enhancing human-computer interaction and enabling efficient communication between Chinese and English speakers.

Influence of Overpressure Breathing on Vowel Formant Frequencies

Voice controlled management systems are based on speech recognition techniques. The use of such systems in combat aircraft is complex due to a number of critical factors which affect the accuracy of speech recognition, such as high level of ambient noise and vibration, use of oxygen masks, serious psycho-physical stress of speakers, etc. One of the specificity of the oxygen mask application is overpressure breathing. The results of the simulations presented in this paper show that the presence of overpressure on the order of 1000 Pa in the vocal tract has a significant influence on the first two formant frequencies. The formants discrimination field is significantly reduced when oxygen mask is used, influencing both perceptive and automatic discrimination of spoken vowels.

Provision of helpful review videos: Effects of video characteristics on perceived helpfulness

With the rapid growth and popularity of YouTube, an increasing number of consumers rely on online product review videos to obtain product‐related information. As the provision of online review videos grows and consumers increasingly rely on them for their purchase decisions, understanding factors that contribute to the perceived helpfulness of video reviews becomes critical for video review management. This paper examines how various visual and vocal characteristics of online review videos are associated with the perceived helpfulness of videos. We collect detailed observational data on 13,840 electronic product review videos posted on YouTube and employ video content analysis, speech recognition, and natural language processing techniques to extract the visual and vocal characteristics of review videos. By using econometric models, we find that the increase in visual stimulation, captured by brightness and visual dynamics, increases the perceived helpfulness of reviews. In addition, featuring reviewers’ faces in review videos increases the perceived helpfulness of videos. Consumers also perceive review videos in which reviewers express more positive facial emotions as more helpful. Furthermore, lower voice pitch and faster speech rates are associated with higher perceived helpfulness of reviews. To complement the empirical analysis and further isolate the causal effects of review brightness and pitch, we conduct controlled experiments. Overall, the findings can facilitate the management and operation of online review videos for product reviewers, businesses, review platforms, and consumers. In particular, the findings provide direct and actionable guidance to content generators who aim to create more helpful product reviews.

An approach for Correcting the Word-level Mispronunciations for non-native English-speaking Indian Children

Speech Recognition and its potential applications in terms of “talking devices” have become indispensable in today’s world. Technological advances like mobiles, smart home assistants or tablets extensively use the techniques of automatic speech recognition that works good for adults but cannot always follow and understand children’s speech. The primary goal of this paper is to bridge the gap of communication between voice assistants and Indian children speaking English as secondary language. The issue of lack of children’s speech corpora with English as non-native language, is addressed by creating a dataset of children in the age group of 5-15 years, speaking Hindi or Marathi as their mother tongue and English as their second language. The analysis and implementation of the proposed work shows the accuracy of approximately 96% and potential for further scope by increasing the size of dataset in lower age group. The key contributions of our work are (i) creating speech dataset of Indian children whose mother-tongue is Hindi or Marathi, (ii) employing and evaluating hybrid Convolutional Neural Network (CNN) as an age classifier, (iii) language modeling to customize children vocabulary, (iv) checking accuracy and performance of the system.

Construction of English Speech Recognition Model by Fusing CNN and Random Deep Factorization TDNN

In current society, speech recognition can perform a variety of functions, such as completing voice commands, enabling speech processing, spoken language translation and facilitating communication. Therefore, the study of speech recognition technology is of high value. However, current speech recognition techniques focus on among clearly expressed spoken words, which poses great challenges for recognition with spoken pronunciation or dialect pronunciation. Some scholars currently use a model combining time-delay neural networks and long and short-term memory networks to build speech recognition systems, but the performance in acoustic recognition is poor. Therefore, the study proposes a convolutional neural network (CNN), time-delay neural network (TDNN) and output-gate projected Gated recurrent by analyzing the deep neural network unit (OPGRU) combined with a composite English speech recognition model. The model can optimize the acoustic model after the introduction of CNN, and the model can accurately recognize pronunciation features and make the model have a wider recognition range. The proposed composite model is compared with the Word error rate (Wer) and runtime metrics in the Mozilla Common Voice dataset. The Wer result of the composite model is 23.42% and the running time is 1418 s. The Wer result of the composite model is 24.61% and the running time is 1385 s. Compared with the TDNN-OPGRU model, the Wer of the composite model decreases by 1.19% but the running time increases by 33 s. The accuracy of the composite model is higher than that of the TDNN-OPGRU model. From a comprehensive consideration, the speech recognition model accuracy has higher priority, so the composite model proposed in the study has better performance.

An automatic speech recognition system in Indian and foreign languages: A state-of-the-art review analysis

Speech Recognition is one of the prominent research topics in the field of Natural Language Processing (NLP). The Speech Recognition technique removes the barriers and makes the system ease for inter-communication between human beings and devices. The aim of this study is to analyze the Automatic Speech Recognition System (ASRS) proposed by different researchers using Machine learning and Deep Learning techniques. In this work, Indian and foreign languages speech recognition systems like Hindi, Marathi, Malayalam, Urdu, Sanskrit, Nepali, Kannada, Chinese, Japanese, Arabic, Italian, Turkish, French, and German are considered. An integrated framework is presented and elaborated with recent advancement. The various platform like Hidden Markov Model Toolkit (HMM Toolkit), CMU Sphinx, Kaldi toolkit are explained which is used for building the speech recognition model. Further, some applications are elaborated which depict the uses of ASRS.

A Perspective Study on Speech Recognition

Emotions play an extremely important role in human mental life. It is a medium of expression of one’s perspective or one’s mental state to others. Speech Emotion Recognition (SER) can be defined as extraction of the emotional state of the speaker from his or her speech signal. There are few universal emotions including Neutral, Anger, Happiness, and Sadness in which any intelligent system with finite computational resources can be trained to identify or synthesize as required. In this work spectral and prosodic features are used for speech emotion recognition because both of these features contain the emotional information. Mel-Frequency Cepstral Coefficients (MFCC) is one of the spectral features. Fundamental frequency, loudness, pitch and speech intensity and glottal parameters are the prosodic features which are used to model different emotions. The potential features are extracted from each utterance for the computational mapping between emotions and speech patterns. Pitch can be detected from the selected features, using which gender can be classified. The audio signal is filtered using a method known as feature extraction technique. In this article, the feature extraction technique for speech recognition and voice classification is analyzed and also centered to comparative analysis of different types of Mel frequency cepstral coefficients (MFCC) feature extraction method. The MFCC technique is used for deduction of noise in voice signals and also used for voice classification and speaker identification. The statistical results of the different MFCC techniques are discussed and finally concluded that the delta-delta MFCC feature extraction technique is better than the other feature extraction techniques..

Noise-Separated Adaptive Feature Distillation for Robust Speech Recognition

This letter makes an improvement on feature-based knowledge distillation for robust speech recognition. The use of distillation techniques in speech recognition has been demonstrated to improve the robustness of the system. In this letter, we propose a noise-separated adaptive feature distillation method, including an adaptive distillation position selection strategy and a noise separation mechanism, assuming that there is a common network structure between the student and teacher. The proposed method has two improvements. First, distillation positions can be adaptively selected in each iteration by comparing loss values computed on intermediate representations of the student and the teacher, increasing the flexibility of knowledge transfer during distillation. Second, a noise separation module is proposed to constrain noise information elimination by explicitly separating the speech information and the noise information in noisy speech, which reduces the interference of noise information during distillation. Therefore, a better recognition performance is demonstrated with the proposed method compared to the standardized feature-based knowledge distillation method.