Human Speech Recognition Research Articles

Interference of mid-level sound statistics underlie human speech recognition sensitivity in natural noise.

Recognizing speech in noise, such as in a busy restaurant, is an essential cognitive skill where the task difficulty varies across environments and noise levels. Although there is growing evidence that the auditory system relies on statistical representations for perceiving 1-5 and coding4,6-9 natural sounds, it's less clear how statistical cues and neural representations contribute to segregating speech in natural auditory scenes. We demonstrate that human listeners rely on mid-level statistics to segregate and recognize speech in environmental noise. Using natural backgrounds and variants with perturbed spectro-temporal statistics, we show that speech recognition accuracy at a fixed noise level varies extensively across natural backgrounds (0% to 100%). Furthermore, for each background the unique interference created by summary statistics can mask or unmask speech, thus hindering or improving speech recognition. To identify the neural coding strategy and statistical cues that influence accuracy, we developed generalized perceptual regression, a framework that links summary statistics from a neural model to word recognition accuracy. Whereas a peripheral cochlear model accounts for only 60% of perceptual variance, summary statistics from a mid-level auditory midbrain model accurately predicts single trial sensory judgments, accounting for more than 90% of the perceptual variance. Furthermore, perceptual weights from the regression framework identify which statistics and tuned neural filters are influential and how they impact recognition. Thus, perception of speech in natural backgrounds relies on a mid-level auditory representation involving interference of multiple summary statistics that impact recognition beneficially or detrimentally across natural background sounds.

An Integrated All-Natural Conductive Supramolecular Hydrogel Wearable Biosensor with Enhanced Biocompatibility and Antibacterial Properties.

Conductive hydrogels exhibit tremendous potential for wearable bioelectronics, biosensing, and health monitoring applications, yet concurrently enhancing their biocompatibility and antimicrobial properties remains a long-standing challenge. Herein, we report an all-natural conductive supramolecular hydrogel (GT5-DACD2-B) prepared via the Schiff base reaction between the biofriendly dialdehyde cyclodextrin and gelatin. The potent antibacterial agent fusidic acid (FA) is incorporated through host-guest inclusion, enabling 100% inhibition of Staphylococcus aureus proliferation. The biocompatibility of our hydrogel is bolstered with tannic acid (TA) facilitating antibacterial effects through interactions with gelatin, while borax augments conductivity. This supramolecular hydrogel not only exhibits stable conductivity and rapid response characteristics but also functions as a flexible sensor for monitoring human movement, facial expressions, and speech recognition. Innovatively integrating biocompatibility, antimicrobial activity, and conductivity into a single system, our work pioneers a paradigm for developing multifunctional biosensors with integrated antibacterial functionalities, paving the way for advanced wearable bioelectronics with enhanced safety and multifunctionality.

Comparing human and machine speech recognition in noise with QuickSIN.

A test is proposed to characterize the performance of speech recognition systems. The QuickSIN test is used by audiologists to measure the ability of humans to recognize continuous speech in noise. This test yields the signal-to-noise ratio at which individuals can correctly recognize 50% of the keywords in low-context sentences. It is argued that a metric for automatic speech recognizers will ground the performance of automatic speech-in-noise recognizers to human abilities. Here, it is demonstrated that the performance of modern recognizers, built using millions of hours of unsupervised training data, is anywhere from normal to mildly impaired in noise compared to human participants.

Comprehensive multiparametric analysis of human deepfake speech recognition

In this paper, we undertake a novel two-pronged investigation into the human recognition of deepfake speech, addressing critical gaps in existing research. First, we pioneer an evaluation of the impact of prior information on deepfake recognition, setting our work apart by simulating real-world attack scenarios where individuals are not informed in advance of deepfake exposure. This approach simulates the unpredictability of real-world deepfake attacks, providing unprecedented insights into human vulnerability under realistic conditions. Second, we introduce a novel metric to evaluate the quality of deepfake audio. This metric facilitates a deeper exploration into how the quality of deepfake speech influences human detection accuracy. By examining both the effect of prior knowledge about deepfakes and the role of deepfake speech quality, our research reveals the importance of these factors, contributes to understanding human vulnerability to deepfakes, and suggests measures to enhance human detection skills.

Biobased, degradable and directional porous carboxymethyl chitosan/lignosulfonate sodium aerogel-based piezoresistive pressure sensor with dual-conductive network for human motion detection

The rapid growth of wearable electronics, healthcare monitoring, and human–computer interaction has sparked growing interest in flexible piezoresistive pressure sensors, which in turn raised considerable concerns about the increasingly serious environmental pollution caused by electronic waste. Herein, a radial inward directional freezing-drying method was proposed to prepare biobased, degradable and directional porous carboxymethyl chitosan/lignosulfonate sodium aerogel with dual-conductive network constructed by carboxylated multiwalled carbon nanotubes (C-CNTs) and MXene for piezoresistive pressure sensor. Owing to the synergistic conductive network of C-CNTs and MXene as well as directional porous structure, the sensor exhibited high sensitivity (2.33 kPa−1 in 0–10 kPa, 1.04 kPa−1 in 10–31 kPa, 0.53 kPa−1 in 31–53 kPa), fast response (response/recovery time of 140/60 ms), and excellent repeatability (2,000 loading–unloading cycles). Moreover, the sensor was successfully applied for human motion detection, healthy monitoring, micro-expression identification, and speech recognition. In addition, the aerogel for sensor was able to be completely degraded within 70 h in 1 M hydrochloric acid solution or partially degraded in boiling water within 2.5 h for the recycling of conductive materials, which was beneficial for not only environment protection but also saving resource. Our findings conceivably stand out as a new strategy to prepare environmentally friendly and high-performance piezoresistive pressure sensor and will promote the further development and application of flexible electronics.

Ultra-Stretchable Composite Organohydrogels Polymerized Based on MXene@Tannic Acid-Ag Autocatalytic System for Highly Sensitive Wearable Sensors.

Conductive hydrogels have attracted widespread attention in the fields of biomedicine and health monitoring. However, their practical application is severely hindered by the lengthy and energy-intensive polymerization process and weak mechanical properties. Here, a rapid polymerization method of polyacrylic acid/gelatin double-network organohydrogel is designed by integrating tannic acid (TA) and Ag nanoparticles on conductive MXene nanosheets as catalyst in a binary solvent of water and glycerol, requiring no external energy input. The synergistic effect of TA and Ag NPs maintains the dynamic redox activity of phenol and quinone within the system, enhancing the efficiency of ammonium persulfate to generate radicals, leading to polymerization within 10min. Also, ternary composite MXene@TA-Ag can act as conductive agents, enhanced fillers, adhesion promoters, and antibacterial agents of organohydrogels, granting them excellent multi-functionality. The organohydrogels exhibit excellent stretchability (1740%) and high tensile strength (184kPa). The strain sensors based on the organohydrogels exhibit ultrahigh sensitivity (GF=3.86), low detection limit (0.1%), and excellent stability (>1000 cycles, >7 days). These sensors can monitor the human limb movements, respiratory and vocal cord vibration, as well as various levels of arteries. Therefore, this organohydrogel holds potential for applications in fields such as human health monitoring and speech recognition.

Self-healing and reprocessable biobased non-isocyanate polyurethane elastomer with dual dynamic covalent adaptive network for flexible strain sensor

Owing to the nontoxicity of monomers, non-isocyanate polyurethane elastomers have attracted considerable attention, yet the synthesis and functionalization are still challenging. Herein, self-healing and reprocessable biobased non-isocyanate polyurethane (NIPU) elastomers with dual dynamic covalent adaptive network (DDCAN) by disulfide bonds and dynamic imine bonds were synthesized as substrate for the construction of flexible strain sensor with MXene as conductive substance. The tensile strength and elongation at break of the NIPU elastomer were 3.22 MPa and 234 %, respectively. Thanks to the formation of DDCAN, the NIPU elastomer showed high healing efficiency of 97.5 % after healing at room temperature for 24 h. Interestingly, the NIPU elastomer possessed superior reprocessing capability and the tensile strength after three reprocessing cycles reached 136.1 % of the original one, which was attributed to the increase of crosslinking points caused by topological rearrangement. In addition, the NIPU elastomer-based flexible strain sensor exhibited fast response (response time = 60 ms) and excellent repeatability (1000 bending-releasing cycles) and was successfully applied for human motion detection and speech recognition. The findings in this work conceivably stand out as a new methodology for the preparation of biobased and functional elastomers, which will greatly promote the sustainable development and application of flexible electronics.

A perceptual similarity space for speech based on self-supervised speech representations.

Speech recognition by both humans and machines frequently fails in non-optimal yet common situations. For example, word recognition error rates for second-language (L2) speech can be high, especially under conditions involving background noise. At the same time, both human and machine speech recognition sometimes shows remarkable robustness against signal- and noise-related degradation. Which acoustic features of speech explain this substantial variation in intelligibility? Current approaches align speech to text to extract a small set of pre-defined spectro-temporal properties from specific sounds in particular words. However, variation in these properties leaves much cross-talker variation in intelligibility unexplained. We examine an alternative approach utilizing a perceptual similarity space acquired using self-supervised learning. This approach encodes distinctions between speech samples without requiring pre-defined acoustic features or speech-to-text alignment. We show that L2 English speech samples are less tightly clustered in the space than L1 samples reflecting variability in English proficiency among L2 talkers. Critically, distances in this similarity space are perceptually meaningful: L1 English listeners have lower recognition accuracy for L2 speakers whose speech is more distant in the space from L1 speech. These results indicate that perceptual similarity may form the basis for an entirely new speech and language analysis approach.

Exploring emergent syllables in end-to-end automatic speech recognizers through model explainability technique

Automatic speech recognition systems based on end-to-end models (E2E-ASRs) can achieve comparable performance to conventional ASR systems while reproducing all their essential parts automatically, from speech units to the language model. However, they hide the underlying perceptual processes modelled, if any, and they have lower adaptability to multiple application contexts, and, furthermore, they require powerful hardware and an extensive amount of training data. Model-explainability techniques can explore the internal dynamics of these ASR systems and possibly understand and explain the processes conducting to their decisions and outputs. Understanding these processes can help enhance ASR performance and reduce the required training data and hardware significantly. In this paper, we probe the internal dynamics of three E2E-ASRs pre-trained for English by building an acoustic-syllable boundary detector for Italian and Spanish based on the E2E-ASRs’ internal encoding layer outputs. We demonstrate that the shallower E2E-ASR layers spontaneously form a rhythmic component correlated with prominent syllables, central in human speech processing. This finding highlights a parallel between the analysed E2E-ASRs and human speech recognition. Our results contribute to the body of knowledge by providing a human-explainable insight into behaviours encoded in popular E2E-ASR systems.

A Comprehensive Review on Machine Learning Approaches for Enhancing Human Speech Recognition

A Comprehensive Review on Machine Learning Approaches for Enhancing Human Speech Recognition

Disposable, strain-insensitive, and room-temperature-operated flexible humidity and VOC sensor with enhanced sensitivity and selectivity through interface control

As people are paying more attention to the health and comfort of the indoor environment, the sensors detecting the relative humidity (RH) and volatile organic compounds (VOCs) have attracted widely research interests. However, how to improve the sensitivity and selectivity remains a challenge, especially for flexible sensors working at room temperature (around 25 ℃). In this work, the controlling of solid-gas interface was found to be able to enhance the sensitivity and the manipulation of solid-solid interface increased the selectivity of the sensor for VOC detection. The flexible sensor based on electrospinning cellulose (EC) show high sensitivity for humidity detection due to its large solid-gas interface, which can be used for real-time monitoring of human breathing and speech recognition, simultaneously, it can also detect a variety of VOCs. By introducing silver nanowires (AgNWs) to establish solid-solid interface between silver and expanded graphite (EG), the selectivity of the sensor for VOCs detection was improved, which shown higher sensitivity for EtOH with faster response/recovery time, as well as long-term cycle stability and bending insensitivity. Furthermore, the sensor also can be decomposed by burning, offering a sustainable disposal method. This versatile, low-cost and environmentally friendly flexible electronic device may be used for wearable air monitoring and environmental controlling.

Reliable and Scalable Piezoresistive Sensors with an MXene/MoS2 Hierarchical Nanostructure for Health Signals Monitoring.

The increased popularity of wearable electronic devices has led to a greater need for advanced sensors. However, fabricating pressure sensors that are flexible, highly sensitive, robust, and compatible with large-scale fabrication technology is challenging. This work investigates a piezoresistive sensor constructed from an MXene/MoS2 hierarchical nanostructure, which is obtained through an easy and inexpensive fabrication process. The sensor exhibits a high sensitivity of 0.42 kPa-1 (0-1.5 kPa), rapid response (∼36 ms), and remarkable mechanical durability (∼10,000 cycles at 13 kPa). The sensor has been demonstrated to be successful in detecting human motion, speech recognition, and physiological signals, particularly in analyzing human pulse. These data can be used to alert and identify irregularities in human health. Additionally, the sensing units are able to construct sensor arrays of various sizes and configurations, enabling pressure distribution imaging in a variety of application scenarios. This research proposes a cost-effective and scalable approach to fabricating piezoresistive sensors and sensor arrays, which can be utilized for monitoring human health and for use in human-machine interfaces.

Multi-Accent Speaker Detection Using Normalize Feature MFCC Neural Network Method

Speaker recognition is a field of research that continues to this day. Various methods have been developed to detect the human voice with greater precision and accuracy. Research on human speech recognition that is quite challenging is accent recognition. Detecting various types of human accents with different accents and ethnicities with high accuracy is a research that is quite difficult to do. According to the results of the research on the data preprocessing stage, feature extraction and selection of the right classification method play a very important role in determining the accuracy results. This study uses a preprocessing approach with normalizing features combined with MFCC as a method to perform feature extraction and the neural network (NN), which is a classification method that works based on the workings of the human brain. Research results obtained using the normalize feature with MFCC and neural network for multiaccent speaker recognition, the accuracy performance reaches 82.68%, precision is 83% and recall is 82.88%.

High-Linearity Flexible Pressure Sensor Based on the Gaussian-Curve-Shaped Microstructure for Human Physiological Signal Monitoring.

Flexible pressure sensors with high-performance show broad application prospects in health monitoring, wearable electronic devices, intelligent robot sensing, and other fields. Although flexible pressure sensors have made significant progress in sensitivity and detection range, most of them still exhibit strong nonlinearity, which leads to significant troubles in signal acquisition and thus limits their popularity in practical applications. It remains a serious challenge for the flexible pressure sensor to achieve high linearity while maintaining high sensitivity. Herein, a doped sensing membrane with a uniformly distributed Gaussian-curve-shaped micropattern array was developed using the micro-electromechanical systems (MEMS) process, and a flexible sensor structure with the doped film as the core was designed and constructed. The prototype sensor has a high sensitivity of 1.77 kPa-1 and a linearity of 0.99 in the full detection range of 20 Pa to 30 kPa. In addition, its excellent performance also includes fast response/recovery times (∼25/50 ms) and long-term endurance (>10,000 cycles at 15 kPa). The prototype sensor has been successfully demonstrated in human pulse monitoring, speech recognition, and gesture recognition. The 2 × 6 sensor array can detect the spatial pressure distribution. Thus, such a microstructure shape design will open a new way to fabricate a high-linearity pressure sensor for potential applications in health monitoring, human-machine interaction, etc.

ICA and IVA bounded multivariate generalized Gaussian mixture based hidden Markov models

Machine learning (ML), a branch of artificial intelligence (AI), is an area of computational science that is concerned with the analysis and interpretation of patterns and structures in data to enable learning and decision-making without the participation of a human. Hidden Markov models (HMMs), which have been acknowledged for decades but have recently made a significant revival in machine learning, are one of the most impressively powerful probabilistic models. HMMs are frequently employed in machine learning to model heterogeneous time series data. In this paper, we integrate independent component analysis (ICA) and ICA with a bounded multivariate generalized Gaussian mixture model (ICA-BMGGMM) into the HMM approach. One limitation of ICA is that it assumes the sources to be independent from each other. This assumption can be relaxed by combining independent vectors analysis (IVA) and IVA with the BMGGMM (IVA-BMGGMM) into the HMM approach to improve their modeling capability. We validate our proposed models using a variety of applications, such as human action recognition, speech recognition, and energy disaggregation. The results presented in the paper demonstrate the effectiveness of the proposed approaches for modeling different types of data. These data include KTH and Weizmann datasets for human action recognition, TIMIT and SDR for speech recognition, REDD dataset for energy disaggregation and EEG dataset for elliptic seizure classification. For all conducted experiments, our proposed models outperform other comparing models for all performance metrics such as accuracy, sensitivity, and precision. The best detection results were found using the IVABMGGMM-HMM for the reported experiments.

Recognition Of Chicken Species Through Sound Using Weierstrass Transform

Voice recognition is an applied technique in the field of digital signal processing that has been widely used, such as technology in the field of telecommunications which is now able to provide data transmission services not only text but can also serve data transmission using voice. Speech recognition studies to date have only focused on human speech recognition, so it is important to develop research on speech recognition in animals. In this research a speech recognition system will be developed for balenggek, pelung and bekisar ornamental chickens using the Delphi 7 programming language and the weiertrass transform method. The performance evaluation measurement of the speech recognition system in this study uses two parameters, namely the detection rate and the false positive rate. The results showed that the speech recognition system for ornamental chickens using the weierstrass transform method had an average detection rate of 83.00% and an average false positive rate of 16%.

A Smart Metasurface for Electromagnetic Manipulation Based on Speech Recognition

In this work, we propose and realize a smart metasurface for programming electromagnetic (EM) manipulations based on human speech recognition. The smart metasurface platform is composed of a digital coding metasurface, a speech-recognition module, a single-chip computer, and a digital-to-analog converter (DAC) circuit, and can control EM waves according to pre-stored voice commands in a smart way. The constructed digital metasurface contains 6 × 6 super unit cells, each of which consists of 4 × 4 active elements with embedded varactor diodes. Together with the DAC and single-chip computer, the speech-recognition module can recognize voice commands and generate corresponding voltage sequences to control the metasurface. In addition, a genetic algorithm (GA) is adopted in the design of the metasurface for efficiently optimizing the phase distributions. To verify the performance of the smart metasurface platform, three typical functions are demonstrated: radar cross-section reduction, vortex beam generation, and beam splitting. The proposed strategy may offer a new avenue for controlling EM waves and establishing a link between EM and acoustic communications.

Deep Learning for the Recognition of Human Speech

Abstract—Humans primarily communicate through speech, and language is their main means of doing so. Emotion is crucial to social connection. Knowing how to recognise the emotions in a speech is important and challenging given that we are dealing with human-machine interaction. Depending on the mood a person is going through, they will express themselves differently. When someone expresses their emotions, each one has a distinctive energy, pitch, and tone fluctuation that is categorised according to the situation. The identification of spoken emotions is thus a future goal for computer vision. The objective of our project is to develop intelligent speechthat utilises a convolutional neural network to recognise emotions. which uses a variety of modules for emotion recognition and classifier. Keywords— Human speech, Emotion, recognition of speech, GMM, Mel-Frequency-Cepstrum Coefficient (MFCC), linear prediction Cepstral coefficients (LPCC) , (LSTM)

Machine learning for Arabic phonemes recognition using electrolarynx speech

<p><span lang="EN-US">Automatic speech recognition system is one of the essential ways of interaction with machines. Interests in speech based intelligent systems have grown in the past few decades. Therefore, there is a need to develop more efficient methods for human speech recognition to ensure the reliability of communication between individuals and machines. This paper is concerned with Arabic phoneme recognition of electrolarynx device. Electrolarynx is a device used by cancer patients having vocal laryngeal cords removed. Speech recognition here is considered to find the preferred machine learning model that can classify phonemes produced by electrolarynx device. The phonemes recognition employs different machine learning schemes, including convolutional neural network, recurrent neural network, artificial neural network (ANN), random forest, extreme gradient boosting (XGBoost), and long short-term memory. Modern standard Arabic is utilized for testing and training phases of the recognition system. The dataset covers both an ordinary speech and electrolarynx device speech recorded by the same person. Mel frequency cepstral coefficients are considered as speech features. The results show that the ANN machine learning method outperformed other methods with an accuracy rate of 75%, a precision value of 77%, and a phoneme error rate (PER) of 21.85%.</span></p>

Emotion recognition of human speech using deep learning method and MFCC features

Subject matter: Speech emotion recognition (SER) is an ongoing interesting research topic. Its purpose is to establish interactions between humans and computers through speech and emotion. To recognize speech emotions, five deep learning models: Convolution Neural Network, Long-Short Term Memory, Artificial Neural Network, Multi-Layer Perceptron, Merged CNN, and LSTM Network (CNN-LSTM) are used in this paper. The Toronto Emotional Speech Set (TESS), Surrey Audio-Visual Expressed Emotion (SAVEE) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) datasets were used for this system. They were trained by merging 3 ways TESS+SAVEE, TESS+RAVDESS, and TESS+SAVEE+RAVDESS. These datasets are numerous audios spoken by both male and female speakers of the English language. This paper classifies seven emotions (sadness, happiness, anger, fear, disgust, neutral, and surprise) that is a challenge to identify seven emotions for both male and female data. Whereas most have worked with male-only or female-only speech and both male-female datasets have found low accuracy in emotion detection tasks. Features need to be extracted by a feature extraction technique to train a deep-learning model on audio data. Mel Frequency Cepstral Coefficients (MFCCs) extract all the necessary features from the audio data for speech emotion classification. After training five models with three datasets, the best accuracy of 84.35 % is achieved by CNN-LSTM with the TESS+SAVEE dataset.