Speech Feature Extraction Research Articles

Speech recognition inspired features for acoustic emission

For the application of acoustic emission methods in the field of condition monitoring, extracting meaningful information out of signals is an essential step in signal processing and mandatory for machine learning. However, when dealing with acoustic emission signals, the computational intensity of extracting features is becoming more important due to its large amount of data. The methodology in this study, hence, investigates established methods from descriptive statistics as typically used in acoustic emission and techniques commonly employed in speech recognition. Thereby, the objective is to improve the feature extraction of acoustic emission signals, emphasizing the synergies between the methods of both approaches. Extracting traditional acoustic emission features based on descriptive statistics alone usually yields robust features capturing the significant patterns within the time and frequency domain. However, traditional acoustic emission features may neglect more detailed information present in the signals due to averaging or mere evaluation of extreme values. To address this limitation, the methodology is extended by incorporating techniques from the field of speech recognition, such as applying window functions and digital filter banks to calculate the so-called cepstral coefficients. When comparing machine learning models that incorporate these cepstral coefficients alongside those that do not, this approach demonstrates that speech recognition techniques can result in more effective models. Here, the basic concept for speech recognition-based feature extraction and its implementation are presented, and necessary considerations and techniques are discussed, which are crucial for performant signal processing, particularly for real-time monitoring applications. The methodology is demonstrated, using acoustic emission signals derived from industrial-scale processes , such as e.g. milling, drilling or friction stir welding.

Detecting schizophrenia, bipolar disorder, psychosis vulnerability and major depressive disorder from 5 minutes of online-collected speech.

Psychosis poses substantial social and healthcare burdens. The analysis of speech is a promising approach for the diagnosis and monitoring of psychosis, capturing symptoms like thought disorder and flattened affect. Recent advancements in Natural Language Processing (NLP) methodologies enable the automated extraction of informative speech features, which has been leveraged for early psychosis detection and assessment of symptomology. However, critical gaps persist, including the absence of standardized sample collection protocols, small sample sizes, and a lack of multi-illness classification, limiting clinical applicability. Our study aimed to (1) identify an optimal assessment approach for the online and remote collection of speech, in the context of assessing the psychosis spectrum and evaluate whether a fully automated, speech-based machine learning (ML) pipeline can discriminate among different conditions on the schizophrenia-bipolar spectrum (SSD-BD-SPE), help-seeking comparison subjects (MDD), and healthy controls (HC) at varying layers of analysis and diagnostic complexity. We adopted online data collection methods to collect 20 minutes of speech and demographic information from individuals. Participants were categorized as "healthy" help-seekers (HC), having a schizophrenia-spectrum disorder (SSD), bipolar disorder (BD), major depressive disorder (MDD), or being on the psychosis spectrum with sub-clinical psychotic experiences (SPE). SPE status was determined based on self-reported clinical diagnosis and responses to the PHQ-8 and PQ-16 screening questionnaires, while other diagnoses were determined based on self-report from participants. Linguistic and paralinguistic features were extracted and ensemble learning algorithms (e.g., XGBoost) were used to train models. A 70%-30% train-test split and 30-fold cross-validation was used to validate the model performance. The final analysis sample included 1140 individuals and 22,650 minutes of speech. Using 5-minutes of speech, our model could discriminate between HC and those with a serious mental illness (SSD or BD) with 86% accuracy (AUC = 0.91, Recall = 0.7, Precision = 0.98). Furthermore, our model could discern among HC, SPE, BD and SSD groups with 86% accuracy (F1 macro = 0.855, Recall Macro = 0.86, Precision Macro = 0.86). Finally, in a 5-class discrimination task including individuals with MDD, our model had 76% accuracy (F1 macro = 0.757, Recall Macro = 0.758, Precision Macro = 0.766). Our ML pipeline demonstrated disorder-specific learning, achieving excellent or good accuracy across several classification tasks. We demonstrated that the screening of mental disorders is possible via a fully automated, remote speech assessment pipeline. We tested our model on relatively high number conditions (5 classes) in the literature and in a stratified sample of psychosis spectrum, including HC, SPE, SSD and BD (4 classes). We tested our model on a large sample (N = 1150) and demonstrated best-in-class accuracy with remotely collected speech data in the psychosis spectrum, however, further clinical validation is needed to test the reliability of model performance.

Building a Speech Dataset and Recognition Model for the Minority Tu Language

Speech recognition technology has many applications in our daily life. However, for many low-resource languages without written forms, acquiring sufficient training data remains a significant challenge for building accurate ASR models. The Tu language, spoken by an ethnic minority group in Qinghai Province in China, is one such example. Due to the lack of written records and the great diversity in regional pronunciations, there has been little previous research on Tu-language speech recognition. This work seeks to address this research gap by creating the first speech dataset for the Tu language spoken in Huzhu County, Qinghai. We first formulated the relevant pronunciation rules for the Tu language based on linguistic analysis. Then, we constructed a new speech corpus, named HZ-TuDs, through targeted data collection and annotation. Based on the HZ-TuDs dataset, we designed several baseline sequence-to-sequence deep neural models for end-to-end Tu-language speech recognition. Additionally, we proposed a novel SA-conformer model, which combines convolutional and channel attention modules to better extract speech features. Experiments showed that our proposed SA-conformer model can significantly reduce the character error rate from 23% to 12%, effectively improving the accuracy of Tu language recognition compared to previous approaches. This demonstrates the effectiveness of our dataset construction and model design efforts in advancing speech recognition technology for this low-resource minority language.

Application of speech recognition algorithm based on interactive artificial intelligence system in English video teaching system

Social robots are intelligent programs that can autonomously run, automatically publish information, and interact on social media platforms. This article focuses on the speech recognition algorithm of interactive artificial intelligence systems, combined with the Hanning window function and the Vitit algorithm, and explores its application value in English video teaching systems through detailed experiments. Adopting speech feature extraction technology provides strong support for the subsequent recognition process. The application of Hanning window function has a positive impact on speech signal preprocessing. The most indispensable algorithm in speech recognition models is the Vitit algorithm, which successfully solves the problem of time variation and continuity of speech signals. This study successfully applied speech recognition algorithms based on interactive artificial intelligence systems to English video teaching systems. By combining speech feature extraction, Hanning window function, and Vitit algorithm, a system with strong recognition ability and interactivity was constructed. The English video teaching system aims to improve students’ learning process. By combining interactive artificial intelligence speech recognition technology with English video teaching, it can improve students’ experience in learning English and also meet their personalized learning habits.

Speech Feature Extraction in Broadcast Hosting Based on Fluctuating Equation Inversion

Speech Feature Extraction in Broadcast Hosting Based on Fluctuating Equation Inversion

Real-time translation of English speech through speech feature extraction

Real-time translation of English speech through speech feature extraction

3D facial animation driven by speech-video dual-modal signals

In recent years, the applications of digital humans have become increasingly widespread. One of the most challenging core technologies is the generation of highly realistic and automated 3D facial animation that combines facial movements and speech. The single-modal 3D facial animation driven by speech typically ignores the weak correlation between speech and upper facial movements as well as head posture. In contrast, the video-driven approach can perfectly solve the posture problem while obtaining natural expressions. However, mapping 2D facial information to 3D facial information may lead to information loss, which make lip synchronization generated by video-driven methods is not as good as the speech-driven methods trained on 4D facial data. Therefore, this paper proposes a dual-modal generation method that uses speech and video information to generate more natural and vivid 3D facial animation. Specifically, the lip movements related to speech are generated by speech-video information, while speech-uncorrelated postures and expressions are generated solely by video information. The speech-driven module is used to extract speech features, and its output lip animation is then used as the foundation for facial animation. The expression and pose module is used to extract temporal visual features for regressing expression and head posture parameters. We fuse speech and video features to obtain chin posture parameters related to lip movements, and use these parameters to fine-tune the lip animation generated form the speech-driven module. This paper introduces multiple consistency losses to enhance the network’s capability to generate expressions and postures. Experiments conducted on the LRS3, TCD-TIMIT and MEAD datasets show that the proposed method achieves better performance on evaluation metrics such as CER, WER, VER and VWER than the current state-of-the-art methods. In addition, a perceptual user study show that over 77% and 70% of cases believe that this paper’s method is better than the comparative algorithms EMOCA and SPECTRE in terms of realism. In terms of lip synchronization, it received over 79% and 66% of cases support, respectively. Both evaluation methods demonstrate the effectiveness of the proposed method.

Research Of Singing With Note-name Recognition Algorithm Based On End-to-End Model

With the rapid development of artificial intelligence, and people are not only seeking material needs, but also pursuing happiness and spiritual aspirations. Especially some singing talent shows appear, but there are many controversial results. In order to solve the problem of singing with note-name evaluation and evaluate the singer's singing level in the professional music education of solfeggio and ear training, the paper uses artificial intelligence algorithms to recognize the note-name. Firstly, preprocess the singing sound data, including pre emphasis, framing, and windowing to reduce the impact of noise, and extract the acoustic features of the singing sound, the acoustic speech features are organized into a form suitable for DCNN input and a DCNN model is designed to obtain the pinyin form of the singing sound, the softmax layer adopts an end-to-end CTC structure, which classifies the input singing speech, finds corresponding phoneme sequences, and obtains the output results. The end-to-end CTC structure is used to classify and optimize the recognition process, and finally the state of the obtained features is output. And then singing pronunciation dictionary generates candidate word sequences based on the mapping relationship between phonemes and notes. Finally, based on the acoustic model score, the candidate note sequence with the highest score is obtained through decoder processing, and obtained the singing with note-name result of the singing sound. In order to further improve recognition performance, the paper introduces a new CTC-DCNN acoustic model. In this model, residual blocks can transfer input features to the output part through shortcuts, allowing the multi-layer convolutional speech features to be preserved as much as possible. At the same time, the deep structure can also better achieve the extraction and analysis of speech features. A new and improved CTC-DCNN acoustic model is obtained by optimizing the maxout function. The algorithm proposed in this paper is fair and equal in obtaining information, and no one participates in the entire process of obtaining scores. It is believed that the scoring results obtained in this way should be more objective.

Predicting Individual Well-Being in Teamwork Contexts Based on Speech Features

Current methods for assessing individual well-being in team collaboration at the workplace often rely on manually collected surveys. This limits continuous real-world data collection and proactive measures to improve team member workplace satisfaction. We propose a method to automatically derive social signals related to individual well-being in team collaboration from raw audio and video data collected in teamwork contexts. The goal was to develop computational methods and measurements to facilitate the mirroring of individuals’ well-being to themselves. We focus on how speech behavior is perceived by team members to improve their well-being. Our main contribution is the assembly of an integrated toolchain to perform multi-modal extraction of robust speech features in noisy field settings and to explore which features are predictors of self-reported satisfaction scores. We applied the toolchain to a case study, where we collected videos of 20 teams with 56 participants collaborating over a four-day period in a team project in an educational environment. Our audiovisual speaker diarization extracted individual speech features from a noisy environment. As the dependent variable, team members filled out a daily PERMA (positive emotion, engagement, relationships, meaning, and accomplishment) survey. These well-being scores were predicted using speech features extracted from the videos using machine learning. The results suggest that the proposed toolchain was able to automatically predict individual well-being in teams, leading to better teamwork and happier team members.

Self-Supervised Open-Set Speaker Recognition with Laguerre-Voronoi Descriptors.

Speaker recognition is a challenging problem in behavioral biometrics that has been rigorously investigated over the last decade. Although numerous supervised closed-set systems inherit the power of deep neural networks, limited studies have been made on open-set speaker recognition. This paper proposes a self-supervised open-set speaker recognition that leverages the geometric properties of speaker distribution for accurate and robust speaker verification. The proposed framework consists of a deep neural network incorporating a wider viewpoint of temporal speech features and Laguerre-Voronoi diagram-based speech feature extraction. The deep neural network is trained with a specialized clustering criterion that only requires positive pairs during training. The experiments validated that the proposed system outperformed current state-of-the-art methods in open-set speaker recognition and cluster representation.

Fuzzy speech emotion recognition considering semantic awareness

Speech emotion recognition is of great significance in the industry such as social robots, health care, and intelligent education. Due to the obscurity of emotional expression in speech, most works on speech emotion recognition (SER) ignore the consistency of speech emotion recognition, leading to fuzzy expression and low accuracy in emotional recognition. In this paper, we propose a semantic aware speech emotion recognition model to alleviate this issue. Specifically, a speech feature extraction module based on CNN and Transformer is designed to extract local and global information from the speech. Moreover, a semantic embedding support module is proposed to use text semantic information as auxiliary information to assist the model in extracting emotional features of speech, and can effectively overcome the problem of low recognition rate caused by emotional ambiguity. In addition, the model uses a key-value pair attention mechanism to fuse the features, which makes the fusion of speech and text features preferable. In experiments on two benchmark corpora IEMOCAP and EMO-DB, the recognition rates of 74.3% and 72.5% were obtained under respectively, which show that the proposed model can significantly improve the accuracy of emotion recognition.

An automated speech recognition and feature selection approach based on improved Northern Goshawk optimization

<p><span lang="EN-US">Automatic speech recognition (ASR) approach is dependent on optimal speech feature extraction, which attempts to get a parametric depiction of an input speech signal. Feature extraction (FE) strategy combined with a feature selection (FS) approach should capture the most important features of the signal while discarding the rest. FS is a crucial process that can affect the pattern classification and recognition system's performance. In this research, we introduce a hybrid supervised learning using metaheuristic technique for optimum FE and FS termed Northern Goshawk optimization (NGO) and opposition-based learning (OBL). Pre-processing, feature extraction and selection, and recognition are the three steps of the proposed technique. The pre-processing is done first to lessen the amount of noise. In the FE stage, we extract features. The OBL-NGO method is used to pick the best collection of extracted characteristics. Finally, these optimised features are utilised to train the k-nearest neighbour (KNN) classifier, and the matching text is shown as the output based on these optimised characteristics of the provided input audio signal. The system's performance is outstanding, and the suggested OBL-NGO is best suited for ASR, according to the testing data.</span></p>

Retracted: SmHeSol (IoT-BC): Smart Healthcare Solution for Future Development Using Speech Feature Extraction Integration Approach with IoT and Blockchain

Retracted: SmHeSol (IoT-BC): Smart Healthcare Solution for Future Development Using Speech Feature Extraction Integration Approach with IoT and Blockchain

Application and Effectiveness Analysis of Multimodal Emotion Recognition Technology in Music Education

Abstract Emotions in music education affect learners’ cognitive activities, and failure to capture learners’ emotional changes in a timely manner can lead to music teachers’ inability to adjust their teaching strategies in a timely manner. In this paper, a convolutional neural network is utilized to extract speech and visual emotion features of students during the process of music education. The spatial plane fusion method is used to fuse the speech and visual emotion modalities. A cross-modal interactive attention mechanism is introduced to optimize the fusion effect of the multimodal emotion features. Then, a support vector machine is used to identify and classify the emotion features. The study shows that the multimodal emotion recognition model proposed in this paper can achieve an emotion recognition accuracy of 88.78%, can accurately recognize the emotional state of students, and can assist teachers in effectively intervening in the negative emotions of students. In the music classroom applying this technology, the average test score of the student’s music education program is 93.70, and their will to learn music education is 95.09% on average. This paper’s multimodal emotion recognition model helps teachers implement effective interventions in music education and establishes the foundation for improving students’ interest in music learning.

Feature Extraction of Assamese Speech Based One Motion Analysis

Feature Extraction of Assamese Speech Based One Motion Analysis

Virtual Speech System Based on Sensing Technology and Teaching Management in Universities

Abstract In this paper, digital speech is compressed using discrete Fourier transform, discrete cosine transform, and improved discrete cosine transform, and compressed sensing technology is proposed. Based on the compressed sensing technology, the frequency domain parameter compression algorithm and the speech coding and decoding algorithm are designed, and the interactive 3D virtual speech system design is completed through the pre-processing of the speech system, the extraction of speech features and the design of speech control commands. The virtual voice system designed in this paper is introduced in the teaching management mode of colleges and universities, and the main functions of the system include four major sections: notification management, online Q&A, virtual voice system interaction, and teaching resource management. The virtual voice system built using sensing technology is simulated and tested, and the practical application effect of the system is studied through empirical analysis. The experimental results show that the amplitude of the sound recorded by the compression sensor in the voice sensing experiment is more concentrated, the range is concentrated between [-0.025,0.025], and the detected voice is smaller and more effective than the amplitude recorded by the cell phone. Students were mainly satisfied and very satisfied with the four system functions designed in this paper, and in terms of the online Q&A function, only one student expressed great dissatisfaction and the total number of satisfied people was 119, and the students were highly satisfied with the teaching management of the system designed in this study.

A New Algorithm for Speech Feature Extraction Using Polynomial Chirplet Transform

A New Algorithm for Speech Feature Extraction Using Polynomial Chirplet Transform

Automated extraction of speech and turn-taking parameters in autism allows for diagnostic classification using a multivariable prediction model.

Autism spectrum disorder (ASD) is diagnosed on the basis of speech and communication differences, amongst other symptoms. Since conversations are essential for building connections with others, it is important to understand the exact nature of differences between autistic and non-autistic verbal behaviour and evaluate the potential of these differences for diagnostics. In this study, we recorded dyadic conversations and used automated extraction of speech and interactional turn-taking features of 54 non-autistic and 26 autistic participants. The extracted speech and turn-taking parameters showed high potential as a diagnostic marker. A linear support vector machine was able to predict the dyad type with 76.2% balanced accuracy (sensitivity: 73.8%, specificity: 78.6%), suggesting that digitally assisted diagnostics could significantly enhance the current clinical diagnostic process due to their objectivity and scalability. In group comparisons on the individual and dyadic level, we found that autistic interaction partners talked slower and in a more monotonous manner than non-autistic interaction partners and that mixed dyads consisting of an autistic and a non-autistic participant had increased periods of silence, and the intensity, i.e. loudness, of their speech was more synchronous.

Application of dynamic time warping optimization algorithm in speech recognition of machine translation

Speech recognition is the foundation of human-computer interaction technology and an important aspect of speech signal processing, with broad application prospects. Therefore, it is very necessary to recognize speech. At present, speech recognition has problems such as low recognition rate, slow recognition speed, and severe interference from other factors. This paper studied speech recognition based on dynamic time warping (DTW) algorithm. By introducing speech recognition, the specific steps of speech recognition were understood. Before performing speech recognition, the speech that needs to be recognized needs to be converted into a speech sequence using an acoustic model. Then, the DTW algorithm was used to preprocess speech recognition, mainly by sampling and windowing the speech. After preprocessing, speech feature extraction was carried out. After feature extraction was completed, speech recognition was carried out. Through experiments, it can be found that the recognition rate of speech recognition on the basis of DTW algorithm was very high. In a quiet environment, the recognition rate was above 93.85 %, and the average recognition rate of the 10 selected testers was 95.8 %. In a noisy environment, the recognition rate was above 91.4 %, and the average recognition rate of the 10 selected testers was 93 %. In addition to high recognition rate, DTW based speech recognition also had a very fast speed for vocabulary recognition. Based on the DTW algorithm, speech recognition not only has a high recognition rate, but also has a faster recognition speed.

Speech emotion recognition via multiple fusion under spatial–temporal parallel network

Speech, as a necessary way to express emotions, plays a vital role in human communication. With the continuous deepening of research on emotion recognition in human–computer interaction, speech emotion recognition (SER) has become an essential task to improve the human–computer interaction experience. When performing emotion feature extraction of speech, the method of cutting the speech spectrum will destroy the continuity of speech. Besides, the method of using the cascaded structure without cutting the speech spectrum cannot simultaneously extract speech spectrum information from both temporal and spatial domains. To this end, we propose a spatial–temporal parallel network for speech emotion recognition without cutting the speech spectrum. To further mix the temporal and spatial features, we design a novel fusion method (called multiple fusion) that combines the concatenate fusion and ensemble strategy. Finally, the experimental results on five datasets demonstrate that the proposed method outperforms state-of-the-art methods.