Speech Processing Applications Research Articles

Multi-label bird species classification from audio recordings using attention framework

For the conservation of avian biodiversity, bird detection is vital since it allows ornithologists to quantify which species exist in a particular area. Analyzing their acoustic signals enables the efficient identification of multiple bird species from overlapping recordings. This paper addresses classifying bird vocalizations in real-time audio recording using acoustic analysis. Schemes based on recurrent neural networks (RNN) are presented in the proposed work. Gated-recurrent units (GRU) are a particular type of RNN that has shown remarkable performance in acoustic classification. We propose a hierarchical Attention-based bidirectional gated recurrent unit (BiGRU) model for classifying acoustic signals of birds by using Mel-frequency cepstral coefficients (MFCC). The attention mechanism has proved its superior efficacy in many acoustic, speech and music processing applications. The attention mechanism is employed to give a different focus to the information outputted from the hidden layers of BiGRU. We adopted a short-time sliding-aggregation approach to decide on the test data, in which probability outcomes are species-wise summed and normalized. Species with the highest probability scores are assumed to be the dominant species in the recording. Our Attention-BiGRU classifier achieves pretty high performance in the Xeno-Canto dataset, with an F1-score of 0.84, with competing performance to the state-of-the-art multi-label classifiers.

Survey on optimization algorithms in speech processing

Nature is a tremendous provenance of resolving hard and complex problems that exist in the field of computer science because it reveals a very diverse, robust, dynamic, and interesting phenomenon. It constantly finds the optimal solution to resolve its problem which accomplishes exact equity among its element. Nature-inspired algorithms are the heuristic high-level procedure that interprets nature to solve the optimization problem which popularized in the new era of computation. The main objective of this paper is to evaluate the modern technology and enhancement in the nature-inspired algorithm, especially in the application of speech processing, speech recognition, and speech feature selection problems. This paper presents broad collections of global optimization algorithms which have been successfully applied to generate recognition systems that are integrated with metaheuristic algorithms.

Clifford-valued linear canonical transform: Convolution and uncertainty principles

Linear canonical transform is a three-parameter class of linear integral transformation, which has found many applications in optics, filter design, speech, image, video, and signal processing due to its flexibility in representing information within the linear canonical domain. In this article, we extend the linear canonical transform to the class of Clifford-valued signals and introduce a novel transform coined as Clifford-valued linear canonical transform. The primary analysis encompasses the derivation of fundamental properties of the proposed transform such as translation and scaling covariance, inner product relation and inversion formula. Subsequently, we investigate the convolution, continuity, and differentiation theorems for the Clifford-valued linear canonical transform. A non-homogeneous partial differential equation has also been solved in the Clifford framework as an application of the differentiation theorem. Finally, we culminate our investigation by deriving several uncertainty principles for the proposed transform.

Self-supervised learning based knowledge distillation framework for automatic speech recognition for hearing impaired

The use of speech processing applications, particularly speech recognition, has got a lot of attention in recent decades. In recent years, research has focused on using deep learning for speech-related applications. This new branch of machine learning has outperformed others in a range of applications, including voice, and has thus become a particularly appealing research subject. Noise, speaker variability, language variability, vocabulary size, and domain remain one of the most significant research difficulties in speech recognition. We investigated on self-supervised algorithm for the unlabelled data. In recent years, these algorithms have progressed significantly, with their efficacy approaching and supervised pre-training alternatives across a variety of data modalities such as image and video. The purpose of this research is to develop powerful models for audio speech recognition that do not require human annotation. We accomplish this by distilling information from an automatic speech recognition (ASR) model that was trained on a large audio-only corpus. We integrate Connectionist Temporal Classification (CTC) loss, KL divergence loss in distillation technique. We demonstrate that distillation significantly speeds up training. We evaluate our model with evaluation metric Word Error Rate (WER).

Application of Speech Processing for Pathological Voice Detection and Analysis

Application of Speech Processing for Pathological Voice Detection and Analysis

Voice Activity Detection: Fusion of Time and Frequency Domain Features with A SVM Classifier

Voice activity detection (VAD) discriminates between segments of an audio signal that has speech content from the ones with either noise or silence. It is deployed as the front-end of some speech processing applications such as voice recognition, and speaker recognition to improve their performance in terms of accuracy and efficiency. It is also used in the communication system to bring about efficient utilization of transmission bandwidth by ensuring only segments of the audio signal with voice activity are encoded and transmitted. In this work, the VAD algorithm was implemented using a features-fusion strategy. In the pre-processing stage, contents outside the human auditory frequency range were removed with the aid of a digital Butterworth bandpass filter. The signal was then fragmented into frames from where time-domain features (zero-crossing, standard deviation, normalized envelope, kurtosis, skewness, and root-mean-square energy.) and frequency-domain features (13MFCCs) were extracted and then combined to form a feature representation of each frame. Recursive feature elimination was applied to the dataset to reduce the features to seven (7) which was used to train a Support Vector Machine (SVM) to be able to distinguish between voiced and unvoiced frames. A State-of-art performance was recorded by this simple SVM-based VAD system with an accuracy of 100%, recall of 100%, precision of 100% and F1 score of 100% which is at par with similar implementations which utilizes a complex architecture of deep neural network with high computational cost and training time. Keywords : Voice activity detection, fusion strategy, support vector machine, frequency domain features, time domain features DOI: 10.7176/CEIS/13-3-03 Publication date: May 31 st 2022

Comparison of discrete transforms for deep‐neural‐networks‐based speech enhancement

In recent studies of speech enhancement, a deep-learning model is trained to predict clean speech spectra from the known noisy spectra of speech. Rather than using the traditional discrete Fourier transform (DFT), this paper considers other well-known transforms to generate the speech spectra for deep-learning-based speech enhancement. In addition to the DFT, seven different transforms were tested: discrete Cosine transform, discrete Sine transform, discrete Haar transform, discrete Hadamard transform, discrete Tchebichef transform, discrete Krawtchouk transform, and discrete Tchebichef-Krawtchouk transform. Two deep-learning architectures were tested: convolutional neural networks (CNN) and fully connected neural networks. Experiments were performed for the NOIZEUS database, and various speech quality and intelligibility measures were adopted for performance evaluation. The quality and intelligibility scores of the enhanced speech demonstrate that discrete Sine transformation is better suited for the front-end processing with a CNN as it outperformed the DFT in this kind of application. The achieved results demonstrate that combining two or more existing transforms could improve the performance in specific conditions. The tested models suggest that we should not assume that the DFT is optimal in front-end processing with deep neural networks (DNNs). On this basis, other discrete transformations should be taken into account when designing robust DNN-based speech processing applications.

MilVAD: A bag-level MNIST modelling of voice activity detection using deep multiple instance learning

Voice Activity Detection (VAD) which is used as an onset step for majority of the applications in Digital Speech Processing (DSP) area is defined as the process of identifying speech region in an audio recording. It is mostly used for automatic speech recognition, speaker identification/verification, speech enhancement, speaker diarization etc. in order to reduce output errors and increase overall effectiveness of the systems. In this study, a bag-level MNIST modelling of VAD was proposed using Deep Multiple Instance Learning (Deep MIL) approach. To the best of our knowledge, because this is the first attempt that the VAD was modelled as a MIL problem in the literature, we named “milVAD”. The MNIST dataset was modified to obtain bag-level classifier model for the VAD framework while the MIL algorithm was implemented inside a Convolutional Neural Network (CNN) as an embedded layer using Noisy-And pooling method. The proposed modelling scenario has surprisingly achieved high training accuracy, which is approx. 99.91%, with only nine epochs via Deep MIL at bag-level. These results proved that the MIL can efficiently be used for the VAD systems in the manner of binary classification.

Increasing Context for Estimating Confidence Scores in Automatic Speech Recognition

Accurate confidence measures for predictions from machine learning techniques play a critical role in the deployment and training of many speech and language processing applications. For example, confidence scores are important when making use of automatically generated transcriptions in training automatic speech recognition (ASR) systems, as well as down-stream applications, such as information retrieval and conversational assistants. Previous work on improving confidence scores for these systems has focused on two main directions: designing features correlated with improved confidence prediction; and employing sequence models to account for the importance of contextual information. Few studies, however, have explored incorporating contextual information more broadly, such as from the future, in addition to the past, or making use of alternative multiple hypotheses in addition to the most likely one. This article introduces two general approaches for encapsulating contextual information from lattices. Experimental results illustrating the importance of increasing contextual information for estimating confidence scores are presented on a range of limited resource languages where word error rates range between 30% and 60%. The results show that the novel approaches provide significant gains in the accuracy of confidence estimation.

Energy-Efficient High-Speed ASIC Implementation of Convolutional Neural Network Using Novel Reduced Critical-Path Design

Convolutional Neural Network (CNN) plays an important role in several machine learning tasks related to speech, image, and video processing applications. The increasing demand for faster processing in real-time applications requires high-speed implementation of CNN. However, in general, CNN involves higher latency due to the computationally intensive behavior of the convolutional layer. While state-of-the-art architecture provides efficient dataflow of the convolutional operations, this paper proposes a hardware-efficient, high-speed convolution block for ASIC implementation of the CNN algorithm. The proposed convolution block is designed using a novel bit-level-multiply-accumulator (BLMAC) with a modified Booth encoder and a Wallace reduction tree. The critical path of the overall architecture is significantly shortened due to the time-optimized implementation of the proposed BLMAC, which is a main component of the convolution process. Critical path analysis and dataflow strategy are also provided to demonstrate the acceleration of the proposed design. The proposed architecture was synthesized using Synopsys Design Compiler to prove its accelerated processing. The ASIC synthesis results of the proposed architecture using a 65nm standard cell library show at least 53% reduction in latency, 52.2% reduction in area-delay product, and 54.2% reduction in power-delay product compared to the state-of-the-art architecture.

Evaluating Self-Supervised Speech Representations for Speech Emotion Recognition

Self-supervised learning has recently been implemented widely in speech processing areas, replacing conventional acoustic feature extraction to extract meaningful information from speech. One of the challenging applications of speech processing is to extract affective information from speech, commonly called speech emotion recognition. Until now, it is not clear the position of these speech representations compared to the classical acoustic feature. This paper evaluates nineteen self-supervised speech representations and one classical acoustic feature for five distinct speech emotion recognition datasets on the same classifier. We calculate the effect size among twenty speech representations to show the magnitude of relative differences from the top to the lowest performance. The top three are WavLM Large, UniSpeech-SAT Large, and HuBERT Large, with negligible effect sizes among them. The significance test supports the difference among self-supervised speech representations. The best prediction for each dataset is shown in the form of a confusion matrix to gain insights into the best performance of speech representations for each emotion category based on the training data from balanced vs. unbalanced datasets, English vs. Japanese corpus, and five vs. six emotion categories. Despite showing their competitiveness, this exploration of self-supervised learning for speech emotion recognition also shows their limitations on models pre-trained on small data and trained on unbalanced datasets.

VACE-WPE: Virtual Acoustic Channel Expansion Based on Neural Networks for Weighted Prediction Error-Based Speech Dereverberation

Speech dereverberation is an important issue for many real-world speech processing applications. Among the techniques developed, the weighted prediction error (WPE) algorithm has been widely adopted and advanced over the last decade, which blindly cancels out the late reverberation component from the reverberant mixture of microphone signals. In this study, we extend the neural-network-based virtual acoustic channel expansion (VACE) framework for the WPE-based speech dereverberation, a variant of the WPE that we recently proposed to enable the use of dual-channel WPE algorithm in a single-microphone speech dereverberation scenario. Based on the previous study, some ablation studies are conducted regarding the constituents of the VACE-WPE in an offline processing scenario. These studies reveal the characteristics of the system, thereby simplifying the architecture and leading to the introduction of new strategies for training the neural network for the VACE. Experimental results demonstrate that VACE-WPE (our PyTorch implementation and pre-trained models are available from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/dreadbird06/vace_wpe</uri> ) considerably outperforms its single-channel counterpart in simulated noisy reverberant environments in terms of objective speech quality and is superior to the single-channel WPE as well as several fully neural speech dereverberation methods when employed as the front-end for the far-field automatic speech recognizer.

Microscopic and Blind Prediction of Speech Intelligibility: Theory and Practice.

Being able to estimate speech intelligibility without the need for listening tests would confer great benefits for a wide range of speech processing applications. Many attempts have therefore been made to introduce an objective, and ideally referencefree measure for this purpose. Most works analyze speech intelligibility prediction (SIP) methods from a macroscopic point of view, averaging over longer time spans. This paper, in contrast, presents a theoretical framework for the microscopic evaluation of SIP methods. Within our framework, a Statistically estimated Accuracy based on Theory (StAT) is derived, which numerically quantifies the statistical limitations inherent in microscopic SIP. A state-of-the-art approach to microscopic SIP, namely, the use of automatic speech recognition (ASR) to directly predict listening test results, is evaluated within this framework. The practical results are in good agreement with the theory. As the final contribution, a fully blind DIscriminative Speech intelligibility Predictor (DISP) is introduced and is also evaluated within the StAT framework. It is shown that this novel, blind estimator can predict intelligibility as well as-and often even with better accuracy than-the non-blind ASR-based approach, and that its results are again in good agreement with its theoretically derived performance potential.

Dialect Identification in Telugu Language Speech Utterance Using Modified Features with Deep Neural Network

Dialect Identification is the process of identifies the dialects of particular standard language. The Telugu Language is one of the historical and important languages. Like any other language Telugu also contains mainly three dialects Telangana, Costa Andhra and Rayalaseema. The research work in dialect identification is very less compare to Language identification because of dearth of database. In any dialects identification system, the database and feature engineering play vital roles because of most the words are similar in pronunciation and also most of the researchers apply statistical approaches like Hidden Markov Model (HMM), Gaussian Mixture Model (GMM), etc. to work on speech processing applications. But in today's world, neural networks play a vital role in all application domains and produce good results. One of the types of the neural networks is Deep Neural Networks (DNN) and it is used to achieve the state of the art performance in several fields such as speech recognition, speaker identification. In this, the Deep Neural Network (DNN) based model Multilayer Perceptron is used to identify the regional dialects of the Telugu Language using enhanced Mel Frequency Cepstral Coefficients (MFCC) features. To do this, created a database of the Telugu dialects with the duration of 5h and 45m collected from different speakers in different environments. The results produced by DNN model compared with HMM and GMM model and it is observed that the DNN model provides good performance.

Remote Speech Analysis in the Evaluation of Hospitalized Patients With Acute Decompensated Heart Failure

ObjectivesThis study assessed the performance of an automated speech analysis technology in detecting pulmonary fluid overload in patients with acute decompensated heart failure (ADHF). BackgroundPulmonary edema is the main cause of heart failure (HF)-related hospitalizations and a key predictor of poor postdischarge prognosis. Frequent monitoring is often recommended, but signs of decompensation are often missed. Voice and sound analysis technologies have been shown to successfully identify clinical conditions that affect vocal cord vibration mechanics. MethodsAdult patients with ADHF (n = 40) recorded 5 sentences, in 1 of 3 languages, using HearO, a proprietary speech processing and analysis application, upon admission (wet) to and discharge (dry) from the hospital. Recordings were analyzed for 5 distinct speech measures (SMs), each a distinct time, frequency resolution, and linear versus perceptual (ear) model; mean change from baseline SMs was calculated. ResultsIn total, 1,484 recordings were analyzed. Discharge recordings were successfully tagged as distinctly different from baseline (wet) in 94% of cases, with distinct differences shown for all 5 SMs in 87.5% of cases. The largest change from baseline was documented for SM2 (218%). Unsupervised, blinded clustering of untagged admission and discharge recordings of 9 patients was further demonstrated for all 5 SMs. ConclusionsAutomated speech analysis technology can identify voice alterations reflective of HF status. This platform is expected to provide a valuable contribution to in-person and remote follow-up of patients with HF, by alerting to imminent deterioration, thereby reducing hospitalization rates. (Clinical Evaluation of Cordio App in Adult Patients With CHF; NCT03266029)

Overview and tutorial on the identification and tracking of pitch and frequency

Many diverse areas of acoustics leverage the identification and tracking of frequency over time. Pitch tracking plays an important role in music information retrieval and performance analysis. Frequency contours are also important features in speech processing and bioacoustics applications, and the identification of frequency content in a signal is widely used. From a signal processing perspective, these applications vary drastically with respect to signal duration and frequency, desired resolutions, and amount of signal interference. In some cases, a single pitch is used to characterize a multicomponent sound, while other applications demand independent tracking of individual components. As such, a wide variety of algorithms exist in both the time and frequency domains. This presentation provides a general overview of commonly used methods and available implementations for tracking pitch and/or frequency. Potential sources of error will be emphasized, focusing on the considerations relevant to selecting and optimizing an algorithm for a given application.

Machine learning improvements to compressive sensing for speech enhancement in hearing aid applications

PurposeSpeech is the primary means of communication for humans. A proper functioning auditory system is needed for accurate cognition of speech. Compressed sensing (CS) is a method for simultaneous compression and sampling of a given signal. It is a novel method increasingly being used in many speech processing applications. The paper aims to use Compressive sensing algorithm for hearing aid applications to reduce surrounding noise.Design/methodology/approachIn this work, the authors propose a machine learning algorithm for improving the performance of compressive sensing using a neural network.FindingsThe proposed solution is able to reduce the signal reconstruction time by about 21.62% and root mean square error of 43% compared to default L2 norm minimization used in CS reconstruction. This work proposes an adaptive neural network–based algorithm to enhance the compressive sensing so that it is able to reconstruct the signal in a comparatively lower time and with minimal distortion to the quality.Research limitations/implicationsThe use of compressive sensing for speech enhancement in a hearing aid is limited due to the delay in the reconstruction of the signal.Practical implicationsIn many digital applications, the acquired raw signals are compressed to achieve smaller size so that it becomes effective for storage and transmission. In this process, even unnecessary signals are acquired and compressed leading to inefficiency.Social implicationsHearing loss is the most common sensory deficit in humans today. Worldwide, it is the second leading cause for “Years lived with Disability” the first being depression. A recent study by World health organization estimates nearly 450 million people in the world had been disabled by hearing loss, and the prevalence of hearing impairment in India is around 6.3% (63 million people suffering from significant auditory loss).Originality/valueThe objective is to reduce the time taken for CS reconstruction with minimal degradation to the reconstructed signal. Also, the solution must be adaptive to different characteristics of the signal and in presence of different types of noises.

Drone classification using RF signal based spectral features

Drone detection and classification, important in military and civilian applications, are performed using different sensor signals. Proposed study handles this task using Radio Frequency (RF) signals utilizing basic machine learning methods. It is composed of two main stages as feature extraction succeeded by training/testing of the model. In feature extraction stage, valuable information for classification, contained in the RF signal, is obtained. For this purpose, spectral features, frequently used in speech processing applications, are employed. Specifically, Power Spectral Density (PSD), Mel-Frequency Cepstral Coefficients (MFCC) and Linear Frequency Cepstral Coefficients (LFCC) are adopted by adjusting filter bank margins and parameters for this task. In the second stage, a Support Vector Machine (SVM) classifier is first trained based on the obtained features and finally tested to measure its performance. All experimental studies are carried out using publicly available DroneRF dataset. This dataset contains 2-Class, 4-Class and 10-Class samples for drone existence vs. background (BG), drone types and drone operation modes, respectively. The best classification results are obtained using, PSD, MFCC and LFCC based features for 2-Class, MFCC and LFCC based features for 4-Class and LFCC based features for 10-Class. Accuracy rates for 2-Class, 4-Class and 10-Class are 100%, 98.67% and 95.15%, respectively. These results show that the proposed method outperforms the results given in the literature for DroneRF dataset.

Comparison Of Different Feature Extraction Techniques In Telugu Dialects Identification

The Telugu language is one of the standard and historical languages in India. Like other spoken languages, the Telugu language also contains different dialects i.e., Telangana, Costa Andhra and Rayalaseema. The research in dialects identification (DI) work is very less as compared to other speech processing applications. For the research Dialect identification (DI) in Telugu language, database is very important, but there is no standard database. To carry out the research work in DI, we created the database of Telugu Language with different dialects and we anlyzed different feature extraction techniques to implement system. In this paper, we compare the performance of different models given by applying the different feature extraction techniques such as spatial, temporal, and prosodic features in Telugu dialect identification. We have applied different classification models i.e., K-Nearest Neighbour (K-NN), Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM). It is observed that GMM model provides good accuracy with MFCC+∆MFCC+∆∆MFCC and MFCC+PITCH +LOUDNESS features compare to HMM model.

A Novel Speech to Mouth Articulation System for Realistic Humanoid Robots

A significant ongoing issue in realistic humanoid robotics (RHRs) is inaccurate speech to mouth synchronisation. Even the most advanced robotic systems cannot authentically emulate the natural movements of the human jaw, lips and tongue during verbal communication. These visual and functional irregularities have the potential to propagate the Uncanny Valley Effect (UVE) and reduce speech understanding in human-robot interaction (HRI). This paper outlines the development and testing of a novel Computer Aided Design (CAD) robotic mouth prototype with buccinator actuators for emulating the fluidic movements of the human mouth. The robotic mouth system incorporates a custom Machine Learning (ML) application that measures the acoustic qualities of speech synthesis (SS) and translates this data into servomotor triangulation for triggering jaw, lip and tongue positions. The objective of this study is to improve current robotic mouth design and provide engineers with a framework for increasing the authenticity, accuracy and communication capabilities of RHRs for HRI. The primary contributions of this study are the engineering of a robotic mouth prototype and the programming of a speech processing application that achieved a 79.4% syllable accuracy, 86.7% lip synchronisation accuracy and 0.1s speech to mouth articulation differential.