Mel Frequency Cepstrum Coefficients Research Articles

An ensemble-based framework for mispronunciation detection of Arabic phonemes

Determination of mispronunciations and ensuring feedback to users are maintained by computer-assisted language learning (CALL) systems. In this work, we introduce an ensemble model that defines the mispronunciation of Arabic phonemes and assists learning of Arabic, effectively. To the best of our knowledge, this is the very first attempt to determine the mispronunciations of Arabic phonemes employing ensemble learning techniques and conventional machine learning models, comprehensively. In order to observe the effect of feature extraction techniques, mel-frequency cepstrum coefficients (MFCC), and Mel-spectrogram are blended with each learning algorithm. To show the success of proposed model, 29 letters in the Arabic phonemes, 8 of which are hafiz, are voiced by a total of 11 different person. The amount of data set has been enhanced employing the methods of adding noise, time shifting, time stretching, pitch shifting. Extensive experiment results demonstrate that the utilization of voting classifier as an ensemble algorithm with Mel-spectrogram feature extraction technique exhibits remarkable classification result with 95.9% of accuracy.

Low Resource Language Analysis Using Deep Learning Algorithm for Gender Classification

Voice signals are the essential input source for applications based on human and computer interaction technology. Gender identification through voice signals is one of the most challenging tasks. For voice signal based analysis, deep learning algorithms provide an alternative to traditional and conventional algorithms for classification. To identify the gender through voice signals of female, male and ‘first-time’ transgender, the deep learning algorithm is used to improve the robustness of the identification model with the Mel Frequency Cepstrum Coefficients (MFCC) as a feature of the voice signals. This article presents the identification accuracy of gender with the help of recorded live voice signals. The voice samples of the third gender are recorded in the Hindi language. These Hindi language voice samples of transgender are very low resources and are unavailable at any recognized sources. The simulation results do not depend on the duration of the signals and are text independent. The recurrent neural network – Bidirectional Long Short-term Memory (RNN – BiLSTM) algorithm has been simulated on the recorded voice signals. The simulation outcome is compared with the earlier reported results in the literature. The gender-wise average accuracy of the proposed model is achieved as 91.44%, 94.94%, and 96.11% for males, females, and transgender, respectively, using voice signals. The identification accuracy of transgender is high in comparison to other genders. On the other hand, the average accuracy of the proposed model is obtained as 94.16%.

Human Age Estimation Through Audio Utilising MFCC and RNN

Age is one of human main attributes. Age is important factor to improve communication experience. Age estimation has been used in several applications to improve user experience. Therefore, an approach is needed to estimate the user age, one of which is through audio. In this study, Mel Frequency Cepstrum Coefficients (MFCC) and Recurrent Neural Network (RNN) will be used to estimate age through audio. MFCC is used to get features from audio data, while RNN is used to estimate age. Dataset used here was taken from corpus of user speech data on the Common Voice website. This study shows that MFCC and RNN methods are able to estimate human age through audio with highest accuracy obtained in SimpleRNN is 0.5647, and 0.7087 in LSTM.

Heart Rate Detection and Classification from Speech Spectral Features Using Machine Learning

Measurement of vital signs of the human body such as heart rate, blood pressure, body temperature and respiratory rate is an important part of diagnosing medical conditions and these are usually measured using medical equipment. In this paper, we propose to estimate an important vital sign – heart rate from speech signals using machine learning algorithms. Existing literature, observation and experience suggest the existence of a correlation between speech characteristics and physiological, psychological as well as emotional conditions. In this work, we estimate the heart rate of individuals by applying machine learning based regression algorithms to Mel frequency cepstrum coefficients, which represent speech features in the spectral domain as well as the temporal variation of spectral features. The estimated heart rate is compared with actual measurement made using a conventional medical device at the time of recording speech. We obtain estimation accuracy close to 94% between the estimated and actual measured heart rate values. Binary classification of heart rate as ‘normal’ or ‘abnormal’ is also achieved with 100% accuracy. A comparison of machine learning algorithms in terms of heart rate estimation and classification accuracy is also presented. Heart rate measurement using speech has applications in remote monitoring of patients, professional athletes and can facilitate telemedicine.

Exploring the ability of vocal biomarkers in distinguishing depression from bipolar disorder, schizophrenia, and healthy controls.

Vocal features have been exploited to distinguish depression from healthy controls. While there have been some claims for success, the degree to which changes in vocal features are specific to depression has not been systematically studied. Hence, we examined the performances of vocal features in differentiating depression from bipolar disorder (BD), schizophrenia and healthy controls, as well as pairwise classifications for the three disorders. We sampled 32 bipolar disorder patients, 106 depression patients, 114 healthy controls, and 20 schizophrenia patients. We extracted i-vectors from Mel-frequency cepstrum coefficients (MFCCs), and built logistic regression models with ridge regularization and 5-fold cross-validation on the training set, then applied models to the test set. There were seven classification tasks: any disorder versus healthy controls; depression versus healthy controls; BD versus healthy controls; schizophrenia versus healthy controls; depression versus BD; depression versus schizophrenia; BD versus schizophrenia. The area under curve (AUC) score for classifying depression and bipolar disorder was 0.5 (F-score = 0.44). For other comparisons, the AUC scores ranged from 0.75 to 0.92, and the F-scores ranged from 0.73 to 0.91. The model performance (AUC) of classifying depression and bipolar disorder was significantly worse than that of classifying bipolar disorder and schizophrenia (corrected p < 0.05). While there were no significant differences in the remaining pairwise comparisons of the 7 classification tasks. Vocal features showed discriminatory potential in classifying depression and the healthy controls, as well as between depression and other mental disorders. Future research should systematically examine the mechanisms of voice features in distinguishing depression with other mental disorders and develop more sophisticated machine learning models so that voice can assist clinical diagnosis better.

The structural tuning of the convolutional neural network forspeaker identification in mel frequency cepstrumcoefficients space

As a result of the literature analysis, the main methods for speaker identification from speech signals were defined. These are statistical methods based on Gaussian mixture model and a universal background model, as well as neural network methods, in particular, using convolutional or Siamese neural networks. The main characteristics of these methods are the recognition performance, a number of parameters, and the training time. High recognition performance is achieved by using convolutional neural networks, but a number of parameters of these networks are much higher than for statistical methods, although lower than for Siamese neural networks. A large number of parameters require a large training set, which is not always available for the researcher. In addition, despite the effectiveness of convolutional neural networks, model size and inference efficiency remain important fordevices with a limited source of computing power, such as peripheral or mobile devices. Therefore, the aspects of tuning of the structure of existing convolutional neural networks are relevant for research. In this work, we have performed a structural tuning of an existing convolutional neural network based on the VGGNet architecture for speaker identification in the space of mel frequency cepstrum coefficients. The aim of the work was to reduce the number of neural network parameters and, as a result, to reduce the network training time, provided that the recognition performance is sufficient (the correct recognition is above 95 %). The neural network proposed as a result of structural tuning has fewer layers than the architecture of the basic neural network. Insteadof the ReLU activation function, the related Leaky ReLU function with a parameter of 0.1 was used. The number of filters and the size of kernels in convolutional layers are changed. The size of kernels for the max pooling layer has been increased. It is proposedto use the averaging of the results of each convolution to input a two-dimensional convolution results to a fully connected layer with the Softmax activation function. The performed experiment showed that the number of parameters of the proposed neural network is 29 % less than the number of parameters of the basic neural network, provided that the speaker recognition performance is almost the same. In addition, the training time of the proposed and basic neural networks was evaluated on five datasets of audio recordings corresponding to different numbers of speakers. The training time of the proposed network was reduced by 10-39 % compared to the basic neural network. The results of the research show the advisability of the structural tuning of the convolutional neural network for devices with a limited source of computing, namely, peripheral or mobile devices.

Speech emotion recognition using multiple classification models based on MFCC feature values

In everyday life, people are often influenced by emotions in our behaviour and language. When people use different emotions to articulate the same text, it can have a completely different effect. With the increasing demand for speech emotion recognition (SER), more machine learning and deep learning methods are being used to perform SER. matlab was used as the experimental tool in this study. The Berlin Database of Emotional Speech was used as the database. Feature extraction was performed by Mel Frequency Cepstrum Coefficients (MFCC) and based on these feature values, Support Vector Machines (SVM), K-Nearest Neighbors Algorithm (KNN), Semi-supervised graph-based classifier, ECOC classification model, Naive Bayes model and long short-term memory for predictive classification. The results surface that among the six classifiers, the best sentiment recognition method is LSTM, with 93.2% and 73.03% accuracies in the training and test groups respectively.

Searchable encryption over encrypted speech retrieval scheme in cloud storage

Aiming at the privacy and security issues of sensitive speech data in cloud storage, and how to achieve efficient retrieval with privacy-preserving for encrypted speech data in cloud storage, a searchable encryption over encrypted speech retrieval scheme in cloud storage was proposed. Firstly, the data owner encrypts the original speech data adopting symmetric encryption algorithm Lorenz chaotic mapping. Secondly, the encrypted speech data is uploaded to the cloud server for storage, and meanwhile the Mel frequency cepstrum coefficient (MFCC) features of original speech are extracted as the input of the convolutional neural network (CNN) to execute the deep semantic feature extraction, which is taken as the speech keywords. Finally, the extracted speech keywords are encrypted and stored in the cloud. When the authorized user sends a retrieval request, the trained CNN is utilized to extract the keywords of the speech to be retrieved, and the search trapdoor is generated and sent to the cloud server. In the process of retrieval, the Euclidean distance is used to match the encrypted keywords with the search trapdoor. The theoretical analysis and experimental results show that the proposed scheme has higher security and retrieval accuracy, and can be suitable for the encrypted storage of speech data and efficient secure retrieval.

Speech Emotion Recognition Using Cascaded Attention Network with Joint Loss for Discrimination of Confusions

Due to the complexity of emotional expression, recognizing emotions from the speech is a critical and challenging task. In most of the studies, some specific emotions are easily classified incorrectly. In this paper, we propose a new framework that integrates cascade attention mechanism and joint loss for speech emotion recognition (SER), aiming to solve feature confusions for emotions that are difficult to be classified correctly. First, we extract the mel frequency cepstrum coefficients (MFCCs), deltas, and delta-deltas from MFCCs to form 3-dimensional (3D) features, thus effectively reducing the interference of external factors. Second, we employ spatiotemporal attention to selectively discover target emotion regions from the input features, where self-attention with head fusion captures the long-range dependency of temporal features. Finally, the joint loss function is employed to distinguish emotional embeddings with high similarity to enhance the overall performance. Experiments on interactive emotional dyadic motion capture (IEMOCAP) database indicate that the method achieves a positive improvement of 2.49% and 1.13% in weighted accuracy (WA) and unweighted accuracy (UA), respectively, compared to the state-of-the-art strategies.

Estimation of habit-related information from male voice data using machine learning-based methods

According to a survey on the cause of death among Japanese people, lifestyle-related diseases (such as malignant neoplasms, cardiovascular diseases, and pneumonia) account for 55.8% of all deaths. Three habits, namely, drinking, smoking, and sleeping, are considered the most important factors associated with lifestyle-related diseases, but it is difficult to measure these habits autonomously and regularly. Here, we propose a machine learning-based approach for detecting these lifestyle habits using voice data. We used classifiers and probabilistic linear discriminant analysis based on acoustic features, such as mel-frequency cepstrum coefficients (MFCCs) and jitter, extracted from a speech dataset we developed, and an X-vector from a pre-trained ECAPA-TDNN model. For training models, we used several classifiers implemented in MATLAB 2021b, such as support vector machines, K-nearest neighbors (KNN), and ensemble methods with some feature-projection options. Our results show that a cubic KNN method using acoustic features performs well on the sleep habit classification, while X-vector-based models perform well on smoking and drinking habit classifications. These results suggest that X-vectors may help estimate factors directly affecting the vocal cords and vocal tracts of the users (e.g., due to smoking and drinking), while acoustic features may help classify chronotypes, which might be informative with respect to the individuals’ vocal cord and vocal tract ultrastructure.

A study on the classification of Chinese folk instruments based on deep learning

Due to the Internet's tremendous expansion in recent years, digital music has seen explosive growth. It is becoming increasingly important to provide fast and accurate music retrieval for users, and music information retrieval has gradually become a research hotspot. Among them, the classification of musical instruments is one of the hot research directions. As an important part of the world's musical instruments, the Chinese folk musical instrument tradition has great research value, but there is less research in this direction. Traditional music classification methods can be roughly divided into two steps: manual feature extraction and traditional machine learning. But traditional music classification has several shortcomings, as follows. Firstly, manual extraction of music features has a high error rate and it is very difficult to ensure the accuracy of the features, and the labor cost is high; secondly, traditional machine learning still has problems with multiple classifications and training on large scale data. Therefore, in this paper, based on previous research, the original data features are extracted from the Mel Frequency Cepstrum Coefficient (MFCC), which is a feature that can represent musical timbre, and deep learning is proposed to be used to classify traditional Chinese folk instruments. This paper finds that the results of identifying and classifying Chinese folk musical instruments using different classical classification methods are compared when the training set and test set data are the same. According to the experimental results, the use of KNN has the best performance among these classification methods.

Sound-Based Abnormal Combustion Classification Model for High Compression Ratio, Spark-Ignition Engines Using Mel-Frequency Cepstrum Coefficients and Ensemble Learning Algorithms

Sound-Based Abnormal Combustion Classification Model for High Compression Ratio, Spark-Ignition Engines Using Mel-Frequency Cepstrum Coefficients and Ensemble Learning Algorithms

Artificial Intelligent for Human Emotion Detection with the Mel-Frequency Cepstral Coefficient (MFCC)

Emotions are an important aspect of human communication. Expression of human emotions can be identified through sound. The development of voice detection or speech recognition is a technology that has developed rapidly to help improve human-machine interaction. This study aims to classify emotions through the detection of human voices. One of the most frequently used methods for sound detection is the Mel-Frequency Cepstrum Coefficient (MFCC) where sound waves are converted into several types of representation. Mel-frequency cepstral coefficients (MFCCs) are the coefficients that collectively represent the short-term power spectrum of a sound, based on a linear cosine transform of a log power spectrum on a nonlinear mel scale of frequency. The primary data used in this research is the data recorded by the author. The secondary data used is data from the "Berlin Database of Emotional Speech" in the amount of 500 voice recording data. The use of MFCC can extract implied information from the human voice, especially to recognize the feelings experienced by humans when pronouncing the sound. In this study, the highest accuracy was obtained when training with epochs of 10000 times, which was 85% accuracy.

Raga classification using enhanced spatial bound whale optimization algorithm

A raga is a unique set of notes with certain rules that carefully followed, retain and protect its purity and produce amazing musical effects. An automated raga transcription and identification is important for computational musicology, which is an important step for musicology for indexing, classifying, and recommending tunes. In the present research, the audio features such as mel frequency cepstrum coefficients (MFCCs), spectral flux, short time energy, audio feature extractor, and spectral centroid features are used for the prediction of a raga. The model showed more complexity which means it required lots of training data. The proposed enhanced spatial bound whale optimization algorithm (ESBWOA) is used that overcome the feature selection problem of high dimensional features. In addition to this, a weighted salp swarm algorithm (SSA) is used for selecting the tone-based features from the ragas based on amplitude or each raga sample. The features were fed for bidirectional long short-term memory (Bi-LSTM) network, which enhanced the success rate for raga identification and classification. The present research uses CompMusic dataset in the research work where 9 classes for Carnatic music and 7 classes in Hindustani music are considered for the classification of ragas.

Coal-gangue recognition via multi-branch convolutional neural network based on MFCC in noisy environment

Traditional coal-gangue recognition methods usually do not consider the impact of equipment noise, which severely limits its adaptability and recognition accuracy. This paper mainly studies the more accurate recognition of coal-gangue in the noise site environment with the operation of shearer, conveyor, transfer machine and other device in the process of top coal caving. Mel Frequency Cepstrum Coefficients (MFCC) smoothing method was introduced to express the intrinsic feature of sound pressure more clearly in the coal-gangue recognition site. Then, a multi-branch convolution neural network (MBCNN) model with three branches was developed, and the smoothed MFCC feature was incorporated into this model to realize the recognition of falling coal and gangue in noisy environment. The sound pressure signal datasets under the operation of different device were constructed through a great deal of laboratory and site data acquisition. Comparative experiments were carried out on noiseless dataset, single noise dataset and simulated site dataset, and the results show that our method can provide higher correct recognition accuracy and better robustness. The proposed coal-gangue recognition approach based on MBCNN and MFCC smoothing can not only recognize the state of falling coal or gangue, but also recognize the operational state of site device.

Identification of oestrus cows based on vocalisation characteristics and machine learning technique using a dual-channel-equipped acoustic tag

Timely and accurate detection of oestrus in cows is an essential element of the good management of dairy farms. At present, the detection of cows in oestrus by acoustic means is impeded by the problems of filtering, incomplete feature selection, and poor recognition accuracy. To overcome these difficulties, this study proposes a sound detection method for cows in oestrus based on machine learning technology using an optimal feature combination and an optimal time window. Firstly, a dual-channel sound detection tag consisting of a unidirectional microphone and an omnidirectional microphone (OM) was developed. The Least Mean Squares adaptive algorithm based on wavelet thresholds was used to filter the signals from the OM, and the dual-channel endpoint detection algorithm was used to identify the lowing of individual cows. The Friedman analysis was then used to select the sound features with significant differences before and after oestrus in terms of time, frequency, and cepstrum, and these were used to determine the most acceptable feature combination. We then analysed the effects of Back Propagation Neural Network (BPNN), Cartesian Regression Tree, Support Vector Machine, and Random Forest classification on the accuracy, precision, sensitivity, specificity, and F1 score of oestrus discrimination. Different time windows were used, and the discrimination performance of these algorithms was evaluated using the Area under Receiver Operating Characteristic Curve to find the most satisfactory match between the time window and the recognition algorithm. The dual-channel acoustic tag’s accuracy, precision, sensitivity, and specificity results were 91.25, 98.83, 91.75, and 83.68%, respectively. BPNN with the 70 ms time window and the feature combination (spectral roll-off + spectral flatness + Mel-Frequency Cepstrum Coefficients) was confirmed as the most suitable oestrus recognition method. The average accuracy, precision, sensitivity, specificity, and F1 score of this method were 97.62, 98.07, 97.17, 97.19, and 97.63%, respectively. Based on these results, the approach was shown to be a feasible means of oestrus detection in dairy cows. Based on its ability to differentiate cows and its consistency, it was demonstrated that sound has the potential to replace accelerometers as an early indicator of oestrus in dairy cows.

Speech to Text Conversion System for Myanmar Alphabet

This paper is aimed to implement the speech to text conversion system for Myanmar alphabet. The Myanmar alphabet consists of 33 characters from ?ka? to ?ah?. The proposed system is software architecture which allows the user to speak against the computer in Myanmar language and the corresponding character is printed on the screen in the Microsoft Office Word Document Format. The system is emphasized on Speaker Independent Isolated Word Recognition System. The proposed system directly acquires and converts speech to text. This system contains two main modules: feature extraction and feature matching. Mel Frequency Cepstrum Coefficients (MFCC) is applied for feature extraction which extracts a small amount of data from the voice signal that can later be used to represent each character. Feature matching involves the actual procedure to identify the unknown character by comparing extracted features from the voice inputs of a set of known characters. In this system, Vector Quantization (VQ) approach using Linde, Buzo and Gray (LBG) clustering algorithm, which reduces the amount of data and complexity, is applied for feature matching. To implement this system MATLAB programming language is used.

Extreme-Low-Speed Heavy Load Bearing Fault Diagnosis by Using Improved RepVGG and Acoustic Emission Signals.

With the worldwide carbon neutralization boom, low-speed heavy load bearings have been widely used in the field of wind power. Bearing failure generates impulses when the rolling element passes the cracked surface of the bearing. Over the past decade, acoustic emission (AE) techniques have been used to detect failure signals. However, the high sampling rates of AE signals make it difficult to design and extract fault features; thus, deep neural network-based approaches have been proposed. In this paper, we proposed an improved RepVGG bearing fault diagnosis technique. The normalized and noise-reduced bearing signals were first converted into Mel frequency cepstrum coefficients (MFCCs) and then inputted into the model. In addition, the exponential moving average method was used to optimize the model and improve its accuracy. Data were extracted from the test bench and wind turbine main shaft bearing. Four damage classes were studied experimentally. The experimental results demonstrated that the improved RepVGG model could be employed for classifying low-speed heavy load bearing states by using MFCCs. Furthermore, the effectiveness of the proposed model was assessed by performing comparisons with existing models.

Fault diagnosis of transformer winding short circuit based on WKPCA-WM and IPOA-CNN

When the winding of the power transformer is short-circuited, the winding will experience constant vibration, which will cause axial instability of the winding, and then lead to winding looseness, deformation, bulge, etc., therefore, a diagnosis method based on the Improved Pelican Optimization Algorithm and Convolutional Neural Network (IPOA-CNN) for short-circuit voiceprint signal of transformer windings is proposed. At the same time, considering the input parameter dimension of deep learning cannot be too high, a new feature parameter selection method is constructed for this model. Firstly, the frequency characteristics of winding acoustic vibration signals are analyzed, and then the characteristic parameters of transformer acoustic signals are extracted by Wavelet Packet Energy Spectrum (WPES) and Mel Frequency Cepstrum Coefficient (MFCC), respectively. Then, the two methods are combined to construct the WM feature extraction algorithm, and the Weighted Kernel Principal Component Analysis (WKPCA) is used to reduce the dimension of the feature to obtain the feature parameters with accurate feature information and low redundancy; Finally, combined with Sobol sequence to optimize the initial population of Pelican Optimization Algorithm (POA), the convolution kernel of Convolutional neural network (CNN) was optimized by IPOA, and the optimal convolution kernel was obtained. The transformer winding short-circuits voiceprint diagnosis models of WKPCA-WM and IPOA-CNN were constructed, which realized the accurate diagnosis of winding short-circuit voiceprint. The validity and feasibility of the method are verified by the acoustic signal data collected in the laboratory.

Research on the mechanical fault diagnosis method based on sound signal and IEMD-DDCNN

PurposeThe purpose of this paper is to provide a shorter time cost, high-accuracy fault diagnosis method for water pumps. Water pumps are widely used in industrial equipment and their fault diagnosis is gaining increasing attention. Considering the time-consuming empirical mode decomposition (EMD) method and the more efficient classification provided by the convolutional neural network (CNN) method, a novel classification method based on incomplete empirical mode decomposition (IEMD) and dual-input dual-channel convolutional neural network (DDCNN) composite data is proposed and applied to the fault diagnosis of water pumps.Design/methodology/approachThis paper proposes a data preprocessing method using IEMD combined with mel-frequency cepstrum coefficient (MFCC) and a neural network model of DDCNN. First, the sound signal is decomposed by IEMD to get numerous intrinsic mode functions (IMFs) and a residual (RES). Several IMFs and one RES are then extracted by MFCC features. Ultimately, the obtained features are split into two channels (IMFs one channel; RES one channel) and input into DDCNN.FindingsThe Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection (MIMII dataset) is used to verify the practicability of the method. Experimental results show that decomposition into an IMF is optimal when taking into account the real-time and accuracy of the diagnosis. Compared with EMD, 51.52% of data preprocessing time, 67.25% of network training time and 63.7% of test time are saved and also improve accuracy.Research limitations/implicationsThis method can achieve higher accuracy in fault diagnosis with a shorter time cost. Therefore, the fault diagnosis of equipment based on the sound signal in the factory has certain feasibility and research importance.Originality/valueThis method provides a feasible method for mechanical fault diagnosis based on sound signals in industrial applications.