Speech Database Research Articles

Grammatical Error Correction Detection of English Conversational Pronunciation Under a Deep Learning Algorithm

Grammar correction in spoken English can enhance proficiency. This paper briefly introduces the gate recurrent unit (GRU) algorithm and its appli- cation in English speech recognition and grammatical error correction of speech recognition results. The GRU algorithm was firstly used to recognize English speech, then transform it into a text, and finally correct the English grammar of the text. Additionally, the attention mechanism was incorporated to enhance the performance of grammatical error correction. Subsequently, simulation experiments were conducted. Firstly, speech recognition and grammatical error correction were independently verified. The performance of the proposed algorithm in correcting grammatical errors in spoken English was evaluated using a self-built speech database. The results demonstrated that the proposed GRU-based algorithm yielded the best performance in independent speech recognition, independent grammatical error correction, and the overall spoken grammatical error correction. The contribution of this study lies in using the GRU algorithm to convert speech into text and perform grammar correction on the text, providing an effective reference for grammar correction in English communication.

Spectral Subtraction Based Weighted Function for Pitch Extraction in Noisy Speech

Many speech-related works employ the pitch period as a crucial component. Speech signals are typically collected in challenging noisy settings for real-world projects. Therefore, it is now more important than ever for the algorithm to be noise resistant in order to estimate pitch accurately. However, when dealing with noisy speech files at a low signal-to-noise ratio (SNR) value, many state-of-the-art algorithms are unable to produce satisfactory results. In this work, a new noise-resistant pitch estimation algorithm based on spectral subtraction is presented, which uses a weighted function to lessen the impact of the vocal tract effect. Furthermore, to enhance the correlation between the pitch estimates and smoothen the pitch contours, we employ a weighted function that combines the spectral subtraction-based technique as the numerator and the circular average magnitude difference function (CAMDF) as the denominator. We have utilized two noisy speech databases using seven different kinds of recorded ambient noise, and we evaluated our system against three cutting-edge methods. The suggested method lowers the Gross Pitch Error (GPE) rate at practically all SNRs in white noise and performs best on the NTT and KEELE databases.

Optimized Multimodal Emotional Recognition Using Long Short-Term Memory

The aim of this project is to research and classification on human emotions. A new method for the recognition of speech signals has been introduced. It’s called LSTM (Long-Short Term Memory). It is a type of Recurrent neural network. RNN is used for analyzing sequential data, hence it is useful for speech signal recognition. Several Datasets were found across the internet for this project. Ex: TESS (Toronto Emotional Speech Set), RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song), SAVEE (Surrey Audio-Visual Expressed Emotion), CREMA-D (Crowd- Sourced Emotional Multimodal Actors Dataset). The Main Dataset used in this project is TESS (Toronto Emotional Speech Set) Dataset and Mel Frequency Cepstral Coefficient (MFCC) is Used for Feature extraction.

Cross-modal learning representation using new margin combination for speech recognition task

Cross-modal retrieval aims to elucidate the fusion of information, mimic human learning, and advance the field. The main challenge in cross-modal matching is to build a shared subspace reflecting semantic proximity. Previous works fail to capture asymmetric relevance by adopting symmetric similarity computations. To overcome these shortcomings, an efficient approach called quaternion representation learning (QRL) is introduced for better cross-modal matching. Thus, a better representation of the shared semantics is offered by virtue of its richer representation capacity of the quaternionic space and its strong expressive power. Transfer learning is a crucial aspect in this context. By leveraging pre-trained models, the knowledge gained from one task or domain can be effectively transferred to another, allowing for improved performance and generalization. In this study, transfer learning is employed to enhance the cross-modal retrieval system. Specifically, a pre-trained ResNet-512 model is utilized in conjunction with the proposed total margin (TM) loss function, which combines the QRL approach with the novel adaptive mean margin (AMM) methodology. The TM loss function, coupled with the pre-trained ResNet-512 model, is evaluated on the Audio-Visual Arabic Speech Database (AVAS) and the Arabic Visual Speech Database (AVSD), along with other audio-visual datasets. Experimental results demonstrate the effectiveness of the TM loss function in consistently improving performance on both databases. The recall scores (R@k) and mean average precision (mAP) values achieved on the AVAS Database are as follows: R@1: 42.1±0.7, R@2: 70.2±0.1, R@5: 78.5±1.0, and mAP: 53.0±1.1. Similarly, on the AVSD Database, the results are R@1: 41.7±0.3, R@2: 69.2±1.1, R@5: 78.0±0.3, and mAP: 52.7±0.5. By incorporating transfer learning and the TM loss function into the cross-modal retrieval framework, this study demonstrates the potential for improving clustering efficiency and enhancing visual and speech understanding. The combination of pre-trained models and the TM loss function offers a promising avenue for advancing crossmodal matching techniques and achieving state-of-the-art performance.

Scalability and diversity of StarGANv2-VC in Arabic emotional voice conversion: Overcoming data limitations and enhancing performance

Emotional Voice Conversion (EVC) for under-resourced languages like Arabic faces challenges due to limited emotional speech data. This study explored strategies to mitigate dataset scarcity and improve Arabic EVC performance. Fundamental experiments (Speaker-Dependent, Gender-Dependent, Gender-Independent) were conducted using the KSUEmotions dataset to analyze speaker, gender, and model impacts. Data augmentation techniques like time stretching and phase shuffling artificially increased data diversity. Attention mechanisms integrated into StarGANv2-VC aimed to better capture emotional cues. Transfer learning leveraged the larger English Emotional Speech Database (ESD) to enhance the Arabic system. A novel “Reordering Speaker-Emotion Data” approach treated each emotion as a separate speaker to expand the emotional variability. Our comprehensive approach, combining transfer learning, data augmentation, and architectural modifications, demonstrates the potential to overcome dataset limitations and enhance the performance of Arabic EVC systems.

Narrowing or in Kálmán's writings

AbstractThe paper investigates the usage of the Hungarian connective vagy ‘or’. Our starting point is Ariel & Mauri's (2018, 2019) and Ariel's (2020) papers about the use of or, where they argue that its core meaning is ‘alternativity’. Our goal is to describe Hungarian vagy ‘or’ by analyzing various corpus data, and compare the results. We examined the personal subcorpus of the Hungarian National Corpus (MNSZ2), and the Hungarian Spontaneous Speech Database (BEA). In this paper, as a tribute to the memory of László Kálmán, we investigated a third corpus that is constructed from Kálmán's very popular informative texts on Qubit.

Combining Transformer, Convolutional Neural Network, and Long Short-Term Memory Architectures: A Novel Ensemble Learning Technique That Leverages Multi-Acoustic Features for Speech Emotion Recognition in Distance Education Classrooms

Speech emotion recognition (SER) is a technology that can be applied to distance education to analyze speech patterns and evaluate speakers’ emotional states in real time. It provides valuable insights and can be used to enhance students’ learning experiences by enabling the assessment of their instructors’ emotional stability, a factor that significantly impacts the effectiveness of information delivery. Students demonstrate different engagement levels during learning activities, and assessing this engagement is important for controlling the learning process and improving e-learning systems. An important aspect that may influence student engagement is their instructors’ emotional state. Accordingly, this study used deep learning techniques to create an automated system for recognizing instructors’ emotions in their speech when delivering distance learning. This methodology entailed integrating transformer, convolutional neural network, and long short-term memory architectures into an ensemble to enhance the SER. Feature extraction from audio data used Mel-frequency cepstral coefficients; chroma; a Mel spectrogram; the zero-crossing rate; spectral contrast, centroid, bandwidth, and roll-off; and the root-mean square, with subsequent optimization processes such as adding noise, conducting time stretching, and shifting the audio data. Several transformer blocks were incorporated, and a multi-head self-attention mechanism was employed to identify the relationships between the input sequence segments. The preprocessing and data augmentation methodologies significantly enhanced the precision of the results, with accuracy rates of 96.3%, 99.86%, 96.5%, and 85.3% for the Ryerson Audio–Visual Database of Emotional Speech and Song, Berlin Database of Emotional Speech, Surrey Audio–Visual Expressed Emotion, and Interactive Emotional Dyadic Motion Capture datasets, respectively. Furthermore, it achieved 83% accuracy on another dataset created for this study, the Saudi Higher-Education Instructor Emotions dataset. The results demonstrate the considerable accuracy of this model in detecting emotions in speech data across different languages and datasets.

Enhancing speech emotion recognition with deep learning using multi-feature stacking and data augmentation

This study evaluates the effectiveness of data augmentation on 1D convolutional neural network (CNN) and transformer models for speech emotion recognition (SER) on the Ryerson audio-visual database of emotional speech and song (RAVDESS) dataset. The results show that data augmentation has a positive impact on improving emotion classification accuracy. Techniques such as noising, pitching, stretching, shifting, and speeding are applied to increase data variation and overcome class imbalance. The 1D CNN model with data augmentation achieved 94.5% accuracy, while the transformer model with data augmentation performed even better at 97.5%. This research is expected to contribute better insights for the development of accurate emotion recognition methods by using data augmentation with these models to improve classification accuracy on the RAVDESS dataset. Further research can explore larger and more diverse datasets and alternative model approaches.

Emotions recognition in audio signals using an extension of the latent block model

Emotion detection in human speech is a significant area of research, crucial for various applications such as affective computing and human–computer interaction. Despite advancements, accurately categorizing emotional states in speech remains challenging due to its subjective nature and the complexity of human emotions. To address this, we propose leveraging Mel frequency cepstral coefficients (MFCCS) and extend the latent block model (LBM) probabilistic clustering technique with a Gaussian multi-way latent block model (GMWLBM). Our objective is to categorize speech emotions into coherent groups based on the emotional states conveyed by speakers. We employ MFCCS from time-series audio data and utilize a variational Expectation Maximization method to estimate GMWLBM parameters. Additionally, we introduce an integrated Classification Likelihood (ICL) model selection criterion to determine the optimal number of clusters, enhancing robustness. Numerical experiments on real data from the Berlin Database of Emotional Speech (EMO-DB) demonstrate our method’s efficacy in accurately detecting and classifying emotional states in human speech, even in challenging real-world scenarios, thereby contributing significantly to affective computing and human–computer interaction applications.

Strategy for developing a speech recognition model specialized for patients with depression or Parkinson's disease with small size speech database.

Most of speech recognition models currently in use have been dealt with speech of normal people. The speech recognition rate for patients with depression or Parkinson's disease (PD) who show differences in speech characteristics compared to normal subjects is lower than that of normal subjects. This study explores the model to enhance accuracy of speech recognition for individuals who have depression or PD, aiming to provide them more accurate service. In this study, considering the speech features of patients with depression or PD, we designed a model with the assumption that understanding the overall meaning and context of speech through the utilization of global information, rather than local information, is more effective in enhancing recognition accuracy. We propose the m-Globalformer, a model based on the Globalformer architecture that combines the squeeze-and-excitation (SE) module with the Transformer. The m-Globalformer enhances the utilization of global information by modifying the base SE module. The model employs pre-training and fine-tuning strategies, considering the limited speech data of the patients. In the initial training phase, a large-scale normal speech dataset was used, followed by fine-tuning the model with a small-scale dataset of depression or PD patients. The m-Globalformer demonstrated superior performance in our experiments, achieved character error rates (CER) of 11.28% for depression and 19.67% for PD.

A Comparative Study of Khasi Speech Recognition Systems with Recurrent Neural Network-Based Language Model

This paper offers a comparative analysis of Khasi speech recognition systems utilizing a recurrent neural network-based language model (RNN-LM). Develop different acoustic models (AMs) to evaluate the optimal performance. This paper observed that using RNN-LM performed best than traditional other models. The wave surfer performs data processing followed by collecting the recorder based continuous speech database. Moreover, a minimization of word error rate (WER) in 2.83.8% range for major speech data and 2.4-3.5% for minor speech data. Additionally, two acoustic features are used, and from the experimental results, the Mel frequency cepstral coefficient (MFCC) yielded improved performance than the perceptual linear prediction (PLP).

A multi-level power grid enhanced identity authentication data management platform based on filtering algorithms

In response to the optimal extraction of DCT coefficients in facial images, the author proposes a DCT coefficient extraction method based on discriminant analysis. Based on the discriminant analysis of DCT coefficients, the DCT coefficients with high discriminant values are selected as features. Comparing the DPA based discrete cosine coefficient selection method proposed by the author with the traditional Zigzag discrete cosine coefficient selection method, experiments were conducted on the ORL face database and the Yale face database, respectively. The recognition performance on the ORL face database was higher than that on the Yale face database, as the facial image expression and lighting changes in the ORL database were relatively few, making it suitable for extracting key features. In response to the problem that the speech parameter MFCC is greatly affected by noise and can only reflect the static characteristics of speech, the author extracted gamma pass filtering cepstrum coefficients with human auditory characteristics and gamma pass sliding differential cepstrum coefficients that can reflect the dynamic characteristics of speech based on gamma tone filters and sliding differential cepstrum. In the NUST603 speech database, under pure background, the recognition rate based on GFSDCC features reached 89.88%, and the recognition effect based on GFCC features was 87.52%, which is 4.66% and 2.36% higher than that based on MFCC features. In noisy environments, the average recognition rates of speaker recognition systems based on GFCC and GFSDCC are 56.06% and 59.07%, while the average recognition rates of speaker recognition systems based on MFCC speech features are 53.89%, 2.17% and 5.18% higher, respectively. The gain in this recognition effect comes from the characteristics of the auditory model, as the Gammatone filter effectively reflects the noise resistance of the human auditory system.

AROA based Pre-trained Model of Convolutional Neural Network for Voice Pathology Detection and Classification

With the demand for better, more user-friendly HMIs, voice recognition systems have risen in prominence in recent years. The use of computer-assisted vocal pathology categorization tools allows for the accurate detection of voice pathology diseases. By using these methods, vocal disorders may be diagnosed early on and treated accordingly. An effective Deep Learning-based tool for feature extraction-based vocal pathology identification is the goal of this project. This research presents the results of using EfficientNet, a pre-trained Convolutional Neural Network (CNN), on a speech pathology dataset in order to achieve the highest possible classification accuracy. An Artificial Rabbit Optimization Algorithm (AROA)-tuned set of parameters complements the model's mobNet building elements, which include a linear stack of divisible convolution and max-pooling layers activated by Swish. In order to make the suggested approach applicable to a broad variety of voice disorder problems, this study also suggests a unique training method along with several training methodologies. One speech database, the Saarbrücken voice database (SVD), has been used to test the proposed technology. Using up to 96% accuracy, the experimental findings demonstrate that the suggested CNN approach is capable of detecting speech pathologies. The suggested method demonstrates great potential for use in real-world clinical settings, where it may provide accurate classifications in as little as three seconds and expedite automated diagnosis and treatment.

Non-intrusive speech quality assessment: A survey

Speech quality is a critical consideration for applications such as speech enhancement, coding, transmission, and synthesis. Accurately evaluating the quality of degraded speech without a reference is particularly challenging. As a result, non-intrusive speech quality assessment has been extensively researched due to its efficiency and ease of deployment. This survey comprehensively reviews these studies. Commonly used speech quality assessment databases are reviewed, as they serve as validation sets for objective metrics. Non-intrusive speech quality assessment methods are categorized and reviewed, and the performance of state-of-the-art and classical speech quality assessment methods is compared. The survey presents an overview of classic algorithms and recent advancements in the field of non-intrusive speech quality assessment.

Performance analysis of a dilated attention fast GAN for speech enhancement

Recent advancements in speech enhancement have witnessed the emergence of generator-based methodologies. However, several of these approaches exhibit complexity in handling input variations, either excelling at low signal-to-noise ratios (SNRs) by utilizing intricate representations of noisy and clean speech or demonstrating superior performance only at higher SNRs. In this work, we investigated speech enhancement using a Dilated Attention Fast Generative Adversarial Network (DAF-GAN). The proposed DAF-GAN framework achieves stability in performance across different SNR conditions by efficiently processing large-scale signal lengths. The DFS-GAN features a dilated discriminator model operating via patches. The generator architecture incorporates multi-decoding and attention gates facilitated through skip-connections, strategically integrated within the Fast-U-Net model to optimize processing speed. An ideal ratio mask was used in the test phase to further refine the enhanced signal by emphasizing target speech while suppressing residual noise or artifacts. The DAF-GAN performance was assessed using objective metrics such as PESQ on a number of noisy speech databases. Results revealed that the DAF-GAN performed modestly in comparison with the state-of-the-art models. For example, analyses of the VoiceBank-DEMAND dataset yielded a PESQ score of 2.50 for the DAF-GAN.

MECOS: A bilingual Manipuri–English spontaneous code-switching speech corpus for automatic speech recognition

In this study, we introduce a new code-switched speech database with 57h of Manipuri–English annotated spontaneous speech. Manipuri is an official language of India and is primarily spoken in the north–eastern Indian state of Manipur. Most Manipur native speakers today are bilingual and frequently use code switching in everyday discussions. By carefully assessing the amount of code-switched speech in each video, recordings from YouTube are gathered. 21,339 utterances and 291,731 instances of code switching are present in the database. Given the code-switching nature of the data, a proper annotation procedure is used, and the data are manually annotated using the Meitei Mayek unicode font and the roman alphabets for Manipuri and English, respectively. The transcription includes the information of the speakers, non-speech information, and the corresponding annotation. The aim of this research is to construct an automatic speech recognition (ASR) system as well as offer a thorough analysis and details of the speech corpus. We believe that our research is the first to use an ASR system for Manipuri–English code-switched speech. To evaluate the performance, ASR systems based on hybrid deep neural network and hidden Markov model (DNN–HMM), time delay neural network (TDNN), hybrid time delay neural network and long short-term memory (TDNN–LSTM) and three end-to-end (E2E) models i.e. hybrid connectionist temporal classification and attention model (CTC-Attention), Conformer, wav2vec XLSR are developed for Manipuri–English language. In comparison to other models, pure TDNN produces outcomes that are clearly superior.

Text-independent speaker identification system using discrete wavelet transform with linear prediction coding

One of the key problems of the modern day is the presentation of an identity verification system that can perform sufficient accuracy in identity verification, is resilient to assaults and noises, and can be recorded in the simplest possible method. In this study, a new speaker feature extraction which based on discrete wavelet transform (DWT) and linear prediction coding (LPC) algorithm (WLPCA) are investigated. This paper's primary objective is to evidence the performance of the new method for speaker identification by a Gaussian mixture model (GMM). The proposed method improves the recognition rate over the Mel-frequency cepstral coefficient (MFCC). Experimental evaluation of the process performance is performed on two speech databases; our recorded database and the publicly available TIMIT database. We show that the speech features derived by the newly proposed method are more suitable for GMM (91.53%), in terms of the time-consumed, by requiring less Gaussian mixtures than MFCC (85.77%). For testing the presented method in a noisy environment, Additive white Gaussian noise (AWGN) was added to the TIMIT database, where a slight improvement in the performance of the presented method (60.02%) over the MFCC (59.89%) was observed.

Convolutional Neural Network Architectures for Gender, Emotional Detection from Speech and Speaker Diarization

This paper introduces three system architectures for speaker identification that aim to overcome the limitations of diarization and voice-based biometric systems. Diarization systems utilize unsupervised algorithms to segment audio data based on the time boundaries of utterances, but they do not distinguish individual speakers. On the other hand, voice-based biometric systems can only identify individuals in recordings with a single speaker. Identifying speakers in recordings of natural conversations can be challenging, especially when emotional shifts can alter voice characteristics, making gender identification difficult. To address this issue, the proposed architectures include techniques for gender, emotion, and diarization at either the segment or group level. The evaluation of these architectures utilized two speech databases, namely VoxCeleb and RAVDESS (Ryerson audio-visual database of emotional speech and song) datasets. The findings reveal that the proposed approach outperforms the strategy level in terms of recognition results, despite the real-time processing advantage of the latter. The challenge of identifying multiple speakers engaging in a conversation while considering emotional changes that impact speech is effectively addressed by the proposed architectures. The data indicates that the gender and emotion classification of diarization achieves an accuracy of over 98 percent. These results suggest that the proposed speech-based approach can achieve highly accurate speaker identification.

AVID: A speech database for machine learning studies on vocal intensity

Vocal intensity, which is quantified typically with the sound pressure level (SPL), is a key feature of speech. To measure SPL from speech recordings, a standard calibration tone (with a reference SPL of 94 dB or 114 dB) needs to be recorded together with speech. However, most of the popular databases that are used in areas such as speech and speaker recognition have been recorded without calibration information by expressing speech on arbitrary amplitude scales. Therefore, information about vocal intensity of the recorded speech, including SPL, is lost. In the current study, we introduce a new open and calibrated speech/electroglottography (EGG) database named Aalto Vocal Intensity Database (AVID). AVID includes speech and EGG produced by 50 speakers (25 males, 25 females) who varied their vocal intensity in four categories (soft, normal, loud and very loud). Recordings were conducted using a constant mouth-to-microphone distance and by recording a calibration tone. The speech data was labelled sentence-wise using a total of 19 labels that support the utilisation of the data in machine learning (ML) -based studies of vocal intensity based on supervised learning. In order to demonstrate how the AVID data can be used to study vocal intensity, we investigated one multi-class classification task (classification of speech into soft, normal, loud and very loud intensity classes) and one regression task (prediction of SPL of speech). In both tasks, we deliberately warped the level of the input speech by normalising the signal to have its maximum amplitude equal to 1.0, that is, we simulated a scenario that is prevalent in current speech databases. The results show that using the spectrogram feature with the support vector machine classifier gave an accuracy of 82% in the multi-class classification of the vocal intensity category. In the prediction of SPL, using the spectrogram feature with the support vector regressor gave an mean absolute error of about 2 dB and a coefficient of determination of 92%. We welcome researchers interested in classification and regression problems to utilise AVID in the study of vocal intensity, and we hope that the current results could serve as baselines for future ML studies on the topic.

Аналитический обзор методов автоматического анализа экстралингвистических компонентов спонтанной речи

The accuracy of automatic spontaneous speech recognition systems is far from that of trained speech recognition systems. This is due to the fact that spontaneous speech is not as smooth and failure-free as spontaneous speech. Spontaneous speech varies from speaker to speaker: the quality of phonemes’ pronunciation, the presence of pauses, speech disruptions and extralinguistic items (laughing, coughing, sneezing, and chuckling when expressing emotions of irritation, etc.) interrupt the fluency of verbal speech. However, it is worth noting that extralinguistic items very often carry important paralinguistic information, so it is crucial for automatic spontaneous speech recognition systems not only to identify such phenomena and distinguish them from the verbal components of speech but also to classify them. This review presents an analysis of works on the topic of automatic detection and analysis of extralinguistic items in spontaneous speech. Both individual methods and approaches to the recognition of extralinguistic items in a speech stream, and works related to the multiclass classification of isolatedly recorded extralinguistic units are considered and described. The most popular methods of extralinguistic units’ analysis are neural networks, such as deep neural networks and networks based on transformer models. The basic concepts related to the term extralinguistic items are given, the original systematization of extralinguistic items in the Russian language is proposed, the corpus and databases of audio spoken speech both in Russian and in other languages are described, the data sets of extralinguistic items recorded isolatedly are also given. The accuracy of extralinguistic items recognition increases with the following conditions of work with the speech signal: pre-processing of audio signals of items has shown an increase in the accuracy of separately recorded extralinguistic items classification; consideration of context (analysis of several frames of speech signal) and use of filters for smoothing the time series after extraction of feature vectors showed an increase in accuracy in frame-by-frame analysis of the speech signal with spontaneous speech.