Voice Activity Detection Method Research Articles

Automatic detection of continuous pig cough in a complex piggery environment

Pig cough sound monitoring is an effective means of early warning for respiratory diseases. Until present, most studies focused on the investigation of high precision pig cough recognition algorithms based on manually segmented individual sound datasets. However, the recognition of continuous sound was mostly ignored, which cannot apply in practical engineering. Meanwhile, less consideration has been given to complex scenarios, such as the overlap of multiple sounds, which occur frequently in large-scale piggeries. To this end, we explored an automatic detection of continuous sound algorithm and proposed a continuous pig cough recognition method which has a significant role in the diagnosis of diseases. Initially, we proposed a voice activity detection (VAD) method to automatically segment continuous sound. Subsequently, we investigated a multi-classifier fusion strategy to promote recognition accuracy. Finally, we proposed a low-complexity continuous pig cough recognition method. The experimental results show that the recall and precision of pig cough in continuous sound is 93.1% and 91.6% respectively, which is much higher than the 67.3% and 90.6% of the baseline detection method. The recognition accuracy of continuous pig cough reached 91.4%. From the perspective of practical application, our algorithm development considered the complex environment of a real pig barn.

Efficient Pause Extraction and Encode Strategy for Alzheimer's Disease Detection Using Only Acoustic Features from Spontaneous Speech.

Clinical studies have shown that speech pauses can reflect the cognitive function differences between Alzheimer's Disease (AD) and non-AD patients, while the value of pause information in AD detection has not been fully explored. Herein, we propose a speech pause feature extraction and encoding strategy for only acoustic-signal-based AD detection. First, a voice activity detection (VAD) method was constructed to detect pause/non-pause feature and encode it to binary pause sequences that are easier to calculate. Then, an ensemble machine-learning-based approach was proposed for the classification of AD from the participants' spontaneous speech, based on the VAD Pause feature sequence and common acoustic feature sets (ComParE and eGeMAPS). The proposed pause feature sequence was verified in five machine-learning models. The validation data included two public challenge datasets (ADReSS and ADReSSo, English voice) and a local dataset (10 audio recordings containing five patients and five controls, Chinese voice). Results showed that the VAD Pause feature was more effective than common feature sets (ComParE: 6373 features and eGeMAPS: 88 features) for AD classification, and that the ensemble method improved the accuracy by more than 5% compared to several baseline methods (8% on the ADReSS dataset; 5.9% on the ADReSSo dataset). Moreover, the pause-sequence-based AD detection method could achieve 80% accuracy on the local dataset. Our study further demonstrated the potential of pause information in speech-based AD detection, and also contributed to a more accessible and general pause feature extraction and encoding method for AD detection.

Supervised Contrastive Learning for Voice Activity Detection

The noise robustness of voice activity detection (VAD) tasks, which are used to identify the human speech portions of a continuous audio signal, is important for subsequent downstream applications such as keyword spotting and automatic speech recognition. Although various aspects of VAD have been recently studied by researchers, a proper training strategy for VAD has not received sufficient attention. Thus, a training strategy for VAD using supervised contrastive learning is proposed for the first time in this paper. The proposed method is used in conjunction with audio-specific data augmentation methods. The proposed supervised contrastive learning-based VAD (SCLVAD) method is trained using two common speech datasets and then evaluated using a third dataset. The experimental results show that the SCLVAD method is particularly effective in improving VAD performance in noisy environments. For clean environments, data augmentation improves VAD accuracy by 8.0 to 8.6%, but there is no improvement due to the use of supervised contrastive learning. On the other hand, for noisy environments, the SCLVAD method results in VAD accuracy improvements of 2.9% and 4.6% for “speech with noise” and “speech with music”, respectively, with only a negligible increase in processing overhead during training.

Voice activity detection for audio signal of voyage data recorder using residue network and attention mechanism

ABSTRACT In this paper, a voice activity detection (VAD) method based on residue network and attention mechanism for VDR audio signal is proposed. First, several frame-wise acoustic features are extracted to eliminate audio data amount. Then, the frame-wise features are reshaped to construct 2-D feature maps. To facilitate parameter optimisation, a residue network is utilised herein to learn the complex mapping function from acoustic features to VAD output. Furthermore, to focus more on local information and suppress useless data, an attention mechanism is combined with the residue network to extract complex hidden information from the 2-D feature maps. Finally, the compressed 2-D features are flattened and refined by a dense layer to cover global information. The proposed method can automatically learn the mapping function efficiently and effectively. Experimental results show that the proposed method achieves the highest AUC, and the second highest ACC and F-measure compared with reference methods on the annotated real-world VDR audio dataset.

Voice activity detection using a local-global attention model

Voice activity detection (VAD) is an essential initial step of speech signal processing, and greatly affects the timeliness and accuracy of the system. Although many novel methods have been proposed to promote the performance of VAD under low signal-to-noise ratios (SNRs), the robustness to very low SNRs and unknown noisy environments has yet to be enhanced. In this paper, we propose a fusion model of local attention and global attention to strengthen the attention mechanism currently applied in VAD methods. First, based on self-attention, the local attention cooperates with long short-term memory networks (LSTMs) to achieve efficient use of local contextual information; and then the global attention measures global contextual information to focus on the most appropriate area of contextual frames. The experimental results show that compared with the state-of-the-art VAD methods, the proposed approach achieves better performance under low SNRs such as −15 dB and non-stationary noisy conditions.

A hybrid HXPLS‐TMFCC parameterization and DCNN‐SFO clustering based speaker diarization system

AbstractThe speaker diarization is considered to be the process by which the speaker signal is segmented, and the speaker identity is grouped into homogenous regions. The central point behind this scheme is the ability to distinguish between the speaker signal and each speaker signal with the label. As mass communication and meetings grow quickly, the diarization of the speakers is burden to improve the readability of the speech transcript. To solve this problem, tangent weighted mel‐frequency cepstral coefficient (TMFCC) and the extended linear prediction with autocorrelation snapshot feature extraction and the speaker diarization approach proposes a deep convolutional neural network (DCNN) for clustering and optimization using sailfish optimizer. A new development in the HXLPS extraction method is the holoentropy with extended linear prediction with autocorrelation snapshot. TMFCC makes more efficient and improves the effectiveness of the proposed scheme using lesser energy frame and higher energy framework. When achieve this, the voice activity detection method can recognize speech and non‐speech signals. Therefore, every segmented signal is represented by the d‐vector. The label of the speaker signal is clustered according to the speaker label used in the DCNN. The evaluation methods, like tracking distance, false alarm rate, diarization error rate examine the effectiveness.

An improved voice activity detection method based on spectral features and neural network

The recognition accuracy of speech signal and noise signal is greatly affected under low signal-to-noise ratio. The neural network with parameters obtained from the training set can achieve good results in the existing data, but is poor for the samples with different the environmental noises. This method firstly extracts the features based on the physical characteristics of the speech signal, which have good robustness. It takes the 3-second data as samples, judges whether there is speech component in the data under low signal-to-noise ratios, and gives a decision tag for the data. If a reasonable trajectory which is like the trajectory of speech is found, it is judged that there is a speech segment in the 3-second data. Then, the dynamic double threshold processing is used for preliminary detection, and then the global double threshold value is obtained by K-means clustering. Finally, the detection results are obtained by sequential decision. This method has the advantages of low complexity, strong robustness, and adaptability to multi-national languages. The experimental results show that the performance of the method is better than that of traditional methods under various signal-to-noise ratios, and it has good adaptability to multi language.

Analysis of derivative of instantaneous frequency and its application to voice activity detection

Recently, the use of speech processing in a time–frequency domain that handles the phase spectrum in addition to the amplitude one has been increasing because many studies have revealed the importance of phase. Inspired by this motivation, this paper presents a new speech signal phase processing. The contributions of this paper include the following two points: a detailed analysis of a speech signal by considering a novel phase feature, a derivative of instantaneous frequency (DIF), and demonstrating a new phase-based voice activity detection (VAD) algorithm as one of the DIF’s applications. These contributions develop on our previous work that briefly introduced the DIF into VAD. In the analysis part, we investigate the DIF from the theoretical aspect and research the statistical distribution of the DIF of speech signals under various conditions. We also propose a new phase-based VAD algorithm via the statistical likelihood ratio, and use the DIF as an auxiliary feature to improve a conventional amplitude-based VAD method. The experimental results confirm the efficacy of the phase feature in the VAD application and the possibility of combining the phase and amplitude for better performance.

Speech enhancement and encoding by combining SS-VAD and LPC

In this paper, the encoding of noisy and enhanced speech data is demonstrated. To encode and enhance the speech data under an uncontrolled environment, the linear predictive coding (LPC) and spectral subtraction with voice activity detection (SS-VAD) methods are studied individually. The noisy speech data is obtained by considering the amalgamation of the clean speech signal and noise model and it is encoded using the LPC technique. The LPC uses a lossy compression procedure to encode the speech data which converts the data rate from 64 to 2.4 Kbps. Due to reverberations and degradations in noisy speech data, the quality of encoded noisy speech data is very less. Therefore, an algorithm is proposed to enhance and encode the speech data by combining SS-VAD and LPC under degraded conditions. In the first step, the encoding of noisy speech data is done using LPC and its performance is evaluated using signal-to-ratio. The noisy speech data is given as input to the SS-VAD algorithm and the output of SS-VAD is given as input to the LPC encoder is followed in the second step. In the LPC encoder, the coefficients are extracted from the input speech data to design all-pole filters. The cross correlation process is also done for differentiating the voiced and unvoiced samples at the analysis step. The pitch information and extracted coefficients are used in the synthesis step. The experiments are conducted for different types of noisy speech data which are degraded by musical noise, F16 noise, factory noise, and car noise. The experimental results show that there is a significant improvement in the quality of enhanced encoded speech data obtained by the proposed method compared to encoded noisy speech data. The schematic representation of outputs of LPC and proposed combined SS-VAD and LPC waveforms are also given in this work.

RealVAD: A Real-World Dataset and A Method for Voice Activity Detection by Body Motion Analysis

We present an automatic voice activity detection (VAD) method that is solely based on visual cues. Unlike traditional approaches processing audio, we show that upper body motion analysis is desirable for the VAD task. The proposed method consists of components for body motion representation, feature extraction from a Convolutional Neural Network (CNN) architecture and unsupervised domain adaptation. The body motion representations as images are used by the feature extraction component, which is generic and person-invariant, thus, can be applied to a subject who has never been seen. The endmost component handles the domain-shift problem, which appears due to the fact that the way people move/ gesticulate while speaking might vary from subject to subject, which results in disparate body motion features and consequently poorer VAD performance. The experimental analyses applied on a publicly available real-world VAD dataset show that the proposed method performs better than the state-of-the-art video-only and multimodal VAD approaches. Moreover, the proposed method has a better generalization ability as VAD results are more consistent across different subjects. As another major contribution, we present a new multimodal dataset (called RealVAD), created from a real-world (no role-plays) panel discussion. This dataset contains many actual situations/ challenges that are missing in the previous VAD datasets. We benchmarked the RealVAD dataset by applying the proposed method as well as cross-dataset analyses. Particularly, the results of cross-dataset experiments highlight the remarkable positive contribution of the unsupervised domain adaptation applied.

Multi-Task Learning U-Net for Single-Channel Speech Enhancement and Mask-Based Voice Activity Detection

In this paper, a multi-task learning U-shaped neural network (MTU-Net) is proposed and applied to single-channel speech enhancement (SE). The proposed MTU-based SE method estimates an ideal binary mask (IBM) or an ideal ratio mask (IRM) by extending the decoding network of a conventional U-Net to simultaneously model the speech and noise spectra as the target. The effectiveness of the proposed SE method was evaluated under both matched and mismatched noise conditions between training and testing by measuring the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). Consequently, the proposed SE method with IRM achieved a substantial improvement with higher average PESQ scores by 0.17, 0.52, and 0.40 than other state-of-the-art deep-learning-based methods, such as the deep recurrent neural network (DRNN), SE generative adversarial network (SEGAN), and conventional U-Net, respectively. In addition, the STOI scores of the proposed SE method are 0.07, 0.05, and 0.05 higher than those of the DRNN, SEGAN, and U-Net, respectively. Next, voice activity detection (VAD) is also proposed by using the IRM estimated by the proposed MTU-Net-based SE method, which is fundamentally an unsupervised method without any model training. Then, the performance of the proposed VAD method was compared with the performance of supervised learning-based methods using a deep neural network (DNN), a boosted DNN, and a long short-term memory (LSTM) network. Consequently, the proposed VAD methods show a slightly better performance than the three neural network-based methods under mismatched noise conditions.

An Efficient Android-Based Multimodal Biometric Authentication System With Face and Voice

Multimodal biometric authentication method can conquer the defects of the unimodal biometric authentication technology. In this paper, we design and develop an efficient Android-based multimodal biometric authentication system with face and voice. Considering the hardware performance restriction of the smart terminal, including the random access memory (RAM), central processing unit (CPU) and graphics processor unit (GPU), etc., which cannot efficiently accomplish the tasks of storing and quickly processing the large amount of data, a face detection method is introduced to efficiently discard the redundant background of the image and reduce the unnecessary information. Furthermore, an improved local binary pattern (LBP) coding method is presented to improve the robustness of the extracted face feature. We also improve the conventional endpoint detection technology, i.e. the voice activity detection (VAD) method, which can efficiently increase the detection accuracy of the voice mute and transition information and boost the voice matching effectiveness. To boost the authentication accuracy and effectiveness, we present an adaptive fusion strategy which organically integrates the merits of the face and voice biometrics simultaneously. The cross-validation experiments with public databases demonstrate encouraging authentication performances compared with some state-of-the-art methods. Extensive testing experiments on Android-based smart terminal show that the developed multimodal biometric authentication system achieves perfect authentication effect and can efficiently content the practical requirements.

Noise Robust Voice Activity Detection Based on Multi-Layer Feed-Forward Neural Network

DOI: 10.26650/electrica.2019.18042 This paper proposes a voice activity detection (VAD) method based on time and spectral domain features using multi-layer feed-forward neural network (MLF-NN) for various noisy conditions. In the proposed method, time features that were short-time energy and zero-crossing rate and spectral features that were entropy, centroid, roll-off, and flux of speech signals were extracted. Clean speech signals were used in training MLF-NN and the network was tested for noisy speech at various noisy conditions. The proposed VAD method was evaluated for six kinds of noises which are white, car, babble, airport, street, and train at four different signal-to-noise ratio (SNR) levels. The proposed method was tested on core TIMIT database and its performance was compared with SOHN, G.729B and Long-Term Spectral Flatness (LSFM) VAD methods in point of correct speech rate, false alarm rate, and overall accuracy rate. Extensive simulation results show that the proposed method gives the most successful average correct speech rate, false alarm rate, and overall accuracy rate in most low and high SNR level conditions for different noise environments. Cite this article as : Arslan O, Engin EZ. Noise Robust Voice Activity Detection Based on Multi-Layer Feed-Forward Neural Network. Electrica 2019; 10.26650/electrica.2019.18042.

RVAD: An unsupervised segment-based robust voice activity detection method

This paper presents an unsupervised segment-based method for robust voice activity detection (rVAD). The method consists of two passes of denoising followed by a voice activity detection (VAD) stage. In the first pass, high-energy segments in a speech signal are detected by using a posteriori signal-to-noise ratio (SNR) weighted energy difference and if no pitch is detected within a segment, the segment is considered as a high-energy noise segment and set to zero. In the second pass, the speech signal is denoised by a speech enhancement method, for which several methods are explored. Next, neighbouring frames with pitch are grouped together to form pitch segments, and based on speech statistics, the pitch segments are further extended from both ends in order to include both voiced and unvoiced sounds and likely non-speech parts as well. In the end, a posteriori SNR weighted energy difference is applied to the extended pitch segments of the denoised speech signal for detecting voice activity. We evaluate the VAD performance of the proposed method using two databases, RATS and Aurora-2, which contain a large variety of noise conditions. The rVAD method is further evaluated, in terms of speaker verification performance, on the RedDots 2016 challenge database and its noise-corrupted versions. Experiment results show that rVAD is compared favourably with a number of existing methods. In addition, we present a modified version of rVAD where computationally intensive pitch extraction is replaced by computationally efficient spectral flatness calculation. The modified version significantly reduces the computational complexity at the cost of moderately inferior VAD performance, which is an advantage when processing a large amount of data and running on low resource devices. The source code of rVAD is made publicly available.

Long-term speech information based threshold for voice activity detection in massive microphone network

Voice activity detection (VAD) is essential for multiple microphone arrays processing, in which massive potential devices, such as microphone devices for far-field voice-based interaction in smart home environments, will be activated when sound sources appear. Therefore, the VAD can save a lot of computing resources in massive microphone arrays processing for the sparsity in sound source activity. However, it may not be feasible to obtain an accurate VAD in harsh environments, such as far-field, time-varying noise field. In this paper, the long-term speech information (LTSI) and the log-energy are modeled for deriving a more accurate VAD. First, the LTSI can be obtained by measuring the differential entropy of long-term smoothed noisy signal spectrum. Then, the LTSI is used to get labeled data for the initialization of a Gaussian mixture model (GMM), which is used to fit the log-energy distribution of noise and (noisy) speech. Finally, combining the LTSI and the GMM parameters of noise and speech distribution, this paper derives an adaptive threshold, which represents a reasonable boundary between noise and speech. Experimental results show that our VAD method has a remarkable improvement for a massive microphone network.

Voice Activity Detection for Transient Noisy Environment Based on Diffusion Nets

We address voice activity detection in acoustic environments of transients and stationary noises, which often occur in real life scenarios. We exploit unique spatial patterns of speech and non-speech audio frames by independently learning their underlying geometric structure. This process is done through a deep encoder-decoder based neural network architecture. This structure involves an encoder that maps spectral features with temporal information to their low-dimensional representations, which are generated by applying the diffusion maps method. The encoder feeds a decoder that maps the embedded data back into the high-dimensional space. A deep neural network, which is trained to separate speech from non-speech frames, is obtained by concatenating the decoder to the encoder, resembling the known Diffusion nets architecture. Experimental results show enhanced performance compared to competing voice activity detection methods. The improvement is achieved in both accuracy, robustness and generalization ability. Our model performs in a real-time manner and can be integrated into audio-based communication systems. We also present a batch algorithm which obtains an even higher accuracy for off-line applications.

Efficient harmonic peak detection of vowel sounds for enhanced voice activity detection

Voice activity detection (VAD) involves discriminating speech segments from background noise and is a critical step in numerous speech-related applications. However, distinguishing speech from noise based on the properties of noise is fallible, because it is difficult to predict and characterise the noise occurring in real life. In this study, the authors instead focus on the intrinsic characteristics of speech. The harmonic peaks of vowel sounds have higher energies than the other spectral components of speech and are the speech features most likely to survive in most cases of severe noise. Therefore, the energy differences between harmonic peaks and other spectral features show promise for enabling robust VAD. To exploit this feature, the harmonic peaks must be accurately located. For this purpose, this study proposes an efficient harmonic peak location detection (HPD) method. Based on extensive experiments conducted in the presence of various noise types and signal-to-noise ratios, we found that VAD with the proposed HPD approach outperforms existing VAD methods and does so with reasonable computational cost and higher robustness.

Enhancement of speech dynamics for voice activity detection using DNN

Voice activity detection (VAD) is an important preprocessing step for various speech applications to identify speech and non-speech periods in input signals. In this paper, we propose a deep neural network (DNN)-based VAD method for detecting such periods in noisy signals using speech dynamics, which are time-varying speech signals that may be expressed as the first- and second-order derivatives of mel cepstra, also known as the delta and delta-delta features. Unlike these derivatives, in this paper, the dynamics are highlighted by speech period candidates, which are calculated based on heuristic rules for the patterns of the first and second derivatives of the input signals. These candidates, together with the log power spectra, are input into the DNN to obtain VAD decisions. In this study, experiments are conducted to compare the proposed method with a DNN-based method, which exclusively utilizes log power spectra by using speech signals smeared with five types of noise (white, babble, factory, car, and pink) with signal-to-noise ratios (SNRs) of 10, 5, 0, and − 5 dB. The experimental results show that the proposed method is superior under all the considered noise conditions, indicating that the speech period candidates improve the log power spectra.

Voice Activity Detection Using an Adaptive Context Attention Model

Voice activity detection (VAD) classifies incoming signal segments into speech or background noise; its performance is crucial in various speech-related applications. Although speech-signal context is a relevant VAD asset, its usefulness varies in unpredictable noise environments. Therefore, its usage should be adaptively adjustable to the noise type. This letter improves the use of context information by using an adaptive context attention model (ACAM) with a novel training strategy for effective attention, which weights the most crucial parts of the context for proper classification. Experiments in real-world scenarios demonstrate that the proposed ACAM-based VAD outperforms the other baseline VAD methods.

Innovative Method for Unsupervised Voice Activity Detection and Classification of Audio Segments

An accurate and noise-robust voice activity detection (VAD) system can be widely used for emerging speech technologies in the fields of audio forensics, wireless communication, and speech recognition. However, in real-life application, the sufficient amount of data or human-annotated data to train such a system may not be available. Therefore, a supervised system for VAD cannot be used in such situations. In this paper, an unsupervised method for VAD is proposed to label the segments of speech-presence and speech-absence in an audio. To make the proposed method efficient and computationally fast, it is implemented by using long-term features that are computed by using the Katz algorithm of fractal dimension estimation. Two databases of different languages are used to evaluate the performance of the proposed method. The first is Texas Instruments Massachusetts Institute of Technology (TIMIT) database, and the second is the King Saud University (KSU) Arabic speech database. The language of TIMIT is English, while the language of the KSU speech database is Arabic. TIMIT is recorded in only one environment, whereas the KSU speech database is recorded in distinct environments using various recording systems that contain sound cards of different qualities and models. The evaluation of the proposed method suggested that it labels voiced and unvoiced segments reliably in both clean and noisy audio.