Gammatone Research Articles

Voice Command Recognition Using Biologically Inspired Time-Frequency Representation and Convolutional Neural Networks.

Voice command is an important interface between human and technology in healthcare, such as for hands-free control of surgical robots and in patient care technology. Voice command recognition can be cast as a speech classification task, where convolutional neural networks (CNNs) have demonstrated strong performance. CNN is originally an image classification technique and time-frequency representation of speech signals is the most commonly used image-like representation for CNNs. Various types of time-frequency representations are commonly used for this purpose. This work investigates the use of cochleagram, utilizing a gammatone filter which models the frequency selectivity of the human cochlea, as the time-frequency representation of voice commands and input for the CNN classifier. We also explore multi-view CNN as a technique for combining learning from different time-frequency representations. The proposed method is evaluated on a large dataset and shown to achieve high classification accuracy.

Evaluation of localization precision by proposed quasi-spherical nested microphone array in combination with multiresolution adaptive steered response power

Abstract Multiple sound source localization in noisy and reverberant conditions is one of the important challenges in the speech signal processing. The aim of this article is three-dimensional sound source localization in undesirable scenarios. For the localization algorithms, the spatial aliasing is one of the destructive factors in reducing the accuracy. Firstly, a 3D quasi-spherical nested microphone array (QSNMA) is proposed for eliminating the spatial aliasing. Since the speech signal has the windowed-disjoint orthogonality property, the speech information differs in terms of the frequency bands. Then, the Gammatone filter bank is introduced for the speech subband processing. In the following, the multiresolution steered response power (SRP) algorithm is adaptively implemented on subbands with the phase transform (PHAT)/maximum likelihood (ML) weighted functions based on the levels of the noise and reverberation. The peaks of the multiresolution adaptive SRP (MASRP) algorithm are extracted in each subband based on the number of speakers for continuous time frames. Finally, the distribution of these peaks are calculated in each subband and they are merged by the use of weighted averaging method. The final 3D speakers locations are estimated by extracting the peaks in the final distribution. The proposed QSNMAMASRP(PHAT/ML) algorithm is evaluated on real and simulated data for 2 and 3 simultaneous speakers in noisy and reverberant conditions. The proposed method is compared with SRP-PHAT, spectral source model-deep neural network, and spherical harmonic temporal extension of multiple response model sparse Bayesian learning algorithms on different range of signal-to-noise ratio and reverberation time. The mean absolute estimation error, averaged standard deviation for absolute estimation error, and computational complexity results show the superiority of the proposed method.

Speech activity detection using time-frequency auditory spectral pattern

Speech activity detection (SAD) is useful to identify human speech from other sounds and ear plays an important role in this. Gammatone filter bank response mimics the human auditory periphery model. We propose an unsupervised SAD based on cochleagram features which are derived from gammatone filter bank-multiple window observation (GTFB-MWO) technique. Spectral clustering of cochleagram features is employed here with two strategies: correlation measure based non-speech input detection and feature winning score to identify the speech and non-speech clusters. The proposed SAD performance is compared with different baseline unsupervised SAD techniques on 600 min of noisy speech corpus over different signal-to-noise ratios (SNRs). The result shows 4.52% increase in overall F1-score and 3.42% reduction in half total error rate (HTER) compared to the recent reported result of rVAD [1].

Environment Sound Classification System Based on Hybrid Feature and Convolutional Neural Network

At present, the environment sound recognition system mainly identifies environment sounds with deep neural networks and a wide variety of auditory features. Therefore, it is necessary to analyze which auditory features are more suitable for deep neural networks based ESCR systems. In this paper, we chose three sound features which based on two widely used filters:the Mel and Gammatone filter banks. Subsequently, the hybrid feature MGCC is presented. Finally, a deep convolutional neural network is proposed to verify which features are more suitable for environment sound classification and recognition tasks. The experimental results show that the signal processing features are better than the spectrogram features in the deep neural network based environmental sound recognition system. Among all the acoustic features, the MGCC feature achieves the best performance than other features. Finally, the MGCC-CNN model proposed in this paper is compared with the state-of-the-art environmental sound classification models on the UrbanSound 8K dataset. The results show that the proposed model has the best classification accuracy.

Improved Auditory-Inspired Signal Processing Algorithm Design for Tracking Multiple Frequency Components

An adaptive synchrony capture filterbank (SCFB) signal processing architecture for tracking signal frequency components was proposed in a previous paper. Detailed analysis and further improvements are presented here. The original SCFB signal processing architecture consisted of a fixed array of traditional, passive linear, gammatone filters in cascade with a bank of three adaptively tunable bandpass filters that form part of a frequency discriminator loop (FDL). The SCFB exhibits many desirable properties for processing speech, music, and other complex sounds. In this article, an improved SCFB architecture is presented along with detailed specification of parameters that yield better frequency tracking. This architecture combines a FDL with a phase-locked loop, a concept borrowed from communication systems, to acquire and track frequency components in a time-varying signal. This modified algorithm estimates frequencies with higher accuracy even in the presence of closely spaced tones in the input. The improved performance of the algorithm is demonstrated by comparing the mean square error in the frequency estimates of synthetic time-varying signals.

A Novel Integrity Authentication Algorithm Based on Perceptual Speech Hash and Learned Dictionaries

Perceptual speech hash and robust watermarking have been widely investigated to solve the problems of authenticating speech integrity. The former generates a watermark and the latter embeds the watermark into the speech signal to implement speech integrity authentication. In this paper, we propose a perceptual speech hash algorithm and a robust watermarking algorithm for speech integrity authentication. To obtain perceptual speech hash values, we propose a gammatone filter model of the speech signal to extract sensitive auditory features (denoted by gammatone features). A random Gaussian matrix is used to reduce the dimensionality of the features of the gammatone to generate perceptual speech hash values. For the watermarking algorithm, we construct learned dictionaries to obtain the robust sparse feature of coefficients of the stationary wavelet transforms, and embed a watermark (perceptual speech hash values) into the sparse feature by patchwork and quantization index modulation. We illustrate the good imperceptibility of the authentication scheme in terms of the signal-to-noise ratio, objective difference grade, and subjective difference grade, and verify its robustness against common signal processing operations while maintaining imperceptibility. Moreover, our proposed method is sensitive to the malicious modification of the watermarked speech. Compared with state-of-the-art algorithms, the proposed algorithm can obtain better comprehensive performance in the detection and localization of tampering with the content of speech.

CNN-Based Learnable Gammatone Filterbank and Equal-Loudness Normalization for Environmental Sound Classification

For environmental sound classification (ESC), this letter presents a learnable auditory filterbank based on a one-dimensional (1D) convolutional neural network with strong psychophysiological inductive bias in the form of a gammatone filterbank and an equal-loudness prompting normalization. In the past, a number of ESC methods based on learnable auditory features obtained by performing plain 1D convolutions on raw input waveforms for outperforming traditional handcrafted features such as a mel-frequency filterbank have been proposed. However, the large number of parameters involved in the convolutions suggests that these methods will not generalize better than a model defined by a smaller number of parameters, which is considered in this letter. Here, a learnable gammatone filterbank layer consisting of 1D kernels represented by a parametric form of the bandpass gammatone filters is proposed for acquiring a time-frequency representation of the raw waveform. A normalization with learnable parameters that control the trade-off between energy equalization and structure preservation in the spectro-temporal domain is proposed. To verify the effectiveness of the considered network and the normalization, ESC experiments on the ESC-50 and UrbanSound8K datasets were conducted. Compared to other state-of-the-art networks, the considered network performed better on the two datasets. In addition, an ensemble architecture achieved further performance improvement.

A Hybrid Approach for Single Channel Speech Enhancement using Deep Neural Network and Harmonic Regeneration Noise Reduction

This paper presents a hybrid approach for single channel speech enhancement using deep neural network (DNN) and harmonic regeneration noise reduction (HRNR). The DNN was used as a supervised algorithm to predict new target mask such as constrained Wiener Filter (cWF) target mask from noisy mixture signal that was transformed into gammatone filter bank features. Meanwhile, HRNR algorithm was applied in the post-filtering strategy to eliminate residual noise. The DNN algorithm is an emerging supervised speech enhancement to overcome heavy nonstationary noise and low signal-to-noise ratio (SNR) issues. To validate the proposed algorithm with new target mask, 600 Malay utterances combining male and female speakers were used in a training session while 120 Malay utterances were used in a prediction session. The short time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ) scores were calculated as the performance metrics. In this work, the proposed target mask outperformed other baseline target masks. Thus, PESQ and STOI scores for the hybrid speech enhancement algorithm is 1.17 and 0.79, respectively, at - 5 dB babble noise SNR.

Ambiguity function analysis of human echolocator waveform by using gammatone filter processing

Human echolocation is a technique used by small group of people (normally blind persons) to observe their surrounding via emitting an active signal (usually tongue click) and analyse the return echo. This skill allows them to navigate safely by ‘seeing through sound’. Remarkable performances upon echolocating have attracted scholars’ attention from multidiscipline field including psychophysical and engineering. Study of waveform diversity has reported the signal is wideband, consisting of multiple frequency components with exponential decay factor that probably made up the entire unique signal. As such, wideband perspective is considered during ambiguity function analysis using wide-band ambiguity function method. The waveform analysis continues and reported optimum autocorrelation function is achieved by adopting a bio-inspired technique, which incorporated gammatone-filter processing. It will be interesting to synthesise the waveform using gammatone-filter approaches prior to relay it into ambiguity function process for analysis. Therefore, it is worth to explore its output results in this paper, which hold new findings that may help to understand how these humans achieve an optimum performance upon echolocating including ambiguity function characteristic. Thus, the new knowledge explored in this paper could be beneficial in emerging concept for new development in radar and sonar system application in near future.

Evaluation of Mel and Gammatone Filter Banks Used for Spectral Analysis in Comparison with the Direct Use of FFT

This paper analyses the development of Automatic Speech Recognition systems in relation to the varied types of spectral analysis methods used. A critical evaluation of Mel and Gammatone filter banks used for spectral analysis in comparison with the direct use of FFT spectral values is considered. Research was based on understanding the effectiveness of existing Automatic Speech Recognition systems are specifically focused on Mel and Gammatone filter banks in comparison with FFT spectral values.

Replay attack detection with auditory filter-based relative phase features

There are many studies on detecting human speech from artificially generated speech and automatic speaker verification (ASV) that aim to detect and identify whether the given speech belongs to a given speaker. Recent studies demonstrate the success of the relative phase (RP) feature in speaker recognition/verification and the detection of synthesized speech and converted speech. However, there are few studies that focus on the RP feature for replay attack detection. In this paper, we improve the discriminating ability of the RP feature by proposing two new auditory filter-based RP features for replay attack detection. The key idea is to integrate the advantage of RP-based features in signal representation with the advantage of two auditory filter-based RP features. For the first proposed feature, we apply a Mel-filter bank to convert the signal representation of conventional RP information from a linear scale to a Mel scale, where the modified representation is called the Mel-scale RP feature. For the other proposed feature, a gammatone filter bank is applied to scale the RP information, where the scaled RP feature is called the gammatone-scale RP feature. These two proposed phase-based features are implemented to achieve better performance than a conventional RP feature because of the scale resolution and. In addition to the use of individual Mel/gammatone-scale RP features, a combination of the scores of these proposed RP features and a standard magnitude-based feature, that is, the constant Q transform cepstral coefficient (CQCC), is also applied to further improve the reliable detection decision. The effectiveness of the proposed Mel-scale RP feature, gammatone-scale RP feature, and their combination are evaluated using the ASVspoof 2017 dataset. On the evaluation dataset, our proposed methods demonstrate significant improvement over the existing feature and baseline CQCC feature. The combination of the CQCC and gammatone-scale RP provides the best performance compared with an individual baseline feature and other combination methods.

Supervised model for Cochleagram feature based fundamental heart sound identification

The efficiency of automated heart sound analysis mostly depends on accurate detection of acoustic events. In this study, an acoustic feature based heart sound segmentation algorithm has been proposed for automatic identification of the fundamental heart sounds (FHS). Gammatone filter bank energy has been introduced to represent the heart sound distinctive features. A supervised artificial neural network (ANN) model is used to detect S1-S2 and non S1-S2 segments of the cardiac cycle. Finally time based information is utilized to identify S1 and S2 positions. Performance of the system is evaluated using 764 real and noisy heart sound cycles (both normal and abnormal domains) from the 2016 PhysioNet/CinC challenge database with annotations provided for heart sound states. The accuracy achieved using Cochleagram feature is more than 95% for both first and second heart sound identification. Proposed technique shows that multilayer perceptron (MLP) neural network using Cochleagram feature improvises the overall S1-S2 identification accuracy compared to the other acoustic features reported earlier.

A strategic approach to recognize the speech of the children with hearing impairment: different sets of features and models

The automatic speech recognition system is developed and tested for recognizing the speeches of a normal person in various languages. This paper mainly emphasizes the need for the development of a more challenging speaker independent speech recognition system for hearing impaired to recognize the speeches uttered by any Hearing Impaired (HI) speaker. In this work, Gamma tone energy features with filters spaced an equivalent rectangular bandwidth (ERB), MEL & BARK scale, and MFPLPC features are used at the front end and vector quantization (VQ) & multivariate hidden Markov models (MHMM) at the back end for recognizing the speeches uttered by any hearing impaired speaker. Performance of the system is compared for the three modeling techniques VQ, FCM (Fuzzy C means) clustering and MHMM for the recognition of isolated digits and simple continuous sentences in Tamil. Recognition accuracy (RA) is 81.5% with speeches of eight speakers considered for training and speeches of the remaining two speakers considered for testing for speaker independent isolated digit recognition system. Accuracy is found to be 91% and 87.5% for considering 90% of the data for training and 10% for testing for speaker independent isolated digit and continuous speech recognition systems respectively. Accuracy can be further enhanced by having an extensive database for creating models/templates. Receiver operating characteristics (ROC) drawn between True Positive Rate and False Positive Rate is used to assess the performance of the system for HI. This system can be utilized to understand the speech uttered by any hearing impaired speaker and the system facilitates the provision of necessary assistance to them. It ultimately improves the social status of the hearing impaired people and their confidence level will be enhanced.

Spectro-temporal features for environmental sound classification

This paper proposes 2N_BILBP feature extraction method based on spectro-temporal features for sound event classification. Spectro-temporal features have a similar pattern to texture features in image processing. So, the concept of texture features is used in this digital signal processing field. Two-neighbour bidirectional local binary pattern (2N_BILBP) is used in this paper for feature extraction. 2N_BILBP is also compared with the previous method called bidirectional local binary pattern. Firstly, the input audio is converted into spectrogram using short-time Fourier transform and then gamma tone is used. The resulting gamma-tone-like spectrogram is then used to extract features. These features are used as feature features. Finally, the input audio is labelled using this feature vector. Evaluation is tested on three datasets called ESC-10 dataset, ESC-50 dataset and UrbanSound8K datasets called benchmark datasets.

Spectro-temporal features for environmental sound classification

This paper proposes 2N_BILBP feature extraction method based on spectro-temporal features for sound event classification. Spectro-temporal features have a similar pattern to texture features in image processing. So, the concept of texture features is used in this digital signal processing field. Two-neighbour bidirectional local binary pattern (2N_BILBP) is used in this paper for feature extraction. 2N_BILBP is also compared with the previous method called bidirectional local binary pattern. Firstly, the input audio is converted into spectrogram using short-time Fourier transform and then gamma tone is used. The resulting gamma-tone-like spectrogram is then used to extract features. These features are used as feature features. Finally, the input audio is labelled using this feature vector. Evaluation is tested on three datasets called ESC-10 dataset, ESC-50 dataset and UrbanSound8K datasets called benchmark datasets.

Ia-PNCC: Noise Processing Method for Underwater Target Recognition Convolutional Neural Network

Underwater target recognition is a key technology for underwater acoustic countermeasure. And how to classify and recognize underwater targets according to the noise information of underwater targets has been a hot topic in the field of underwater acoustic signals. In this paper, the deep learning model is applied in the underwater target recognition and the improved anti-noise Power-Normalized Cepstral Coefficients (ia-PNCC) is proposed based on PNCC oriented to underwater target noise features. In this coefficient, multitaper and normalized Gammatone filter group are used to improve the anti-noise capacity of PNCC in underwater target recognition, and it is combined with the convolutional neural network to recognize the underwater target. The experiment results show that the acoustic feature presented by ia-PNCC is of higher anti-noise capacity and more adaptive to the underwater target recognition model of convolutional neural network. Compared with the combination of convolutional neural network with single acoustic feature such as MFCC (Mel-scale frequency cepstral coefficients) or LPCC (Linear Prediction PLP) and so on, the combination of ia-PNCC with convolutional neural network improves the underwater target recognition accuracy greatly.

Visibility enhancement via optimal two-piece gamma tone mapping for optical see-through displays under ambient light

Visibility of the overlaid virtual image is highly sensitive to surrounding illumination in optical see-through (OST) displays. Ambient light can especially deteriorate the visibility of low gray-levels of comparable luminance. Therefore, we first derive a luminance model based on actual measurements under varied ambient light and use it to extract low gray-level regions (LGR), which suffer from severe contrast loss. It was experimentally found that gamma tone mapping is the most appropriate method for LGR contrast enhancement of OST displays. Luminance boosting before gamma tone mapping allows better expression of very low value gray levels in the output AR image. The gamma values and luminance boosting parameter are optimally determined by cost minimization. Visibility enhancement is verified experimentally on a practical setup using a variety of ambient light levels and images.

Robust emotion recognition from speech: Gamma tone features and models

Affective computing is gaining paramount importance in ensuring the better and effective human–machine interaction. As glottal and speech signals depict the characteristics of the emotional nature of the speaker in addition to the linguistic information, speaker’s emotions are needed to be recognised to give meaningful response by the system. This paper emphasises the effectiveness and efficiency in selecting the energy features by passing the speech through the Gamma tone filters spaced in Equivalent rectangular bandwidth (ERB), MEL and BARK scale. Various modelling techniques are used to develop the robust multi-speaker independent speaker’s emotion/stress recognition system. Since EMO-DB Berlin database and SAVEE emotional audio-visual database used in this work contain the only limited set of speech utterances uttered by 10/4 actors/speakers in different emotions, it has become challenging to improve the performance of the stress/emotion recognition system. Speaker independent emotion recognition is done by extracting the Gamma tone energy features and cepstral features by passing the concatenated speech considered for training through the Gamma tone filters spaced in ERB, MEL and BARK scales. Subsequently, VQ/Fuzzy clustering models and continuous density hidden Markov models are created for all emotions and evaluation is done with the utterances of a speaker independent of speeches considered for training. The proposed features for test utterances are captured and applied to the VQ/Fuzzy/MHMM/SVM models and testing is performed by using minimum distance criterion/maximum log-likelihood criterion. The proposed Gamma tone energy/cepstral features and modelling techniques provide complementary evidence in assessing the performance of the system. This algorithm offers 96%, 79%, and 95.3% as weighted accuracy recall for the stress recognition system with respect to the classification done on emotion-specific group VQ/Fuzzy/MHMM/SVM models for GTF energy features with Gamma tone filters spaced in ERB, MEL and BARK scale respectively for the system evaluated for the EMO-DB database. Weighted accuracy recall is found to be 91%, 93% and 94% for the classification done on emotion-specific group models for GTF energy features with Gamma tone filters spaced in ERB, MEL and BARK scale respectively for the evaluation done on the utterances chosen from the SAVEE database. Gamma tone Cepstral features provide the overall accuracy of 92%, 90% and 92% for filters spaced in ERB, MEL and BARK scale for Berlin EMO-DB. Decision level fusion classification based on GTF energy features and modelling techniques provides the overall accuracy as 99.8% for EO-DB database and 100% for SAVEE database.

Perception of noise-vocoded tone complexes: A time domain analysis based on an auditory filterbank model

When a wideband harmonic tone complex (wHTC) is passed through a noise vocoder, the resulting sounds can have spectra with large peak-to-valley ratios, but little or no periodicity strength in the autocorrelation functions. We measured judgments of pitch strength for normal-hearing listeners for noise-vocoded wideband harmonic tone complexes (NV-wHTCs) relative to standard and anchor stimuli. The standard was a 1-channel NV-wHTC and the anchor was either the unprocessed wHTC or an infinitely-iterated rippled noise (IIRN). Although there is variability among individuals, the magnitude judgment functions obtained with the IIRN anchor suggest different listening strategies among individuals. In order to gain some insight into possible listening strategies, test stimuli were analyzed at the output of an auditory filterbank model based on gammatone filters. The weak periodicity strengths of NV-wHTCs observed in the stimulus autocorrelation functions are augmented at the output of the gammatone filterbank model. Six analytical models of pitch strength were evaluated based on summary correlograms obtained from the gammatone tone filterbank. The results of the filterbank analysis suggest that, contrary to the weak or absent periodicity strengths in the stimulus domain, temporal cues contribute to pitch strength perception of noise-vocoded harmonic stimuli such that listeners' judgments of pitch strength reflect a nonlinear, weighted average of the temporal information between the fine structure and the envelope.

Pseudo-color cochleagram image feature and sequential feature selection for robust acoustic event recognition

This work proposes the use of pseudo-color cochleagram image of sound signals for feature extraction for robust acoustic event recognition. A cochleagram is a variation of the spectrogram. It utilizes a gammatone filter and has been shown to better reveal spectral information. We propose mapping of the grayscale cochleagram image to higher dimensional color space for improved characterization from environmental noise. The resulting time-frequency representation is referred as pseudo-color cochleagram image and the resulting feature, which captures the statistical distribution, as pseudo-color cochleagram image feature (PC-CIF). In addition, sequential backward feature selection is applied for selecting the most useful feature dimensions, thereby reducing the feature dimension and improving the classification performance. We evaluate the effectiveness of the proposed methods using two classifiers, k-nearest neighbor and support vector machines. The performance is evaluated on a dataset containing 50 sound classes, taken from the Real World Computing Partnership Sound Scene Database in Real Acoustical Environments, with the addition of environmental noise at various signal-to-noise ratios. The experimental results show that the proposed techniques give significant improvement in classification performance over baseline methods. The most improved results were observed at low signal-to-noise ratios.