Acoustic Modeling Techniques Research Articles

An automated technique to generate phone-to-articulatory label mapping

Recent studies have shown that in the case of under-resourced languages, use of articulatory features (AF) emerging from an articulatory model results in improved automatic speech recognition (ASR) compared to conventional mel frequency cepstral coefficient (MFCC) features. Articulatory features are more robust to noise and pronunciation variability compared to conventional acoustic features. To extract articulatory features, one method is to take conventional acoustic features like MFCC and build an articulatory classifier that would output articulatory features (known as pseudo-AF). However, these classifiers require a mapping from phone to different articulatory labels (AL) (e.g., place of articulation and manner of articulation), which is not readily available for many of the under-resourced languages. In this article, we have proposed an automated technique to generate phone-to-articulatory label (phone-to-AL) mapping for a new target language based on the knowledge of phone-to-AL mapping of a well-resourced language. The proposed mapping technique is based on the center-phone capturing property of interpolation vectors emerging from the recently proposed phone cluster adaptive training (Phone-CAT) method. Phone-CAT is an acoustic modeling technique that belongs to the broad category of canonical state models (CSM) that includes subspace Gaussian mixture model (SGMM). In Phone-CAT, the interpolation vector belonging to a particular context-dependent state has maximum weight for the center-phone in case of monophone clusters or by the AL of the center-phone in case of AL clusters. These relationships from the various context-dependent states are used to generate a phone-to-AL mapping. The Phone-CAT technique makes use of all the speech data belonging to a particular context-dependent state. Therefore, multiple segments of speech are used to generate the mapping, which makes it more robust to noise and other variations. In this study, we have obtained a phone-to-AL mapping for three under-resourced Indian languages namely Assamese, Hindi and Tamil based on the phone-to-AL mapping available for English. With the generated mappings, articulatory features are extracted for these languages using varying amounts of data in order to build an articulatory classifier. Experiments were also performed in a cross-lingual scenario assuming a small training data set (≈ 2 h) from each of the Indian languages with articulatory classifiers built using a lot of training data (≈ 22 h) from other languages including English (Switchboard task). Interestingly, cross-lingual performance is comparable to that of an articulatory classifier built with large amounts of native training data. Using articulatory features, more than 30% relative improvement was observed over the conventional MFCC features for all the three languages in a DNN framework.

Improvements in the Detection of Vowel Onset and Offset Points in a Speech Sequence

Detecting the vowel regions in a given speech signal has been a challenging area of research for a long time. A number of works have been reported over the years to accurately detect the vowel regions and the corresponding vowel onset points (VOPs) and vowel end points (VEPs). Effectiveness of the statistical acoustic modeling techniques and the front-end signal processing approaches has been explored in this regard. The work presented in this paper aims at improving the detection of vowel regions as well as the VOPs and VEPs. A number of statistical modeling approaches developed over the years have been employed in this work for the aforementioned task. To do the same, three-class classifiers (vowel, nonvowel and silence) are developed on the TIMIT database employing the different acoustic modeling techniques and the classification performances are studied. Using any particular three-class classifier, a given speech sample is then forced-aligned against the trained acoustic model under the constraints of first-pass transcription to detect the vowel regions. The correctly detected and spurious vowel regions are analyzed in detail to find the impact of semivowel and nasal sound units on the detection of vowel regions as well as on the determination of VOPs and VEPs. In addition to that, a novel front-end feature extraction technique exploiting the temporal and spectral characteristics of the excitation source information in the speech signal is also proposed. The use of the proposed excitation source feature results in the detection of vowel regions that are quite different from those obtained through the mel-frequency cepstral coefficients. Exploiting those differences in the obtained evidences by using the two kinds of features, a technique to combine the evidences is also proposed in order to get a better estimate of the VOPs and VEPs. When the proposed techniques are evaluated on the vowel---nonvowel classification systems developed using the TIMIT database, significant improvements are noted. Moreover, the improvements are noted to hold across all the acoustic modeling paradigms explored in the presented work.

Evaluation of Loudspeaker-Based Virtual Sound Environments for Testing Directional Hearing Aids.

Assessments of hearing aid (HA) benefits in the laboratory often do not accurately reflect real-life experience. This may be improved by employing loudspeaker-based virtual sound environments (VSEs) that provide more realistic acoustic scenarios. It is unclear how far the limited accuracy of these VSEs influences measures of subjective performance. Verify two common methods for creating VSEs that are to be used for assessing HA outcomes. A cocktail-party scene was created inside a meeting room and then reproduced with a 41-channel loudspeaker array inside an anechoic chamber. The reproduced scenes were created either by using room acoustic modeling techniques or microphone array recordings. Participants were 18 listeners with a symmetrical, sloping, mild-to-moderate hearing loss, aged between 66 and 78 yr (mean = 73.8 yr). The accuracy of the two VSEs was assessed by comparing the subjective performance measured with two-directional HA algorithms inside all three acoustic environments. The performance was evaluated by using a speech intelligibility test and an acceptable noise level task. The general behavior of the subjective performance seen in the real environment was preserved in the two VSEs for both directional HA algorithms. However, the estimated directional benefits were slightly reduced in the model-based VSE, and further reduced in the recording-based VSE. It can be concluded that the considered VSEs can be used for testing directional HAs, but the provided sensitivity is reduced when compared to a real environment. This can result in an underestimation of the provided directional benefit. However, this minor limitation may be easily outweighed by the high realism of the acoustic scenes that these VSEs can generate, which may result in HA outcome measures with a significantly higher ecological relevance than provided by measures commonly performed in the laboratory or clinic.

An open-access interactive textbook for teaching room acoustics modeling

There are several different techniques for modeling room acoustics. Interactive visualizations can be used to ease learning the main principles and algorithms for that. This presentation will demonstrate a new interactive open-access textbook aiming to support learning the basics of room acoustics modeling techniques. The visualizations illustrate how different algorithms work in simple 2-D geometries, and what are the underlying assumptions in each of them. Interactivity here means that the students have control over both various parameters that affect the modeling performance and accuracy, and the geometries. They can study how the modeling results change in response to such changes. The main emphasis here is on geometrical room acoustics, and the techniques based on that: image-source model, ray-tracing, and acoustic radiance transfer, but the longer term aim is to cover all the main room acoustic modeling principles and techniques.

Comparison of Several Acoustic Modeling Techniques for Speech Emotion Recognition

Automatic Speech Emotion Recognition (SER) is a current research topic in the field of Human Computer Interaction (HCI) with a wide range of applications. The purpose of speech emotion recognition system is to automatically classify speaker's utterances into different emotional states such as disgust, boredom, sadness, neutral, and happiness. The speech samples in this paper are from the Berlin emotional database. Mel Frequency cepstrum coefficients (MFCC), Linear prediction coefficients (LPC), linear prediction cepstrum coefficients (LPCC), Perceptual Linear Prediction (PLP) and Relative Spectral Perceptual Linear Prediction (Rasta-PLP) features are used to characterize the emotional utterances using a combination between Gaussian mixture models (GMM) and Support Vector Machines (SVM) based on the Kullback-Leibler Divergence Kernel. In this study, the effect of feature type and its dimension are comparatively investigated. The best results are obtained with 12-coefficient MFCC. Utilizing the proposed features a recognition rate of 84% has been achieved which is close to the performance of humans on this database.

Overview of geometrical room acoustic modeling techniques.

Computerized room acoustics modeling has been practiced for almost 50 years up to date. These modeling techniques play an important role in room acoustic design nowadays, often including auralization, but can also help in the construction of virtual environments for such applications as computer games, cognitive research, and training. This overview describes the main principles, landmarks in the development, and state-of-the-art for techniques that are based on geometrical acoustics principles. A focus is given to their capabilities to model the different aspects of sound propagation: specular vs diffuse reflections, and diffraction.

Advanced acoustic modelling techniques in MP3 speech recognition

The automatic recognition of MP3 compressed speech presents a challenge to the current systems due to the lossy nature of compression which causes irreversible degradation of the speech wave. This article evaluates the performance of a recognition system optimized for MP3 compressed speech with current state-of-the-art acoustic modelling techniques and one specific front-end compensation method. The article concentrates on acoustic model adaptation, discriminative training, and additional dithering as prominent means of compensating for the described distortion in the task of phoneme and large vocabulary continuous speech recognition (LVCSR). The experiments presented on the phoneme task show a dramatic increase of the recognition error for unvoiced speech units as a direct result of compression. The application of acoustic model adaptation has proved to yield the highest relative contribution while the gain of discriminative training diminished with decreasing bit-rate. The application of additional dithering yielded a consistent improvement only for the MFCC features, but the overall results were still worse than those for the PLP features.

수중 음향재료의 반향음 감소성능 연구

The requirement of acoustic material which is used in underwater environment more increases. The material is used to reduce acoustic signature and radiate noise for underwater vehicle. Underwater acoustic material was made by viscoelastic material such as rubber and polyurethane etc. The mechanical and acoustic characteristics of these material change with hydrostatic pressure. In order to improve an acoustic performance according to hydrostatic pressure, several kinds of scatterers were added to viscoelastic material. In this paper, acoustic modelling and analysis techniques of underwater acoustic material with hydrostatic pressure were introduced and proposed. The specimens for pulse tube test were made and echo reductions were calculated and measured with hydrostatic pressure. Also the characteristics of echo reduction of the specimens with hydrostatic pressure were obtained and discussed.

Acceleration Strategies for Speech Recognition Based on Deep Neural Networks

The Context-Dependent Deep-Neural-Network HMM, or CD-DNN-HMM, is a powerful acoustic modeling technique for HMM-based speech recognition systems. The CD-DNN-HMM can greatly outperform against the conventional Gaussian-mixture HMMs. Therefore, we build a CD-DNN-HMM LVCSR system by modifying a mature GMM-HMM system. The baseline CD-DNN-HMM system achieve word-error rate of 18.6% that is far better than 24.9% achieved by the GMM-HMM system. However, the speed of the baseline CD-DNN-HMM system becomes a major roadblock for its real-time rate reaches 0.72 on the standard NIST 2000 Hub5 evaluation set. In this paper, we realize several optimization algorithms in our baseline system to accelerate the recognition speed. Testing the optimized system on the same evaluation set, we achieve real-time rate of 0.39, a relative reduction of 45.8%.

Cross-Lingual Phone Mapping for Large Vocabulary Speech Recognition of Under-Resourced Languages

SUMMARY This paper presents a novel acoustic modeling technique of large vocabulary automatic speech recognition for under-resourced languages by leveraging well-trained acoustic models of other languages (called source languages). The idea is to use source language acoustic model to score the acoustic features of the target language, and then map these scores to the posteriors of the target phones using a classifier. The target phone posteriors are then used for decoding in the usual way of hybrid acoustic modeling. The motivation of such a strategy is that human languages usually share similar phone sets and hence it may be easier to predict the target phone posteriors from the scores generated by source language acoustic models than to train from scratch an under-resourced language acoustic model. The proposed method is evaluated using on the Aurora-4 task with less than 1 hour of training data. Two types of source language acoustic models are considered, i.e. hybrid HMM/MLP and conventional HMM/GMM models. In addition, we also use triphone tied states in the mapping. Our experimental results show that by leveraging well trained Malay and Hungarian acoustic models, we achieved 9.0% word error rate (WER) given 55 minutes of English training data. This is close to the WER of 7.9% obtained by using the full 15 hours of training data and much better than the WER of 14.4% obtained by conventional acoustic modeling techniques with the same 55 minutes of training data.

A model for underwater sound levels generated by marine impact pile driving

Marine impact pile driving generates very high underwater sound pressures that can be harmful to aquatic life. Environmental assessments for pile driving projects typically require advance estimates of acoustic impact zones for marine mammals and fish. A computer model has been developed to predict the radiated acoustic field from impact driving of cylindrical piles. The stress wave in the pile is predicted using a force-generator model of the hammer-pile system. The force-generator model is coupled to a 1-D finite-difference model of longitudinal stress waves in a cylindrical pile. The radiated pressure is computed by matching the velocity boundary condition at the pile wall using a superposition of monopole sources distributed over the length of the pile in a layered 2-D fluid medium. The transfer function for the monopoles is computed using the near-field Hankel transform for radial particle velocity at the pile wall. Standard ocean acoustic modeling techniques are used to compute the Mach wave propagating away from the pile. As an example, predictions of the model are compared to field measurements obtained in a riverine environment.

수중 음향재료의 음향성능 설계기법 연구

The requirement of acoustic performance about underwater acoustic material which is used in underwater environment more increases. Underwater acoustic material was made by viscoelastic material such as a rubber and a polyurethane etc. In order to increase an acoustic performance, several kinds of inclusions were added to viscoelastic material. In this paper, acoustic modelling and analysis techniques were introduced and the acoustic characteristics of underwater acoustic material were studied. Echo reduction and transmission loss were calculated with volume fraction of inclusion in the material. Also the characteristic impedance and the input impedance of underwater acoustic material were obtained and effects on the echo reduction and transmission loss of material were discussed.

A hybrid acoustic model for room impulse response synthesis

The prediction and synthesis of room impulse responses (RIR) has wide application from computer gaming to architectural acoustics. When a level of physical accuracy is important, a single acoustic modeling technique is usually limited by its computational load. Hybrid acoustic models target different time/frequency regions of the RIR with different modeling techniques. This paper introduces a hybrid acoustic model consisting of a physical FDTD model for low-mid frequencies, beam-tracing, and the acoustic radiance transfer method in the early part and late parts at high frequencies respectively. In this work, attention is given to establishing the equivalence of the boundary characteristics in each modeling domain. Good agreement is demonstrated indicating that mixing the separate model responses leads to an energetically consistent RIR.

Using out-of-language data to improve an under-resourced speech recognizer

Under-resourced speech recognizers may benefit from data in languages other than the target language. In this paper, we report how to boost the performance of an Afrikaans automatic speech recognition system by using already available Dutch data. We successfully exploit available multilingual resources through (1) posterior features, estimated by multilayer perceptrons (MLP) and (2) subspace Gaussian mixture models (SGMMs). Both the MLPs and the SGMMs can be trained on out-of-language data. We use three different acoustic modeling techniques, namely Tandem, Kullback–Leibler divergence based HMMs (KL-HMM) as well as SGMMs and show that the proposed multilingual systems yield 12% relative improvement compared to a conventional monolingual HMM/GMM system only trained on Afrikaans. We also show that KL-HMMs are extremely powerful for under-resourced languages: using only six minutes of Afrikaans data (in combination with out-of-language data), KL-HMM yields about 30% relative improvement compared to conventional maximum likelihood linear regression and maximum a posteriori based acoustic model adaptation.

Three-Dimensional Digital Waveguide Mesh Simulation of Cylindrical Vocal Tract Analogs

3D time-domain acoustic modeling techniques have the potential to produce more accurate simulation of the vocal tract than previously implemented 1D or 2D solutions, although the variability of human voice renders it a problematic benchmark for validation of its resynthesis. This study uses acoustic measurement of acrylic cylindrical vocal tract models derived from X-Ray data to assess the validity of comparable 3D digital waveguide mesh simulations. It is found that for more simple structures the 3D digital waveguide mesh is able to reproduce the acoustic behavior up to 10 kHz with only slight errors in resonant frequencies. As the simulated structures become more geometrically complex, this shifting becomes more severe.

A Bayesian Framework Using Multiple Model Structures for Speech Recognition

This paper proposes an acoustic modeling technique based on Bayesian framework using multiple model structures for speech recognition. The aim of the Bayesian approach is to obtain good prediction of observation by marginalizing all variables related to generative processes. Although the effectiveness of marginalizing model parameters was recently reported in speech recognition, most of these systems use only “one” model structure, e.g., topologies of HMMs, the number of states and mixtures, types of state output distributions, and parameter tying structures. However, it is insufficient to represent a true model distribution, because a family of such models usually does not include a true distribution in most practical cases. One of solutions of this problem is to use multiple model structures. Although several approaches using multiple model structures have already been proposed, the consistent integration of multiple model structures based on the Bayesian approach has not seen in speech recognition. This paper focuses on integrating multiple phonetic decision trees based on the Bayesian framework in HMM based acoustic modeling. The proposed method is derived from a new marginal likelihood function which includes the model structures as a latent variable in addition to HMM state sequences and model parameters, and the posterior distributions of these latent variables are obtained using the variational Bayesian method. Furthermore, to improve the optimization algorithm, the deterministic annealing EM (DAEM) algorithm is applied to the training process. The proposed method effectively utilizes multiple model structures, especially in the early stage of training and this leads to better predictive distributions and improvement of recognition performance.

Quantifying the effects of roughness scattering on reflection loss measurements

Seafloor reflection loss and roughness measurements were taken at the Experimental Validation of Acoustic Modeling Techniques experiment in 2006. The magnitude and phase of the reflection loss was measured at frequencies from 5 to 80 kHz and grazing angles from 7° to 77°. Approximately 1500 samples were taken for each angle. The roughness was measured with a laser profiler. Geoacoustic parameters such as water and sediment sound speed and density were measured concurrently. The reflection loss data were compared with three models: A flat interface elastic model based on geoacoustic measurements; a flat interface poro-elastic model based on the Biot/Stoll model; and a rough interface model based on the measured interface roughness power spectrum. The data were most consistent with the poro-elastic model including scattering. The elastic model consistently predicted values for the reflection loss which were higher than measured. The data exhibited more variability than the model due to layering and fluctuations in the propagating medium.

Propagation modeling techniques for marine mammal management studies

Acoustic propagation modeling techniques are often used to estimate the impact of anthropogenic sound sources on the marine environment, and to estimate population density and detection ranges of marine mammals from their vocalizations. Sophisticated propagation models can be used to accurately calculate acoustic transmission loss as a function of range and depth. This is often done along a number of uniformly spaced radials surrounding a sound source and results are interpolated to each simulated animal location. Computational time, detailed input parameters, and interpolation over complex bathymetry can be a hindrance to efficient and accurate results. This work investigates the impact of using simple propagation modeling that avoids interpolation between radials. Differences will be compared between direct and interpolated values as well as between coherent and incoherent transmission loss solutions. The accuracy and efficiency of the different approaches are evaluated by comparing the number of animal...

Room Acoustic Modelling Techniques: A Comparison of a Scale Model and a Computer Model for a New Opera Theatre

Today most acoustic consultants are using room acoustic computer models as a basis for their acoustic design. However, room acoustic scale modelling is still being used for the design in some major projects, although the costs and the time needed are significantly larger than those related to computer modelling. Both techniques were used by the author in a project for a new opera theatre; first the acoustical design was based on computer simulations using the Odeon software, and next a 1:20 scale model was built and tested. In the paper the results obtained with the two different modelling techniques are compared, and in general a satisfactory agreement has been found. The advantages and drawbacks related to each of the modelling techniques are discussed.

A Theoretical Approach to Room Acoustic Simulations Based on a Radiative Transfer Model

A theoretical approach to room acoustic simulations based on a radiative transfer model is developed by adapting the classical radiative transfer theory from optics to acoustics. The proposed acoustic radiative transfer model expands classical geometrical room acoustic modeling algorithms by incorporating a propagation medium that absorbs and scatters radiation, handling both diffuse and non-diffuse reflections on boundaries and objects in the room. The main scope of this model is to provide a proper foundation for a wide number of room acoustic simulation models, in order to establish and unify their principles. It is shown that this room acoustic modeling technique establishes the basis of two recently proposed algorithms, the acoustic diffusion equation and the room acoustic rendering equation. Both methods are derived in detail using an analytical approximation and a simplified integral equation of the proposed method, respectively, allowing a clear definition of the underlying assumptions, limitations, advantages and disadvantages.