Acoustic Modeling Techniques Research Articles

Advancing Accessibility through Automatic Speech Recognition and NLP Integration

Abstract: The integration of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) technologies has the potential to revolutionize accessibility and inclusive communication. This article explores the fundamentals of ASR, including acoustic modeling, language modeling, and speech signal processing techniques, and discusses the challenges posed by language diversity in developing accurate ASR systems. The advent of ASR technology has opened up numerous possibilities for enhancing accessibility across various domains, such as assistive technology, education, healthcare, and media. The integration of ASR with NLP techniques enables the processing and analysis of spoken language data, leading to the development of voice-enabled virtual assistants, conversational AI systems, and cross-lingual communication tools. However, several challenges remain, including the need for robust and accurate ASR systems, privacy and security concerns, and ethical considerations in the development and deployment of these technologies. The article also presents future directions, such as the integration of ASR and NLP with emotion recognition and sentiment analysis, advances in deep learning techniques, and the application of these technologies in healthcare and accessibility. Overall, the integration of ASR and NLP holds immense promise for creating more natural, empathetic, and inclusive communication systems, but their development and deployment must be approached with care to ensure fairness, transparency, and user privacy.

Evaluation of Interpretable Speech Biomarkers for Monitoring Alzheimer’s Disease and Mild Cognitive Impairment Progression

AbstractBackgroundAlzheimer’s disease (AD) is a progressive neurodegenerative disease. Like for other dementias, biomarkers may help characterize distinct aspects of the underlying pathology, predict decline, and monitor disease progression. We extracted a composite array of 24 speech‐based biomarkers using machine learning and signal processing techniques. We then determined which of the given biomarkers was predictive of MCI and AD progression from baseline by conducting a correlation analysis.MethodSpoken responses to a Cookie Theft Picture description task were recorded from 2 subjects with AD, 4 subjects with Mild Cognitive Impairment (MCI) due to AD (biomarker confirmed AD), and 4 subjects with MCI. We automatically extracted acoustic, linguistic, and cognitive biomarkers using speech recognition, acoustic, and language modeling techniques. We later computed Kendall’s tau‐b (tb) correlation between the biomarker values and Montreal Cognitive Assessment (MoCA) scores. Recordings and MoCA scores were collected at baseline, at 6 months after baseline, and at 12 months for a subgroup of the subjects. To perform the correlation analysis, we used scipy.stats.kendalltau library in Python.ResultWith respect to the acoustic biomarkers, a strong negative correlation (tb = ‐0.50) was observed between the MoCA scores and features such as pause time, pause percentage, and pause speech ratio, while a moderate positive correlation (tb = 0.2) was found for rhythm standard deviation. With respect to the linguistic biomarkers a strong positive correlation(tb = 0.50) for the features of corrected type‐token ratio, average sentence length in words, and noun count was reported, while a moderate positive correlation (tb = 0.27) was found for the features of word type count, word token count, and moving average type‐token ratio. The other acoustic, linguistic, and cognitive biomarkers showed a very weak (tb = 0.00 ‐ 0.10) or weak correlation (tb = 0.10 ‐ 0.20).ConclusionAltogether, these results suggest that speech‐based interpretable biomarkers may help clinicians to diagnose AD at earlier stages and monitor disease progression. Our preliminary data suggest that AD patients encounter more problems delivering longer and linguistically elaborated narratives as the disease progresses.

Evaluation of Interpretable Speech Biomarkers for Monitoring Alzheimer’s Disease and Mild Cognitive Impairment Progression

AbstractBackgroundAlzheimer’s disease (AD) is a progressive neurodegenerative disease. Like for other dementias, biomarkers may help characterize distinct aspects of the underlying pathology, predict decline, and monitor disease progression. We extracted a composite array of 24 speech‐based biomarkers using machine learning and signal processing techniques. We then determined which of the given biomarkers was predictive of MCI and AD progression from baseline by conducting a correlation analysis.MethodSpoken responses to a Cookie Theft Picture description task were recorded from 2 subjects with AD, 4 subjects with Mild Cognitive Impairment (MCI) due to AD (biomarker confirmed AD), and 4 subjects with MCI. We automatically extracted acoustic, linguistic, and cognitive biomarkers using speech recognition, acoustic, and language modeling techniques. We later computed Kendall’s tau‐b (τb ) correlation between the biomarker values and Montreal Cognitive Assessment (MoCA) scores. Recordings and MoCA scores were collected at baseline, at 6 months after baseline, and at 12 months for a subgroup of the subjects. To perform the correlation analysis, we used scipy.stats.kendalltau library in Python.ResultWith respect to the acoustic biomarkers, a strong negative correlation (τb= ‐0.50) was observed between the MoCA scores and features such as pause time, pause percentage, and pause speech ratio, while a moderate positive correlation (τb= 0.2) was found for rhythm standard deviation. With respect to the linguistic biomarkers a strong positive correlation(τb= 0.50) for the features of corrected type‐token ratio, average sentence length in words, and noun count was reported, while a moderate positive correlation (τb= 0.27) was found for the features of word type count, word token count, and moving average type‐token ratio. The other acoustic, linguistic, and cognitive biomarkers showed a very weak (τb= 0.00 ‐ 0.10) or weak correlation (τb= 0.10 ‐ 0.20).ConclusionAltogether, these results suggest that speech‐based interpretable biomarkers may help clinicians to diagnose AD at earlier stages and monitor disease progression. Our preliminary data suggest that AD patients encounter more problems delivering longer and linguistically elaborated narratives as the disease progresses.

Evolution of acoustic methods for assessing and managing exposure of gray whales to sound pulses from seismic surveys off Sakhalin Island, Russian Far East

An effective program of estimation and mitigation of sound exposure from airgun array pulses is of vital importance when an endangered whale population forages in coastal waters near oil and gas production fields, where geophysical exploration campaigns regularly take place. Starting in 2001, increasingly sophisticated acoustic measurement and modeling techniques have been used for the environmental management of airgun array seismic surveys adjacent to the nearshore feeding grounds of the gray whale (Eschrichtius robustus) off the NE coast of Sakhalin Island, Russia. These monitoring data are also used in combination with visual observations for the analysis of the effects of sound exposure on whale behavior, distribution, and energetics. This talk follows two decades of evolution of the acoustic monitoring technology used in the Sakhalin studies, from rudimentary analog sonobuoys with limited bandwidth VHF telemetry, to digital stations capable of onboard signal processing and relaying of data via satellite to anywhere on the globe. In parallel, it traces the advancement of modeling techniques capable to fill the gaps between sparse measurement locations to yield high resolution estimates of the sound field from seismic sources operating concurrently at different, moving locations and in various regimes of ramp-up, production, or mitigation pulses.

Amalgamation of noise elimination and TDNN acoustic modelling techniques for the advancements in continuous Kannada ASR system

Amalgamation of noise elimination and TDNN acoustic modelling techniques for the advancements in continuous Kannada ASR system

Development of Small Vocabulary Continuous Speech-to-Text System for Kannada Language/Dialects

Objectives: To develop a speech-to-text (STT) system using Kaldi speech recognition toolkit for continuous Kannada language/dialects. Methods: A continuous Kannada speech data is collected from 100 speakers/farmers of Karnataka state in field. The lexicon/dictionary and set of phonemes for Kannada language/dialects are created and transcribed the collected speech data using transcriber tool. The ASR models are developed at different phoneme levels using Kaldi. Findings: In this work, an effort is made to develop a robust small vocabulary STT system for continuous Kannada language using Kaldi. The various acoustic modelling techniques are used to develop a robust ASR model and achieved a word error rate (WER) of 0.23%. The performance of the developed ASR model is compared with existing works and analyzed by offline speech recognition. Novelty: Many STT systems have been developed for Indian and International languages/dialects, but not for Kannada language. This work is first of its kind using Kaldi in Kannada language under the constraints of limited data. The developed ASR model could be used further in the development of end-to-end ASR system for speech processing applications. Keywords: Automatic Speech Recognition (ASR); Word Error Rate (WER); Continuous Kannada Speech Data; Kannada Language/Dialects; Lexicon

Analysis of an acoustic propagation model for sources of noise with directivity in indoor environments

Small caliber firearm (SCF) noise sources are typically impulsive in nature, possess a large amount of acoustic energy (consequently a large hearing damage risk), and are not omnidirectional. These sources are often operated in indoor shooting ranges where the potential for noise exposure risk is greater due to the reflective surfaces in the room. Indoor sound propagation models require inputs such as geometry, wall material properties, and some quantified source level description. One such room acoustic modeling technique is the Image-Source Method (ISM). ISM typically assumes specular reflections off the walls and represents those reflections as image sources. Many ISM algorithms can operate with high computational efficiency for simple omnidirectional source models, usually represented by a single quantity: sound power. However, ISM models using an omnidirectional source assumption can produce high errors in some scenarios involving highly directional sources, such as SCFs. In this work, an ISM algorithm has been modified to predict listener exposure levels from non-omnidirectional sources in complicated room designs, and has been validated against measured data from an SCF on an indoor Air Force shooting range.

Performance Analysis of various Front-end and Back End Amalgamations for Noise-robust DNN-based ASR

Introduction: An Automatic Speech Recognition (ASR) system enables to recognize the speech utterances and thus can be used to convert speech into text for various purposes. These systems are deployed in different environments such as clean or noisy and are used by all ages or types of people. These also present some of the major difficulties faced in the development of an ASR system. Thus, an ASR system need to be efficient, while also being accurate and robust. Our main goal is to minimize the error rate during training as well as testing phases, while implementing an ASR system. Performance of ASR depends upon different combinations of feature extraction techniques and back-end techniques. In this paper, using a continuous speech recognition system, the performance comparison of different combinations of feature extraction techniques and various types of back-end techniques has been presented Methods: Hidden Markov Models (HMMs), Subspace Gaussian Mixture Models (SGMMs) and Deep Neural Networks (DNNs) with DNN-HMM architecture, namely Karel’s, Dan’s and Hybrid DNN-SGMM architecture are used at the back-end of the implemented system. Mel frequency Cepstral Coefficient (MFCC), Perceptual Linear Prediction (PLP), and Gammatone Frequency Cepstral coefficients (GFCC) are used as feature extraction techniques at the front-end of the proposed system. Kaldi toolkit has been used for the implementation of the proposed work. The system is trained on the Texas Instruments-Massachusetts Institute of Technology (TIMIT) speech corpus for English language Results: The experimental results show that MFCC outperforms GFCC and PLP in noiseless conditions, while PLP tends to outperform MFCC and GFCC in noisy conditions. Furthermore, the hybrid of Dan’s DNN implementation along with SGMM performs the best for the back-end acoustic modeling. The proposed architecture with PLP feature extraction technique in the front end and hybrid of Dan’s DNN implementation along with SGMM at the back end outperforms the other combinations in a noisy environment. Conclusion: Automatic Speech recognition has numerous applications in our lives like Home automation, Personal assistant, Robotics etc. It is highly desirable to build an ASR system with good performance. The performance Automatic Speech Recognition is affected by various factors which include vocabulary size, whether system is speaker dependent or independent, whether speech is isolated, discontinuous or continuous, adverse conditions like noise. The paper presented an ensemble architecture that uses PLP for feature extraction at the front end and a hybrid of SGMM + Dan’s DNN in the backend to build a noise robust ASR system Discussion: The presented work in this paper discusses the performance comparison of continuous ASR systems developed using different combinations of front-end feature extraction (MFCC, PLP, and GFCC) and back-end acoustic modeling (mono-phone, tri-phone, SGMM, DNN and hybrid DNN-SGMM) techniques. Each type of front-end technique is tested in combination with each type of back-end technique. Finally, it compares the results of the combinations thus formed, to find out the best performing combination in noisy and clean conditions

"The Diagnostic Evaluation of Switchboard-corpus Automatic Speech Recognition Systems"

To see whether the related mistake patterns can be linked to a particular set of variables, a Eight Control equipment recognizing (and six forcedalignment) algorithms were evaluated for clinical diagnosis. Each recognizing service's result was converted to a standardized way and evaluated to a comparative record made from pronunciations labelled data (which included 54 minutes of information from several hundred speakers). A job evaluation was used to relate a combination of acoustic, morphological, etc. speaker attributes to acknowledgment occurrences throughout this reference data. The decision trees show that correct categorization of phonetic segments and characteristics is one of the most constant variables linked with better recognition performance. These findings indicate that enhancing the pronouncing modelling used in verbal pairing, including the acoustic modeling techniques utilized for morphological classification, might improve future-generation recognition systems.

A hybrid CNN-LiGRU acoustic modeling using raw waveform sincnet for Hindi ASR

Deep Neural Network (DNN) is currently playing the most vital role in Automatic Speech Recognition (ASR). Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) are the advanced versions of DNN. CNN and RNN are right to deal with spatial and temporal properties of the speech signal, respectively, and both properties have a higher impact on accuracy. In today’s scenario, many acoustic modeling techniques often switches due to the battle of CNNs and RNNs. In the last few years, CNN, with raw speech signal, shows their superiority over precomputed acoustic features. Recently, a novel first convolution layer named SincNet was proposed to produce the interpretable filters with better accuracy. In this work, we proposed a hybrid SincNet-CNN-RNN architecture with low computation cost and high accuracy. Different configurations of the hybrid model were extensively examined to achieve this goal. All experiments were performed on the Hindi speech dataset.

In silico feasibility assessment of extracorporeal delivery of low-intensity pulsed ultrasound to intervertebral discs within the lumbar spine

Low intensity pulsed ultrasound (LIPUS) may have utility for non-invasive treatment of discogenic lower back pain through stimulating, remodeling and accelerating healing of injured or degenerated intervertebral disc (IVD) tissues. This study investigates the feasibility of delivering LIPUS to lumbar IVDs between L2 and S1 spine vertebra using a planar extracorporeal phased array (8 × 8 cm, 1024 elements, 500 kHz). Three 3D anatomical models with heterogenous tissues were generated from patient CT image sets and used in the simulation-based analysis. Time-reversal acoustic modeling techniques were applied to optimize posterior-lateral placement of the array with respect to the body to facilitate energy deposition in discrete target regions spanning the annulus fibrosus and central nucleus of each IVD. Forward acoustic and biothermal simulations were performed with time-reversal optimized array placements and driving amplitude/phase settings to predict LIPUS intensity distributions at target sites and to investigate off-target energy deposition and heating potential. Simulation results demonstrate focal intensity gain of 5–168 across all IVD targets and anatomical models, with greater average intensity gain (>50) and energy localization in posterior, posterolateral, and lateral target sites of IVDs. Localized LIPUS delivery was enhanced in thinner patient anatomies and in the high lumbar levels (L2-L3 and L3-L4). Multiple amplitude/phasing illumination patterns could be sequenced at a fixed array position for larger regional energy coverage in the IVD. Biothermal simulations demonstrated that LIPUS-appropriate exposures of 100 mW cm−2 ISPTA to the target disc region would result in <1 °C global peak temperature elevation for all cases. Hence, simulations suggest that spatially-precise extracorporeal delivery of therapeutically relevant LIPUS doses to discrete regions of lumbar IVDs is feasible and may be useful in clinical management of discogenic back pain.

Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition.

Artificial neural networks (ANN) have become the mainstream acoustic modeling technique for large vocabulary automatic speech recognition (ASR). A conventional ANN features a multi-layer architecture that requires massive amounts of computation. The brain-inspired spiking neural networks (SNN) closely mimic the biological neural networks and can operate on low-power neuromorphic hardware with spike-based computation. Motivated by their unprecedented energy-efficiency and rapid information processing capability, we explore the use of SNNs for speech recognition. In this work, we use SNNs for acoustic modeling and evaluate their performance on several large vocabulary recognition scenarios. The experimental results demonstrate competitive ASR accuracies to their ANN counterparts, while require only 10 algorithmic time steps and as low as 0.68 times total synaptic operations to classify each audio frame. Integrating the algorithmic power of deep SNNs with energy-efficient neuromorphic hardware, therefore, offer an attractive solution for ASR applications running locally on mobile and embedded devices.

Acoustic Modeling in Speech Recognition: A Systematic Review

The paper presents a systematic review of acoustic modeling (AM) techniques in speech recognition(SR). Acoustic modeling establishes a relationship between acoustic information and language construct in SR. Over the past decades, researchers presented studies addressing specific concerns in AM. However, all previous research works lack a systematic and comprehensive review of acoustic modeling issues. A systematic review is introduced to understand the acoustic modeling issues in speech recognition. This paper provides an extensive and comprehensive inspection of various researches that have been performed since 1984. The extensive investigation and analysis into AM was performed by getting the relevant data from 73 research works chose after the screening process between the years from 1984 to 2020. The systematic review process was divided into different parts to investigate acoustic modeling issues. Main issues in acoustic modeling such as feature extraction techniques, acoustic modeling units, speech corpora, classification methods, different tools used, language issues applied, and evaluation parameters were investigated. This study helps the reader to understand various acoustic modeling issues with comprehensive details. The research outcomes presented in this study depict research trends and shed light on new research topics in AM. The result of this review can be used to build a better speech recognition system by choosing a suitable acoustic modeling construct in SR.

Efficient Design Optimization of Acoustic Liners for Engine Noise Reduction

Engine noise reduction using acoustic liner panels is an established technology in the design of bypass turbofan engines. Using acoustic impedance modeling techniques in aeroacoustic simulations, quick assessment of different liner configurations can be made. To minimize the engine noise in the best possible way, optimal liner configurations are ideally determined by employing numerical optimization methods. In general, to exhaust the full potential of the lining technology, it is desired to use a large number of design parameters in the optimization process. In the present work, a multilevel optimization framework is presented for the design of acoustic liner panels, which can efficiently handle large design vectors. The multilevel method combines a global search strategy with a local gradient-based optimization method. The discrete adjoint solver, which is used to evaluate the liner sensitivities required in the local search, has been developed using algorithmic differentiation techniques. This approach allows consistent treatment of boundary conditions in the adjoint part, thereby increasing the robustness and accuracy of the adjoint solver. In this way, the discrete adjoint solver fully incorporates the features of the underlying aeroacoustic solver, such as the impedance modeling and nonreflecting boundary conditions. The feasibility and the efficiency of the multilevel optimization strategy are demonstrated by finding the optimal liner parameters in a turbofan engine bypass duct configuration.

Acoustic camera modeling and compensation technique based on SONAR equation

Acoustic camera is a type of ultrasonic equipment used to detect or classify nearby objects located in water. The distortion of SONAR images acquired from conventional acoustic camera equipment inevitably occurs depending on the platform speed and operating frequency. This has not been huge concern because the platform speeds were low. However, more recently, a high accuracy of acoustic camera images has become essential in accordance with the development of unmanned autonomous systems. Additionally, a high speed of operating platform is also required. Therefore, there is a need for the study of image correction to reduce the distortion of SONAR images acquired from the equipment. In this study, we simulate virtual acoustic camera SONAR images by performing numerical modeling based on the SONAR equation to improve the accuracy of acoustic camera images. To verify the validity of the simulated images, we use the literature data and water-tank experiment data. Finally, we conduct the image compensation both in the direction of motion relative to the platform speed, and in the transverse direction relative to the platform speed and operating frequency.Acoustic camera is a type of ultrasonic equipment used to detect or classify nearby objects located in water. The distortion of SONAR images acquired from conventional acoustic camera equipment inevitably occurs depending on the platform speed and operating frequency. This has not been huge concern because the platform speeds were low. However, more recently, a high accuracy of acoustic camera images has become essential in accordance with the development of unmanned autonomous systems. Additionally, a high speed of operating platform is also required. Therefore, there is a need for the study of image correction to reduce the distortion of SONAR images acquired from the equipment. In this study, we simulate virtual acoustic camera SONAR images by performing numerical modeling based on the SONAR equation to improve the accuracy of acoustic camera images. To verify the validity of the simulated images, we use the literature data and water-tank experiment data. Finally, we conduct the image compensation both...

Addressing noise and pitch sensitivity of speech recognition system through variational mode decomposition based spectral smoothing

In this paper, we propose a novel front-end speech parameterization technique for automatic speech recognition (ASR) that is less sensitive towards ambient noise and pitch variations. First, using variational mode decomposition (VMD), we break up the short-time magnitude spectrum obtained by discrete Fourier transform into several components. In order to suppress the ill-effects of noise and pitch variations, the spectrum is then sufficiently smoothed. The desired spectral smoothing is achieved by discarding the higher-order variational mode functions and reconstructing the spectrum using the first-two modes only. As a result, the smoothed spectrum closely resembles the spectral envelope. Next, the Mel-frequency cepstral coefficients (MFCC) are extracted using the VMD-based smoothed spectra. The proposed front-end acoustic features are observed to be more robust towards ambient noise and pitch variations than the conventional MFCC features as demonstrated by the experimental evaluations presented in this study. For this purpose, we developed an ASR system using speech data from adult speakers collected under relatively clean recording conditions. State-of-the-art acoustic modeling techniques based on deep neural networks (DNN) and long short-term memory recurrent neural networks (LSTM-RNN) were employed. The ASR systems were then evaluated under noisy test conditions for assessing the noise robustness of the proposed features. To assess robustness towards pitch variations, experimental evaluations were performed on another test set consisting of speech data from child speakers. Transcribing children's speech helps in simulating an ASR task where pitch differences between training and test data are significantly large. The signal domain analyses as well as the experimental evaluations presented in this paper support our claims.

Investigation of Various Hybrid Acoustic Modeling Units via a Multitask Learning and Deep Neural Network Technique for LVCSR of the Low-Resource Language, Amharic

Multitask learning (MTL) is helpful for improving the performance of related tasks when the training dataset is limited and sparse, especially for low-resource languages. Amharic is a low-resource language and suffers from the problems of training data scarcity, sparsity, and unevenness. Consequently, fundamental acoustic units-based speech recognizers perform worse compared with the speech recognizers of technologically favored languages. This paper presents the results of our contributions to the use of various hybrid acoustic modeling units for the Amharic language. The fundamental acoustic units, namely, syllable, phone, and rounded phone units-based deep neural network (DNN) models have been developed. Various hybrid acoustic units have been investigated by jointly training the fundamental acoustic units via the MTL technique. Those hybrid units and the fundamental units are discussed and compared. The experimental results demonstrate that all the fundamental units-based DNN models outperform the Gaussian mixture models (GMM) with relative performance improvements of 14.14%-23.31%. All the hybrid units outperform the fundamental acoustic units with relative performance improvements of 1.33%-4.27%. The syllable and phone units exhibit higher performance under sufficient and limited training datasets, respectively. All the hybrid units are useful with both sufficient and limited training datasets and outperformed the fundamental units. Overall, our results show that DNN is an effective acoustic modeling technique for the Amharic language. The context-dependent (CD) syllable is the more suitable unit if a sufficient training corpus is available and the accuracy of the recognizer is prioritized. The CD phone is a superior unit if the available training dataset is limited and realizes the highest accuracy and fast recognition speed. The hybrid acoustic units perform the best under both sufficient and limited training datasets and achieve the highest accuracy.

Comparison of Phonemic and Graphemic Word to Sub-Word Unit Mappings for Lithuanian Phone-Level Speech Transcription

Conventional large vocabulary automatic speech recognition (ASR) systems require a mapping from words into sub-word units to generalize over the words that were absent in the training data and to enable the robust estimation of acoustic model parameters. This paper surveys the research done during the last 15 years on the topic of word to sub-word mappings for Lithuanian ASR systems. It also compares various phoneme and grapheme based mappings across a broad range of acoustic modelling techniques including monophone and triphone based Hidden Markov models (HMM), speaker adaptively trained HMMs, subspace gaussian mixture models (SGMM), feed-forward time delay neural network (TDNN), and state-of-the-art low frame rate bidirectional long short term memory (LFR BLSTM) recurrent deep neural network. Experimental comparisons are based on a 50-hour speech corpus. This paper shows that the best phone-based mapping significantly outperforms a grapheme-based mapping. It also shows that the lowest phone error rate of an ASR system is achieved by the phoneme-based lexicon that explicitly models syllable stress and represents diphthongs as single phonetic units.

Spatial mapping of underwater noise radiated from passing vessels using automatic identification system data

Information regarding the spatial distribution of noise radiated by ships in specific areas is used as basic data for the protection of marine organisms and establishment of marine spatial planning. This study derived the spatial distribution of the noise radiated from ships navigating the southern region of Korea by acoustic modeling using Automatic Identification System data. The acoustic energy propagated from a ship to all spatial points in the experimental area was modeled using a ray-based model, and the noise level at a particular point was incoherently summed over all spatial points and averaged over time. By applying the acoustic modeling technique, we derived the average noise level at frequencies below 1 kHz radiated from ships in the ocean, and compared it with the acoustic data measured during the same period. Discrepancy between both results was found at frequencies below 250 Hz, and the possible cause of this discrepancy is related to the power-law model used to calculate the source level of the ship based on an empirical formula. The difference in current can be reliably reduced by applying a more accurate ship source model.

Room-localized spoken command recognition in multi-room, multi-microphone environments

The paper focuses on the design of a practical system pipeline for always-listening, far-field spoken command recognition in everyday smart indoor environments that consist of multiple rooms equipped with sparsely distributed microphone arrays. Such environments, for example domestic and multi-room offices, present challenging acoustic scenes to state-of-the-art speech recognizers, especially under always-listening operation, due to low signal-to-noise ratios, frequent overlaps of target speech, acoustic events, and background noise, as well as inter-room interference and reverberation. In addition, recognition of target commands often needs to be accompanied by their spatial localization, at least at the room level, to account for users in different rooms, providing command disambiguation and room-localized feedback. To address the above requirements, the use of parallel recognition pipelines is proposed, one per room of interest. The approach is enabled by a room-dependent speech activity detection module that employs appropriate multichannel features to determine speech segments and their room of origin, feeding them to the corresponding room-dependent pipelines for further processing. These consist of the traditional cascade of far-field spoken command detection and recognition, the former based on the detection of “activating” key-phrases. Robustness to the challenging environments is pursued by a number of multichannel combination and acoustic modeling techniques, thoroughly investigated in the paper. In particular, channel selection, beamforming, and decision fusion of single-channel results are considered, with the latter performing best. Additional gains are observed, when the employed acoustic models are trained on appropriately simulated reverberant and noisy speech data, and are channel-adapted to the target environments. Further issues investigated concern the inter-dependencies of the various system components, demonstrating the superiority of joint optimization of the component tunable parameters over their separate or sequential optimization. The proposed approach is developed for the Greek language, exhibiting promising performance in real recordings in a four-room apartment, as well as a two-room office. For example, in the latter, a 76.6% command recognition accuracy is achieved on a speaker-independent test, employing a 180-sentence decoding grammar. This result represents a 46% relative improvement over conventional beamforming.