Separation Front-end Research Articles

Brain-informed speech separation (BISS) for enhancement of target speaker in multitalker speech perception

Hearing-impaired people often struggle to follow the speech stream of an individual talker in noisy environments. Recent studies show that the brain tracks attended speech and that the attended talker can be decoded from neural data on a single-trial level. This raises the possibility of “neuro-steered” hearing devices in which the brain-decoded intention of a hearing-impaired listener is used to enhance the voice of the attended speaker from a speech separation front-end. So far, methods that use this paradigm have focused on optimizing the brain decoding and the acoustic speech separation independently. In this work, we propose a novel framework called brain-informed speech separation (BISS)11BISS: brain-informed speech separation. in which the information about the attended speech, as decoded from the subject’s brain, is directly used to perform speech separation in the front-end. We present a deep learning model that uses neural data to extract the clean audio signal that a listener is attending to from a multi-talker speech mixture. We show that the framework can be applied successfully to the decoded output from either invasive intracranial electroencephalography (iEEG) or non-invasive electroencephalography (EEG) recordings from hearing-impaired subjects. It also results in improved speech separation, even in scenes with background noise. The generalization capability of the system renders it a perfect candidate for neuro-steered hearing-assistive devices.

Correlator beamforming for multipath mitigation in high‐fidelity GNSS monitoring applications

Safety and security critical global navigation satellite system (GNSS) applications ensure system integrity by performing long-term monitoring to quantify and characterize GNSS signal anomalies. Multipath significantly hinders these high-fidelity signal quality monitoring applications. Besides multipath limiting antennas and sophisticated receiver signal processing, any remaining multipath is currently removed by antenna siting and azimuth/elevation masking. As urban development around airports and monitoring installations continues, these solutions may not suffice and more aggressive but cost-effective mitigation technologies are needed. This article explores the application of correlator beamforming as a potential solution. Unlike traditional multielement beamforming that requires a separate receiver front-end for each antenna element, correlator beamforming time multiplexes the received signals and performs beamforming as part of receiver correlation processing. This article shows that the multipath mitigation performance of correlator beamforming significantly exceeds a geodetic antenna and approaches that of traditional multi-element beamforming. The results presented are relevant for all GNSS monitoring applications.

Building APMv3 Map Visualization Using Nagios Host Data

Nowadays, having a network monitoring is becoming an important thing for a growing or big company which should also have a network infrastructure inside it. Network monitoring is the use of a system that constantly monitors a computer network for slow or failing components and that notifies the network administrator in case of outages. Acknowledge Proactive Monitoring (APM), one of the sub-departments in Acknowledge, focuses on developing a network monitoring system using the Nagios tool. However, one of its main features called Map Visualization is not really efficient in terms of its map management. The current stable version of the system is called APMv2 and the team is developing a new version called APMv3 which needs a better Map Visualization. The application made in this project was a proof of concept that will later be used on APMv3. It is a web-based application which has a separate frontend written in HTML and JavaScript, has a separate RESTful backend written in PHP, and uses some frameworks such as jQuery UI, jCanvas, Twitter Typeahead, and Bootstrap 3. All necessary data within this application are exchanged asynchronously using AJAX.

A Joint Training Framework for Robust Automatic Speech Recognition

Robustness against noise and reverberation is critical for ASR systems deployed in real-world environments. In robust ASR, corrupted speech is normally enhanced using speech separation or enhancement algorithms before recognition. This paper presents a novel joint training framework for speech separation and recognition. The key idea is to concatenate a deep neural network (DNN) based speech separation frontend and a DNN-based acoustic model to build a larger neural network, and jointly adjust the weights in each module. This way, the separation fron-tend is able to provide enhanced speech desired by the acoustic model and the acoustic model can guide the separation frontend to produce more discriminative enhancement. In addition, we apply sequence training to the jointly trained DNN so that the linguistic information contained in the acoustic and language models can be back-propagated to influence the separation frontend at the training stage. To further improve the robustness, we add more noise- and reverberation-robust features for acoustic modeling. At the test stage, utterance-level unsupervised adaptation is performed to adapt the jointly trained network by learning a linear transformation of the input of the separation frontend. The resulting sequence-discriminative jointly-trained multistream system with run-time adaptation achieves 10.63% average word error rate (WER) on the test set of the reverberant and noisy CHiME-2 dataset (task-2), which represents the best performance on this dataset and a 22.75% error reduction over the best existing method.

Exploring rapid radiochemical separations at the University of Tennessee Radiochemistry Center of Excellence

The University of Tennessee formed its Radiochemistry Center of Excellence (RCoE) in 2013 with support from the U.S. National Nuclear Security Administration. One of the major thrusts of the RCoE is to develop deeper understanding of rapid methods for radiochemical separations that are relevant to both general radiochemical analyses as well as post-detonation nuclear forensics. Early work has included the development and demonstration of rapid separations of lanthanide elements in the gas phase, development of a gas-phase separation front-end for ICP-TOF-MS analysis, and the development of realistic analytical surrogates for post-detonation debris to support methods development.

A separation-based UI architecture with a DSL for role specialization

This paper proposes an architecture and associated methodology to separate front end UI concerns from back end coding concerns to improve the platform flexibility, shorten the development time, and increase the productivity of developers. Typical UI development is heavily dependent upon the underlying platform, framework, or tool used to create it, which results in a number of problems. We took a separation-based UI architecture and modified it with a domain specific language to support the independence of UI creation thereby resolving some of the aforementioned problems. A methodology incorporating this architecture into the development process is proposed. A climate science application was created to verify the validity of the methodology using modern practices of UX, DSLs, code generation, and model-driven engineering. Analyzing related work provides an overview of other methods similar to our method. Subsequently we evaluate the climate science application, conclude, and detail future work.

Investigation of Speech Separation as a Front-End for Noise Robust Speech Recognition

Recently, supervised classification has been shown to work well for the task of speech separation. We perform an in-depth evaluation of such techniques as a front-end for noise-robust automatic speech recognition (ASR). The proposed separation front-end consists of two stages. The first stage removes additive noise via time-frequency masking. The second stage addresses channel mismatch and the distortions introduced by the first stage; a non-linear function is learned that maps the masked spectral features to their clean counterpart. Results show that the proposed front-end substantially improves ASR performance when the acoustic models are trained in clean conditions. We also propose a diagonal feature discriminant linear regression (dFDLR) adaptation that can be performed on a per-utterance basis for ASR systems employing deep neural networks and HMM. Results show that dFDLR consistently improves performance in all test conditions. Surprisingly, the best average results are obtained when dFDLR is applied to models trained using noisy log-Mel spectral features from the multi-condition training set. With no channel mismatch, the best results are obtained when the proposed speech separation front-end is used along with multi-condition training using log-Mel features followed by dFDLR adaptation. Both these results are among the best on the Aurora-4 dataset.

Estimating Uncertainty to Improve Exemplar-Based Feature Enhancement for Noise Robust Speech Recognition

We present a method of improving automatic speech recognition performance under noisy conditions by using a source separation approach to extract the underlying clean speech signal. The feature enhancement processing is complemented with heuristic estimates of the uncertainty of the source separation, that are used to further assist the recognition. The uncertainty heuristics are converted to estimates of variance for the extracted clean speech using a Gaussian Mixture Model based mapping, and applied in the decoding stage under the observation uncertainty framework. We propose six heuristics, and evaluate them using both artificial and real-world noisy data, and with acoustic models trained on clean speech, a multi-condition noisy data set, and the multi-condition set processed with the source separation front-end. Taking the uncertainty of the enhanced features into account is shown to improve recognition performance when the acoustic models are trained on unenhanced data, while training on enhanced noisy data yields the lowest error rates.

Real-time 6-DOF multi-session visual SLAM over large-scale environments

This paper describes a system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs.The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide experimental results for real-time multi-session visual mapping on wheeled and handheld datasets in the MIT Stata Center. These results demonstrate key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.

A New Evidence Model for Missing Data Speech Recognition With Applications in Reverberant Multi-Source Environments

Conventional hidden Markov model (HMM) decoders often experience severe performance degradations in practice due to their inability to cope with uncertain data in time-varying environments. In order to address this issue, we propose the bounded-Gauss-Uniform mixture probability density function (pdf) as a new class of evidence model for missing data speech recognition. Exemplary for a hands-free speech recognition scenario, we illustrate how the parameters of the new mixture pdf can be estimated with the help of a multi-channel source separation front-end. In comparison with other models the new evidence pdf retains a fuller description of the available data and provides a more effective link between source separation and recognition. The superiority of the bounded-Gauss-Uniform mixture pdf over conventional approaches is demonstrated for a connected digits recognition task under varying test conditions.

Quantitative Analysis of a Common Audio Similarity Measure

For music information retrieval tasks, a nearest neighbor classifier using the Kullback-Leibler divergence between Gaussian mixture models of songs' melfrequency cepstral coefficients is commonly used to match songs by timbre. In this paper, we analyze this distance measure analytically and experimentally by the use of synthesized MIDI files, and we find that it is highly sensitive to different instrument realizations. Despite the lack of theoretical foundation, it handles the multipitch case quite well when all pitches originate from the same instrument, but it has some weaknesses when different instruments play simultaneously. As a proof of concept, we demonstrate that a source separation frontend can improve performance. Furthermore, we have evaluated the robustness to changes in key, sample rate, and bitrate.

Problems in expediting protocol processing: performance analysis of kernel mode priority scheduling and front end protocol processors

Computer communication networks are used to support a wide variety of applications ranging from distributed data bases to time-critical process control systems. Communications protocol processing may be expedited by giving it high CPU priority while being run in kernel mode or by off-loading it to a separate front end processor. The purpose of the latter design is to allow protocol handling and application processing to be done concurrently. The state spaces of such systems are complex and difficult to enumerate. We develop and use computationally inexpensive approximate analytic techniques to model them. We examine the influence of the following factors on performance: kernel mode priority scheduling, the need for a buffered communication mechanism between the host CPU and the FEP, and the speed of the FEP. We find that CPU priority scheduling of protocol activity is beneficial to performance as long as high priority processing does not dominate the CPU. We also find that a communication mechanism between the FEP and the CPU with only one or two packet-sized buffers seriously impedes performance to the extent that it necessitates the use of a FEP with a processor at least as fast as that at the host.