Residual physical modeling and semi-supervised source localization

Abstract

Source localization in reverberant environments remains an open challenge which ML techniques have shown promise in addressing. Real-world acoustic environments contain irregular boundaries, scattering, and diffracting elements (e.g., furniture and uneven surfaces) which are impractical to model using acoustic propagation software. We develop a hybrid approach to acoustic source localization which utilizes both analytical, signal processing-based localization, and machine learning (ML) to provide a data-driven source localization model. In this approach, the output of an analytical source localization model (e.g., SRP-PHAT or MUSIC) is augmented by a neural network (NN) system. This augmented system consists of a variational autoencoder and localization network, which is designed to correct the analytic estimate. The networks are trained via semi-supervised learning, on both labeled and unlabeled inputs. The NNs in this approach parameterize stochastic functions, which facilitate a statistically principled learning approach via variational inference (VI). The stochasticity also helps quantify source location uncertainty under the VI approximate posterior. For simulated and real acoustic data, the hybrid approach generalizes better and is more efficient than ML or analytic approaches alone.