Time-frequency Masking Technique Research Articles

A Time–Frequency Masking Based Random Finite Set Particle Filtering Method for Multiple Acoustic Source Detection and Tracking

Considering that multiple talkers may appear simultaneously, a time–frequency (TF) masking based random finite set (RFS) particle filtering (PF) method is developed for multiple acoustic source detection and tracking. The time-delay of arrival (TDOA) measurements of multiple sources are extracted by using a time–frequency masking technique, by which each source’s TF bins are clustered and separated in a joint gain-ratio and time-delay histogram. Since a joint detection and tracking problem is considered, both source positions and source numbers are time-varying and need to be estimated. The tracker is built within a RFS Bayesian filtering framework. Essentially, an RFS process is used to characterize the source dynamics that include source appearance/dissappearance and motion trajectories. Latent variables are also introduced to indicate source dynamics and measurement-source associations. Subsequently, a Rao–Blackwellization PF technique is employed so that the source position state can be marginalized and only the latent variables are estimated by using the PF. The main advantage of the proposed approach is that hypothesis-pruning is formulated in a full probabilistic sense. The performance of the proposed approach is demonstrated in real speech recordings as well as in simulated room environments.

Multimodal (audio–visual) source separation exploiting multi-speaker tracking, robust beamforming and time–frequency masking

A novel multimodal source separation approach is proposed for physically moving and stationary sources which exploits a circular microphone array, multiple video cameras, robust spatial beamforming and time-frequency masking. The challenge of separating moving sources, including higher reverberation time (RT) even for physically stationary sources, is that the mixing filters are time varying; as such the unmixing filters should also be time varying but these are difficult to determine from only audio measurements. Therefore in the proposed approach, visual modality is used to facilitate the separation for both stationary and moving sources. The movement of the sources is detected by a three-dimensional tracker based on a Markov Chain Monte Carlo particle filter. The audio separation is performed by a robust least squares frequency invariant data-independent beamformer. The uncertainties in source localisation and direction of arrival information obtained from the 3D video-based tracker are controlled by using a convex optimisation approach in the beamformer design. In the final stage, the separated audio sources are further enhanced by applying a binary time-frequency masking technique in the cepstral domain. Experimental results show that using the visual modality, the proposed algorithm cannot only achieve performance better than conventional frequency-domain source separations algorithms, but also provide acceptable separation performance for moving sources.

Group-normalized processing of complex wavelet packets

Linear phase is not possible for real valued FIR QMF, while linear phase FIR biorthogonal wavelet filter banks make the mean sqared error of the constructed signal exceed that of the quantization error. W. Lawton’s method for complex valued wavelets construction is extended to generate the complex valued compactly supported wavelet packets that are symmetrical and unitary orthogonal; then well-defined wavelet packets are chosen by the analysis remarks on their time-frequency characteristics. Since the traditional wavelet packets transform coefficients do not exactly represent the strength of signal components, a modified adaptive wavelets transform, group-normalized wavelet packet transform (GNWPT), is presented and utilized for target extraction from formidable clutter or noises with the time-frequency masking technique. The extended definition oflp-norm entropy improves the performance of GNW-PT. Similar method can also be applied to image enhancement, clutter and noise suppression, optimal detection and radar imaging.