A novel target decoupling framework based on waveform-spectrum fusion network for monaural speech enhancement

Abstract

Speech enhancement refers to the technique of restoring clean speech from the corrupted noisy speech to reduce or suppress noise and interference. In recent years, the target decoupling strategy in the time-frequency (T-F) domain is proposed to alleviate the compensation problem between magnitude and phase. However, such target decoupling methods lack the utilization of waveform representations, so the neural networks cannot learn the complementary advantages between time domain and T-F domain. In this paper, we propose a cross-domain target decoupling system leveraging both waveform-domain and T-F domain semantic information to resynthesize the estimated spectrum. In order to fully use the information from both domains, the final reconstruction scheme is extended to each middle layer through lightweight semantic aggregation modules. Based on these above methods, the encoder-decoder-based waveform-spectrum fusion network (WSFNet) is proposed to concretely achieve cross-domain causal monaural speech enhancement. In WSFNet, the dual-path RNN blocks are embedded into the bottleneck layer to model both intra-frame and inter-frame long-range contextual correlations. Extensive experimental results demonstrate that our WSFNet outperforms the most advanced target decoupling systems and other previous state-of-the-art (SOTA) systems in terms of speech quality and intelligibility.