Speech-Oriented Sparse Attention Denoising for Voice User Interface Toward Industry 5.0

Abstract

The adoption of voice user interface (VUI) will promote network automation with enhanced efficiency with reduced simplicity and operating expense in Industry 5.0. Given the noisy environments, speech denoising is indispensable for the VUI in Internet of Things (IoT) or Industrial IoT (IIoT). Despite Transformer's recent success in speech denoising, the adopted full self-attention suffers from quadratic complexity, which challenges the computational power of the IoT/IIoT components. Considering the strong local correlations of speech signals, a speech-oriented sparse attention denoising scheme is developed to keep the meaningful local and global dependencies while mitigating the redundant attentions, resulting in a significant reduction in computational complexity. With the full self-attention as the baseline, experimental results revealed that the proposed scheme achieves a better denoising performance and yields a lower computational cost, indicating the strong potential for various VUI application scenarios in IoT and IIoT toward Industry 5.0.