Time-frequency network combining batch attention and spatial attention for speech bandwidth extension

Abstract

The field of speech bandwidth extension (BWE) has witnessed remarkable advancements through the implementation of convolutional neural networks (CNNs). However, one limitation of CNNs is the lack of consideration for the strong correlation between different frames of speech signals. Effectively utilizing the inter-frame features and sample points has become an urgent challenge that needs to be addressed in this context. To address this problem, we propose an improved time–frequency network that combines spatial attention module and batch attention module. Our proposed approach involves a spatial attention module that uses multi-layer dilated convolutions to capture spatial dependencies in feature maps. Additionally, a batch attention module utilizes pooling and fully connected operations to leverage the correlation between speech frames. We introduce a time–frequency (T-F) loss to optimize the amplitude and phase of reconstructed speech signals, reducing artificial artifacts. Experimental results demonstrate the superiority of our network compared to other baseline BWE networks on four datasets, as measured by SNR, LSD, PESQ, and MOS metrics. Notably, on the TIMIT dataset, our network with the T-F loss achieves superior LSD and SNR values over other loss functions.