Compression loss-based spatial-temporal attention module for compressed video quality enhancement

Abstract

Recently, deep learning technology has achieved remarkable progress in compressed video quality enhancement. However, the existing methods fail to consider the fact that the regions with different compression losses contain varied effective information. To address this issue, this paper proposes a Compression Loss-based Spatial-Temporal Attention (CLSTA) module that predicts the compression loss-based attention of each pixel when restoring target frames. Based on this, the pixels with lower compression losses contribute more to restoring target frames. Meanwhile, the compression-domain information in the compressed video stream is exploited to overcome the difficulty in inferring compression loss-based attention directly from the pixel-domain. With a slight modification, the CLSTA module can be easily integrated into the existing methods for end-to-end training. A series of experiments have been conducted in this study to validate the effectiveness and generality of the CLSTA module for compressed video quality enhancement. By adding the CLSTA module to the state-of-the-art method STDF, the proposed method achieves an average PSNR improvement of 0.96 dB and a BD-rate reduction of 26.85% under the Low-Delay P configuration, which is respectively 0.06 dB and 1.83% better than the baseline STDF. In the best case, the proposed method obtains 0.19 dB higher PSNR than STDF for a single frame.