Learned image compression via neighborhood-based attention optimization and context modeling with multi-scale guiding

Abstract

In recent years, learned image compression has witnessed significant advancements. However, many existing learned image compression models predominantly rely on Convolutional Neural Networks (CNNs) and predicting the distribution of latent features to use ever more expensive entropy models, which cannot fully capture detail redundancy and lead to prohibitively slow decoding. To address this challenge, this paper introduces novel image compression models, MGIC and RMGIC, which consist of the Residual Neighborhood-based Attention Module (RNAM), Multi-Scale Guide Entropy Model (MGEM) and Refined Multi-Scale Guide Entropy Model (RMGEM). Specifically, Neighborhood Attention (NA) is combined with global feature learning to construct the RNAM, which can accurately capture the correlation of spatial neighborhoods and is more suitable for image compression. The proposed RNAM is flexible and could work as a plug-and-play component to enhance CNN-based models. Additionally, MGEM employs two mask convolutions with two kernel sizes to leverage multi-scale information for context modeling and eliminating the effect of parallel decoding on model performance. Building upon MGEM, RMGEM incorporates a refined two-stage spatial context modeling technique to enhance the utilization of spatial neighborhoods and achieve more precise entropy prediction. Furthermore, MGIC and RMGIC are derived by integrating RNAM, MGEM, and RMGEM into the variational autoencoder framework. The proposed methods can save more bandwidth and improve the quality of information storage. Extensive experimental evaluations have shown that the proposed method is effective and achieves state-of-the-art performance.