Deep Fuzzy Clustering Transformer: Learning the General Property of Corruptions for Degradation-Agnostic Multitask Image Restoration

Abstract

For the sake of eliminating multiple degradations, most existing multi-task image restoration methods prefer to learn the properties of each degradation type, which is often accompanied by a bloated model size and a heavy learning burden. To tackle the aforementioned issues, we propose to treat multiple degradations uniformly to achieve degradation type-agnostic multi-task image restoration. We observe that the degradations in different spatial locations are always morphologically similar while the background sceneries vary greatly. In accordance with the above observation, we decouple the degradation features and the background features by an efficient fuzzy clustering method. The degradation features contain all the diverse degradation information, while the images are recovered from the decoupled background features. In practice, we discover a uniformity between the Fuzzy C-means algorithm and Cross-Attention and propose a Deep Fuzzy Clustering Transformer to achieve degradation type-agnostic background extraction via feature map clustering based on spatial distribution characteristics. Furthermore, to capture the spatial distribution properties of an image, an efficient global attention tree is devised to provide a global spatial receptive field for the clustering process. By virtue of the quadtree structure, the proposed global attention trees enable more efficient global modeling than existing methods. Our experimental analysis showed that the proposed method outperformed the state-of-the-art models in terms of both efficiency and performance.