Lightweight Patch-Wise Casformer for dynamic scene deblurring

Abstract

In dynamic scenes, motion blur can often occur, which is non-uniform and can be difficult to remove. Recently, the Transformer has shown excellent performance in various image-related tasks such as classification, recognition, and segmentation. Using a Transformer-based backbone network has also shown potential advantages in image deblurring. However, the computational complexity of Transformers increases quadratically with spatial resolution, making it difficult to apply to high-resolution images. To address the above issue, we propose a cascade Transformer (Casformer) that consists of two key modules: Deep Separable Attention (DSA) and Double-Flow Gate (DFG). Our approach effectively reduces computational complexity while suppressing blurry information. Additionally, we discovered an inconsistency between training and testing images during the image restoration process. We addressed this issue by experimentally verifying an inference aggregation method (IAM) that independently predicts patches during inference to address the problem of imbalanced information distribution. Experimental results demonstrate that our design performs well on GoPro and other datasets, e.g. 29.20 dB PSNR on RealBlur-J, exceeding the previous state-of-the-art (SOTA) 0.14 dB.