MWA-MNN: Multi-patch Wavelet Attention Memristive Neural Network for image restoration

Abstract

Adverse weather conditions can severely reduce the image quality captured by outdoor imaging devices. However, most existing image restoration algorithms are designed for specific tasks, lack generalization capabilities, and are unable to balance spatial details with contextual feature. Moreover, outdoor images are typically captured using edge devices with limited computing resources, rendering it challenging to deploy image restoration algorithms on edge devices. To tackle these problems, we propose a memristive-based image restoration algorithm, named Multi-patch Wavelet Attention Memristive Neural Network (MWA-MNN). Multiple image restoration tasks can be achieved by implementing Convolutional Neural Network (CNN) based on memristor crossbar arrays, as well as specified memristive circuits. Our proposed network comprises three sub-networks with multi-patch design enabling each sub-network to focus on different scales, thereby preserving more spatial details while fusing rich contextual feature. We propose the Discrete Wavelet Attention Block (DWAB) and the Coordinate Channel Attention Block (CCAB) to extract rich contextual information and the Supervised Selective Kernel Block (SSKB) to effectively filter the extracted features. Our proposed memristor-based circuit implementation provides an effective solution for deploying image restoration algorithms on edge devices. Experiment results on multiple benchmark datasets for image deraining, dehazing, and low-light image enhancement show that our proposed method outperforms over 20 state-of-the-art methods.