Transformer-based adaptive 3D residual CNN with sparse representation for PAN-sharpening of multispectral images

Abstract

ABSTRACT Panchromatic (PAN)-sharpening is an important process for most of the practical appliances in the domain of geospatial analysis. Because of the balance among instantaneous field of view and Signal-to-Noise Ratio (SNR), the Multi-Spectral (MS) and PAN images with complementary features are captured by the satellite sensors. Fusing PAN illustrations with maximal dimensional resolution and MS illustrations with maximal spectral resolution is known as PAN-sharpening, which is widely utilized to improve spatial and spectral resolutions. The crucial point in the PAN-sharpening task is to maintain a balance between the extraction of data and the insertion of data. If a misbalance occurs, it may lead to a distortion of the intensity. To address these issues, a new PAN-sharpening model is developed to combine the MS and PAN illustrations to increase the image’s spatial resolution. In addition, the spatial and spectral distortions of the illustrations are highly decreased by using this PAN-sharpening model. Originally, the PAN and MS illustrations were collected from traditional online sources. The MS and PAN images are initially combined and given to the sparse representation process. This provides a simulated PAN-sharpened image. This image is further provided to the intelligent deep learning network, which is the Transformer-based Adaptive 3D Residual Convolutional Neural Network (TA-3DRCNN) and hence the higher-resolution images are obtained, and it is useful in various applications. The parameters present in TA-3DRCNN are tuned using the Modified Uniform Number-based Gannet Optimization Algorithm (MUN-GOA) for upgrading the effectiveness of PAN-sharpening. The resultant outcome is contrasted with the customary PAN-sharpening approaches to validate the system’s effectiveness with respect to various measures.