Residual Channel Attention Network Research Articles

Light field image coding using a residual channel attention network–based view synthesis

PurposeLight field images (LFIs) have gained popularity as a technology to increase the field of view (FoV) of plenoptic cameras since they can capture information about light rays with a large FoV. Wide FoV causes light field (LF) data to increase rapidly, which restricts the use of LF imaging in image processing, visual analysis and user interface. Effective LFI coding methods become of paramount importance. This paper aims to eliminate more redundancy by exploring sparsity and correlation in the angular domain of LFIs, as well as mitigate the loss of perceptual quality of LFIs caused by encoding.Design/methodology/approachThis work proposes a new efficient LF coding framework. On the coding side, a new sampling scheme and a hierarchical prediction structure are used to eliminate redundancy in the LFI's angular and spatial domains. At the decoding side, high-quality dense LF is reconstructed using a view synthesis method based on the residual channel attention network (RCAN).FindingsIn three different LF datasets, our proposed coding framework not only reduces the transmitted bit rate but also maintains a higher view quality than the current more advanced methods.Originality/value(1) A new sampling scheme is designed to synthesize high-quality LFIs while better ensuring LF angular domain sparsity. (2) To further eliminate redundancy in the spatial domain, new ranking schemes and hierarchical prediction structures are designed. (3) A synthetic network based on RCAN and a novel loss function is designed to mitigate the perceptual quality loss due to the coding process.

Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks.

Convolutional neural networks (CNNs) have become instrumental in advancing multi-frame image super-resolution (SR), a technique that merges multiple low-resolution images of the same scene into a high-resolution image. In this paper, a novel deep learning multi-frame SR algorithm is introduced. The proposed CNN model, named Exponential Fusion of Interpolated Frames Network (EFIF-Net), seamlessly integrates fusion and restoration within an end-to-end network. Key features of the new EFIF-Net include a custom exponentially weighted fusion (EWF) layer for image fusion and a modification of the Residual Channel Attention Network for restoration to deblur the fused image. Input frames are registered with subpixel accuracy using an affine motion model to capture the camera platform motion. The frames are externally upsampled using single-image interpolation. The interpolated frames are then fused with the custom EWF layer, employing subpixel registration information to give more weight to pixels with less interpolation error. Realistic image acquisition conditions are simulated to generate training and testing datasets with corresponding ground truths. The observation model captures optical degradation from diffraction and detector integration from the sensor. The experimental results demonstrate the efficacy of EFIF-Net using both simulated and real camera data. The real camera results use authentic, unaltered camera data without artificial downsampling or degradation.

Single image super-resolution via global aware external attention and multi-scale residual channel attention network

Single image super-resolution via global aware external attention and multi-scale residual channel attention network

Fast Brillouin optical time domain sensing based on two-dimension sparse sampling and deep learning recovery

Brillouin optical time domain sensing (BOTDS) is a technique for acquiring the Brillouin scattering spectrum (BSS) of a fiber under test (FUT) by utilizing multiple frequency scans. However, the time-consuming nature of this process limits its applicability in scenarios that require rapid measurements. In order to enhance measurement efficiency and decrease storage capacity, without compromising measurement accuracy or spatial resolution, we propose a novel fast BOTDS method based on two-dimensional (2D) sparse sampling and residual channel attention network (RCAN). The proposed method employs sparse sampling to measure the BSS signal, which is then accurately restored to the BSS with normal frequency sampling intervals by using deep learning techniques. Temperature and strain information is extracted from the restored BSS, facilitating rapid and precise measurements. The effectiveness of the proposed method has been verified through simulation and experiments. In a proof-of-concept experiment, we employed a 3000 m bend-insensitive fiber as the sensing fiber and reduced the sampling frequency by a factor of 10. Our proposed method successfully recovered the Brillouin scattering spectrum signal, resulting in a temperature measurement uncertainty of 0.25 °C. Moreover, the measurement accuracy and spatial resolution almost remained unaffected, surpassing that of interpolation-based approaches.

XVDPU: A High Performance CNN Accelerator on Versal Platform Powered by AI Engine

Nowadays, convolution neural networks (CNNs) are widely used in computer vision applications. However, the trends of higher accuracy and higher resolution generate larger networks. The requirements of computation or I/O are the key bottlenecks. In this paper, we propose XVDPU: the AI-Engine (AIE)-based CNN accelerator on Versal chips to meet heavy computation requirements. To resolve IO bottleneck, we adopt several techniques to improve data-reuse and reduce I/O requirements. An Arithmetic Logic Unit (ALU) is further proposed which can better balance resource utilization, new feature support, and efficiency of the whole system. We have successfully deployed more than 100 CNN models with our accelerator. Our experimental results show that the 96-AIE-core implementation can achieve 1653 frames per second (FPS) for ResNet50 on VCK190, which is 9.8 × faster than the design on ZCU102 running at 168.5 FPS. The 256-AIE-core implementation can further achieve 4050 FPS. We propose a tilling strategy to achieve feature-map-stationary (FMS) for high-definition CNN (HD-CNN) with the accelerator, achieving 3.8 × FPS improvement on Residual Channel Attention Network (RCAN) and 3.1 × on Super-Efficient Super-Resolution (SESR). This accelerator can also solve the 3D convolution task in disparity estimation, achieving end-to-end (E2E) performance of 10.1FPS with all the optimizations.

Deep recurrent residual channel attention network for single image super-resolution

Deep recurrent residual channel attention network for single image super-resolution

Design of a Wind Power Forecasting System Based on Deep Learning

In recent years, wind energy, as a ubiquitous, easy-to-capture cost-effective clean energy, is accounting for a sharp increase in the installed capacity of China’s new energy grid. However, its stochastic volatility has always brought challenges to the generation scheduling of wind farms. To better optimize the energy management level of wind farms and improve the stability of wind power grid connection, this paper proposes a design of a wind power forecasting system based on deep learning. Our system architecture mainly includes data pre-processing, power prediction, and data application modules. The data pre-processing module will correct the numerical weather forecast (NWP) updated twice daily to get the wind turbine generator (WTG) hub height weather forecast and resample the SCADA data in a 15-minute time resolution. The power prediction module will periodically perform the ultra-short-term and short-term forecasting results and record them in the database. On one hand, the data application module will present the power prediction results and SCADA data to the web page for display. On the other hand, it will provide data to the energy dispatching department for reference. The experiment shows the effect of using a residual channel attention network (RCAN) to correct the NWP and the influence of two different RNN cores, including GRU and LSTM, on the prediction of ultra-short-term wind power results of the DeepAR model. The experimental results show that the proposed RCAN can effectively correct the results of the numerical weather forecast to a single WTG, and the DeepAR model using the GRU core can achieve better performance in the test set than the DeepAR model using the LSTM core. Thus, we choose GRU as the core of the model of our power prediction module.

Single Image Deraining Using Residual Channel Attention Networks

Single Image Deraining Using Residual Channel Attention Networks

Seismic full-waveform inversion based on superresolution for high-precision prediction of reservoir parameters

The frequency band limitation of seismic data limits the resolution of full-waveform inversion (FWI) results. In addition, the high computational cost seriously affects the practical application of FWI. To alleviate these concerns, an FWI method based on superresolution (SR-FWI) is developed to improve the prediction efficiency and accuracy of the reservoir parameters. A channel attention mechanism is introduced for the multifrequency characteristics of the model images. A constrained residual channel attention network (CRCAN) is built for superresolution (SR) by adding structural constraints to the loss function of a deep learning network. A total of 65,000 sets of geologic models and natural images constitute the network training data, 90% of which are used for training with the rest used for testing. The iterative calculation for FWI is time-consuming; hence, SR is applied to the iterative process to reduce the number of iterations and accelerate the model update. Low-resolution images along with the synthetic and field data are used for the evaluation of the CRCAN and SR-FWI algorithms, respectively. The test results find that CRCAN can effectively improve the image resolution, whereas SR-FWI is beneficial due to its high efficiency and precision, especially in predicting the stratum edge and small-scale anomalies. Therefore, SR-FWI is a powerful means of reservoir static and dynamic detection and can provide high-resolution information for projects, such as resource development and CO2 storage.

Image Super-resolution Methods for FY-3E X-EUVI 195 Å Solar Images

Solar eruptions and the solar wind are sources of space weather disturbances, and extreme-ultraviolet (EUV) observations are widely used to research solar activity and space weather forecasts. Fengyun-3E is equipped with the Solar X-ray and Extreme Ultraviolet Imager, which can observe EUV imaging data. Limited by the lower resolution, however, we research super-resolution techniques to improve the data quality. Traditional image interpolation methods have limited expressive ability, while deep-learning methods can learn to reconstruct high-quality images through training on paired data sets. There is a wide variety of super-resolution models. We try these three representative models: Real-ESRGAN combined with generative adversarial networks, residual channel-attention networks (RCAN) based on channel attention, and SwinIR, based on self-attention. Instruments on different satellites differ in observation time, angle, and resolution, so we selected Solar Dynamics Observatory/Atmospheric Imaging Assembly (SDO/AIA) 193 Å images with similar wavelengths as a reference and used a feature-based method for image registration to eliminate slight deformations to build training data sets. Finally, we compare the above methods in their evaluation metrics and visual quality. RCAN has the highest peak signal-to-noise ratio and structural similarity evaluation. Real-ESRGAN model is the best in the Learned Perceptual Image Patch Similarity index, and its results visually show that it has more highly detailed textures. The corrected super-resolution results can complement the SDO/AIA data to provide solar EUV images with a higher temporal resolution for space weather forecasting and solar physics research.

Denoising and wavefield separation method for DAS VSP via deep learning

Distributed acoustic sensing (DAS) vertical seismic profiling (VSP) is, a new combined method of borehole seismic exploration, that has the advantages of low cost, high resistance to temperature and pressure, and high sampling rate. However, the collected data generally suffer from high noise, detracting from the potential high-precision imaging benefits and limiting the full application of the technique. The quality of wavefield separation is also crucial to VSP processing. To mitigate the various problems, a regularized residual channel attention network (RRCAN) is proposed for improved denoising and wavefield separation. Based on two prediction tasks, the networks are built (i.e., RRCAN_1 and RRCAN_2) by adding a regularization term to the loss function. To reduce the loss of effective signals during the denoising process, a wavefield separation method that includes secondary effective information extraction is adopted. The final separation results reflect the denoised and removed noise data, a total of 500 geological models are constructed using a random method and RRCAN_1 is trained with synthetic and field noise data. The conventional plane-wave destruction method is improved by using velocity information, and high-precision separation results for upgoing and downgoing waves are obtained for RRCAN_2 training. The test results obtained for the synthetic and field data show that the proposed network effectively suppresses DAS VSP noise while yielding excellent separation results for downgoing and upgoing waves. Therefore, denoising and wavefield separation methods based on RRCAN provide effective high-precision DAS VSP imaging.

Mask R-CNN based automated identification and extraction of oil well sites

• A new CNN based instance segmentation method. • The first study for instance segmentation-based oil well sites identification and extraction. • A complete framework including data fusion, object detection and extraction, and postprocessing for oil well sites for the task. Fine-scale land disturbances due to mining development modify the land surface cover and have cumulative detrimental impacts on the environment. Understanding the distribution of fine-scale land disturbances related to mining activities, such as oil well sites, in mining regions is of vital importance to sustainable mining development. For efficient mapping, automated identification and extraction of the oil well sites using high-resolution satellite images are required. In this work, we proposed the Oil Well Site extraction (OWS) Mask R-CNN based on the original Mask R-CNN (Region-based Convolutional Neural Networks), to accurately extract well sites using multi-sensor remote sensing images. For improvement of mapping efficiency, two modifications were made to Mask R-CNN: (1) replacing the backbone of Mask R-CNN with D-LinkNet, and (2) adding a semantic segmentation branch to Mask R-CNN to force the whole network to focus on the relationship between line objects and oil well sites. As imagery data were from multiple sensors (RapidEye 2/3 and WorldView 3), a pre-trained Residual Channel Attention Network (RCAN) was applied to super-resolve the images with different resolutions. Several key spatial features, such as nearby roads and area size, have also been used in the oil well site mapping process. The experimental results indicate that our OWS Mask R-CNN considerably improves the average precision (AP) and the F 1 score of Mask R-CNN from 51.26% and 25.7% to 60.93% and 61.59%, respectively.

Residual Channel Attention Network for Brain Glioma Segmentation.

A glioma is a malignant brain tumor that seriously affects cognitive functions and lowers patients' life quality. Segmentation of brain glioma is challenging because of inter-class ambiguities in tumor regions. Recently, deep learning approaches have achieved outstanding performance in the automatic segmentation of brain glioma. However, existing al-gorithms fail to exploit channel-wise feature interdependence to select semantic attributes for glioma segmentation. In this study, we implement a novel deep neural network that integrates residual channel attention modules to calibrate intermediate features for glioma segmentation. The proposed channel at-tention mechanism adaptively weights feature channel-wise to optimize the latent representation of gliomas. We evaluate our method on the established dataset BraTS2017. Experimental results indicate the superiority of our method. Clinical relevance - While existing glioma segmentation approaches do not leverage channel-wise feature dependence for feature selection our method can generate segmentation masks with higher accuracies and provide more insights on graphic patterns in brain MRI images for further clinical reference.

Super-resolving and composing building dataset using a momentum spatial-channel attention residual feature aggregation network

Model generalizability is crucial in the deployment of deep learning (DL) techniques. When trained on specific datasets, generalizability problems arise across many applications of DL including building extractions. Apart from regularizing the training process, collecting data with distinctive characteristics or distributions can be a promising solution. Over the past decade, several open building datasets have been released. However, in practice, a single dataset cannot overcome the generalization error. By unifying the spatial resolution and spectral bands of different datasets, those datasets could be integrated to relieve the generalization error in building footprint extraction. In this work, we focused on the difference in the spatial resolution between different building datasets. We first examined state-of-the-art super-resolution methods and proposed our own method based on Residual Feature Aggregation Network (RFANet), which we named Momentum and Spatial-Channel Attention RFANet (MSCA-RFANet). We then benchmarked our MSCA-RFANet in a comparative study; our new method achieved higher performance on spatial resolution enhancement. Specifically, in the four times spatial resolution enhancement on the SWOOP 2010 Dataset, our MSCA-RFANet result’s peak signal-to-noise ratio (PSNR) of 30.72 dB exceeded that of RFANet (30.66 dB). Likewise, we achieved a lower mean squared error (MSE) of 36.64 compared to RFANet’s 36.94. With detailed benchmarks against Second-order Attention Network (SAN) and Residual Channel Attention Network (RCAN), we confirmed the superior performance of our method in enhancing the spatial resolution of high-spatial-resolution images. Then, we explored the impact of super-resolution resolution and data composition on building footprint extraction. Our building footprint extraction experiments demonstrated the positive impact of super-resolution and data composition. These promising results showed that our method is suitable to integrating existing public building dataset to overcome generalization error in DL-based building footprint extraction.

IMPACT OF DEEP LEARNING-BASED SUPER-RESOLUTION ON BUILDING FOOTPRINT EXTRACTION

Abstract. Automated building footprints extraction from High Spatial Resolution (HSR) remote sensing images plays important roles in urban planning and management, and hazard and disease control. However, HSR images are not always available in practice. In these cases, super-resolution, especially deep learning (DL)-based methods, can provide higher spatial resolution images given lower resolution images. In a variety of remote sensing applications, DL based super-resolution methods are widely used. However, there are few studies focusing on the impact of DL-based super-resolution on building footprint extraction. As such, we present an exploration of this topic. Specifically, we first super-resolve the Massachusetts Building Dataset using bicubic interpolation, a pre-trained Super-Resolution CNN (SRCNN), a pre-trained Residual Channel Attention Network (RCAN), a pre-trained Residual Feature Aggregation Network (RFANet). Then, using the dataset under its original resolution, as well as the four different super-resolutions of the dataset, we employ the High-Resolution Network (HRNet) v2 to extract building footprints. Our experiments show that super-resolving either training or test datasets using the latest high-performance DL-based super-resolution method can improve the accuracy of building footprints extraction. Although SRCNN based building footprint extraction gives the highest Overall Accuracy, Intersection of Union and F1 score, we suggest using the latest super-resolution method to process images before building footprint extraction due to the fixed scale ratio of pre-trained SRCNN and low speed of convergence in training.

Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network

Although the existing deep-learning-based hyperspectral image (HSI) denoising methods have achieved tremendous success, recovering high-quality HSIs in complex scenes that contain mixed noise is still challenging. Besides, these methods have not fully explored the local and global spatial–spectral information of HSIs. To address the above issues, a novel HSI mixed noise removal network called subspace projection attention and residual channel attention network (SPARCA-Net) is proposed. Specifically, we propose an orthogonal subspace projection attention (OSPA) module to adaptively learn to generate bases of the signal subspace and project the input into such space to remove noise. By leveraging the local and global spatial relations, OSPA is able to reconstruct the local structure of the feature maps more precisely. We further propose a residual channel attention (RCA) module to emphasize the interdependence between feature maps and exploit the global channel correlation of them, which could enhance the channel-wise adaptive learning. In addition, multiscale joint spatial–spectral input and residual learning strategies are employed to capture multiscale spatial–spectral features and reduce the degradation problem, respectively. Synthetic and real HSI data experiments demonstrated that the proposed HSI denoising network outperforms many of the advanced methods in both quantitative and qualitative assessments.

LCRCA: image super-resolution using lightweight concatenated residual channel attention networks

LCRCA: image super-resolution using lightweight concatenated residual channel attention networks

An Effective Network Integrating Residual Learning and Channel Attention Mechanism for Thin Cloud Removal

Cloud-contaminated images seriously disturb the effective information in ground observations and reduce the availability of remote sensing images. This letter presents a thin cloud removal method for cloud-contaminated images based on a residual channel attention network. Compared with the present thin cloud removal methods not showing different attention to cloud components and ground details, the proposed method introduces channel attention mechanism into the residual learning (RL) path, which achieves suppressing thin clouds and enhancing the details of ground scenes simultaneously. Based on this, both residual channel attention module (RCAM) and residual group (RG) are constructed, playing the role of basic modules to stack an encoder–decoder-based residual channel attention network, which effectively avoids the loss of ground information in the deep layers. Compared with recent state-of-the-art (SOTA) methods, experimental results on both actual and synthesized cloudy images show the proposed method’s superiority in reconstructing rich details of ground scenes.

An Adversarial Learning Approach for Super-Resolution Enhancement Based on AgCl@Ag Nanoparticles in Scanning Electron Microscopy Images.

Scanning electron microscopy (SEM) plays a crucial role in the characterization of nanoparticles. Unfortunately, due to the limited resolution, existing imaging techniques are insufficient to display all detailed characteristics at the nanoscale. Hardware-oriented techniques are troubled with costs and material properties. Computational approaches often prefer blurry results or produce a less meaningful high-frequency noise. Therefore, we present a staged loss-driven neural networks model architecture to transform low-resolution SEM images into super-resolved ones. Our approach consists of two stages: first, residual channel attention network (RCAN) with mean absolute error (MAE) loss was used to get a better peak signal-to-noise ratio (PSNR). Then, discriminators with adversarial losses were activated to reconstruct high-frequency texture features. The quantitative and qualitative evaluation results indicate that compared with other advanced approaches, our model achieves satisfactory results. The experiment in AgCl@Ag for photocatalytic degradation confirms that our proposed method can bring realistic high-frequency structural detailed information rather than meaningless noise. With this approach, high-resolution SEM images can be acquired immediately without sample damage. Moreover, it provides an enhanced characterization method for further directing the preparation of nanoparticles.

TCRAN: Multivariate time series classification using residual channel attention networks with time correction

Currently, the most popular and effective approach to solve multivariate time series classification(MTSC) tasks is based on deep learning technology. However, the existing deep learning-based algorithms ignore the unique time characteristics of time series in the process of network training, and do not consider the features correlation in different convolutional layers. So they cannot obtain the convincing feature representation ability and result in unsatisfactory classification accuracy. To solve this problem, we propose a new time corrected residual attention network(TCRAN) which can fully extract the long-term time-dependence information to enhance the discriminative power of the network. The hallmark of TCRAN is that we employ the time residual channel attention block(TRCAB) as the basic structure, which incorporates the adaptive channel feature adjustment mechanism(AFM) and the bi-directional gated recurrent unit(Bi-GRU) into the deep residual structure to adaptively extract time-dependent features. Meanwhile, to integrate the overall dependency information between different layers, we also employ an inter-module adaptive feature adjustment mechanism(IAM) in the TCRAN. The experiments results with 15 multivariate time series datasets illustrate that the proposed TCRAN can achieve the highest average classification accuracy of 0.7276 and improve accuracy by 1.64% compared to the state-of-the-art method. All these verify the effectiveness of TCRAN.