Hybrid Loss Function Research Articles

Global Guided Cross-Modal Cross-Scale Network for RGB-D Salient Object Detection.

RGB-D saliency detection aims to accurately localize salient regions using the complementary information of a depth map. Global contexts carried by the deep layer are key to salient objection detection, but they are diluted when transferred to shallower layers. Besides, depth maps may contain misleading information due to the depth sensors. To tackle these issues, in this paper, we propose a new cross-modal cross-scale network for RGB-D salient object detection, where the global context information provides global guidance to boost performance in complex scenarios. First, we introduce a global guided cross-modal and cross-scale module named G2CMCSM to realize global guided cross-modal cross-scale fusion. Then, we employ feature refinement modules for progressive refinement in a coarse-to-fine manner. In addition, we adopt a hybrid loss function to supervise the training of G2CMCSNet over different scales. With all these modules working together, G2CMCSNet effectively enhances both salient object details and salient object localization. Extensive experiments on challenging benchmark datasets demonstrate that our G2CMCSNet outperforms existing state-of-the-art methods.

A Multi Brain Tumor Region Segmentation Model Based on 3D U-Net

Accurate segmentation of different brain tumor regions from MR images is of great significance in the diagnosis and treatment of brain tumors. In this paper, an enhanced 3D U-Net model was proposed to address the shortcomings of 2D U-Net in the segmentation tasks of brain tumors. While retaining the U-shaped characteristics of the original U-Net network, an enhanced encoding module and decoding module were designed to increase the extraction and utilization of image features. Then, a hybrid loss function combining the binary cross-entropy loss function and dice similarity coefficient was adopted to speed up the model’s convergence and to achieve accurate and fast automatic segmentation. The model’s performance in the segmentation of brain tumor’s whole tumor region, tumor core region, and enhanced tumor region was studied. The results showed that the proposed 3D U-Net model can achieve better segmentation performance, especially for the tumor core region and enhanced tumor region tumor regions.

Multiscale Feature Aggregation Network for Salient Object Detection in Optical Remote Sensing Images

Optical remote sensing images (ORSI) have various applications in different fields, and salient target detection (ORSI-SOD) of ORSI has become an important research topic in recent years. However, ORSI-SOD is a challenging problem due to the variable and complex backgrounds, large differences in levels, mixed backgrounds, and diverse topological shapes of ORSI. In this paper, we propose a novel model called multi-scale feature refinement aggregation network (MFANet), which consists of a multi-scale feature refinement module (MFR) and a context aggregation module (CFA). The MFR module extracts semantic information of ORSI across different dimensions in the multi-scale feature extraction stage. In the feature refinement stage, we use the proposed self-refinement module under the guidance of attention and reverse attention to progressively refine the prediction results. The CFA module introduces the hybrid attention module to gradually aggregate and extract salient regions from the context extraction module. To adapt to dense scenes, we develop a hybrid loss function that enables network optimization of multi-scale objectives in a self-adaptive manner. Our method outperforms most state-of-the-art salient object detection methods proposed in recent years in terms of accuracy.

Self-adaption and texture generation: A hybrid loss function for low-dose CT denoising.

Deep learning has been successfully applied to low-dose CT (LDCT) denoising. But the training of the model is very dependent on an appropriate loss function. Existing denoising models often use per-pixel loss, including mean abs error (MAE) and mean square error (MSE). This ignores the difference in denoising difficulty between different regions of the CT images and leads to the loss of large texture information in the generated image. In this paper, we propose a new hybrid loss function that adapts to the noise in different regions of CT images to balance the denoising difficulty and preserve texture details, thus acquiring CT images with high-quality diagnostic value using LDCT images, providing strong support for condition diagnosis. We propose a hybrid loss function consisting of weighted patch loss (WPLoss) and high-frequency information loss (HFLoss). To enhance the model's denoising ability of the local areas which are difficult to denoise, we improve the MAE to obtain WPLoss. After the generated image and the target image are divided into several patches, the loss weight of each patch is adaptively and dynamically adjusted according to its loss ratio. In addition, considering that texture details are contained in the high-frequency information of the image, we use HFLoss to calculate the difference between CT images in the high-frequency information part. Our hybrid loss function improves the denoising performance of several models in the experiment, and obtains a higher peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Moreover, through visual inspection of the generated results of the comparison experiment, the proposed hybrid function can effectively suppress noise and retain image details. We propose a hybrid loss function for LDCT image denoising, which has good interpretation properties and can improve the denoising performance of existing models. And the validation results of multiple models using different datasets show that it has good generalization ability. By using this loss function, high-quality CT images with low radiation are achieved, which can avoid the hazards caused by radiation and ensure the disease diagnosis for patients.

Bidirectional Attentional Interaction Networks for RGB-D salient object detection

Aiming at the issues of insufficient cross-modality feature interaction and ineffective utilization of cross-modality data in RGB-D saliency object detection (SOD) tasks, we propose a Bidirectional Attentional Interaction Network (BAINet) for RGB-D SOD, which employs an encoder-decoder structure for bidirectional interaction of cross-modality features through a dual-branch progressive fusion approach. To begin with, based on the fact that RGB and depth information streams can complement each other, the bidirectional attention interaction module accomplishes bidirectional interaction between cross-modality features by capturing complementary cues from different modality data. In order to enhance the expressiveness of the fused RGB-D features, the global feature perception module endows the features with rich multi-scale contextual semantic information by enlarging the field of perception. In addition, exploring the correlation of cross-level features is vital to achieve accurate salient inference. Specifically, We introduce a cross-level guidance aggregation module to capture inter-layer dependencies and complete the integration of cross-level features, which effectively suppresses shallow cross-modality features and refines the saliency map during decoding. To improve the model training speed, a hybrid loss function is employed to train multi-branch saliency inference maps simultaneously. Extensive experiments on five publicly available datasets clearly show that the proposed model outperforms 18 state-of-the-art methods.

Recognition of multivariate geochemical anomalies using a geologically-constrained variational autoencoder network with spectrum separable module – A case study in Shangluo District, China

This study has developed a novel variational autoencoder architecture by incorporating the spectrum separable module, termed SSM-VAE, so as to recognize the multi-mineral-species geochemical patterns over the Shangluo district, central China, and then facilitate a better understanding of the regional metallogenesis. The primary advantage of SSM-VAE is its ability to explore the interlayer correlations and then integrate them into the reconstructed features, which greatly improved the geological interpretability of the model products. In the training process, a hybrid loss function with a geologically-constrained term derived from the fractal analysis was designed to guide the model to focus on anomalies related to mineralization. Afterward, the factor analysis together with the EM-MML thresholding algorithm were integrated as a post-processing tool. Finally, the mineral-spot identification rate (δ) versus the size of the anomalous area (S) are used to validate the ore-bearing potential of different anomaly divisions. Comparing to other state-of-the-art models without regard to the interlayer correlations, SSM-VAE can produce well-zoned, mineral species-specific, and more metalliferous anomalous patches. To be specific, we separated out 10 zoned anomaly divisions, and on average, when S = 1% there are 8.5013 mineral spots falling within.

Dynamic controllable residual generative adversarial network for low-dose computed tomography imaging.

Computed tomography (CT) imaging technology has become an indispensable auxiliary method in medical diagnosis and treatment. In mitigating the radiation damage caused by X-rays, low-dose computed tomography (LDCT) scanning is becoming more widely applied. However, LDCT scanning reduces the signal-to-noise ratio of the projection, and the resulting images suffer from serious streak artifacts and spot noise. In particular, the intensity of noise and artifacts varies significantly across different body parts under a single low-dose protocol. To improve the quality of different degraded LDCT images in a unified framework, we developed a generative adversarial learning framework with a dynamic controllable residual. First, the generator network consists of the basic subnetwork and the conditional subnetwork. Inspired by the dynamic control strategy, we designed the basic subnetwork to adopt a residual architecture, with the conditional subnetwork providing weights to control the residual intensity. Second, we chose the Visual Geometry Group Network-128 (VGG-128) as the discriminator to improve the noise artifact suppression and feature retention ability of the generator. Additionally, a hybrid loss function was specifically designed, including the mean square error (MSE) loss, structural similarity index metric (SSIM) loss, adversarial loss, and gradient penalty (GP) loss. The results obtained on two datasets show the competitive performance of the proposed framework, with a 3.22 dB peak signal-to-noise ratio (PSNR) margin, 0.03 SSIM margin, and 0.2 contrast-to-noise ratio margin on the Challenge data and a 1.0 dB PSNR margin and 0.01 SSIM margin on the real data. Experimental results demonstrated the competitive performance of the proposed method in terms of noise decrease, structural retention, and visual impression improvement.

Contour Information-Guided Multi-Scale Feature Detection Method for Visible-Infrared Pedestrian Detection.

Infrared pedestrian target detection is affected by factors such as the low resolution and contrast of infrared pedestrian images, as well as the complexity of the background and the presence of multiple targets occluding each other, resulting in indistinct target features. To address these issues, this paper proposes a method to enhance the accuracy of pedestrian target detection by employing contour information to guide multi-scale feature detection. This involves analyzing the shapes and edges of the targets in infrared images at different scales to more accurately identify and differentiate them from the background and other targets. First, we propose a preprocessing method to suppress background interference and extract color information from visible images. Second, we propose an information fusion residual block combining a U-shaped structure and residual connection to form a feature extraction network. Then, we propose an attention mechanism based on a contour information-guided approach to guide the network to extract the depth features of pedestrian targets. Finally, we use the clustering method of mIoU to generate anchor frame sizes applicable to the KAIST pedestrian dataset and propose a hybrid loss function to enhance the network's adaptability to pedestrian targets. The extensive experimental results show that the method proposed in this paper outperforms other comparative algorithms in pedestrian detection, proving its superiority.

DARMF-UNet: A dual-branch attention-guided refinement network with multi-scale features fusion U-Net for gland segmentation

Accurate gland segmentation is critical in determining adenocarcinoma. Automatic gland segmentation methods currently suffer from challenges such as less accurate edge segmentation, easy mis-segmentation, and incomplete segmentation. To solve these problems, this paper proposes a novel gland segmentation network Dual-branch Attention-guided Refinement and Multi-scale Features Fusion U-Net (DARMF-UNet), which fuses multi-scale features using deep supervision. At the first three layers of feature concatenation, a Coordinate Parallel Attention (CPA) is proposed to guide the network to focus on the key regions. A Dense Atrous Convolution (DAC) block is used in the fourth layer of feature concatenation to perform multi-scale features extraction and obtain global information. A hybrid loss function is adopted to calculate the loss of each segmentation result of the network to achieve deep supervision and improve the accuracy of segmentation. Finally, the segmentation results at different scales in each part of the network are fused to obtain the final gland segmentation result. The experimental results on the gland datasets Warwick-QU and Crag show that the network improves in terms of the evaluation metrics of F1 Score, Object Dice, Object Hausdorff, and the segmentation effect is better than the state-of-the-art network models.

Deep learning framework to improve the quality of cone-beam computed tomography for radiotherapy scenarios.

The application of cone-beam computed tomography (CBCT) in image-guided radiotherapy and adaptive radiotherapy remains limited due to its poor image quality. In this study, we aim to develop a deep learning framework to generate high-quality CBCT images for therapeutic applications. The synthetic CT (sCT) generation from the CBCT was proposed using a transformer-based network with a hybrid loss function. The network was trained and validated using the data from 176 patients to produce a general model that can be extensively applied to enhance CBCT images. After the first therapy, each patient can receive paired CBCT/planning CT (pCT) scans, and the obtained data were used to fine-tune the general model for further improvement. For subsequent treatment, a patient-specific, personalized model was made available. In total, 34 patients were examined for general model testing, and another six patients who underwent rescanned pCT scan were used for personalized model training and testing. The general model decreased the mean absolute error (MAE) from 135 HU to 59 HU as compared to the CBCT. The hybrid loss function demonstrated superior performance in CT number correction and noise/artifacts reduction. The proposed transformer-based network also showed superior power in CT number correction compared to the classical convolutional neural network. The personalized model showed improvement based on the general model in some details, and the MAE was reduced from 59 HU (for the general model) to 57 HU (p<0.05 Wilcoxon signed-rank test). We established a deep learning framework based on transformer for clinical needs. The deep learning model demonstrated potential for continuous improvement with the help of a suggested personalized training strategy compatible with the clinical workflow.

Implementation of a machine-learned gas optics parameterization in the ECMWF Integrated Forecasting System: RRTMGP-NN 2.0

Abstract. Radiation schemes are physically important but computationally expensive components of weather and climate models. This has spurred efforts to replace them with a cheap emulator based on neural networks (NNs), obtaining large speed-ups, but at the expense of accuracy, energy conservation and generalization. An alternative approach, which is slower but more robust than full emulation, is to use NNs to predict optical properties but keep the radiative transfer equations. Recently, NNs were developed to replace the RRTMGP (Rapid Radiative Transfer Model for General circulation model applications–Parallel) gas optics scheme and shown to be accurate while improving speed. However, the evaluations were based solely on offline radiation computations. In this paper, we describe the implementation and prognostic evaluation of RRTMGP-NN in the Integrated Forecasting System (IFS) of the European Centre for Medium-Range Weather Forecasts (ECMWF). The new gas optics scheme was incorporated into ecRad, the modular ECMWF radiation scheme. Using two new methods to improve accuracy – a hybrid loss function designed to reduce radiative forcing errors and an early stopping method based on monitoring fluxes and heating rates with respect to a line-by-line benchmark – we train NN models on RRTMGP k distributions with reduced spectral resolutions. Offline evaluation of the new NN gas optics, RRTMGP-NN 2.0, shows a very high level of accuracy for clear-sky fluxes and heating rates. For instance, the RMSE in the shortwave surface downwelling flux is 0.78 W m−2 for RRTMGP and 0.80 W m−2 for RRTMGP-NN in a present-day scenario, while upwelling flux errors are actually smaller for the NN. Because our approach does not affect the treatment of clouds, no additional errors will be introduced for cloudy profiles. RRTMGP-NN closely reproduces radiative forcings for five important greenhouse gases across a wide range of concentrations such as 8×CO2. To assess the impact of different gas optics schemes in the IFS, four 1-year coupled ocean–atmosphere simulations were performed for each configuration. The results show that RRTMGP-NN and RRTMGP produce very similar model climates, with the differences being smaller than those between existing schemes and statistically insignificant for zonal means of single-level quantities such as surface temperature. The use of RRTMGP-NN speeds up ecRad by a factor of 1.5 compared to RRTMGP (the gas optics being almost 3 times faster) and is also faster than the older and less accurate RRTMG, which is used in the current operational cycle of the IFS.

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion.

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

DRU-Net: Pulmonary Artery Segmentation via Dense Residual U-Network with Hybrid Loss Function

The structure and topology of the pulmonary arteries is crucial to understand, plan, and conduct medical treatment in the thorax area. Due to the complex anatomy of the pulmonary vessels, it is not easy to distinguish between the arteries and veins. The pulmonary arteries have a complex structure with an irregular shape and adjacent tissues, which makes automatic segmentation a challenging task. A deep neural network is required to segment the topological structure of the pulmonary artery. Therefore, in this study, a Dense Residual U-Net with a hybrid loss function is proposed. The network is trained on augmented Computed Tomography volumes to improve the performance of the network and prevent overfitting. Moreover, the hybrid loss function is implemented to improve the performance of the network. The results show an improvement in the Dice and HD95 scores over state-of-the-art techniques. The average scores achieved for the Dice and HD95 scores are 0.8775 and 4.2624 mm, respectively. The proposed method will support physicians in the challenging task of preoperative planning of thoracic surgery, where the correct assessment of the arteries is crucial.

Research of segmentation recognition of small disease spots on apple leaves based on hybrid loss function and CBAM.

Identification technology of apple diseases is of great significance in improving production efficiency and quality. This paper has used apple Alternaria blotch and brown spot disease leaves as the research object and proposes a disease spot segmentation and disease identification method based on DFL-UNet+CBAM to address the problems of low recognition accuracy and poor performance of small spot segmentation in apple leaf disease recognition. The goal of this paper is to accurately prevent and control apple diseases, avoid fruit quality degradation and yield reduction, and reduce the resulting economic losses. DFL-UNet+CBAM model has employed a hybrid loss function of Dice Loss and Focal Loss as the loss function and added CBAM attention mechanism to both effective feature layers extracted by the backbone network and the results of the first upsampling, enhancing the model to rescale the inter-feature weighting relationships, enhance the channel features of leaf disease spots and suppressing the channel features of healthy parts of the leaf, and improving the network's ability to extract disease features while also increasing model robustness. In general, after training, the average loss rate of the improved model decreases from 0.063 to 0.008 under the premise of ensuring the accuracy of image segmentation. The smaller the loss value is, the better the model is. In the lesion segmentation and disease identification test, MIoU was 91.07%, MPA was 95.58%, F1 Score was 95.16%, MIoU index increased by 1.96%, predicted disease area and actual disease area overlap increased, MPA increased by 1.06%, predicted category correctness increased, F1 Score increased by 1.14%, the number of correctly identified lesion pixels increased, and the segmentation result was more accurate. Specifically, compared to the original U-Net model, the segmentation of Alternaria blotch disease, the MIoU value increased by 4.41%, the MPA value increased by 4.13%, the Precision increased by 1.49%, the Recall increased by 4.13%, and the F1 Score increased by 2.81%; in the segmentation of brown spots, MIoU values increased by 1.18%, MPA values by 0.6%, Precision by 0.78%, Recall by 0.6%, and F1 Score by 0.69%. The spot diameter of the Alternaria blotch disease is 0.2-0.3cm in the early stage, 0.5-0.6cm in the middle and late stages, and the spot diameter of the brown spot disease is 0.3-3cm. Obviously, brown spot spots are larger than Alternaria blotch spots. The segmentation performance of smaller disease spots has increased more noticeably, according to the quantitative analysis results, proving that the model's capacity to segment smaller disease spots has greatly improved. The findings demonstrate that for the detection of apple leaf diseases, the method suggested in this research has a greater recognition accuracy and better segmentation performance. The model in this paper can obtain more sophisticated semantic information in comparison to the traditional U-Net, further enhance the recognition accuracy and segmentation performance of apple leaf spots, and address the issues of low accuracy and low efficiency of conventional disease recognition methods as well as the challenging convergence of conventional deep convolutional networks.

Longitudinal prediction of postnatal brain magnetic resonance images via a metamorphic generative adversarial network

Missing scans are inevitable in longitudinal studies due to either subject dropouts or failed scans. In this paper, we propose a deep learning framework to predict missing scans from acquired scans, catering to longitudinal infant studies. Prediction of infant brain MRI is challenging owing to the rapid contrast and structural changes particularly during the first year of life. We introduce a trustworthy metamorphic generative adversarial network (MGAN) for translating infant brain MRI from one time point to another. MGAN has three key features: (i) Image translation leveraging spatial and frequency information for detail-preserving mapping; (ii) Quality-guided learning strategy that focuses attention on challenging regions. (iii) Multi-scale hybrid loss function that improves translation of image contents. Experimental results indicate that MGAN outperforms existing GANs by accurately predicting both tissue contrasts and anatomical details.

Improving building rooftop segmentation accuracy through the optimization of UNet basic elements and image foreground-background balance

Building rooftop segmentation using deep learning techniques is a popular yet challenging area of research in computer vision and remote sensing image processing. While recent studies have developed various deep learning models, there has been less focus on investigating the basic elements of network architecture and foreground-background balance in sampled images. To address this research gap, this study proposes optimizing the UNet basic elements and image foreground-background balance to improve building rooftop segmentation accuracy. The Inria dataset is used for model training and accuracy evaluation. The results show that the impact of network backbone on segmentation accuracy depends on its ability to extract lower-level features and whether it is pretrained by large amounts of data. This study also finds that residual structures of network backbone can negatively affect performance regardless of the network depth. Among the decoding network upsampling strategies, the proposed super-resolution method achieves the best segmentation results, but does not significantly differ from the commonly used bilinear interpolation and deconvolution methods. Additionally, this study proposes a hybrid loss function that considers area, arithmetic, shape, spectral, and texture discrepancy, which further improves the segmentation performance. Finally, this study highlights the significant impact of image foreground-background balance on building rooftop segmentation and proposes an ensemble model to mitigate image foreground-background imbalance and improve segmentation performance.

AGHRNet: An attention ghost-HRNet for confirmation of catch‐and‐shake locations in jujube fruits vibration harvesting

The development of an intelligent jujube fruit harvesting device is a critical step in achieving the whole mechanization process. Catch‐and‐shake harvesting, as an efficient and stable vibration harvesting method, has widely been used to save labor and improve harvesting efficiency in large-scale jujube orchards. However, existing catch‐and‐shake harvesters still rely heavily on the operator's naked eyes to determine the shaking position, which is subjective, highly inefficient and highly labor intensive. To address this problem, this study proposes a computer vision system including truck and branches segmentation, skeleton extraction and vibration position selection to automatically identify appropriate shaking positions. A lightweight attention Ghost-HRNet (AGHRNet) based on deep learning is designed to separate the truck and branches from the complex orchard background. In AGHRNet, an attention Ghost block is proposed to lighten the model and improve segmentation accuracy. Moreover, we also construct a hybrid loss function to solve the pixel-level category imbalance of the trunks, branches and background. To verify the effectiveness of the AGHRNet, ten state-of-the-art semantic segmentation methods, including the FCN8s, SegNet, U-Net, PSPNet, DeepLabv3+, BiSeNet, DDRNet, DANet, SegFormer and HRNet, are compared. Comparative experimental results show that compared with the other state-of-the-art methods, the proposed AGHRNet has higher segmentation accuracy (77.79 % mIoU, 89.46 % mPA) and a smaller model size (9.53 MB) on our datasets. Finally, mask medial axis extraction and the minimum inscribed circle operation are applied to obtain the appropriate vibration location. The computer vision system can provide technical references for catch‐and‐shake harvesting and developing jujube fruit-harvesting robots.

Automated Skeletal Bone Age Assessment with Two-Stage Convolutional Transformer Network Based on X-ray Images.

Human skeletal development is continuous and staged, and different stages have various morphological characteristics. Therefore, bone age assessment (BAA) can accurately reflect the individual's growth and development level and maturity. Clinical BAA is time consuming, highly subjective, and lacks consistency. Deep learning has made considerable progress in BAA in recent years by effectively extracting deep features. Most studies use neural networks to extract global information from input images. However, clinical radiologists are highly concerned about the ossification degree in some specific regions of the hand bones. This paper proposes a two-stage convolutional transformer network to improve the accuracy of BAA. Combined with object detection and transformer, the first stage mimics the bone age reading process of the pediatrician, extracts the hand bone region of interest (ROI) in real time using YOLOv5, and proposes hand bone posture alignment. In addition, the previous information encoding of biological sex is integrated into the feature map to replace the position token in the transformer. The second stage extracts features within the ROI by window attention, interacts between different ROIs by shifting the window attention to extract hidden feature information, and penalizes the evaluation results using a hybrid loss function to ensure its stability and accuracy. The proposed method is evaluated on the data from the Pediatric Bone Age Challenge organized by the Radiological Society of North America (RSNA). The experimental results show that the proposed method achieves a mean absolute error (MAE) of 6.22 and 4.585 months on the validation and testing sets, respectively, and the cumulative accuracy within 6 and 12 months reach 71% and 96%, respectively, which is comparable to the state of the art, markedly reducing the clinical workload and realizing rapid, automatic, and high-precision assessment.

Boosting Adversarial Transferability with Shallow-Feature Attack on SAR Images

Adversarial example generation on Synthetic Aperture Radar (SAR) images is an important research area that could have significant impacts on security and environmental monitoring. However, most current adversarial attack methods on SAR images are designed for white-box situations by end-to-end means, which are often difficult to achieve in real-world situations. This article proposes a novel black-box targeted attack method, called Shallow-Feature Attack (SFA). Specifically, SFA assumes that the shallow features of the model are more capable of reflecting spatial and semantic information such as target contours and textures in the image. The proposed SFA generates ghost data packages for input images and generates critical features by extracting gradients and feature maps at shallow layers of the model. The feature-level loss is then constructed using the critical features from both clean images and target images, which is combined with the end-to-end loss to form a hybrid loss function. By fitting the critical features of the input image at specific shallow layers of the neural network to the target critical features, our attack method generates more powerful and transferable adversarial examples. Experimental results show that the adversarial examples generated by the SFA attack method improved the success rate of single-model attack under a black-box scenario by an average of 3.73%, and 4.61% after combining them with ensemble-model attack without victim models.

RaGMAN: A relativistic average generative multi-adversarial network for pansharpening

Pansharpening aims to reconstruct a high-resolution multi-spectral image (HRMS) from a low spatial resolution multi-spectral (MS) image and a high spatial resolution single band panchromatic (PAN) image acquired from the same satellite. To solve the spatial information and spectral information of the fused image that are not evenly preserved, this paper proposes a relativistic average generative multi-adversarial network (RaGMAN) for pansharpening. The RaGMAN consists of two parts. One builds a two-stream generator network to generate HRMS images by extracting features from PAN and MS images. The other is a dual discriminator to preserve the spectral and spatial information of the input when performing fusion. To make the generated image contain more spatial information, we propose multiple residual dense block in the generator. At the same time, in order to improve the overall quality of the fused images, two relativistic average discriminators are used in the network. Furthermore, a novel hybrid loss function is introduced to optimize training. Compared with seven state-of-the-art methods on GF-2, QB and WV-3 datasets, experimental results show that the proposed RaGMAN method can produce excellent pansharpening performance in terms of spatial and spectral fidelity.