Optical Remote Sensing Images Research Articles

Transformer-Based Object Detection with Deep Feature Fusion Using Carafe Operator in Remote Sensing Image

Recently, broad applications can be found in optical remote sensing images (ORSI), such as in urban planning, military mapping, field survey, and so on. Target detection is one of its important applications. In the past few years, with the wings of deep learning, the target detection algorithm based on CNN has harvested a breakthrough. However, due to the different directions and target sizes in ORSI, it will lead to poor performance if the target detection algorithm for ordinary optical images is directly applied. Therefore, how to improve the performance of the object detection model on ORSI is thorny. Aiming at solving the above problems, premised on the one-stage target detection model-RetinaNet, this paper proposes a new network structure with more efficiency and accuracy, that is, a Transformer-Based Network with Deep Feature Fusion Using Carafe Operator (TRCNet). Firstly, a PVT2 structure based on the transformer is adopted in the backbone and we apply a multi-head attention mechanism to obtain global information in optical images with complex backgrounds. Meanwhile, the depth is increased to better extract features. Secondly, we introduce the carafe operator into the FPN structure of the neck to integrate the high-level semantics with the low-level ones more efficiently to further improve its target detection performance. Experiments on our well-known public NWPU-VHR-10 and RSOD show that mAP increases by 8.4% and 1.7% respectively. Comparison with other advanced networks also witnesses that our proposed network is effective and advanced.

Multiscale Feature Aggregation Network for Salient Object Detection in Optical Remote Sensing Images

Optical remote sensing images (ORSI) have various applications in different fields, and salient target detection (ORSI-SOD) of ORSI has become an important research topic in recent years. However, ORSI-SOD is a challenging problem due to the variable and complex backgrounds, large differences in levels, mixed backgrounds, and diverse topological shapes of ORSI. In this paper, we propose a novel model called multi-scale feature refinement aggregation network (MFANet), which consists of a multi-scale feature refinement module (MFR) and a context aggregation module (CFA). The MFR module extracts semantic information of ORSI across different dimensions in the multi-scale feature extraction stage. In the feature refinement stage, we use the proposed self-refinement module under the guidance of attention and reverse attention to progressively refine the prediction results. The CFA module introduces the hybrid attention module to gradually aggregate and extract salient regions from the context extraction module. To adapt to dense scenes, we develop a hybrid loss function that enables network optimization of multi-scale objectives in a self-adaptive manner. Our method outperforms most state-of-the-art salient object detection methods proposed in recent years in terms of accuracy.

Aggregating transformers and CNNs for salient object detection in optical remote sensing images

Salient object detection (SOD) in optical remote sensing images (RSIs) plays a significant role in many areas such as agriculture, environmental protection, and the military. However, since the difference in imaging mode and image complexity between RSIs and natural scene images (NSIs), it is difficult to achieve remarkable results by directly extending the saliency method targeting NSIs to RSIs. Besides, we note that the convolutional neural networks (CNNs) based U-Net cannot effectively acquire the global long-range dependency, and the Transformer doesn’t adequately characterize the spatial local details of each patch. Therefore, to conduct salient object detection in RSIs, we propose a novel two-branch architecture based network for Aggregating the Transformers and CNNs, namely ATC-Net, where the local spatial details and the global semantic information are fused into the final high-quality saliency map. Specifically, our saliency model adopts an encoder-decoder architecture including two parallel encoder branches and a decoder. Firstly, the two parallel encoder branches extract global and local features by using Transformer and CNNs, respectively. Then, the decoder employs a series of feature-enhanced fusion (FF) modules to aggregate multi-level global and local features by interactive guidance and enhance the fused feature via attention mechanism. Finally, the decoder deploys the read out (RO) module to fuse the aggregated feature of FF module and the low-level CNN feature, steering the feature to focus more on spatial local details. Extensive experiments are performed on two public optical RSIs datasets, and the results show that our saliency model consistently outperforms 30 state-of-the-art methods.

Alignment Integration Network for Salient Object Detection and Its Application for Optical Remote Sensing Images.

Salient object detection has made substantial progress due to the exploitation of multi-level convolutional features. The key point is how to combine these convolutional features effectively and efficiently. Due to the step by step down-sampling operations in almost all CNNs, multi-level features usually have different scales. Methods based on fully convolutional networks directly apply bilinear up-sampling to low-resolution deep features and then combine them with high-resolution shallow features by addition or concatenation, which neglects the compatibility of features, resulting in misalignment problems. In this paper, to solve the problem, we propose an alignment integration network (ALNet), which aligns adjacent level features progressively to generate powerful combinations. To capture long-range dependencies for high-level integrated features as well as maintain high computational efficiency, a strip attention module (SAM) is introduced into the alignment integration procedures. Benefiting from SAM, multi-level semantics can be selectively propagated to predict precise salient objects. Furthermore, although integrating multi-level convolutional features can alleviate the blur boundary problem to a certain extent, it is still unsatisfactory for the restoration of a real object boundary. Therefore, we design a simple but effective boundary enhancement module (BEM) to guide the network focus on boundaries and other error-prone parts. Based on BEM, an attention weighted loss is proposed to boost the network to generate sharper object boundaries. Experimental results on five benchmark datasets demonstrate that the proposed method can achieve state-of-the-art performance on salient object detection. Moreover, we extend the experiments on the remote sensing datasets, and the results further prove the universality and scalability of ALNet.

Enhancing Remote Sensing Image Super-Resolution with Efficient Hybrid Conditional Diffusion Model

Recently, optical remote-sensing images have been widely applied in fields such as environmental monitoring and land cover classification. However, due to limitations in imaging equipment and other factors, low-resolution images that are unfavorable for image analysis are often obtained. Although existing image super-resolution algorithms can enhance image resolution, these algorithms are not specifically designed for the characteristics of remote-sensing images and cannot effectively recover high-resolution images. Therefore, this paper proposes a novel remote-sensing image super-resolution algorithm based on an efficient hybrid conditional diffusion model (EHC-DMSR). The algorithm applies the theory of diffusion models to remote-sensing image super-resolution. Firstly, the comprehensive features of low-resolution images are extracted through a transformer network and CNN to serve as conditions for guiding image generation. Furthermore, to constrain the diffusion model and generate more high-frequency information, a Fourier high-frequency spatial constraint is proposed to emphasize high-frequency spatial loss and optimize the reverse diffusion direction. To address the time-consuming issue of the diffusion model during the reverse diffusion process, a feature-distillation-based method is proposed to reduce the computational load of U-Net, thereby shortening the inference time without affecting the super-resolution performance. Extensive experiments on multiple test datasets demonstrated that our proposed algorithm not only achieves excellent results in quantitative evaluation metrics but also generates sharper super-resolved images with rich detailed information.

RAIH-Det: An End-to-End Rotated Aircraft and Aircraft Head Detector Based on ConvNeXt and Cyclical Focal Loss in Optical Remote Sensing Images

In airport ground-traffic surveillance systems, the detection of an aircraft and its head (AIH) is an important task in aircraft trajectory judgment. However, accurately detecting an AIH in high-resolution optical remote sensing images is a challenging task due to the difficulty in effectively modeling the features of aircraft objects, such as changes in appearance, large-scale differences, complex compositions, and cluttered background. In this paper, we propose an end-to-end rotated aircraft and aircraft head detector (RAIH-Det) based on ConvNeXt-T (Tiny) and cyclical local loss. Firstly, a new U-shaped network based on ConvNeXt-T with the same performance as the Local Vision Transformer (e.g., Swin Transformer) is presented to assess the relationships among aircraft in the spatial domain. Then, in order to enhance the sharing of more mutual information, the extended BBAVectors with six vectors captures the oriented bounding box (OBB) of the aircraft in any direction, which can assist in head keypoint detection by exploiting the relationship between the local and overall structural information of aircraft. Simultaneously, variant cyclical focal loss is adopted to regress the heatmap location of keypoints on the aircraft head to focus on more reliable samples. Furthermore, to perform a study on AIH detection and simplify aircraft head detection, the OBBs of the “plane” category in the DOTA-v1.5 dataset and the corresponding head keypoints annotated by our volunteers were integrated into a new dataset called DOTA-Plane. Compared with other state-of-the-art rotated object and keypoint detectors, RAIH-Det, as evaluated on DOTA-Plane, offered superior performance.

An Optical Remote Sensing Image Matching Method Based on the Simple and Stable Feature Database

Satellite remote sensing has entered the era of big data due to the increase in the number of remote sensing satellites and imaging modes. This presents significant challenges for the processing of remote sensing systems and will result in extremely high real-time data processing requirements. The effective and reliable geometric positioning of remote sensing images is the foundation of remote sensing applications. In this paper, we propose an optical remote sensing image matching method based on a simple stable feature database. This method entails building the stable feature database, extracting local invariant features that are comparatively stable from remote sensing images using an iterative matching strategy, and storing useful information about the features. Without reference images, the feature database-based matching approach potentially saves storage space for reference data while increasing image processing speed. To evaluate the performance of the feature database matching method, we train the feature database with various local invariant feature algorithms on different time phases of Gaofen-2 (GF-2) images. Furthermore, we carried out matching comparison experiments with various satellite images to confirm the viability and stability of the feature database-based matching method. In comparison with direct matching using the classical feature algorithm, the feature database-based matching method in this paper can essentially improve the correct rate of feature point matching by more than 30% and reduce the matching time by more than 40%. This method improves the accuracy and timeliness of image matching, potentially solves the problem of large storage space occupied by the reference data, and has great potential for fast matching of optical remote sensing images.

Corn Land Extraction Based on Integrating Optical and SAR Remote Sensing Images

Corn is an important food crop worldwide, and its yield is directly related to Chinese food security. Accurate remote sensing extraction of corn can realize the rational application of land resources, which is of great significance to the sustainable development of modern agriculture. In the field of large-scale crop remote sensing classification, single-period optical remote sensing images often cannot achieve high-precision classification. To improve classification accuracy, multiple time series image combinations have gradually been adopted. However, due to the influence of cloudy and rainy weather, it is often difficult to obtain complete time series optical images. Synthetic aperture radar (SAR) data are imaged by microwaves, which have strong penetrating power and are not affected by clouds. A critical way to solve this problem is to use SAR images to compensate for the lack of optical images and obtain a complete time series image in the corn-growing season. However, SAR images have limited wavelengths and cannot provide important wavelengths, such as visible light bands and near-infrared information. To solve this problem, this study took Zhaodong City, a vital corn-planting base in China, as the research area; took GF-6/GF-3 and Sentinel-1/Sentinel-2 as remote sensing data sources; designed12 classification scenarios; analyzed the best classification period and the best time series combination of corn classification; studied the influence of SAR images on the classification results of time series images; and compared the classification differences between GF-6/GF-3 and Sentinel-1/Sentinel-2. The results show that the classification accuracy of time series combinations is much higher than that of single-period images. The polarization characteristics of SAR images can improve the classification accuracy with time series images. The classification accuracy of GF series images from China is obviously higher than that of Sentinel series images. The research performed in this paper can provide a reference for agricultural classification by using remote sensing data.

Applicability of UAV-based optical imagery and classification algorithms for detecting pine wilt disease at different infection stages

ABSTRACT As a quarantine disease with a rapid spread tendency in the context of climate change, accurate detection and location of pine wilt disease (PWD) at different infection stages is critical for maintaining forest health and being highly productivity. In recent years, unmanned aerial vehicle (UAV)-based optical remote-sensing images have provided new instruments for timely and accurate PWD monitoring. Numerous corresponding analysis algorithms have been proposed for UAV-based image classification, but their applicability of detecting different PWD infection stages has not yet been evaluated under a uniform conditions and criteria. This research aims to systematically assess the performance of multi-source images for detecting different PWD infection stages, analyze effective classification algorithms, and further analyze the validity of thermal images for early detection of PWD. In this study, PWD infection was divided into four stages: healthy, chlorosis, red and gray, and UAV-based hyperspectral (HSI), multispectral (MSI), and MSI with a thermal band (MSI&TIR) datasets were used as the data sources. Spectral analysis, support vector machine (SVM), random forest (RF), two- and three-dimensional convolutional network (2D- and 3D-CNN) algorithms were applied to these datasets to compare their classification abilities. The results were as follows: (I) The classification accuracy of the healthy, red, and gray stages using the MSI dataset was close to that obtained when using the MSI&TIR dataset with the same algorithms, whereas the HSI dataset displayed no obvious advantages. (II) The RF and 3D-CNN algorithms were the most accurate for all datasets (RF: overall accuracy = 94.26%, 3D-CNN: overall accuracy = 93.31%), while the spectral analysis method is also valid for the MSI&TIR dataset. (III) Thermal band displayed significant potential in detection of the chlorosis stage, and the MSI&TIR dataset displayed the best performance for detection of all infection stages. Considering this, we suggest that the MSI&TIR dataset can essentially satisfy PWD identification requirements at various stages, and the RF algorithm provides the best choice, especially in actual forest investigations. In addition, the performance of thermal imaging in the early monitoring of PWD is worthy of further investigation. These findings are expected to provide insight into future research and actual surveys regarding the selection of both remote sensing datasets and data analysis algorithms for detection requirements of different PWD infection stages to detect the disease earlier and prevent losses.

Attention-Enhanced Region Proposal Networks for Multi-Scale Landslide and Mudslide Detection from Optical Remote Sensing Images

Detecting areas where a landslide or a mudslide might occur is critical for emergency response, disaster recovery, and disaster cost estimation. Previous works have reported that a variety of convolutional neural networks (CNNs) significantly outperform traditional approaches for landslide/mudslide detection. These approaches always consider features from the local window and neighborhood information. The CNNs mainly focus on the features derived at a local scale, which might be inefficient for recognizing complex landslide and mudslide scenes. To effectively identify landslide and mudslide risks at a local and global scale, this paper integrates attentions into the architecture of state-of-the-art CNNs—including Faster RCNN—to develop an attention-enhanced region proposal network for multi-scale landslide/mudslide detection. In detail, we employed the attentions to process the region proposals generated by a region proposal network and then combined the results obtained from the attentions and region proposal network to identify whether the object included in a region proposal was a landslide/mudslide. Based on our developed dataset and the Bijie dataset, the experimental results prove that: (1) although the state-of-the-art CNNs for object detection can precisely detect landslides and mudslides, they are inadequate in dealing with similarity to non-landslide/non-mudslide regions; and (2) the proposed method, which integrates global features from attention layers into local features derived from CNNs, outperforms the unmodified CNNs in detecting non-landslides and non-mudslides. Our findings prove that the representations at the local and global scale might be significant for precise landslide and mudslide detection.

Method of Building Detection in Optical Remote Sensing Images Based on SegFormer.

An appropriate detection network is required to extract building information in remote sensing images and to relieve the issue of poor detection effects resulting from the deficiency of detailed features. Firstly, we embed a transposed convolution sampling module fusing multiple normalization activation layers in the decoder based on the SegFormer network. This step alleviates the issue of missing feature semantics by adding holes and fillings, cascading multiple normalizations and activation layers to hold back over-fitting regularization expression and guarantee steady feature parameter classification. Secondly, the atrous spatial pyramid pooling decoding module is fused to explore multi-scale contextual information and to overcome issues such as the loss of detailed information on local buildings and the lack of long-distance information. Ablation experiments and comparison experiments are performed on the remote sensing image AISD, MBD, and WHU dataset. The robustness and validity of the improved mechanism are demonstrated by control groups of ablation experiments. In comparative experiments with the HRnet, PSPNet, U-Net, DeepLabv3+ networks, and the original detection algorithm, the mIoU of the AISD, the MBD, and the WHU dataset is enhanced by 17.68%, 30.44%, and 15.26%, respectively. The results of the experiments show that the method of this paper is superior to comparative methods such as U-Net. Furthermore, it is better for integrity detection of building edges and reduces the number of missing and false detections.

Edge-Guided Recurrent Positioning Network for Salient Object Detection in Optical Remote Sensing Images.

Optical remote sensing images (RSIs) have been widely used in many applications, and one of the interesting issues about optical RSIs is the salient object detection (SOD). However, due to diverse object types, various object scales, numerous object orientations, and cluttered backgrounds in optical RSIs, the performance of the existing SOD models often degrade largely. Meanwhile, cutting-edge SOD models targeting optical RSIs typically focus on suppressing cluttered backgrounds, while they neglect the importance of edge information which is crucial for obtaining precise saliency maps. To address this dilemma, this article proposes an edge-guided recurrent positioning network (ERPNet) to pop-out salient objects in optical RSIs, where the key point lies in the edge-aware position attention unit (EPAU). First, the encoder is used to give salient objects a good representation, that is, multilevel deep features, which are then delivered into two parallel decoders, including: 1) an edge extraction part and 2) a feature fusion part. The edge extraction module and the encoder form a U-shape architecture, which not only provides accurate salient edge clues but also ensures the integrality of edge information by extra deploying the intraconnection. That is to say, edge features can be generated and reinforced by incorporating object features from the encoder. Meanwhile, each decoding step of the feature fusion module provides the position attention about salient objects, where position cues are sharpened by the effective edge information and are used to recurrently calibrate the misaligned decoding process. After that, we can obtain the final saliency map by fusing all position attention cues. Extensive experiments are conducted on two public optical RSIs datasets, and the results show that the proposed ERPNet can accurately and completely pop-out salient objects, which consistently outperforms the state-of-the-art SOD models.

Adaptive Spatial Tokenization Transformer for Salient Object Detection in Optical Remote Sensing Images

Convolutional neural networks (CNN)-based salient object detection (SOD) models have achieved promising performance in optical remote sensing images (ORSIs) in recent years. However, the restriction concerning the local sliding window operation of CNN has caused many existing CNN-based ORSI SOD models to still struggle with learning long-range relationships. To this end, a novel transformer framework is proposed for ORSI SOD which is inspired by the powerful global dependency relationships of transformer networks. This is the first attempt to explore global and local details using transformer architecture for SOD in ORSIs. Concretely, we design an adaptive spatial tokenization transformer encoder to extract global-local features, which can accurately sparsify tokens for each input image and achieve competitive performance in ORSI SOD tasks. Then, a specific dense token aggregation decoder (DTAD) is proposed to generate saliency results, including three cascade decoders to integrate the global-local tokens and contextual dependencies. Extensive experiments indicate that the proposed model greatly surpasses twenty state-of-the-art (SOTA) SOD approaches on two standard ORSI SOD datasets under seven evaluation metrics. We also report a comparison results to demonstrate the generalization capacity on the latest challenging ORSI datasets. In addition, we validate the contributions of different modules through a series of ablation analyses, especially the proposed adaptive spatial tokenization module (ASTM), which can halve the computational budget.

Threatening Patch Attacks on Object Detection in Optical Remote Sensing Images

Advanced Patch Attacks (PAs) on object detection in natural images have pointed out the great safety vulnerability in methods based on deep neural networks. However, little attention has been paid to this topic in Optical Remote Sensing Images (O-RSIs). To this end, we focus on this research, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., PAs on object detection in O-RSIs, and propose a more Threatening PA without the scarification of the visual quality, dubbed TPA. Specifically, to address the problem of inconsistency between local and global landscapes in existing patch selection schemes, we propose leveraging the First-Order Difference (FOD) of the objective function before and after masking to select the sub-patches to be attacked. Further, considering the problem of gradient inundation when applying existing coordinate-based loss to PAs directly, we design an IoU-based objective function specific for PAs, dubbed Bounding box Drifting Loss (BDL), which pushes the detected bounding boxes far from the initial ones until there are no intersections between them. Finally, on two widely used benchmarks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., DIOR and DOTA, comprehensive evaluations of our TPA with four typical detectors (Faster R-CNN, FCOS, RetinaNet, and YOLO-v4) witness its remarkable effectiveness. To the best of our knowledge, this is the first attempt to study the PAs on object detection in O-RSIs, and we hope this work can get our readers interested in studying this topic.

Adjacent Context Coordination Network for Salient Object Detection in Optical Remote Sensing Images.

Salient object detection (SOD) in optical remote sensing images (RSIs), or RSI-SOD, is an emerging topic in understanding optical RSIs. However, due to the difference between optical RSIs and natural scene images (NSIs), directly applying NSI-SOD methods to optical RSIs fails to achieve satisfactory results. In this article, we propose a novel adjacent context coordination network (ACCoNet) to explore the coordination of adjacent features in an encoder-decoder architecture for RSI-SOD. Specifically, ACCoNet consists of three parts: 1) an encoder; 2) adjacent context coordination modules (ACCoMs); and 3) a decoder. As the key component of ACCoNet, ACCoM activates the salient regions of output features of the encoder and transmits them to the decoder. ACCoM contains a local branch and two adjacent branches to coordinate the multilevel features simultaneously. The local branch highlights the salient regions in an adaptive way, while the adjacent branches introduce global information of adjacent levels to enhance salient regions. In addition, to extend the capabilities of the classic decoder block (i.e., several cascaded convolutional layers), we extend it with two bifurcations and propose a bifurcation-aggregation block (BAB) to capture the contextual information in the decoder. Extensive experiments on two benchmark datasets demonstrate that the proposed ACCoNet outperforms 22 state-of-the-art methods under nine evaluation metrics, and runs up to 81 fps on a single NVIDIA Titan X GPU. The code and results of our method are available at https://github.com/MathLee/ACCoNet.

Lightweight Salient Object Detection in Optical Remote-Sensing Images via Semantic Matching and Edge Alignment

Recently, relying on convolutional neural networks (CNNs), many methods for salient object detection in optical remote-sensing images (ORSI-SOD) are proposed. However, most methods ignore the number of parameters and computational cost brought by CNNs, and only a few pay attention to portability and mobility. To facilitate practical applications, in this article, we propose a novel lightweight network for ORSI-SOD based on semantic matching and edge alignment, termed SeaNet. Specifically, SeaNet includes a lightweight MobileNet-V2 for feature extraction, a dynamic semantic matching module (DSMM) for high-level features, an edge self-alignment module (ESAM) for low-level features, and a portable decoder for inference. First, the high-level features are compressed into semantic kernels. Then, semantic kernels are used to activate salient object locations in two groups of high-level features through dynamic convolution operations in DSMM. Meanwhile, in ESAM, cross-scale edge information extracted from two groups of low-level features is self-aligned through <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{2}$ </tex-math></inline-formula> loss and used for detail enhancement. Finally, starting from the highest level features, the decoder infers salient objects based on the accurate locations and fine details contained in the outputs of the two modules. Extensive experiments on two public datasets demonstrate that our lightweight SeaNet not only outperforms most state-of-the-art lightweight methods, but also yields comparable accuracy with state-of-the-art conventional methods, while having only 2.76 M parameters and running with 1.7 G floating point operations (FLOPs) for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$288 \times 288$ </tex-math></inline-formula> inputs. Our code and results are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MathLee/SeaNet</uri> .

Boundary-Aware Network With Two-Stage Partial Decoders for Salient Object Detection in Remote Sensing Images

Salient object detection is a binary pixel-wise classification to distinguish objects in an image, and also have attracted many research interests in the optical Remote Sensing Images (RSIs). The existing state-of-the-art method exploits the full encoder-decoder architecture to predict salient objects in the optical RSIs, suffering from the problem of unsmooth edges and incomplete structures. To address these problems, in this paper, we propose a Boundary-Aware Network (BANet) with two-stage partial decoders sharing the same encoders for salient object detection in RSIs. Specifically, a Boundary-Aware Partial Decoder (BAD) is introduced at the first stage to focus on learning clear edges of salient objects. To solve the pixel-imbalance problem between boundary and background, an edge-aware loss is proposed to guide learning the BAD network. The resulting features are then employed in turn to enhance high-level features. Afterwards, the Structure-Aware Partial Decoder (SAD) is further introduced at the second stage to improve the structure integrity of salient objects. To alleviate the problem of incomplete structures, the structural similarity loss is further proposed to supervise learning the SAD network. In a consequence, our proposed BANet can predict salient objects with clear edges and complete structure, while reducing model parameters due to the discardment of low-level features. Besides, training a deep neural network requires a large amount of images, and the current benchmark datasets for optical remote sensing images are not large enough. Therefore, we also create a large-scale challenging dataset for salient object detection in RSIs. Extensive experiments demonstrate that our proposed BANet outperforms previous RSI SOD models on all existing benchmark datasets and our new presented dataset available at https://github.com/QingpingZheng/RSISOD.

An Effective Multimodel Fusion Method for SAR and Optical Remote Sensing Images

An Effective Multimodel Fusion Method for SAR and Optical Remote Sensing Images

Salient Object Detection in Optical Remote Sensing Images Driven by Transformer.

Existing methods for Salient Object Detection in Optical Remote Sensing Images (ORSI-SOD) mainly adopt Convolutional Neural Networks (CNNs) as the backbone, such as VGG and ResNet. Since CNNs can only extract features within certain receptive fields, most ORSI-SOD methods generally follow the local-to-contextual paradigm. In this paper, we propose a novel Global Extraction Local Exploration Network (GeleNet) for ORSI-SOD following the global-to-local paradigm. Specifically, GeleNet first adopts a transformer backbone to generate four-level feature embeddings with global long-range dependencies. Then, GeleNet employs a Direction-aware Shuffle Weighted Spatial Attention Module (D-SWSAM) and its simplified version (SWSAM) to enhance local interactions, and a Knowledge Transfer Module (KTM) to further enhance cross-level contextual interactions. D-SWSAM comprehensively perceives the orientation information in the lowest-level features through directional convolutions to adapt to various orientations of salient objects in ORSIs, and effectively enhances the details of salient objects with an improved attention mechanism. SWSAM discards the direction-aware part of D-SWSAM to focus on localizing salient objects in the highest-level features. KTM models the contextual correlation knowledge of two middle-level features of different scales based on the self-attention mechanism, and transfers the knowledge to the raw features to generate more discriminative features. Finally, a saliency predictor is used to generate the saliency map based on the outputs of the above three modules. Extensive experiments on three public datasets demonstrate that the proposed GeleNet outperforms relevant state-of-the-art methods. The code and results of our method are available at https://github.com/MathLee/GeleNet.

EFM-Net: An Essential Feature Mining Network for Target Fine-Grained Classification in Optical Remote Sensing Images

Target fine-grained classification has been the research hotspot in remote sensing image interpretation, which has received general attention in application fields. One challenge of the fine-grained classification task is to learn the most discriminative feature using the deep convolutional neural network (DCNN). At present, many works of fine-grained image classification obtain target features by optimizing the feature extraction and enhancement, which are not accurate enough in remote sensing images. In this paper, we propose an essential feature mining network (EFM-net for short) based on DCNN to address this issue. Its major motivation is to obtain the essential feature which is fine enough to distinguish between similar instances. The proposed pipeline includes the Miner for purifying the essential feature and the Refiner for data augmentation. These two modules can work in a mutually reinforcing way, and extract the essential feature of targets. We evaluate EFM-Net on two public fine-grained classification datasets in remote sensing, FGSC-23 and FGSCR-42, and our Aircraft-16. The results show that the proposed method consistently outperforms existing alternatives. We have released our source code in Github https://github.com/JACYI/EFM-Net-Pytorch.git.