Optical Remote Sensing Imagery Research Articles

Ship-VNet: An Algorithm for Ship Velocity Analysis Based on Optical Remote Sensing Imagery Containing Kelvin Wakes

Extracting ship velocity vectors from optical remote sensing images is a very challenging task, and ship wakes are the only motion features of ships. However, because the sensor’s field of view is not sufficiently bright and the brightness is not uniform, the image contains noise, which makes it difficult to define and extract the wake of the ship. Velocity analysis of the extracted wake makes the whole process complicated and slow. Therefore, considering the above problems, this paper proposes Ship-VNet, an optical remote sensing image ship velocity analysis algorithm based on Kelvin wakes. In this model, the rotating target detection algorithm is used to detect the ship, and then, the classical relationship between the kinematic characteristics of the ship’s Kelvin wake and the velocity of the ship is studied and experimentally analyzed in the frequency domain. In addition, based on optical remote sensing images and corresponding real AIS data, a ship dataset with Kelvin wakes marked with heading velocity was constructed to verify the effectiveness of the proposed method. Compared with the ship velocity analysis method based on the frequency domain, which was also used in the previous research, the experiment demonstrates the superiority of the method in terms of analysis accuracy.

HODet: A New Detector for Arbitrary-Oriented Rectangular Object in Optical Remote Sensing Imagery

HODet: A New Detector for Arbitrary-Oriented Rectangular Object in Optical Remote Sensing Imagery

Parcel-Based Mapping Framework of Corn Harvest Progress by Combining Optical and Radar Remote Sensing Imagery

Parcel-Based Mapping Framework of Corn Harvest Progress by Combining Optical and Radar Remote Sensing Imagery

WaterFormer: A coupled transformer and CNN network for waterbody detection in optical remotely-sensed imagery

As one of the most significant components of the ecosystem, waterbody needs to be highly monitored at different spatial and temporal scales. Nevertheless, waterbody variations in shape, size, and reflectivity, complicated and varied types of land covers, and environmental scene diversity, present colossal challenges in achieving accurate waterbody detection (WD). In this paper, we propose a novel network coupled with the Transformer and convolutional neural network (CNN), termed WaterFormer, to automatically, efficiently, and accurately delineate waterbodies from optical high-resolution remotely sensed (HR-RS) images. This network mainly includes a dual-stream CNN, a cross-level Vision Transformer, a light-weight attention module, and a sub-pixel up-sampling module. First, the dual-stream network abstracts waterbody features at multi-views and different levels. Then, to exploit the long-range dependencies between low-level spatial information and high-order semantic features, the cross-level Vision Transformer is embedded into the dual-stream, aiming at improving WD accuracy. Afterwards, the light-weight attention module is adopted to provide semantically strong feature abstractions by enhancing discrimination neurons, and the sub-pixel up-sampling module is employed to further generate high-resolution and high-quality class-specific representations. Quantitative and qualitative evaluations demonstrated that the WaterFormer provided a promising means for detecting waterbody areas in satellite images under complex scene conditions. Moreover, comparative analyses with the state-of-the-art (SOTA) alternatives, e.g., MSFENet, MSAFNet, and BiSeNet, also verified the generalization and superiority of the WaterFormer in WD tasks. The assessment results exhibited that the WaterFormer gained an average accuracy of 97.24%, average precision of 94.59%, average recall of 91.95%, average F1-score of 93.24%, and average Kappa index of 0.9133, respectively. Additionally, we presented an open-access HR satellite imagery waterbody dataset, a mesoscale dataset with high-quality and high-precision waterbody annotation to facilitate future research in this field. The dataset has been released at https://github.com/NJdeuK/WD_Dataset.

Combining Gaussian Process Regression with Poisson Blending for Seamless Cloud Removal from Optical Remote Sensing Imagery for Cropland Monitoring

Constructing optical image time series for cropland monitoring requires a cloud removal method that accurately restores cloud regions and eliminates discontinuity around cloud boundaries. This paper describes a two-stage hybrid machine learning-based cloud removal method that combines Gaussian process regression (GPR)-based predictions with image blending for seamless optical image reconstruction. GPR is employed in the first stage to generate initial prediction results by quantifying temporal relationships between multi-temporal images. GPR predictive uncertainty is particularly combined with prediction values to utilize uncertainty-weighted predictions as the input for the next stage. In the second stage, Poisson blending is applied to eliminate discontinuity in GPR-based predictions. The benefits of this method are illustrated through cloud removal experiments using Sentinel-2 images with synthetic cloud masks over two cropland sites. The proposed method was able to maintain the structural features and quality of the underlying reflectance in cloud regions and outperformed two existing hybrid cloud removal methods for all spectral bands. Furthermore, it demonstrated the best performance in predicting several vegetation indices in cloud regions. These experimental results indicate the benefits of the proposed cloud removal method for reconstructing cloud-contaminated optical imagery.

Semantic Attention and Structured Model for Weakly Supervised Instance Segmentation in Optical and SAR Remote Sensing Imagery

Instance segmentation in remote sensing (RS) imagery aims to predict the locations of instances and represent them with pixel-level masks. Thanks to the more accurate pixel-level information for each instance, instance segmentation has enormous potential applications in resource planning, urban surveillance, and military reconnaissance. However, current RS imagery instance segmentation methods mostly follow the fully supervised paradigm, relying on expensive pixel-level labels. Moreover, remote sensing imagery suffers from cluttered backgrounds and significant variations in target scales, making segmentation challenging. To accommodate these limitations, we propose a semantic attention enhancement and structured model-guided multi-scale weakly supervised instance segmentation network (SASM-Net). Building upon the modeling of spatial relationships for weakly supervised instance segmentation, we further design the multi-scale feature extraction module (MSFE module), semantic attention enhancement module (SAE module), and structured model guidance module (SMG module) for SASM-Net to enable a balance between label production costs and visual processing. The MSFE module adopts a hierarchical approach similar to the residual structure to establish equivalent feature scales and to adapt to the significant scale variations of instances in RS imagery. The SAE module is a dual-stream structure with semantic information prediction and attention enhancement streams. It can enhance the network’s activation of instances in the images and reduce cluttered backgrounds’ interference. The SMG module can assist the SAE module in the training process to construct supervision with edge information, which can implicitly lead the model to a representation with structured inductive bias, reducing the impact of the low sensitivity of the model to edge information caused by the lack of fine-grained pixel-level labeling. Experimental results indicate that the proposed SASM-Net is adaptable to optical and synthetic aperture radar (SAR) RS imagery instance segmentation tasks. It accurately predicts instance masks without relying on pixel-level labels, surpassing the segmentation accuracy of all weakly supervised methods. It also shows competitiveness when compared to hybrid and fully supervised paradigms. This research provides a low-cost, high-quality solution for the instance segmentation task in optical and SAR RS imagery.

Dual Parallel Branch Fusion Network for Road Segmentation in High-Resolution Optical Remote Sensing Imagery

Road segmentation from high-resolution (HR) remote sensing images plays a core role in a wide range of applications. Due to the complex background of HR images, most of the current methods struggle to extract a road network correctly and completely. Furthermore, they suffer from either the loss of context information or high redundancy of details information. To alleviate these problems, we employ a dual branch dilated pyramid network (DPBFN), which enables dual-branch feature passing between two parallel paths when it is merged to a typical road extraction structure. A DPBFN consists of three parts: a residual multi-scaled dilated convolutional network branch, a transformer branch, and a fusion module. Constructing pyramid features through parallel multi-scale dilated convolution operations with multi-head attention block can enhance road features while suppressing redundant information. Both branches after fusing can solve shadow or vision occlusions and maintain the continuity of the road network, especially on a complex background. Experiments were carried out on three datasets of HR images to showcase the stable performance of the proposed method, and the results are compared with those of other methods. The OA in the three data sets of Massachusetts, Deep Globe, and GF-2 can reach more than 98.26%, 95.25%, and 95.66%, respectively, which has a significant improvement compared with the traditional CNN network. The results and explanation analysis via Grad-CAMs showcase the effective performance in accurately extracting road segments from a complex scene.

Automated object recognition in high-resolution optical remote sensing imagery.

This paper reports the background and results of the Automated Object Recognition in Optical Remote Sensing Imagery, which is one of the tracks in 2022 International Algorithm Case Competition, as well as summarize the challenges, champion solutions, and future directions.

Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation

Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation

Contour Modeling Arbitrary-Oriented Ship Detection From Very High-Resolution Optical Remote Sensing Imagery

Under the multiscale distribution, due to dramatic aspect ratio variance leading to prominent arbitrary-oriented character of ships in very high-resolution (VHR) optical remote sensing imagery, how to generate accurate oriented bounding box (OBB) becomes a hot research topic for arbitrary-oriented ship detection. Consequently, in this letter, a concise and effective one-stage anchor-free contour modeling detector called CMDet is proposed for accurate arbitrary-oriented ship detection. Different from currently existed methods via carefully decoupling several independently characteristic parameters for OBB modeling and regression, we resolve the OBB modeling by jointly regressing the contour information. Specifically, the contour information is expressed as a series of Fourier transform coefficients, which are generated by setting up the mapping relation of 1-D Fourier contour coefficients and spatial OBB contour. In addition, a new inherent geometry loss is designed to make detector better learn the geometry information in training phase. After that, the proposed CMDet only needs to predict the correct center point of ships and regress the corresponding entire 1-D Fourier contour coefficients to generate accurate OBB for ship detection. Finally, extensive experiments are carried out on two public OBB ship detection datasets (e.g., HRSC2016 and DIOR-ship), and comparison results demonstrate that the proposed CMDet can obtain the competitive result than the state-of-the-art (SOTA) detectors.

Posterior Instance Injection Detector for Arbitrary-Oriented Object Detection From Optical Remote-Sensing Imagery

Posterior Instance Injection Detector for Arbitrary-Oriented Object Detection From Optical Remote-Sensing Imagery

Full Semantic Constructed Network for Urban Use Classification From Very High-Resolution Optical Remote Sensing Imagery

Recently, semantic segmentation technology has been a research hotspot in optical remote sensing urban use classification. However, since coupled semantic relations in very high resolution and complex urban scenes, a more effective semantic description for pixelwise urban use interpretation has become a challenge. Then, aiming to set up a more effective semantic description, the effective receptive field (ERF) is analyzed in general convolutional neural networks, and there is an unreasonable ERF distribution in the stacked convolutional layers of the encoder. This would lead to a large amount of small ERFs and fewer not large enough ERFs to form a naive semantic description at decoder. Therefore, in this article, a novel full semantic constructed network (FSCNet) is proposed to improve the naive semantic description and set up an effective semantic description. First, to avoid noise from shallow feature layers, a residual refinement convolution is designed to optimize the full scale skip connections based on the U-shaped encoder-decoder. Second, an interscale fusion module is newly designed for multiscale feature fusion, which can generate three initial semantic modalities that are prepared for redefining the full semantic description. Third, a multiscale local context spatial attention module and boundary supervision are designed for an initial shallow semantic modality to capture the pure boundary information, and then pyramid spatial pooling is employed for an initial deep semantic modality to further enlarge the ERF and obtain a more abstract global information. Next, a self-calibration convolution combined with the atrous spatial pyramid pooling is designed to rectify and enrich an initial middle semantic modality, which can improve the naive semantic description and bridge the semantic gap between the redefined shallow and deep semantic modalities to advance the full semantic feature fusion. Finally, extensive experiments are carried out on three benchmarks (e.g., ISPRS Vaihinge, Potsdam and DLRSD), and comparative results show that the proposed FSCNet can get remarkable performance compared to state-of-the-art (SOTA) methods. Besides, the code is available at https://github.com/DorisCV/FSCNet.

RACDNet: Resolution- and Alignment-Aware Change Detection Network for Optical Remote Sensing Imagery

Change detection (CD) methods work on the basis of co-registered multi-temporal images with equivalent resolutions. Due to the limitation of sensor imaging conditions and revisit period, it is difficult to acquire the desired images, especially in emergency situations. In addition, accurate multi-temporal images co-registration is largely limited by vast object changes and matching algorithms. To this end, a resolution- and alignment-aware change detection network (RACDNet) is proposed for multi-resolution optical remote-sensing imagery CD. In the first stage, to generate high-quality bi-temporal images, a light-weighted super-resolution network is proposed by fully considering the construction difficulty of different regions, which facilitates to detailed information recovery. Adversarial loss and perceptual loss are further adopted to improve the visual quality. In the second stage, deformable convolution units are embedded in a novel Siamese–UNet architecture for bi-temporal deep features alignment; thus, robust difference features can be generated for change information extraction. We further use an atrous convolution module to enlarge the receptive field, and an attention module to bridge the semantic gap between the encoder and decoder. To verify the effectiveness of our RACDNet, a novel multi-resolution change detection dataset (MRCDD) is created by using Google Earth. The quantitative and qualitative experimental results demonstrate that our RACDNet is capable of enhancing the details of the reconstructed images significantly, and the performance of CD surpasses other state-of-the-art methods by a large margin.

A Robust LCSE-ResNet for Marine Man-made Target Classification Based on Optical Remote Sensing Imagery

It is of great need to accurately locate vessels and oil platforms because huge numbers of these marine man-made targets can cause oil spills or illegal invasion problems. Many different types of marine targets in the complex marine environment make it very difficult to deal with the target detection and classification process accurately. This paper proposes a robust Laplacian of Gaussian operator connected domain controller squeeze excitation residual network (LCSE-ResNet) for marine man-made target classification based on optical remote sensing imagery. Vessel and oil platform candidate regions in remote sensing images are extracted by the Laplacian of Gaussian (LoG) operator and the connected domain controller, which are input into the classification neural networks in the following. It is a whole End-to-End structure from original remote sensing images to the precise target information. That clouds, ripples, and the other man-made target disturbances can be excluded effectively means the better robust character in the actual application. Some shape and size features of vessels and oil platforms are considered in the LCSE-ResNet structure, which improves the explanation of the final results. Experimental results demonstrate that the proposed LCSE-ResNet can effectively detect marine man-made targets and effectively distinguish vessels and oil platforms.

A Multi-Feature Framework for Quantifying Information Content of Optical Remote Sensing Imagery

Quantifying the information content of remote sensing images is considered to be a fundamental task in quantitative remote sensing. Traditionally, the grayscale entropy designed by Shannon’s information theory cannot capture the spatial structure of images, which has prompted successive proposals of a series of neighborhood-based improvement schemes. However, grayscale or neighborhood-based spatial structure is only a basic feature of the image, and the spatial structure should be divided into the overall structure and the local structure and separately characterized. For this purpose, a multi-feature quantification framework for image information content is proposed. Firstly, the information content of optical remote sensing images is measured based on grayscale, contrast, neighborhood-based topology, and spatial distribution features instead of simple grayscale or spatial structure. Secondly, the entropy metrics of the different features are designed to quantify the uncertainty of images in terms of both pixel and spatial structure. Finally, a weighted model is used to calculate the comprehensive information content of the image. The experimental results confirm that the proposed method can effectively measure the multi-feature information content, including the overall and local spatial structure. Compared with state-of-the-art entropy models, our approach is the first study to systematically consider the multiple features of image information content based on Shannon entropy. It is comparable to existing models in terms of thermodynamic consistency. This work demonstrates the effectiveness of information theory methods in measuring the information content of optical remote sensing images.

Cloud Removal for Optical Remote Sensing Imagery Using Distortion Coding Network Combined with Compound Loss Functions

Optical remote sensing (RS) satellites perform imaging in the visible and infrared electromagnetic spectrum to collect data and analyze information on the optical characteristics of the objects of interest. However, optical RS is sensitive to illumination and atmospheric conditions, especially clouds, and multiple acquisitions are typically required to obtain an image of sufficient quality. To accurately reproduce surface information that has been contaminated by clouds, this work proposes a generative adversarial network (GAN)-based cloud removal framework using a distortion coding network combined with compound loss functions (DC-GAN-CL). A novel generator embedded with distortion coding and feature refinement mechanisms is applied to focus on cloudy regions and enhance the transmission of optical information. In addition, to achieve feature and pixel consistency, both coherent semantics and local adaptive reconstruction factors are considered in our loss functions. Extensive numerical evaluations on RICE1, RICE2, and Paris datasets are performed to validate the good performance achieved by the proposed DC-GAN-CL in both peak signal-to-noise ratio (PSNR) and visual perception. This system can thus restore images to obtain similar quality to cloud-free reference images, in a dynamic range of over 30 dB. The restoration effect on the coherence of image semantics produced by this technique is competitive compared with other methods.

Enhancing the Accuracy and Temporal Transferability of Irrigated Cropping Field Classification Using Optical Remote Sensing Imagery

Mapping irrigated areas using remotely sensed imagery has been widely applied to support agricultural water management; however, accuracy is often compromised by the in-field heterogeneity of and interannual variability in crop conditions. This paper addresses these key issues. Two classification methods were employed to map irrigated fields using normalized difference vegetation index (NDVI) values derived from Landsat 7 and Landsat 8: a dynamic thresholding method (method one) and a random forest method (method two). To improve the representativeness of field-level NDVI aggregates, which are the key inputs in our methods, a Gaussian mixture model (GMM)-based filtering approach was adopted to remove noncrop pixels (e.g., trees and bare soils) and mixed pixels along the field boundary. To improve the temporal transferability of method one we dynamically determined the threshold value to account for the impact of interannual weather variability based on the dynamic range of NDVI values. In method two an innovative training sample pool was designed for the random forest modeling to enable automatic calibration for each season, which contributes to consistent performance across years. The irrigated field mapping was applied to a major irrigation district in Australia from 2011 to 2018, for summer and winter cropping seasons separately. The results showed that using GMM-based filtering can markedly improve field-level data quality and avoid up to 1/3 of omission errors for irrigated fields. Method two showed superior performance, exhibiting consistent and good accuracy (kappa > 0.9) for both seasons. The classified maps in wet winter seasons should be used with caution, because rainfall alone can largely meet plant water requirements, leaving the contribution of irrigation to the surface spectral signature weak. The approaches introduced are transferable to other areas, can support multiyear irrigated area mapping with high accuracy, and significantly reduced model development effort.

LWCDNet: A Lightweight Fully Convolution Network for Change Detection in Optical Remote Sensing Imagery

Change detection (CD) is an important task in remote sensing image processing. The main research goal is to identify whether the target area has changed. Recently, the rise of deep learning has provided many novel methods for change detection, and some excellent models have been proposed. However, most of the available methods have a large number of parameters, and the traditional loss function does not perform well when tackling the problem of unbalanced sample number. In order to solve the above problems, we propose a lightweight fully convolution network for change detection, namely LWCDNet. Specifically, LWCDNet is a typical encoder-decoder structure, and it realizes more detailed information transmission and feature extraction by using the artificial padding convolution (APC) module as the convolution unit of the encoder. In addition, a convolutional block attention module (CBAM) is added between encoder and decoder to boost the model’s performance even more by emphasizing critical information. To deal with the sample imbalance in the change detection task, we propose Lov-wce loss. The experimental results on two actual remote sensing datasets show the effectiveness of LWCDNet.

Research on Building Target Detection Based on High-Resolution Optical Remote Sensing Imagery

High-resolution remote sensing image building target detection has wide application value in the fields of land planning, geographic monitoring, smart cities and other fields. However, due to the complex background of remote sensing imagery, some detailed features of building targets are less distinguishable from the background. When carrying out the detection task, it is prone to problems such as distortion and the missing of the building outline. To address this challenge, we developed a novel building target detection method. First, a building detection method based on rectangular approximation and region growth was proposed, and a saliency detection model based on the foreground compactness and local contrast of manifold ranking is used to obtain the saliency map of the building region. Then, the boundary prior saliency detection method based on the improved manifold ranking algorithm was proposed for the target area of buildings with low contrast with the background in remote sensing imagery. Finally, fusing the results of the rectangular approximation-based and saliency-based detection, the proposed fusion method improved the Recall and F1 value of building detection, indicating that this paper provides an effective and efficient high-resolution remote sensing image building target detection method.

High-resolution representations and multistage region-based network for ship detection and segmentation from optical remote sensing images

Ship detection and segmentation in optical remote sensing (ORS) images is a current concern actively addressed by the academic community owing to its vast applications. Most present methods only detect the location of the ship but do not segment it at the pixel level. Considering the complex background of ORS images, identifying ships from interferences, such as clouds, waves, and some land architectures similar to ships, proves to be difficult. To address this issue, we propose a high-resolution representation and multistage region-based network (HR-MSRN) for ship detection and segmentation from ORS images. HR-MSRN mainly consists of three parts: the high-resolution feature pyramid network (HRFPN), region proposal network (RPN), and multistage detection and segmentation network (MSDSN). First, HRFPN is built as a backbone network to extract and fuse multilevel image feature maps. Second, ship candidate boxes are generated by defining numerous anchors through RPN. Third, using the idea of a cascade mask R-CNN as the reference method, the MSDSN is proposed to obtain the ship localization and mask shape. We utilize the proposed framework to evaluate an Airbus-ship dataset, and the experiments indicate that (1) HRFPN provides better feature representation ability than the ResNet-FPN when maintaining the same detection framework, especially for small ships; (2) the direct flow between mask branches refines the mask information, and the semantic segmentation branch enhances context information, which indicates that MSDSN is effective and promotes further improvements in ship detection and segmentation from ORS images; (3) in comparison to other region-based methods, HR-MSRN obtains superior performance of ship detection and segmentation in the ORS imagery.