Optical Remote Sensing Images Research Articles

Deformable Faster R-CNN with Aggregating Multi-Layer Features for Partially Occluded Object Detection in Optical Remote Sensing Images

The region-based convolutional networks have shown their remarkable ability for object detection in optical remote sensing images. However, the standard CNNs are inherently limited to model geometric transformations due to the fixed geometric structures in its building modules. To address this, we introduce a new module named deformable convolution that is integrated into the prevailing Faster R-CNN. By adding 2D offsets to the regular sampling grid in the standard convolution, it learns the augmenting spatial sampling locations in the modules from target tasks without additional supervision. In our work, a deformable Faster R-CNN is constructed by substituting the standard convolution layer with a deformable convolution layer in the last network stage. Besides, top-down and skip connections are adopted to produce a single high-level feature map of a fine resolution, on which the predictions are to be made. To make the model robust to occlusion, a simple yet effective data augmentation technique is proposed for training the convolutional neural network. Experimental results show that our deformable Faster R-CNN improves the mean average precision by a large margin on the SORSI and HRRS dataset.

Optical remote-sensing data based research on detecting soil salinity at different depth in an arid-area oasis, Xinjiang, China

This study discusses measures for improving the precision of optical remote-sensing detection of soil salinity and the possibility of soil salinity detection at different depths of 0–10 cm, 10–30 cm and 30–50 cm using optical remote-sensing data, and analyzes the mechanism by which deep-layer soil salinity influences the soil spectrum. The findings show that there is a high and significant correlation between soil-spectral reflectance and soil salinity, and that the correlation between soil-spectral reflectance and soil salinity decreases gradually from the blue band to the shortwave infrared band of ETM + images. The partial least squares regression model is used to estimate soil salinity in the 0–10-cm surface-layer, confirming that the selected soil-salinity-detecting bands of Band 1 and Band 4, the established difference soil salinity index, the derivative of the normalized differential vegetation index, and the deep-layer soil moisture can improve the precision of remote-sensing detection of surface-layer soil salinity. The precise estimation of the 0–10-cm surface-layer soil salinity with variables features an R2 = 0.752, an RMSE = 26.84 g/Kg, and a p = 0.000. There is a strong mediating effect between deep-layer soil salinity, 0–10-cm surface-layer soil salinity, and soil spectral reflectance in the study area; namely, deep-layer soil salinity influences soil spectral reflectance by influencing surface-layer soil salinity. There is a significant and very strong power function relation between 0 and 10-cm surface-layer soil salinity and deep-layer soil salinity. Based on this relationship, this study estimates deep-layer soil salinity using optical remote-sensing images.

Spectral analysis of wetlands using multi-source optical satellite imagery

The separability of wetland types using different spectral bands is an important subject, which has not yet been well studied in most countries. This is particularly of interest in Canada because it contains approximately one-fourth of the total global wetlands. In this study, the spectral separability of five wetland classes, namely Bog, Fen, Marsh, Swamp, and Shallow Water, was investigated in Newfoundland and Labrador (NL), Canada, using field data and multi-source optical Remote Sensing (RS) images. The objective was to select the most useful spectral bands for wetland studies from four commonly used optical satellites: RapidEye, Sentinel 2A, ASTER, and Landsat 8. However, because the ultimate objective was the classification of wetlands in the province, the separability of wetland classes was also evaluated using several other features, including various spectral indices, as well as textural and ratio features to obtain a high level of classification accuracy. For this purpose, two separability measures were used: The T-statistics, calculated from the parametric t-test method, and the U-statistics, derived from the non-parametric Mann-Whitney U-test. The results indicated that the Near Infrared (NIR) band was the best followed by the Red Edge (RE) band for the discrimination of wetland class pairs. The red band was also the third most useful band for separation of wetland classes, especially for the delineation of the Bog class from the other types. Although the Shortwave Infrared (SWIR) and green bands demonstrated poor separability, they were comparatively more informative than the Thermal Infrared (TIR) and blue bands. This study also demonstrated that ratio features and some spectral indices had high potential to differentiate the wetland species. Finally, wetlands in five study areas in NL were classified by inserting the best spectral bands and features into an object-based Random Forest (RF) classifier. By doing so, the mean Overall Accuracy (OA) and Kappa coefficient in the study areas were 86% and 0.82, respectively.

Scene Recognition From Optical Remote Sensing Images Using Mid-Level Deep Feature Mining

We solve the problem of scene recognition from very high-resolution optical satellite remote sensing (RS) images by exploring the notion of mid-level feature mining. The existing mid-level feature extraction techniques are based on applying feature encodings over a set of discriminatively selected localized feature descriptors from the images. Such techniques inherently suffer from two shortcomings: 1) the local descriptors are not enough discriminative, since they are mostly based on scale invariant feature transform (SIFT) like ad hoc features and 2) the definition of a robust ranking function to select discriminative local features is nontrivial. As a remedy, we propose a pattern mining-based approach for an efficient discovery of mid-level visual elements, which considers convolutional neural network features of the category-independent region proposals extracted from the images as the local descriptors. While the region proposals depict better semantic information than the SIFT like features, the proposed pattern mining strategy can efficiently highlight the correlations between such local descriptors and the class labels. Experimental results suggest that the proposed technique outperforms a number of existing mid-level feature descriptors for the standard optical RS data sets.

Arbitrary-Oriented Ship Detection Framework in Optical Remote-Sensing Images

Ship detection is a challenging problem in complex optical remote-sensing images. In this letter, an effective ship detection framework in remote-sensing images based on the convolutional neural network is proposed. The framework is designed to predict bounding box of ship with orientation angle information. Note that the angle information which is added to bounding box regression makes bounding box accurately fit into the ship region. In order to make the model adaptable to the detection of multiscale ship targets, especially small-sized ships, we design the network with feature maps from the layers of different depths. The whole detection pipeline is a single network and achieves real-time detection for a $704 \times 704$ image with the use of Titan X GPU acceleration. Through experiments, we validate the effectiveness, robustness, and accuracy of the proposed ship detection framework in complex remote-sensing scenes.

Progressive Dual-Domain Filter for Enhancing and Denoising Optical Remote-Sensing Images

Enhancement and denoising have always been a pair of conflicting problems in image processing of computer vision. Inspired by an earlier dual-domain filter (DDF), this letter proposes a progressive DDF to simultaneously enhance and denoise low-quality optical remote-sensing images. The main procedure of the proposed enhancement filter has two parts. First, a bilateral filter is exploited as a guide filter to obtain high-contrast images, which are enhanced by a histogram modification method. Then, low-contrast useful structures are restored by a short-time Fourier transform and are enhanced using an adaptive correction parameter. Both the quantitative and qualitative results of experiments on synthetic and real-world low-quality remote-sensing images demonstrate that the proposed method performs well on contrast enhancement, structure preservation, and noise reduction. Moreover, its satisfactory computation time resulting from its simple implementation makes it suitable for extensive application.

Registration for Optical Multimodal Remote Sensing Images Based on FAST Detection, Window Selection, and Histogram Specification

In recent years, digital frame cameras have been increasingly used for remote sensing applications. However, it is always a challenge to align or register images captured with different cameras or different imaging sensor units. In this research, a novel registration method was proposed. Coarse registration was first applied to approximately align the sensed and reference images. Window selection was then used to reduce the search space and a histogram specification was applied to optimize the grayscale similarity between the images. After comparisons with other commonly-used detectors, the fast corner detector, FAST (Features from Accelerated Segment Test), was selected to extract the feature points. The matching point pairs were then detected between the images, the outliers were eliminated, and geometric transformation was performed. The appropriate window size was searched and set to one-tenth of the image width. The images that were acquired by a two-camera system, a camera with five imaging sensors, and a camera with replaceable filters mounted on a manned aircraft, an unmanned aerial vehicle, and a ground-based platform, respectively, were used to evaluate the performance of the proposed method. The image analysis results showed that, through the appropriate window selection and histogram specification, the number of correctly matched point pairs had increased by 11.30 times, and that the correct matching rate had increased by 36%, compared with the results based on FAST alone. The root mean square error (RMSE) in the x and y directions was generally within 0.5 pixels. In comparison with the binary robust invariant scalable keypoints (BRISK), curvature scale space (CSS), Harris, speed up robust features (SURF), and commercial software ERDAS and ENVI, this method resulted in larger numbers of correct matching pairs and smaller, more consistent RMSE. Furthermore, it was not necessary to choose any tie control points manually before registration. The results from this study indicate that the proposed method can be effective for registering optical multimodal remote sensing images that have been captured with different imaging sensors.

Ship Detection in Optical Remote Sensing Images Based on Saliency and a Rotation-Invariant Descriptor

Major challenges for automatic ship detection in optical remote sensing (ORS) images include cloud, wave, island, wake clutters, and even the high variability of targets. This paper presents a practical ship detection scheme to resolve these existing issues. The scheme contains two main coarse-to-fine stages: prescreening and discrimination. In the prescreening stage, we construct a novel visual saliency detection method according to the difference of statistical characteristics between highly non-uniform regions which allude to regions of interest (ROIs) and homogeneous backgrounds. It can serve as a guide for locating candidate regions. In this way, not only can the targets be precisely detected, but false alarms are also significantly reduced. In the discrimination stage, to get a better representation of the target, both shape and texture features characterizing the ship target are extracted and concatenated as a feature vector for subsequent classification. Moreover, the combined feature is invariant to the rotation. Finally, a trainable Gaussian support vector machine (SVM) classifier is performed to validate real ships out of ship candidates. We demonstrate the superior performance of the proposed hierarchical detection method with detailed comparisons to existing efforts.

Denoising Very High Resolution Optical Remote Sensing Images: Application and Optimization of Nonlocal Bayes method

Very high resolution optical remote sensing images (RSI) are often corrupted by noise. Among popular denoising methods in the state of the art, nonlocal Bayes (NLB) has led to successful results on real datasets, with high quality and reasonable computation time. However, its computation time remains prohibitive with respect to requirements of operational RSI pipelines, such as Pleiades one. In this paper, we tackle such an issue and introduce several optimizations aiming to significantly reduce the computation time required by NLB while keeping the best denoising quality (i.e., preserving edges, textures, and homogeneous areas). More precisely, our improvements consist of reducing multiple estimations of a same pixel with a masking technique and modifying the spatial extent of the similar patch search area (i.e., one of the main parts of nonlocal algorithms, such as NLB). We report several experiments and discuss optimal settings for these parameters, allowing a gain in computation time of 50% (resp. 15%) with optimized masking strategy (resp. spatial extent of the search area). When both contributions are combined, we achieve the same denoising quality as standard NLB while doubling the computation efficiency, the latter being increased fivefold if we accept a very small (lower than 0.1%) loss in quality.

Satellite remote-sensing monitoring of a railway construction project

ABSTRACTCurrent satellite remote-sensing technologies enable the timely and detailed monitoring of human activities on the Earth’s surface. Sub-metre spatial resolution satellite images can picture the ongoing works of railway and highway construction. The synoptic view of satellite images is useful to assist the monitoring and management of such construction projects. In this article, we present an integrated remote-sensing change detection framework applied to monitoring the Light Rail Transport construction in Kuala Lumpur, Malaysia, using sub-metre optical remote-sensing images, Pleiades. Focusing on the known local area surrounding a construction site, the recognition process starts from the completion stage, checking conditions based on pre-defined rules, applying the recognition process corresponding to that stage, or moving to the immediately preceding stage if appropriate. The process ends when a new stage is identified and recorded into a spatio-temporal database or it reaches the previously detected stage as retrieved from the database. The experiment proved the effectiveness of the proposed framework, with an overall accuracy of about 80% and a low false detection rate. The proposed framework can be extended to monitor other similar scheduled works. Future studies will integrate multi-sensor satellite images including synthetic aperture radar images to expand further the practicality of the framework.

Optimal Illumination and Color Consistency for Optical Remote-Sensing Image Mosaicking

Illumination and color consistency are very important for optical remote-sensing image mosaicking. In this letter, we propose a simple but effective technique that simultaneously performs image illumination and color correction for multiview images. In this framework, we first present an uneven illumination removal algorithm based on bright channel prior, which guarantees the illumination consistency inside a single image. We then adapt a pairwise color-correction method to coarsely align the color tone between source and reference images. In this stage, we give a new single-image quality metric which combines brightness deviation, color cast, and entropy together for automatic reference-image selection. Finally, we perform a least-squares adjustment (LSA) procedure to obtain optimal illumination and color consistency among multiview images. In detail, we first perform a pairwise image matching by using SIFT algorithm; once sparse local patch correspondences obtained, the illumination and color relationship between images can be established based on a global gamma correction model; the illumination and color errors can then be minimized by LSA. Extensive experiments on both challenging synthetic and real optical remote-sensing image data sets show that it significantly outperforms the compared state-of-the-art approaches. All the source code and data sets used in this letter are made public. 1 1 https://sites.google.com/site/jiayuanli2016whu/home

A deep-learning model for the amplitude inversion of internal waves based on optical remote-sensing images

ABSTRACTThe amplitude of internal waves is an important parameter for the remote-sensing detection. However, there is no analytic method developed for the amplitude inversion of internal waves based on optical remote-sensing images. In this article, the deep-learning model is introduced to inverse the amplitude of internal waves based on a large number of optical remote-sensing images. The peak-to-peak distance and 15 types of texture characteristic parameters of images are computed, and the relationship between the amplitude of internal waves and the characteristic parameters of remote-sensing images are also investigated. In addition, based on the correlation difference between these parameters and the amplitude, three types of deep-learning model are established by selecting different parameters as input variables. Results show that the inverted amplitude of internal waves from our models shows good agreement with the in-situ data of internal waves, and the average errors between them are about 6.7%, 4.9%, and 3.6%, respectively, which indicates that the deep-learning model is effective for amplitude inversion of internal waves based on optical remote-sensing images.

Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images

Ship detection in optical remote sensing imagery has drawn much attention in recent years, especially with regards to the more challenging inshore ship detection. However, recent work on this subject relies heavily on hand-crafted features that require carefully tuned parameters and on complicated procedures. In this letter, we utilize a fully convolutional network (FCN) to tackle the problem of inshore ship detection and design a ship detection framework that possesses a more simplified procedure and a more robust performance. When tackling the ship detection problem with FCN, there are two major difficulties: 1) the long and thin shape of the ships and their arbitrary direction makes the objects extremely anisotropic and hard to be captured by network features and 2) ships can be closely docked side by side, which makes separating them difficult. Therefore, we implement a task partitioning model in the network, where layers at different depths are assigned different tasks. The deep layer in the network provides detection functionality and the shallow layer supplements with accurate localization. This approach mitigates the tradeoff of FCN between localization accuracy and feature representative ability, which is of importance in the detection of closely docked ships. The experiments demonstrate that this framework, with the advantages of FCN and the task partitioning model, provides robust and reliable inshore ship detection in complex contexts.

Ship Detection in Optical Remote Sensing Images Based on Wavelet Transform and Multi-Level False Alarm Identification

Ship detection by Unmanned Airborne Vehicles (UAVs) and satellites plays an important role in a spectrum of related military and civil applications. To improve the detection efficiency, accuracy, and speed, a novel ship detection method from coarse to fine is presented. Ship targets are viewed as uncommon regions in the sea background caused by the differences in colors, textures, shapes, or other factors. Inspired by this fact, a global saliency model is constructed based on high-frequency coefficients of the multi-scale and multi-direction wavelet decomposition, which can characterize different feature information from edge to texture of the input image. To further reduce the false alarms, a new and effective multi-level discrimination method is designed based on the improved entropy and pixel distribution, which is robust against the interferences introduced by islands, coastlines, clouds, and shadows. The experimental results on optical remote sensing images validate that the presented saliency model outperforms the comparative models in terms of the area under the receiver operating characteristic curves core and the accuracy in the images with different sizes. After the target identification, the locations and the number of the ships in various sizes and colors can be detected accurately and fast with high robustness.

Airport Detection Based on a Multiscale Fusion Feature for Optical Remote Sensing Images

Automatically detecting airports from remote sensing images has attracted significant attention due to its importance in both military and civilian fields. However, the diversity of illumination intensities and contextual information makes this task difficult. Moreover, auxiliary features both within and surrounding the regions of interest are usually ignored. To address these problems, we propose a novel method that uses a multiscale fusion feature to represent the complementary information of each region proposal, which is extracted by constructing a GoogleNet with a light feature module model that has an additional light fully connected layer. Then, the fusion feature is input to a support vector machine whose performance is enhanced using a hard negative mining method. Finally, a simplified localization method is applied to tackle the problem of box redundancy and to optimize the locations of airports. An experiment demonstrates that the fusion feature outperforms other features on airport detection tasks from remote sensing images containing complicated contextual information.

Multi-layer sparse coding model-based ship detection for optical remote-sensing images

ABSTRACTShip detection plays an important role in remote-sensing image processing. In this article, we propose a multi-layer sparse coding model-based ship detection (MSCMSD) method, integrating bottom-up and top-down mechanisms, for ship detection with high-resolution remote-sensing images. The multi-layer sparse coding model was designed to reveal the way how information is processed by human visual system. It is adopted in MSCMSD to detect candidate regions containing ships before any further processing. To detect ships from candidate regions, an omnidirectional solution is also proposed for deformable parts model-based ship detection. As demonstrated in the experiments, MSCMSD can detect ships from optical remote-sensing images with a higher accuracy than other state-of-the-art algorithms.

Affine Invariant Description and Large-Margin Dimensionality Reduction for Target Detection in Optical Remote Sensing Images

A novel target detection method based on affine invariant interest point detection, feature encoding, and large-margin dimensionality reduction (LDR) is proposed for optical remote sensing images. First, four types of interest point detectors are introduced, and their performance in extracting low-level affine invariant descriptors using affine shape estimation is compared. Such a description can deal with significant affine transformations, including viewpoints. Second, feature encoding, which extends bag-of-words (BOW) by encoding high-order statistics, is selected to generate mid-level representation. Finally, LDR based on the large-margin constraint and stochastic subgradient is introduced to make the high-dimensional mid-level representation applicable for target detection. The experiments on aircraft and vehicle detections illustrate the effectiveness of the affine invariant description and LDR (compared with principal component analysis) in improving the detection performance. The experiments also demonstrate the effectiveness of the proposed method compared with popular approaches including Gabor, HOG, LBP, BOW, and R-CNN.

Multi-feature sea–land segmentation based on pixel-wise learning for optical remote-sensing imagery

ABSTRACTRobust sea–land segmentation in optical remote-sensing images is challenging because of the complex sea–land environment and scene diversity. Here, we propose a novel multi-feature sea–land segmentation method via pixel-wise learning for optical remote-sensing images. Multiple features such as greyscale, local statistical information, edge, texture, and structure are first extracted from each pixel in training images and then used to learn a multi-feature sea–land classifier, which transforms the segmentation issue into pixel-wise binary classification problem. In our approach, a new multi-feature sea–land segmentation algorithm is put forward based on the approximation of Newton method. Experiments on Google-Earth, Venezuelan Remote Sensing Satellite-1 (VRSS-1) and Gaofen-1 images demonstrate that the proposed approach yields more robust and accurate sea–land segmentation results.

Maritime Semantic Labeling of Optical Remote Sensing Images with Multi-Scale Fully Convolutional Network

In current remote sensing literature, the problems of sea-land segmentation and ship detection (including in-dock ships) are investigated separately despite the high correlation between them. This inhibits joint optimization and makes the implementation of the methods highly complicated. In this paper, we propose a novel fully convolutional network to accomplish the two tasks simultaneously, in a semantic labeling fashion, i.e., to label every pixel of the image into 3 classes, sea, land and ships. A multi-scale structure for the network is proposed to address the huge scale gap between different classes of targets, i.e., sea/land and ships. Conventional multi-scale structure utilizes shortcuts to connect low level, fine scale feature maps to high level ones to increase the network’s ability to produce finer results. In contrast, our proposed multi-scale structure focuses on increasing the receptive field of the network while maintaining the ability towards fine scale details. The multi-scale convolution network accommodates the huge scale difference between sea-land and ships and provides comprehensive features, and is able to accomplish the tasks in an end-to-end manner that is easy for implementation and feasible for joint optimization. In the network, the input forks into fine-scale and coarse-scale paths, which share the same convolution layers to minimize network parameter increase, and then are joined together to produce the final result. The experiments show that the network tackles the semantic labeling problem with improved performance.

A Novel Method of Aircraft Detection Based on High-Resolution Panchromatic Optical Remote Sensing Images.

In target detection of optical remote sensing images, two main obstacles for aircraft target detection are how to extract the candidates in complex gray-scale-multi background and how to confirm the targets in case the target shapes are deformed, irregular or asymmetric, such as that caused by natural conditions (low signal-to-noise ratio, illumination condition or swaying photographing) and occlusion by surrounding objects (boarding bridge, equipment). To solve these issues, an improved active contours algorithm, namely region-scalable fitting energy based threshold (TRSF), and a corner-convex hull based segmentation algorithm (CCHS) are proposed in this paper. Firstly, the maximal variance between-cluster algorithm (Otsu’s algorithm) and region-scalable fitting energy (RSF) algorithm are combined to solve the difficulty of targets extraction in complex and gray-scale-multi backgrounds. Secondly, based on inherent shapes and prominent corners, aircrafts are divided into five fragments by utilizing convex hulls and Harris corner points. Furthermore, a series of new structure features, which describe the proportion of targets part in the fragment to the whole fragment and the proportion of fragment to the whole hull, are identified to judge whether the targets are true or not. Experimental results show that TRSF algorithm could improve extraction accuracy in complex background, and that it is faster than some traditional active contours algorithms. The CCHS is effective to suppress the detection difficulties caused by the irregular shape.