Heterogeneous Remote Sensing Research Articles

SUNet: Change Detection for Heterogeneous Remote Sensing Images from Satellite and UAV Using a Dual-Channel Fully Convolution Network

Change Detection in heterogeneous remote sensing images plays an increasingly essential role in many real-world applications, e.g., urban growth tracking, land use monitoring, disaster evaluation and damage assessment. The objective of change detection is to identify changes of geo-graphical entities or phenomena through two or more bitemporal images. Researchers have invested a lot in the homologous change detection and yielded fruitful results. However, change detection between heterogenous remote sensing images is still a great challenge, especially for change detection of heterogenous remote sensing images obtained from satellites and Unmanned Aerial Vehicles (UAV). The main challenges in satellite-UAV change detection tasks lie in the intensive difference of color for the same ground objects, various resolutions, the parallax effect and image distortion caused by different shooting angles and platform altitudes. To address these issues, we propose a novel method based on dual-channel fully convolution network. First, in order to alleviate the influence of differences between heterogeneous images, we employ two different channels to map heterogeneous remote sensing images from satellite and UAV, respectively, to a mutual high dimension latent space for the downstream change detection task. Second, we adopt Hough method to extract the edge of ground objects as auxiliary information to help the change detection model to pay more attention to shapes and contours, instead of colors. Then, IoU-WCE loss is designed to deal with the problem of imbalanced samples in change detection task. Finally, we conduct extensive experiments to verify the proposed method using a new Satellite-UAV heterogeneous image data set, named HTCD, which is annotated by us and has been open to public. The experimental results show that our method significantly outperforms the state-of-the-art change detection methods.

Distributed Fusion of Heterogeneous Remote Sensing and Social Media Data: A Review and New Developments

Despite the wide availability of remote sensing big data from numerous different Earth Observation (EO) instruments, the limitations in the spatial and temporal resolution of such EO sensors (as well as atmospheric opacity and other kinds of interferers) have led to many situations in which using only remote sensing data cannot fully meet the requirements of applications in which a (near) real-time response is needed. Examples of these applications include floods, earthquakes, and other kinds of natural disasters, such as typhoons. To address this issue, social media data have gradually been adopted to fill possible gaps in the analysis when remote sensing data are lacking or incomplete. In this case, the fusion of heterogeneous big data streams from multiple data sources introduces significant demands from a computational viewpoint. In order to meet these challenges, distributed computing is increasingly viewed as a feasible solution to parallelize the analysis of massive data coming from different sources (e.g., remote sensing and social media data). In this article, we provide an overview of available and new distributed strategies to address the computational challenges brought by massive heterogeneous data processing and fusion for real-time environmental monitoring and decision-making. The 2013 Boulder (Colorado) flood event is taken as a case study to evaluate several new distributed data fusion frameworks. Experimental results demonstrate that the proposed distributed frameworks are suitable in terms of response time and computational requirements for fusing large-volume heterogeneous data sources.

Iterative Robust Graph for Unsupervised Change Detection of Heterogeneous Remote Sensing Images.

This work presents a robust graph mapping approach for the unsupervised heterogeneous change detection problem in remote sensing imagery. To address the challenge that heterogeneous images cannot be directly compared due to different imaging mechanisms, we take advantage of the fact that the heterogeneous images share the same structure information for the same ground object, which is imaging modality-invariant. The proposed method first constructs a robust K -nearest neighbor graph to represent the structure of each image, and then compares the graphs within the same image domain by means of graph mapping to calculate the forward and backward difference images, which can avoid the confusion of heterogeneous data. Finally, it detects the changes through a Markovian co-segmentation model that can fuse the forward and backward difference images in the segmentation process, which can be solved by the co-graph cut. Once the changed areas are detected by the Markovian co-segmentation, they will be propagated back into the graph construction process to reduce the influence of changed neighbors. This iterative framework makes the graph more robust and thus improves the final detection performance. Experimental results on different data sets confirm the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/IRG-McS.

Partly Uncoupled Siamese Model for Change Detection from Heterogeneous Remote Sensing Imagery

This paper addresses the problematic of detecting changes in bitemporal heterogeneous remote sensing image pairs. In different disciplines, multimodality is the key solution for performance enhancement in a collaborative sensing context. Particularly, in remote sensing imagery there is still a research gap to fill with the multiplication of sensors, along with data sharing capabilities, and multitemporal data availability. This study is aiming to explore the multimodality in a multi-temporal set-up for a better understanding of the collaborative sensor wide information completion; we propose a pairwise learning approach consisting on a pseudo-Siamese network architecture based on two partly uncoupled parallel network streams. Each stream represents itself a Convolutional Neural Network (CNN) that encodes the input patches. The overall Change Detector (CD) model includes a fusion stage that concatenates the two encodings in a single multimodal feature representation which is then reduced to a lower dimension using fully connected layers and finally a loss function based on the binary cross entropy is used as a decision layer. The proposed pseudo-Siamese pairwise learning architecture allows to the CD model to capture the spatial and the temporal dependencies between multimodal input image pairs. The model processes the two multimodal input patches at one-time under different spatial resolutions. The evaluation performances on different real multimodal datasets reflecting a mixture of CD conditions with different spatial resolutions, confirm the effectiveness of the proposed CD architecture.

NASA NeMO-Net's Convolutional Neural Network: Mapping Marine Habitats with Spectrally Heterogeneous Remote Sensing Imagery

Recent advances in machine learning and computer vision have enabled increased automation in benthic habitat mapping through airborne and satellite remote sensing. Here, we applied deep learning and neural network architectures in NASA NeMO-Net, a novel neural multimodal observation and training network for global habitat mapping of shallow benthic tropical marine systems. These ecosystems, particularly coral reefs, are undergoing rapid changes as a result of increasing ocean temperatures, acidification, and pollution, among other stressors. Remote sensing from air and space has been the primary method in which changes are assessed within these important, often remote, ecosystems at a global scale. However, such global datasets often suffer from large spectral variances due to the time of observation, atmospheric effects, water column properties, and heterogeneous instruments and calibrations. To address these challenges, we developed an object-based fully convolutional network (FCN) to improve upon the spatial-spectral classification problem inherent in multimodal datasets. We showed that with training upon augmented data in conjunction with classical methods, such as K-nearest neighbors, we were able to achieve better overall classification and segmentation results. This suggests FCNs are able to effectively identify the relative applicable spectral and spatial spaces within an image, whereas pixel-based classical methods excel at classification within those identified spaces. Our spectrally invariant results, based on minimally preprocessed WorldView-2 and Planet satellite imagery, show a total accuracy of approximately 85% and 80%, respectively, over nine classes when trained and tested upon a chain of Fijian islands imaged under highly variable day-to-day spectral inputs.

A Reliable Mixed-Norm-Based Multiresolution Change Detector in Heterogeneous Remote Sensing Images

Analysis of heterogeneous remote sensing image is a challenging and complex problem due to the fact that the local statistics of the data to be processed can be radically different. In this article, we present a novel and reliable unsupervised change detection (CD) method to analyze heterogeneous remotely sensed image pairs. The proposed method is based on an imaging modality-invariant operator that detects at different scale levels the differences in terms of high-frequency pattern of each structural region existing in the two heterogeneous satellite images. First, this new detector is based upon a dual-norm formulation that makes our underlying CD estimation particularly robust in terms of a sensitivity/specificity tradeoff. Second, the detection process, embedded in a multiresolution framework, allows us to estimate a robust similarity or difference map that is then filtered out by a superpixel-based spatially adaptive filter to further increase its reliability against noise. Finally, changes are then identified from this similarity map by a simple binary clustering process that also takes into account the spatial contextual information around each pixel. Experimental results involving different types of heterogeneous remotely sensed image pairs confirm the robustness of the proposed approach.

异源遥感影像的真三维模型构建方法

异源遥感影像的真三维模型构建方法

Determination of Heterogeneous Remote Sensing of the Earth System Rational Structure on Stage of External Design

Determination of Heterogeneous Remote Sensing of the Earth System Rational Structure on Stage of External Design

Collaborative Regression on Aerosol Optical Thickness from Heterogeneous Remote Sensing Data

Previous studies of aerosol optical thickness (AOT) estimations were generally based on observations from a single satellite sensor. Due to the limited observations from one instrument, the observations yielded AOT estimations with a system bias. In this paper, we combined two heterogeneous data sources, Moderate Resolution Imaging Spectroradiometer (MODIS) and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), together and proposed collaborative regression models to achieve more accurate AOT estimations than a single sensor does. These two independent remote sensors in the A-train satellite constellation both provide global AOT retrievals and they scan the same location on the Earth surface within a two-minute interval. However, the two remote sensors have different design principles respectively and their heterogeneous observation data streams pose challenges for information fusion. In our study, we proposed two types of heterogeneous collaborative regression approaches. One type of collaborative regression approach fuses information in a feature level. The other type of collaborative approach combines information in a model level. In our study, in each level, we apply a linear regression collaboration model and a neural network collaboration model. The proposed approaches are evaluated based on global observation data from MODIS and CALIOP during April 2, 2009 and April 1, 2011. The encouraging experimental results show that the regression approach collaborating in a model level achieves significantly more accurate AOT estimations than the results from the collaborative regression approach in a feature level. It also obtains significantly superior results to the deterministic AOT retrievals from any single satellite sensor.

One-Class Oriented Feature Selection and Classification of Heterogeneous Remote Sensing Images

Information extraction from spatial big data faces challenges in data relevancy analysis and heterogeneous data modeling. When the interested targets are more than one, the relevant analysis is often compromised. In this paper, a one-class oriented approach for effective feature selection and classification of remote sensing images is proposed. Mutual information (MI) is used as the feature selection criterion to cope with a wide range of data types. Then a cluster space (CS) representation is applied to model multimodal data and classifies each target class in turn. Hyperspectral and LiDAR data sets were used in the experiments. The test results demonstrate the advantage in terms of classification accuracies by focusing on one class at a time as compared to considering all classes simultaneously in classification.

A Bayesian Nonparametric Model Coupled with a Markov Random Field for Change Detection in Heterogeneous Remote Sensing Images

In recent years, remote sensing of the Earth surface using images acquired from aircraft or satellites has gained a lot of attention. The acquisition technology has been evolving fast and, as a consequence, many different kinds of sensors (e.g., optical, radar, multispectral, and hyperspectral) are now available to capture different features of the observed scene. One of the main objectives of remote sensing is to monitor changes on the Earth surface. Change detection has been thoroughly studied in the case of images acquired by the same sensors (mainly optical or radar sensors). However, due to the diversity and complementarity of the images, change detection between images acquired with different kinds of sensors (sometimes referred to as heterogeneous sensors) is clearly an interesting problem. A statistical model and a change detection strategy were recently introduced in [J. Prendes, M. Chabert, F. Pascal, A. Giros, and J.-Y. Tourneret, Proceedings of the IEEE Inter- national Conference on Acoustics, Speech and Signal Processing, Florence, Italy, 2014; IEEE Trans. Image Process., 24 (2015), pp. 799-812] to deal with images captured by heterogeneous sensors. The main idea of the suggested strategy was to model the objects contained in an analysis window by mixtures of distributions. The manifold defined by these mixtures was then learned using training data belonging to unchanged areas. The changes were finally detected by thresholding an appropriate distance to the estimated manifold. This paper goes a step further by introducing a Bayesian nonparametric framework allowing us to deal with an unknown number of objects in analysis windows without specifying an upper bound for this number. A Markov random field is also introduced to account for the spatial correlation between neighboring pixels. The proposed change detector is validated using different sets of synthetic and real images (including pairs of optical images and pairs of optical and radar images) showing a significant improvement when compared to existing algorithms.

Change Detection in Heterogeneous Remote Sensing Images Based on Multidimensional Evidential Reasoning

We present a multidimensional evidential reasoning (MDER) approach to estimate change detection from the fusion of heterogeneous remote sensing images. MDER is based on a multidimensional (M-D) frame of discernment composed by the Cartesian product of the separate frames of discernment used for the classification of each image. Every element of the M-D frame is a basic joint state that allows to describe precisely the possible change occurrences between the heterogeneous images. Two kinds of rules of combination are proposed for working either with the free model, or with a constrained model depending on the integrity constraints one wants to take into account in the scenario under study. We show the potential interest of the MDER approach for detecting changes due to a flood in the Gloucester area in the U.K. from two real ERS and SPOT images.

Conditional Copulas for Change Detection in Heterogeneous Remote Sensing Images

A new preprocessing technique is presented in this paper to automatically highlight changes in multitemporal strongly heterogeneous remotely sensed images. The proposed technique is devoted to the case where the two acquisitions, before and after a given event, are significantly different, due, for instance, to different sensors, acquisition modalities, or climatic conditions. In a previous study, it was proven that the local statistics of the images acquired at the two dates could be used to extract a relevant change indicator. Nevertheless, this measure is valid when the two observations have been derived from similar acquisitions. When the acquisition modalities differ, local statistics tend to be too different from one image to the other one to be relevant in highlighting the ground evolution without mixing with the changes at ground. The technique proposed in this paper to overcome this limitation is based on the assumption that some dependence indeed exists between the two images in unchanged areas. This dependence is modeled by quantile regression applied according to the copula theory and used to perform an estimation of the local statistics that would have been observed if the acquisition conditions of the first image had been similar to the ones of the second image. The method yields an estimate of the local statistics of the first image through the point of view of the second one. Then, usual Kullback-Leibler-based comparisons of those statistics are applied to define a change measure, which may be analyzed (e.g., by thresholding) in order to detect changes. Experimental results are shown to validate the proposed method by using a pair of Synthetic Aperture Radar (SAR) images onboard European Remote Sensing (ERS) Satellite images and a pair of optical-SAR images (from the High Resolution Visible (HRV) sensor onboard Satellite Pour l'Observation de la Terre (SPOT) satellite and from ERS-SAR) acquired before and after a flood.