Bitemporal Multispectral Images Research Articles

A Change Detection Method Based on Multi-Scale Adaptive Convolution Kernel Network and Multimodal Conditional Random Field for Multi-Temporal Multispectral Images

Multispectral image change detection is an important application in the field of remote sensing. Multispectral images usually contain many complex scenes, such as ground objects with diverse scales and proportions, so the change detection task expects the feature extractor is superior in adaptive multi-scale feature learning. To address the above-mentioned problems, a multispectral image change detection method based on multi-scale adaptive kernel network and multimodal conditional random field (MSAK-Net-MCRF) is proposed. The multi-scale adaptive kernel network (MSAK-Net) extends the encoding path of the U-Net, and designs a weight-sharing bilateral encoding path, which simultaneously extracts independent features of bi-temporal multispectral images without introducing additional parameters. A selective convolution kernel block (SCKB) that can adaptively assign weights is designed and embedded in the encoding path of MSAK-Net to extract multi-scale features in images. MSAK-Net retains the skip connections in the U-Net, and embeds an upsampling module (UM) based on the attention mechanism in the decoding path, which can give the feature map a better expression of change information in both the channel dimension and the spatial dimension. Finally, the multimodal conditional random field (MCRF) is used to smooth the detection results of the MSAK-Net. Experimental results on two public multispectral datasets indicate the effectiveness and robustness of the proposed method when compared with other state-of-the-art methods.

Tensor-based keypoint detection and switching regression model for relative radiometric normalization of bitemporal multispectral images

ABSTRACT In some remote sensing applications, such as unsupervised change detection, bitemporal multispectral images must be first aligned/harmonized radiometrically. For doing so, Many Relative Radiometric Normalization (RRN) algorithms exist; however, most suffer from misregistration problems and can only operate on geo/co-registered image pairs, while unregistered multispectral pairs are required. To tackle this situation, keypoint-based RRN methods were introduced, which can radiometrically calibrate unregistered/registered image pairs using keypoint matching algorithms. However, they ignore the spatial and spectral characteristics of spectral bands of input images, resulting in potential RRN errors. They also employ a linear mapping function for RRN modelling, which can not handle non-linear radiometric distortions. To address these limitations, this paper proposes a robust algorithm for RRN of bitemporal multispectral images, using a new extension of SURF detector for multispectral images, namely the Weighted Spectral Structure Tensor SURF (WSST-SURF), and a flexible Switching Regression (SR) model. Taking advantage of the tensor theory, WSST-SURF efficiently preserves both spatial and spectral information distributed over all bands of multispectral images for keypoint detection, resulting in extracting reliable inliers (or keypoints) for RRN. An adaptive SR model is introduced based on the normalized mutual information, accurately approximating a linear/non-linear relationship between inliers in multispectral images. Six unregistered multispectral image pairs captured by inter/intra remote sensing sensors were employed to validate the efficacy of the proposed method. The results indicate that adopting spectral tensor-based SURF methods in the RRN process exhibits better local and global performance than using the original SURF. Furthermore, the proposed method outperforms the existing conventional RRN methods in terms of accuracy and visual quality, indicating its competence for RRN of bitemporal multispectral images with high illumination, viewpoint, and scale differences.

Unsupervised Change Detection Using Convolutional-Autoencoder Multiresolution Features

The use of deep learning (DL) methods for change detection (CD) is currently dominated by supervised models that require a large number of labeled samples. However, these samples are difficult to acquire in the multitemporal case. A possible alternative is leveraging methods that exploit transfer learning for CD by reusing DL models pretrained for other tasks. However, the performance of the transfer-learning-based models decreases as much as the target images differ from the ones used for training the model. To overcome this limit, we propose an unsupervised CD method that exploits multiresolution deep feature maps derived by a convolutional autoencoder (CAE). It automatically learns spatial features from the input during the training phase without requiring any labeled data. The proposed method processes the bitemporal images to obtain and compare multiresolution bitemporal feature maps. These feature maps are then analyzed by a feature-selection technique to select the most discriminant ones. Furthermore, an aggregated multiresolution difference image is computed and used for a detail-preserving multiscale CD. In the context of this CD approach, we propose two alternative strategies to retrieve multiscale reliability maps. We tested the proposed method on bitemporal multispectral images acquired by Landsat-5 and Landsat-8 representing burned areas and Sentinel-2 images representing deforested areas. Results confirm the effectiveness of the proposed CD technique.

Multispectral Image Change Detection Based on Single-Band Slow Feature Analysis

Due to differences in external imaging conditions, multispectral images taken at different periods are subject to radiation differences, which severely affect the detection accuracy. To solve this problem, a modified algorithm based on slow feature analysis is proposed for multispectral image change detection. First, single-band slow feature analysis is performed to process bitemporal multispectral images band by band. In this way, the differences between unchanged pixels in each pair of single-band images can be sufficiently suppressed to obtain multiple feature-difference images containing real change information. Then, the feature-difference images of each band are fused into a grayscale distance image using the Euclidean distance. After Gaussian filtering of the grayscale distance image, false detection points can be further reduced. Finally, the k-means clustering method is performed on the filtered grayscale distance image to obtain the binary change map. Experiments reveal that our proposed algorithm is less affected by radiation differences and has obvious advantages in time complexity and detection accuracy.

Random Sampling-Based Relative Radiometric Normalization

Relative radiometric normalization (RRN) is widely used for radiometric calibration of bitemporal multispectral images prior to any temporal analysis such as change detection. However, standard RRN methods are not robust against anomalous (or changed) pixels, which warp the calibration and decrease the spectral similarity of processed images. This letter proposes a novel random sample consensus-based RRN method, which only uses small pixel subsets to implement the linear mapping relationship for RRN, and does not require the calibration of its parameters. The experimental results show that the proposed method performs favorably against the widely used RRN methods in all metrics considered in this letter.

Detection of Urban Built-Up Area Change From Sentinel-2 Images Using Multiband Temporal Texture and One-Class Random Forest

Detection of urban land expansion is important for understanding the urbanization process and improving urban planning. Spatio-temporal contextual information derived from multitemporal high-resolution imagery is useful for highlighting urban land cover changes. This article proposes a new method for detecting urban built-up area change from multitemporal high spatial resolution imagery by combining spectral and spatio-temporal features. A multiband temporal texture measured using pseudo cross multivariate variogram (PCMV) is adopted to quantify the local spatio-temporal dependence between bitemporal multispectral images. The PCMV textures at multiple scales, bitemporal spectral features, and normalized difference vegetation indices are together input to an improved one-class random forest classifier for urban built-up area change mapping. The proposed method is evaluated in urban built-up area change detection using multitemporal Sentinel-2 images of Tianjin area acquired from 2015 to 2019. It is also compared with three feature combinations and an existing postclassification comparison method based on one-class support vector machine. Experimental results demonstrate that the proposed method outperformed the traditional ones, with increases of 2.15%-7.38%, 2.07%-5.45%, 1.93%-6.76%, and 5.98%-13.11% in overall accuracy. Moreover, the proposed method also achieves the best performance using the bitemporal Sentinel-2 images over the east of Beijing area. The proposed method is promising as a simple and reliable way to detect urban built-up area change with multitemporal Sentinel-2 imagery.

Comparison of Keypoint Detectors and Descriptors for Relative Radiometric Normalization of Bitemporal Remote Sensing Images

This article compares the performances of the most commonly used keypoint detectors and descriptors (SIFT, SURF, KAZE, AKAZE, ORB, and BRISK) in keypoint-based relative radiometric normalization (RRN) of unregistered bitemporal multispectral images. The keypoints matched between subject and reference images represent possible unchanged regions and form a radiometric control set (RCS). The initial RCS is further refined by removing the matched keypoints with a low cross-correlation. The final RCS is used to approximate a linear mapping between the corresponding bands of the subject and reference images. This procedure is validated on five datasets of unregistered multispectral image pairs acquired by inter/intra sensors in terms of RRN accuracy, visual quality, quality, and quantity of the samples in the RCS, and computational time. The experimental results show that keypoint-based RRN is robust against variations in spatial-resolution, illumination, and sensors. The blob detectors (SURF, SIFT, KAZE, and AKAZE) are more accurate on average than the corner detectors (ORB and BRISK) in RRN, with an expense of higher computational cost. The source code and samples of datasets used in this study are made available at https://github.com/ArminMoghimi/keypoint-based-RRN to support reproducible research in remote sensing.

Change detection of bitemporal multispectral images based on teacher student model

Ordinary multispectral (MS) image change detection (CD) techniques have not excavated the inherent information of MS images. Deep learning methods are a type of effective method that grabs the changed and unchanged information by deep feature extraction. However, a larger number of labeled samples are difficult to obtain for CD in MS images. An unsupervised MS image CD method is proposed based on a teacher student (TS) model. First, pseudotrain samples are produced by an expectation–maximization algorithm, and then the reliable training samples are selected based on the information of local windows. Second, the pseudotraining samples are used as input of the TS model, which is a self-ensembling model and includes a student model and a teacher model. Then the better teacher model of the TS model is updated by the exponential moving average of the student’s weight at each training step. Finally, the trained teacher model is utilized to generate the CD map. The experiments on four sets of bitemporal MS images demonstrate that the proposed CD method performs well in comprehensive indices compared with existing methods.

A Classified Adversarial Network for Multi-Spectral Remote Sensing Image Change Detection

Adversarial training has demonstrated advanced capabilities for generating image models. In this paper, we propose a deep neural network, named a classified adversarial network (CAN), for multi-spectral image change detection. This network is based on generative adversarial networks (GANs). The generator captures the distribution of the bitemporal multi-spectral image data and transforms it into change detection results, and these change detection results (as the fake data) are input into the discriminator to train the discriminator. The results obtained by pre-classification are also input into the discriminator as the real data. The adversarial training can facilitate the generator learning the transformation from a bitemporal image to a change map. When the generator is trained well, the generator has the ability to generate the final result. The bitemporal multi-spectral images are input into the generator, and then the final change detection results are obtained from the generator. The proposed method is completely unsupervised, and we only need to input the preprocessed data that were obtained from the pre-classification and training sample selection. Through adversarial training, the generator can better learn the relationship between the bitemporal multi-spectral image data and the corresponding labels. Finally, the well-trained generator can be applied to process the raw bitemporal multi-spectral images to obtain the final change map (CM). The effectiveness and robustness of the proposed method were verified by the experimental results on the real high-resolution multi-spectral image data sets.

Multispectral Change Detection With Bilinear Convolutional Neural Networks

Recently, deep learning has been demonstrated to be an effective tool to detect changes in bitemporal remote sensing images. However, most existing methods based on deep learning obtain the ultimate change map by analyzing the difference image (DI) or the stacked feature vectors of input images, which cannot sufficiently capture the relationship between the two input images to obtain the change information. In this letter, a new method named bilinear convolutional neural networks (BCNNs) is proposed to detect changes in bitemporal multispectral images. The model can be trained end to end with two symmetric convolutional neural networks (CNNs), which are capable of learning the feature representation from bitemporal images and utilizing the relations between the two input images by a linear outer product operation in an effective way. Specifically, two sets of patches obtained from two multispectral images of different times are first input into two CNNs to extract deep features, respectively. Then, the matrix outer product is applied on the output feature maps to obtain the combined bilinear features. Finally, the ultimate change detected result can be produced by applying the softmax classifier on the combined features. Experimental results on real multispectral data sets demonstrate the superiority of the proposed method over several well-known change-detection approaches.

Integrating spectral and textural attributes to measure magnitude in object-based change vector analysis

ABSTRACTWith the increasing accessibility of high or very high spatial resolution remote sensing images, more and more change detection works are conducted on the unit of image object, i.e. the object-based change detection (OBCD). Change vector analysis (CVA) is a promising tool for unsupervised OBCD because it can provide reasonable interpretation of the change and an insight into the type of change. However, various features and attributes of image object produce complex high dimensional feature space that poses new challenges to measure the overall change magnitude and direction in CVA. This paper presents a new approach to measure the overall change magnitude of an image object by integrating its spectral and textural attributes, self-adaptively. The new approach was compared with the standard CVA magnitude, Mahalanobis distance, and a recent Self-Adaptive Weight CVA (SAW-CVA) magnitude for unsupervised OBCD with the same OTSU auto-thresholding method. Two cases were investigated where Case I used bi-temporal WorldView multispectral images (4-band) and Case II employed bi-temporal three-band images obtained from Google Earth. Results in the two cases both demonstrated that the new approach outperforms the others for unsupervised OBCD. The percentage correct classification of the new approach, SAW-CVA, standard CVA, and Mahalanobis are 80.28%, 77.93%, 72.77%, and 61.50% in Case I, and 91.80%, 90.16%, 88.52%, and 90.16% in Case II. Further analysis indicated that the new approach gives more reasonable self-adaptive weights and it does not require the empirical self-adaptive index as compared with the recent SAW-CVA method.

A Generative Discriminatory Classified Network for Change Detection in Multispectral Imagery

Multispectral image change detection based on deep learning generally needs a large amount of training data. However, it is difficult and expensive to mark a large amount of labeled data. To deal with this problem, we propose a generative discriminatory classified network (GDCN) for multispectral image change detection, in which labeled data, unlabeled data, and new fake data generated by generative adversarial networks are used. The GDCN consists of a discriminatory classified network (DCN) and a generator. The DCN divides the input data into changed class, unchanged class, and extra class, i.e., fake class. The generator recovers the real data from input noises to provide additional training samples so as to boost the performance of the DCN. Finally, the bitemporal multispectral images are input to the DCN to get the final change map. Experimental results on the real multispectral imagery datasets demonstrate that the proposed GDCN trained by unlabeled data and a small amount of labeled data can achieve competitive performance compared with existing methods.

Change detection in Landsat images based on local neighbourhood information

In this study, a novel technique is proposed to detect the changes in bitemporal multispectral images. Utilisation of the local neighbourhood information in any image processing task may provide good noise immunity and reduces false alarms. Motivated by this, Otsu's thresholding of local information based approach is proposed in this work. It shows the effective performance in change detection of bitemporal Landsat images which suffer from different atmospheric and sunlight conditions. To get the local information around each pixel, both bitemporal images are partitioned into overlapping image blocks. Every block of the first image is concatenated with the corresponding block of the second image for each pixel position. Thus, the information of the concatenated block is considered as inter-block information. Further, Otsu's method is applied on the concatenated block for threshold calculation. Depending on the threshold, binary values are generated. Finally, binary values of both images for all bands are compared by XOR operation to detect it as the background i.e. unchanged pixel or foreground i.e. changed pixel. On the basis of majority class present in XOR output, binary change map is generated. Experiments conducted on Landsat images show that the proposed method provides better performance compared to reported techniques.

Bitemporal multispectral images unsupervised change detection based on undecimated wavelet transform and chi-squared transform

Bitemporal multispectral images unsupervised change detection based on undecimated wavelet transform and chi-squared transform

Automatic change detection using multiindex information map on high-resolution remote sensing images

The problem of high quality training samples and high-dimensional data is encountered in high-resolution image change detection. To address these problems, a novel automatic change detection algorithm in bitemporal multispectral images of the same scene using multiindex information is presented. The conspicuous advantages of the proposed algorithm are: (i) the complicated urban scenes are represented by a set of low-level semantic information index (e.g., textural and structural features), the information indices can directly indicate the primitive urban classes and (ii) change detection is carried out automatically using unsupervised approach. The multiindex information map contains vegetation, water and building extracted using enhanced vegetation index, normalized difference water index and developed efficient morphological building index respectively. The proposed algorithm is validated on the multitemporal Landsat ETM+ images over Coimbatore, Tamilnadu, India where auspicious results were achieved by the proposed method. Moreover, the traditional method based on the pixel based change detection has also been implemented for the purpose of comparison to further validate the advantages of the proposed model.

Superpixel-Based Difference Representation Learning for Change Detection in Multispectral Remote Sensing Images

With the rapid technological development of various satellite sensors, high-resolution remotely sensed imagery has been an important source of data for change detection in land cover transition. However, it is still a challenging problem to effectively exploit the available spectral information to highlight changes. In this paper, we present a novel change detection framework for high-resolution remote sensing images, which incorporates superpixel-based change feature extraction and hierarchical difference representation learning by neural networks. First, highly homogenous and compact image superpixels are generated using superpixel segmentation, which makes these image blocks adhere well to image boundaries. Second, the change features are extracted to represent the difference information using spectrum, texture, and spatial features between the corresponding superpixels. Third, motivated by the fact that deep neural network has the ability to learn from data sets that have few labeled data, we use it to learn the semantic difference between the changed and unchanged pixels. The labeled data can be selected from the bitemporal multispectral images via a preclassification map generated in advance. And then, a neural network is built to learn the difference and classify the uncertain samples into changed or unchanged ones. Finally, a robust and high-contrast change detection result can be obtained from the network. The experimental results on the real data sets demonstrate its effectiveness, feasibility, and superiority of the proposed technique.

Change detection of bitemporal multispectral images based on FCM and D-S theory

In this paper, we propose a change detection method of bitemporal multispectral images based on the D-S theory and fuzzy c-means (FCM) algorithm. Firstly, the uncertainty and certainty regions are determined by thresholding method applied to the magnitudes of difference image (MDI) and spectral angle information (SAI) of bitemporal images. Secondly, the FCM algorithm is applied to the MDI and SAI in the uncertainty region, respectively. Then, the basic probability assignment (BPA) functions of changed and unchanged classes are obtained by the fuzzy membership values from the FCM algorithm. In addition, the optimal value of fuzzy exponent of FCM is adaptively determined by conflict degree between the MDI and SAI in uncertainty region. Finally, the D-S theory is applied to obtain the new fuzzy partition matrix for uncertainty region and further the change map is obtained. Experiments on bitemporal Landsat TM images and bitemporal SPOT images validate that the proposed method is effective.

Prediction And Analysis Of Urban Heat Island Effect In Dangshan By Remote Sensing

Abstract Vegetation index (NDVI) was extracted from bi-temporal multispectral images based on the data obtained from Landsat ETM+on 14th, September, 2000, Landsat ETM+on 9th, September, 2004, Landsat ETM+on 15th, May, 2008 and Landsat-8 on 21st, May 2013 for Dangshan County in Anhui Province. Analysis and data extraction was carried out using the ENVI 5.0 software. Normalized values of thermal radiation brightness temperature and surface brightness temperature were inverted from the bi-temporal thermal infrared band images using the mono-window algorithm. Urban heat island effect in Dangshan County was divided into strong green island zone, green island zone, normal zone, heat island zone and strong heat island zone according based on arithmetic progression. Using regression analysis, quantitative relationship between NDVI and the heat island effect was determined. Results showed an acceleration in urbanization of Dangshan County over the years resulted in a gradual increase in the heat island effect from 2000 to 2013. In addition, area of the heat island and strong heat island increased was observed to increase rapidly, while the area of the green island and strong green island reduced by 46%. Also, using the Markov model, urban heat island effect in Dangshan County was predicted over the next 40 years. This model was feasible in predicting the urban heat island effect with small errors. Finally, it was determine that heat island effect was in negative correlation to the vegetation index (NDVI), and increasing green land appropriately would have a positive effect in alleviating the urban heat island effect

Improving Change Detection Results of IR-MAD by Eliminating Strong Changes

This letter examines the effect of the prior elimination of strong changes on the results of change detection in bitemporal multispectral images using the previously published iteratively reweighted multivariate alteration detection (IR-MAD) method. An initial change mask is calculated by identifying strong changes between two images. By using the mask and hence eliminating the strong changes from the analysis, the IR-MAD method is able to identify a better no-change background. This effect is demonstrated on a multitemporal Landsat Enhanced Thematic Mapper Plus data set from an agricultural region in Germany with substantial improvement in the results even for the scenes which have a large number of changes.