Time Series Of Remote Sensing Images Research Articles

Tomato Production Prediction Based on Deep Learning Algorithm-Cascade-PSPNET and Bayes

Accounting the problem of small sample size in tomato yield statistics, a dual-fusion prediction analysis model based on Bayes theory and deep learning algorithm Cascade-PSPNET is proposed. The tomato yield prediction is conducted based on remote sensing image time series analysis and multi-source information fusion theory with Bayesian theory and credibility weighting. First, the area of tomato planting is analyzed through semantic segmentation algorithms of remote sensing images during the tomato planting process. The yield is predicted by variables such as planting area fluctuation, disaster cycle fluctuation, etc. Through analyzing the reduced yield affected by disasters in different time periods of remote sensing images during planting process, the value chain of tomato industry is calculated by comprehensively analyzing price-value system, inflation coefficient, and unit area yield. At the same time, the annual patent application volume is used to predict the change of tomato yield year by year according to Bayesian theory, and the relationship between annual patent application volume and tomato yield year by year under different confidence levels is analyzed. The results show that it is feasible to use the Bayes method with semantic segmentation algorithms of remote sensing images to predict tomato yield. Next, the experiment of fusion prediction between two prediction models in the same target area will be carried out for verification.

Temporal Sequence Object-based CNN (TS-OCNN) for crop classification from fine resolution remote sensing image time-series

Accurate crop distribution mapping is required for crop yield prediction and field management. Due to rapid progress in remote sensing technology, fine spatial resolution (FSR) remotely sensed imagery now offers great opportunities for mapping crop types in great detail. However, within-class variance can hamper attempts to discriminate crop classes at fine resolutions. Multi-temporal FSR remotely sensed imagery provides a means of increasing crop classification from FSR imagery, although current methods do not exploit the available information fully. In this research, a novel Temporal Sequence Object-based Convolutional Neural Network (TS-OCNN) was proposed to classify agricultural crop type from FSR image time-series. An object-based CNN (OCNN) model was adopted in the TS-OCNN to classify images at the object level (i.e., segmented objects or crop parcels), thus, maintaining the precise boundary information of crop parcels. The combination of image time-series was first utilized as the input to the OCNN model to produce an ‘original’ or baseline classification. Then the single-date images were fed automatically into the deep learning model scene-by-scene in order of image acquisition date to increase successively the crop classification accuracy. By doing so, the joint information in the FSR multi-temporal observations and the unique individual information from the single-date images were exploited comprehensively for crop classification. The effectiveness of the proposed approach was investigated using multitemporal SAR and optical imagery, respectively, over two heterogeneous agricultural areas. The experimental results demonstrated that the newly proposed TS-OCNN approach consistently increased crop classification accuracy, and achieved the greatest accuracies (82.68% and 87.40%) in comparison with state-of-the-art benchmark methods, including the object-based CNN (OCNN) (81.63% and 85.88%), object-based image analysis (OBIA) (78.21% and 84.83%), and standard pixel-wise CNN (79.18% and 82.90%). The proposed approach is the first known attempt to explore simultaneously the joint information from image time-series with the unique information from single-date images for crop classification using a deep learning framework. The TS-OCNN, therefore, represents a new approach for agricultural landscape classification from multi-temporal FSR imagery. Besides, it is readily generalizable to other landscapes (e.g., forest landscapes), with a wide application prospect.

Concept-driven extraction of the Antarctic marginal sea ice zone from remote sensing image time series

Sea ice plays a significant role in global climate change. Marginal ice zone (MIZ) is defined as the transition zone between open ocean and pack ice where intensive air–ice–ocean–wave​ interactions between the ocean and the atmosphere occur. This definition of MIZ is rather vague, which affects its mapping. Previous data-driven methods extracted MIZ from single source data and conveniently used a single classification threshold. In this study, we propose to apply a concept-driven top down extraction method. We use a fuzzy inference system (FIS) to integrate multiple ice properties based upon the MIZ definition, and use membership functions to quantify the uncertainty of the threshold for several ice characteristics. This concept-driven method is applied to mapping the Antarctic MIZ between 2010 and 2021. Results show that a FIS successfully combines the ice concentration and ice thickness, while it allowed us to map the MIZ as objects with vague boundaries, thus well-presenting its indeterminate nature in space.

SwinSTFM: Remote Sensing Spatiotemporal Fusion Using Swin Transformer

Remote sensing images with high temporal and spatial resolutions have broad market demands and various application scenarios. This paper aims to generate high-quality remote sensing image time series for feature mining of the growth quality of traditional Chinese medicine. Spatiotemporal fusion is a flexible method that combines two types of satellite images with high temporal resolution or high spatial resolution to generate high-quality remote sensing images. In recent years, many spatiotemporal fusion algorithms have been proposed, and deep learning-based methods show extraordinary talents in this field. However, the current deep learning-based methods have three problems: 1) most algorithms do not support models with large-scale learnable parameters; 2) the model structure based on convolutional neural networks will bring noise to the image fusion process; 3) current deep learning-based methods ignore some excellent modules in traditional spatiotemporal fusion algorithms. For the above problems and challenges, this paper creatively proposes a new algorithm based on Swin Transformer and linear spectral mixing theory. The algorithm makes full use of the advantages of Swin Transformer in feature extraction, and integrates the unmixing theories into the model based on the self-attention mechanism, which greatly improves the quality of generated images. In the experimental part, the proposed algorithm achieves state-of-the-art results on three well-known public datasets, and has been proved effective and reasonable in ablation study.

A Multilayered Adaptive Recurrent Incremental Network Model for Heterogeneity-Aware Prediction of Derived Remote Sensing Image Time Series

<i>Catastrophic forgetting</i> of previously acquired knowledge is a major setback suffered by the neural networks (NNs) when these are trained on tasks in sequential fashion. The NN variants, including the deep network models, as commonly used in remote sensing data prediction are also not free from this limitation. The issue becomes more prominent when the prediction is performed over data collected from spatial zones with a large degree of subregional variations or heterogeneity. In order to tackle this problem, in this article, we propose a multilayered adaptive recurrent incremental network (MARINE) model, offering a spatial heterogeneity-aware self-adaptation scheme that resolves the catastrophic forgetting issue in an autonomous manner. Typically, the proposed MARINE is equipped with an intrinsic mechanism of clustering the spatial subregions as per their heterogeneity levels and auto-constructing the recurrent network layers in an ensemble fashion so that the knowledge acquired about one group of heterogeneity level does not overwrite that acquired about other groups. With respect to spatio-temporal prediction of normalized difference vegetation index (NDVI) time series, as derived from MODIS Terra satellite remote sensing imagery, we demonstrate that our proposed MARINE achieves competitive results when compared with the state-of-the-arts. More significantly, in the presence of higher degree of spatial heterogeneity, MARINE outperforms others by better <i>avoiding the catastrophic forgetting issue</i>.

A method of water change monitoring in remote image time series based on long short time memory

ABSTRACT This paper proposes convolutional neural network jointed with long short-time memory (CNN_LSTM) and Seq2Seq based on convolutional operation (Convolutional Seq2Seq), which the fully connected operation of Seq2Seq is replaced by convolution, and the attention mechanism of Seq2Seq is improved to monitor changes in water bodies. Convolutional Seq2Seq and CNN_LSTM can extract the temporal and spatial characteristics of remote sensing image time series. We also propose downsampling and resolution recovery (DDR) modules to reduce the computational resource consumption of the two models. Compared with the popular full convolutional network (FCN) −8s, DeepLab v2 with a baseline of ResNet101, and long short time memory (LSTM) methods, the water change monitoring results based on Convolutional Seq2Seq and CNN_LSTM have lower noise and higher accuracy. The CNN_LSTM method also allows fewer hidden layer features of LSTM with high-precision change monitoring results.

Earth Observation Data Cubes for Brazil: Requirements, Methodology and Products

Recently, remote sensing image time series analysis has being widely used to investigate the dynamics of environments over time. Many studies have combined image time series analysis with machine learning methods to improve land use and cover change mapping. In order to support image time series analysis, analysis-ready data (ARD) image collections have been modeled and organized as multidimensional data cubes. Data cubes can be defined as sets of time series associated with spatially aligned pixels. Based on lessons learned in the research project e-Sensing, related to national demands for land use and cover monitoring and related to state-of-the-art studies on relevant topics, we define the requirements to build Earth observation data cubes for Brazil. This paper presents the methodology to generate ARD and multidimensional data cubes from remote sensing images for Brazil. We describe the computational infrastructure that we are developing in the Brazil Data Cube project, composed of software applications and Web services to create, integrate, discover, access, and process the data sets. We also present how we are producing land use and cover maps from data cubes using image time series analysis and machine learning techniques.

Investigating the Influence of Registration Errors on the Patch-Based Spatio-Temporal Fusion Method

Spatio-temporal fusion is a common approach in remote sensing, used to create time-series image data with both fine spatial and temporal resolutions. However, geometric registration error, which is a common problem in remote sensing relative to the ground reference, is a particular problem for multiresolution remote sensing data, especially for images with very different spatial resolutions (e.g., Landsat and MODIS images). Registration error can, thus, have a significant impact on the accuracy of spatio-temporal fusion. To the best of our knowledge, however, almost no effective solutions have been provided to-date to cope with this important issue. This article demonstrates the robustness to registration error of the existing SParse representation-based spatio-temporal reflectance fusion model (SPSTFM). Different to conventional methods that are performed on a per-pixel basis, SPSTFM utilizes image patches as the basic unit. We demonstrate theoretically that the effect of registration error on patch-based methods is smaller than for pixel-based methods. Experimental results show that SPSTFM is highly robust to registration error and is far more accurate under various registration errors relative to pixel-based methods. The advantage is shown to be greater for heterogeneous regions than for homogeneous regions, and is large for the fusion of normalized difference vegetation index data. SPSTFM, thus, offers the remote sensing community a crucial tool to overcome one of the longest standing challenges to the effective fusion of remote sensing image time-series.

A Novel Relational-Based Transductive Transfer Learning Method for PolSAR Images via Time-Series Clustering

The combination of transfer learning and remote sensing image processing technology can effectively improve the automation level of image information extraction from a remote sensing time series. However, in the processing of polarimetric synthetic aperture radar (PolSAR) time-series images, the existing transfer learning methods often cannot make full use of the time-series information of the images, relying too much on the labeled samples in the target domain. Furthermore, the speckle noise inherent in synthetic aperture radar (SAR) imagery aggravates the difficulty of the manual selection of labeled samples, so these methods have difficulty in meeting the processing requirements of large data volumes and high efficiency. In lieu of these problems and the spatio-temporal relational knowledge of objects in time-series images, this paper introduces the theory of time-series clustering and proposes a new three-phase time-series clustering algorithm. Due to the full use of the inherent characteristics of the PolSAR images, this algorithm can accurately transfer the labels of the source domain samples to those samples that have not changed in the whole time series without relying on the target domain labeled samples, so as to realize transductive sample label transfer for PolSAR time-series images. Experiments were carried out using three different sets of PolSAR time-series images and the proposed method was compared with two of the existing methods. The experimental results showed that the transfer precision of the proposed method reaches a high level with different data and different objects and it performs significantly better than the existing methods. With strong reliability and practicability, the proposed method can provide a new solution for the rapid information extraction of remote sensing image time series.

Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series

ABSTRACTLand cover maps are significant in assisting agricultural decision making. However, the existing workflow of producing land cover maps is very complicated and the result accuracy is ambiguous. This work builds a long short-term memory (LSTM) recurrent neural network (RNN) model to take advantage of the temporal pattern of crops across image time series to improve the accuracy and reduce the complexity. An end-to-end framework is proposed to train and test the model. Landsat scenes are used as Earth observations, and some field-measured data together with CDL (Cropland Data Layer) datasets are used as reference data. The network is thoroughly trained using state-of-the-art techniques of deep learning. Finally, we tested the network on multiple Landsat scenes to produce five-class and all-class land cover maps. The maps are visualized and compared with ground truth, CDL, and the results of SegNet CNN (convolutional neural network). The results show a satisfactory overall accuracy (> 97% for five-class and > 88% for all-class) and validate the feasibility of the proposed method. This study paves a promising way for using LSTM RNN in the classification of remote sensing image time series.

Object-based land-cover change detection applied to Brazilian seasonal savannahs using geostatistical features

ABSTRACTA new method for remote-sensing land-use/land-cover (LULC) change detection is proposed to eliminate the effects of forest phenology on classification results. This method is insensitive to spectral changes caused by vegetation seasonality and uses an object-based approach to extract geostatistical features from bitemporal Landsat TM (Thematic Mapper) images. We first create image objects by multiresolution segmentation to extract geostatistical features (semivariogram parameters and indices) and spectral information (average values) from NDVI (normalized difference vegetation index), acquired in the wet and dry seasons, as input data to train a Support Vector Machine algorithm. We also used the image difference traditional change-detection method to validate the effectiveness of the proposed method. We used two classes: (1) LULC change class and (2) seasonal change class. Using the most geostatistical features, the change detection results are considerably improved compared with the spectral features and image differencing technique. The highest accuracy was achieved by the sill (σ2 overall variability) semivariogram parameter (95%) and the AFM (area first lag–first maximum) semivariogram index (88.33%), which were not affected by vegetation seasonality. The results indicate that the geostatistical context makes possible the use of bitemporal NDVI images to address the challenge of accurately detecting LULC changes in Brazilian seasonal savannahs, disregarding changes caused by phenological differences, without using a dense time series of remote-sensing images. The challenge of extracting accurate semivariogram curves from objects of long and narrow shapes requires further study, along with the relationship between the scale of segmentation and image spatial resolution, including the type of change and the initial land-cover class.

Detecting spatio-temporal and typological changes in land use from Landsat image time series

With the development of earth observation technology, the spatial and temporal resolution of remote sensing images have improved dramatically, providing abundant data for detecting land use change in detail. However, extracting spatio-temporal and typological changes in land use from remote sensing image time series is still challenging. Landsat image time series and land survey data are combined to develop a method to detect in detail the time and types of land use change. From the Landsat images, the time series of normalized difference vegetation index variation slope (TSNVS) on each pixel was constructed. The average TSNVS of sample pixels was used as the reference series. Based on TSNVS, the change in land use at six time points (S1 to S6) was detected by measuring the similarity of the time series on undetermined pixels and reference series. Land survey data were used to identify detailed types of land use change. The 16 different types of land use change were identified. The results indicate that the proposed method is effective at detecting spatio-temporal and typological changes in land use from Landsat image time series. After verification, the overall accuracy was 92%, and the kappa coefficient was 0.9.

Cloud masking and removal in remote sensing image time series

Automatic cloud masking of Earth observation images is one of the first required steps in optical remote sensing data processing since the operational use and product generation from satellite image time series might be hampered by undetected clouds. The high temporal revisit of current and forthcoming missions and the scarcity of labeled data force us to cast cloud screening as an unsupervised change detection problem in the temporal domain. We introduce a cloud screening method based on detecting abrupt changes along the time dimension. The main assumption is that image time series follow smooth variations over land (background) and abrupt changes will be mainly due to the presence of clouds. The method estimates the background surface changes using the information in the time series. In particular, we propose linear and nonlinear least squares regression algorithms that minimize both the prediction and the estimation error simultaneously. Then, significant differences in the image of interest with respect to the estimated background are identified as clouds. The use of kernel methods allows the generalization of the algorithm to account for higher-order (nonlinear) feature relations. After the proposed cloud masking and cloud removal, cloud-free time series at high spatial resolution can be used to obtain a better monitoring of land cover dynamics and to generate more elaborated products. The method is tested in a dataset with 5-day revisit time series from SPOT-4 at high resolution and with Landsat-8 time series. Experimental results show that the proposed method yields more accurate cloud masks when confronted with state-of-the-art approaches typically used in operational settings. In addition, the algorithm has been implemented in the Google Earth Engine platform, which allows us to access the full Landsat-8 catalog and work in a parallel distributed platform to extend its applicability to a global planetary scale.

A Time-Weighted Dynamic Time Warping Method for Land-Use and Land-Cover Mapping

This paper presents a time-weighted version of the dynamic time warping (DTW) method for land-use and land-cover classification using remote sensing image time series. Methods based on DTW have achieved significant results in time-series data mining. The original DTW method works well for shape matching, but is not suited for remote sensing time-series classification. It disregards the temporal range when finding the best alignment between two time series. Since each land-cover class has a specific phenological cycle, a good time-series land-cover classifier needs to balance between shape matching and temporal alignment. To that end, we adjusted the original DTW method to include a temporal weight that accounts for seasonality of land-cover types. The resulting algorithm improves on previous methods for land-cover classification using DTW. In a case study in a tropical forest area, our proposed logistic time-weighted version achieves the best overall accuracy of 87.32%. The accuracy of a version with maximum time delay constraints is 84.66%. A time-warping method without time constraints has a 70.14% accuracy. To get good results with the proposed algorithm, the spatial and temporal resolutions of the data should capture the properties of the landscape. The pattern samples should also represent well the temporal variation of land cover.

Sequential pattern mining of land cover dynamics based on time-series remote sensing images

Remote sensing images constitute a new type of multimedia data well suited to land cover change detection tasks, as they can repetitively provide information about the land surface and its changes over large and inaccessible areas. With plans for more missions and higher resolution earth observation systems, the challenge is increasingly going to be the efficient usability of the millions of collected images, especially the decades of remote sensing image time series, to describe land cover and/or scene evolution and dynamics. In contrast to traditional land cover change measures using pair-wise comparisons that emphasize the compositional or configurational changes between dates, this research focuses on the analysis of the temporal sequence of land cover dynamics, which refers to the succession of land cover types for a given area over more than two observational periods. The expected novel significance of this study is the generalization of the application of the sequential pattern mining method for capturing the spatial variability of landscape patterns and their trajectories of change to reveal information regarding process regularities with satellite imagery. Experimental results showed that this approach not only quantifies land cover changes in terms of the percentage area affected and maps the spatial distribution of these land cover changes but also reveals possibly interesting or useful information regarding the trajectories of change. This method is a valuable complement to existing bi-temporal change detection methods.

ESTIMATING RELIABILITY OF DISTURBANCES IN SATELLITE TIME SERIES DATA BASED ON STATISTICAL ANALYSIS

Abstract. Normally, the status of land cover is inherently dynamic and changing continuously on temporal scale. However, disturbances or abnormal changes of land cover — caused by such as forest fire, flood, deforestation, and plant diseases — occur worldwide at unknown times and locations. Timely detection and characterization of these disturbances is of importance for land cover monitoring. Recently, many time-series-analysis methods have been developed for near real-time or online disturbance detection, using satellite image time series. However, the detection results were only labelled with “Change/ No change” by most of the present methods, while few methods focus on estimating reliability (or confidence level) of the detected disturbances in image time series. To this end, this paper propose a statistical analysis method for estimating reliability of disturbances in new available remote sensing image time series, through analysis of full temporal information laid in time series data. The method consists of three main steps. (1) Segmenting and modelling of historical time series data based on Breaks for Additive Seasonal and Trend (BFAST). (2) Forecasting and detecting disturbances in new time series data. (3) Estimating reliability of each detected disturbance using statistical analysis based on Confidence Interval (CI) and Confidence Levels (CL). The method was validated by estimating reliability of disturbance regions caused by a recent severe flooding occurred around the border of Russia and China. Results demonstrated that the method can estimate reliability of disturbances detected in satellite image with estimation error less than 5% and overall accuracy up to 90%.

ESTIMATING RELIABILITY OF DISTURBANCES IN SATELLITE TIME SERIES DATA BASED ON STATISTICAL ANALYSIS

Normally, the status of land cover is inherently dynamic and changing continuously on temporal scale. However, disturbances or abnormal changes of land cover — caused by such as forest fire, flood, deforestation, and plant diseases — occur worldwide at unknown times and locations. Timely detection and characterization of these disturbances is of importance for land cover monitoring. Recently, many time-series-analysis methods have been developed for near real-time or online disturbance detection, using satellite image time series. However, the detection results were only labelled with “Change/ No change” by most of the present methods, while few methods focus on estimating reliability (or confidence level) of the detected disturbances in image time series. To this end, this paper propose a statistical analysis method for estimating reliability of disturbances in new available remote sensing image time series, through analysis of full temporal information laid in time series data. The method consists of three main steps. (1) Segmenting and modelling of historical time series data based on Breaks for Additive Seasonal and Trend (BFAST). (2) Forecasting and detecting disturbances in new time series data. (3) Estimating reliability of each detected disturbance using statistical analysis based on Confidence Interval (CI) and Confidence Levels (CL). The method was validated by estimating reliability of disturbance regions caused by a recent severe flooding occurred around the border of Russia and China. Results demonstrated that the method can estimate reliability of disturbances detected in satellite image with estimation error less than 5% and overall accuracy up to 90%.

Remote sensing time series analysis for crop monitoring with the SPIRITS software: new functionalities and use examples

Monitoring crop and natural vegetation conditions is highly relevant, particularly in the food insecure areas of the world. Data from remote sensing image time series at high temporal and medium to low spatial resolution can assist this monitoring as they provide key information about vegetation status in near real-time over large areas. The Software for the Processing and Interpretation of Remotely sensed Image Time Series (SPIRITS) is a stand-alone flexible analysis environment created to facilitate the processing and analysis of large image time series and ultimately for providing clear information about vegetation status in various graphical formats to crop production analysts and decision makers. In this paper we present the latest functional developments of SPIRITS and we illustrate recent applications. The main new developments include: HDF5 importer, Image re-projection, additional options for temporal Smoothing and Periodicity conversion, computation of a rainfall-based probability index (Standardized Precipitation Index) for drought detection and extension of the Graph composer functionalities. In particular,. The examples of operational analyses are taken from several recent agriculture and food security monitoring reports and bulletins. We conclude with considerations on future SPIRITS developments also in view of the data processing requirements imposed by the coming generation of remote sensing products at high spatial and temporal resolution, such as those provided by the Sentinel sensors of the European Copernicus programme.

Detecting Landscape Changes in High Latitude Environments Using Landsat Trend Analysis: 2. Classification

Mapping landscape dynamics is necessary to assess cumulative impacts due to climate change and development in Arctic regions. Landscape changes produce a range of temporal reflectance trajectories that can be obtained from remote sensing image time-series. Mapping these changes assumes that their trajectories are unique and can be characterized by magnitude and shape. A companion paper in this issue describes a trajectory visualization method for assessing a range of landscape disturbances. This paper focusses on generating a change map using a time-series of calibrated Landsat Tasseled Cap indices from 1985 to 2011. A reference change database covering the Mackenzie Delta region was created using a number of ancillary datasets to delineate polygons describing 21 natural and human-induced disturbances. Two approaches were tested to classify the Landsat time-series and generate change maps. The first involved profile matching based on trajectory shape and distance, while the second quantified profile shape with regression coefficients that were input to a decision tree classifier. Results indicate that classification of robust linear trend coefficients performed best. A final change map was assessed using bootstrapping and cross-validation, producing an overall accuracy of 82.8% at the level of 21 change classes and 87.3% when collapsed to eight underlying change processes.

Classification of Time Series of Multispectral Images With Limited Training Data

Image classification usually requires the availability of reliable reference data collected for the considered image to train supervised classifiers. Unfortunately when time series of images are considered, this is seldom possible because of the costs associated with reference data collection. In most of the applications it is realistic to have reference data available for one or few images of a time series acquired on the area of interest. In this paper, we present a novel system for automatically classifying image time series that takes advantage of image(s) with an associated reference information (i.e., the source domain) to classify image(s) for which reference information is not available (i.e., the target domain). The proposed system exploits the already available knowledge on the source domain and, when possible, integrates it with a minimum amount of new labeled data for the target domain. In addition, it is able to handle possible significant differences between statistical distributions of the source and target domains. Here, the method is presented in the context of classification of remote sensing image time series, where ground reference data collection is a highly critical and demanding task. Experimental results show the effectiveness of the proposed technique. The method can work on multimodal (e.g., multispectral) images.