Time Series Of Images Research Articles

Incorporating Phenological Patterns and Multi-source Remote Sensing Imagesfor Cropland Change Detection

Abstract. Accurate information on cropland changes is critical for monitoring arable land minimum, ensuring national food security, and grasping the situation of agricultural production and supply. The change detection using remote sensing images is one of the main methods for quickly extracting cropland changes. However, existing methods were highly susceptible to seasonal differences due to the high heterogeneity of cultivated land. In this study, an integrated framework was proposed to perform change detection by incorporating phenological patterns and multi-source remote sensing images.There were two improvements in this proposed cropland change detection method: 1) the multi-source remote sensing images were utilized to fill the missing data within a time-series image stack by considering phenological patterns of cropland and 2) the Seq2Seq model considering phenological patterns was developed to extract changes in cropland directly. Compared to the traditional change detection methods, the proposed strategy was able to detect change process. Experiments demonstrated that the proposed method can significantly improved change detection accuracies, given a limited number of labeled samples.

Contrasting topologies of synchronous and asynchronous functional brain networks

ABSTRACT We generated asynchronous functional networks (aFNs) using a novel method called optimal causation entropy and compared aFN topology with the correlation-based synchronous functional networks (sFNs), which are commonly used in network neuroscience studies. Functional magnetic resonance imaging (fMRI) time series from 212 participants of the National Consortium on Alcohol and Neurodevelopment in Adolescence study were used to generate aFNs and sFNs. As a demonstration of how aFNs and sFNs can be used in tandem, we used multivariate mixed effects models to determine whether age interacted with node efficiency to influence connection probabilities in the two networks. After adjusting for differences in network density, aFNs had higher global efficiency but lower local efficiency than the sFNs. In the aFNs, nodes with the highest outgoing global efficiency tended to be in the brainstem and orbitofrontal cortex. aFN nodes with the highest incoming global efficiency tended to be members of the default mode network in sFNs. Age interacted with node global efficiency in aFNs and node local efficiency in sFNs to influence connection probability. We conclude that the sFN and aFN both offer information about functional brain connectivity, which the other type of network does not.

Discovery of 118 New Ultracool Dwarf Candidates Using Machine-learning Techniques

We present the discovery of 118 new ultracool dwarf candidates, discovered using a new machine-learning tool, named SMDET, applied to time-series images from the Wide-field Infrared Survey Explorer. We gathered photometric and astrometric data to estimate each candidate’s spectral type, distance, and tangential velocity. This sample has a photometrically estimated spectral class distribution of 28 M dwarfs, 64 L dwarfs, and 18 T dwarfs. We also identify a T-subdwarf candidate, two extreme T-subdwarf candidates, and two candidate young ultracool dwarfs. Five objects did not have enough photometric data for any estimations to be made. To validate our estimated spectral types, spectra were collected for two objects, yielding confirmed spectral types of T5 (estimated T5) and T3 (estimated T4). Demonstrating the effectiveness of machine-learning tools as a new large-scale discovery technique.

Deep learning-based detection of affected body parts in Parkinson’s disease and freezing of gait using time-series imaging

We proposed a deep learning method using a convolutional neural network on time-series (TS) images to detect and differentiate affected body parts in people with Parkinson’s disease (PD) and freezing of gait (FOG) during 360° turning tasks. The 360° turning task was performed by 90 participants (60 people with PD [30 freezers and 30 nonfreezers] and 30 age-matched older adults (controls) at their preferred speed. The position and acceleration underwent preprocessing. The analysis was expanded from temporal to visual data using TS imaging methods. According to the PD vs. controls classification, the right lower third of the lateral shank (RTIB) on the least affected side (LAS) and the right calcaneus (RHEE) on the LAS were the most relevant body segments in the position and acceleration TS images. The RHEE marker exhibited the highest accuracy in the acceleration TS images. The identified markers for the classification of freezers vs. nonfreezers vs. controls were the left lateral humeral epicondyle (LELB) on the more affected side and the left posterior superior iliac spine (LPSI). The LPSI marker in the acceleration TS images displayed the highest accuracy. This approach could be a useful supplementary tool for determining PD severity and FOG.

MeerKAT Observations of Procyon at 815.5 MHz

We have conducted observations of the nearby (11.46 ly) star system Procyon, using MeerKAT’s UHF (544–1087 MHz) receivers. We produce full-Stokes time and frequency integrated continuum images, as well as total intensity time series imaging at 8 s cadence, and full-Stokes vector-averaged dynamic spectra from the visibilities in order to search for transient activity such as flaring events. We detect no significant radio emission from the system, and estimate an upper limit on the circular polarization fraction of 65% (3σ confidence level). A comparison with previous Very Large Array observations places a 3σ lower limit on the spectral index between 815.5 and 8400 MHz of 0.26, however long-term significant variability over the last 33 yr cannot be ruled out without further, regular radio monitoring of the system.

A phenological-knowledge-independent method for automatic paddy rice mapping with time series of polarimetric SAR images

Paddy rice, which sustains more than half of the global population, requires accurate and efficient mapping to ensure food security. Synthetic aperture radar (SAR) has become indispensable in this process due to its remarkable ability to operate effectively in adverse weather conditions and its sensitivity to paddy rice growth. Phenological-knowledge-based (PKB) methods have been commonly employed in conjunction with time series of SAR images for paddy rice mapping, primarily because they eliminate the need for training datasets. However, PKB methods possess inherent limitations, primarily stemming from their reliance on precise phenological information regarding paddy rice growth. This information varies across regions and paddy rice varieties, making it challenging to use PKB methods effectively on a large spatial scale, such as the national or global scale, where collecting comprehensive phenological data becomes impractical. Moreover, variations in farming practices and field conditions can lead to differences in paddy rice growth stages even within the same region. Using a generalized set of phenological knowledge in PKB methods may not be suitable for all paddy fields, potentially resulting in errors in paddy rice extraction. To address the challenges posed by PKB methods, this study proposed an innovative approach known as the phenological-knowledge-independent (PKI) method for mapping paddy rice using time series of Sentinel-1 SAR images. The central innovation of the PKI method lies in its capability to map paddy rice without relying on specific knowledge of paddy rice phenology or the need for a training dataset. This was made possible by the incorporation of three novel metrics: VH and VV normalized maximum temporal changes (NMTC) and VH temporal mean, derived from the distinctions between paddy rice and other land cover types in time series of SAR images. The PKI method was rigorously evaluated across three regions in China, each featuring different paddy rice varieties. Additionally, the PKI method was compared with two prevalent phenological-knowledge-based techniques: the automated paddy rice mapping method using SAR flooding signals (ARM-SARFS) and the manual interpretation of unsupervised clustering results (MI-UCR). The PKI method achieved an average overall accuracy of 97.99%, surpassing the ARM-SARFS, which recorded an accuracy of 89.65% due to errors stemming from phenological disparities among different paddy fields. Furthermore, the PKI method delivered results on par with the MI-UCR, which relied on the fusion of SAR and optical image time series, achieving an accuracy of 97.71%. As demonstrated by these findings, the PKI method proves highly effective in mapping paddy rice across diverse regions, all without the need for phenological knowledge or a training dataset. Consequently, it holds substantial promise for efficiently mapping paddy rice on a large spatial scale. The source code used in this study is available at https://code.earthengine.google.com/f82cf10cad64fa3f971ae99027001a6e.

Multitemporal UAV study of phenolic compounds in slash pine canopies

Phenolic compounds (PC) are important secondary metabolites in plants, playing a crucial role in plant defense mechanisms against pathogens and other plants. Monitoring PC levels is important for understanding tree stress and implementing effective breeding programs. However, traditional methods for monitoring PC are time-consuming, prone to altering the phenolic composition, and mostly applicable only on a small scale. In this study, we evaluated the performance of Unoccupied Aerial Vehicles (UAV) multispectral imaging in estimating the canopy phenolic content in slash pine over an 11-month period in 2021 and a seven-month period in 2022. Three machine learning models including Partial least squares regression (PLSR), Random forest (RF) and Support Vector Machine (SVM) were compared to determine the optimal predictive model for canopy PC. The RF model provided the best predictive results, with R2 values of 0.82 for the validation set and 0.94 for the calibration set. Additionally, the study assesses the heritable variation in canopy PC over time, with the monthly heritability (h2) of PC ranging from 0 to 0.26 in 2021 and from 0 to 0.35 in 2022; The highest h2 levels were observed in July and September 2021and July 2022. The findings demonstrate significant genetic control over the variation of PC. Furthermore, we observed higher breeding values and genetic gains in July and November, which further supports the strong correlation between PC and environmental factors such as temperature and light intensity. To the best of our knowledge, this is the first study to employ time-series UAV multispectral imaging to predict secondary metabolites in pine trees and estimate their genetic variation over time. As a proof of concept, these findings provide more reliable information for tree breeding programs, ultimately enhancing their overall performance.

Removing thick clouds in landsat 8 OLI images with cloudy time-series auxiliary data

Abstract Thick cloud occlusion severely limits the subsequent application of optical remote-sensing images. Existing mainstream cloud removal methods often use cloud-free homologous temporal images to provide auxiliary information. However, due to the limitation of cloud occlusion and satellite revisit cycle, the time interval between the cloud-free auxiliary image and the target cloudy image is often too large, and the possible land cover changes during the period increase the uncertainty of effective information on the auxiliary images. In this case, more accurate reconstructions may be obtained by using temporally closer and partially cloud-contaminated images. In this paper, a deep network, convLSTM, is developed for cloud removal (i.e., CR-convLSTM), which makes full use of the complementary effective information on partially cloud-contaminated time-series images. Experiments based on simulated clouds indicated that CR-convLSTM can obtain more accurate predictions than the classical cloud removal method MNSPI. CR-convLSTM is an effective cloud removal method with high computational efficiency while ensuring predicting accuracy.

Digital mapping of dates of transplanting and accumulated thermal requirement of rice (Oryza sativa L.) in the subtropics of North Eastern Hill Region, India

ABSTRACT Accurate and timely mapping of rice areas is crucial for effective agricultural planning and food security management in the North Eastern Hill Region of India, especially amid changing climate conditions. This study developed a method to estimate rice transplanting dates by utilizing VH polarized backscattering-coefficients rom Sentinel-1A SAR data and also assessed thermal requirements for rice productivity in Ri-Bhoi district, Meghalaya during kharif 2021. The methodology involved 3 steps- (i) preprocessing data to construct time-series SAR images (ii) rule-based classification of SAR data integrating phenological and topographical information to identify transplanting windows (iii) assessing mapping accuracy. The results when compared with field data, revealed five distinct transplanting windows from early June to early August across 9031 ha of rice area. Overall accuracy was 85%, with a kappa coefficient of 0.74. User and producer accuracies ranged from 0.74 to 0.93 and 0.75 to 0.92, respectively for various transplanting windows. GDD varied between 1432.8°C day and 3290.4°C day, with higher values corresponding to earlier planting dates. This study formulates a robust methodology for rice mapping in hilly, small-field landscapes with multiple transplanting dates by integrating Sentinel-1A SAR data with ancillary and crop phenology information, contributing to improved crop management and climate change adaptations.

The Time Series Classification of Discrete-Time Chaotic Systems Using Deep Learning Approaches

Discrete-time chaotic systems exhibit nonlinear and unpredictable dynamic behavior, making them very difficult to classify. They have dynamic properties such as the stability of equilibrium points, symmetric behaviors, and a transition to chaos. This study aims to classify the time series images of discrete-time chaotic systems by integrating deep learning methods and classification algorithms. The most important innovation of this study is the use of a unique dataset created using the time series of discrete-time chaotic systems. In this context, a large and unique dataset representing various dynamic behaviors was created for nine discrete-time chaotic systems using different initial conditions, control parameters, and iteration numbers. The dataset was based on existing chaotic system solutions in the literature, but the classification of the images representing the different dynamic structures of these systems was much more complex than ordinary image datasets due to their nonlinear and unpredictable nature. Although there are studies in the literature on the classification of continuous-time chaotic systems, no studies have been found on the classification of discrete-time chaotic systems. The obtained time series images were classified with deep learning models such as DenseNet121, VGG16, VGG19, InceptionV3, MobileNetV2, and Xception. In addition, these models were integrated with classification algorithms such as XGBOOST, k-NN, SVM, and RF, providing a methodological innovation. As the best result, a 95.76% accuracy rate was obtained with the DenseNet121 model and XGBOOST algorithm. This study takes the use of deep learning methods with the graphical representations of chaotic time series to an advanced level and provides a powerful tool for the classification of these systems. In this respect, classifying the dynamic structures of chaotic systems offers an important innovation in adapting deep learning models to complex datasets. The findings are thought to provide new perspectives for future research and further advance deep learning and chaotic system studies.

Metis Observation of the Onset of Fully Developed Turbulence in the Solar Corona

This Letter reports the first observation of the onset of fully developed turbulence in the solar corona. Long time series of white-light coronal images, acquired by Metis aboard Solar Orbiter at 2 minutes cadence and spanning about 10 hr, were studied to gain insight into the statistical properties of fluctuations in the density of the coronal plasma in the time domain. From pixel-by-pixel spectral frequency analysis in the whole Metis field of view, the scaling exponents of plasma fluctuations were derived. The results show that, over timescales ranging from 1 to 10 hr and corresponding to the photospheric mesogranulation-driven dynamics, the density spectra become shallower moving away from the Sun, resembling a Kolmogorov-like spectrum at 3 R ⊙. According to the latest observation and interpretive work, the observed 5/3 scaling law for density fluctuations is indicative of the onset of fully developed turbulence in the corona. Metis observation-based evidence for a Kolmogorov turbulent form of the fluctuating density spectrum casts light on the evolution of 2D turbulence in the early stages of its upward transport from the low corona.

Enhanced Monitoring of Sub-Seasonal Land Use Dynamics in Vietnam’s Mekong Delta through Quantile Mapping and Harmonic Regression

In response to economic and environmental challenges like sea-level rise, salinity intrusion, groundwater extraction, sand mining, and sinking delta phenomena, the demand for solutions to adapt to changing conditions in riverine environments has increased significantly. High-quality analyses of land use and land cover (LULC) dynamics play a critical role in addressing these challenges. This study introduces a novel high-spatial resolution satellite-based approach to identify sub-seasonal LULC dynamics in the Mekong River Delta (MRD), employing a three-year (2021–2023) Sentinel-1 and Sentinel-2 satellite data time series. The primary obstacle is discerning detailed vegetation dynamics, particularly the seasonality of rice crops, answered through quantile mapping, harmonic regression with Fourier transform, and phenological metrics as inputs to a random forest machine learning classifier. Due to the substantial data volume, Google’s cloud computing platform Earth Engine was utilized for the analysis. Furthermore, the study evaluated the relative significance of various input features. The overall accuracy of the classification is 82.6% with a kappa statistic of 0.81, determined using comprehensive reference data collected in Vietnam. While the purely pixel-based approach has limitations, it proves to be a viable method for high-spatial resolution satellite image time series classification of the MRD.

A Deep Learning-based Approach to Predict the Flood Patterns Using Sentinel-1A Time Series Images

A Deep Learning-based Approach to Predict the Flood Patterns Using Sentinel-1A Time Series Images

New ecological knowledge of the bubblegum coral Paragorgia arborea (L. 1758) using computer vision

IntroductionModern stationary observatories, equipped with cameras and other sensors, make it possible to observe and study cold-water corals in situ over increasingly long periods of time. In this study we investigate data collected for a bubblegum coral (Paragorgia arborea) with the LoVe (Lofoten Vesteralen) observatory for a period of 117 days, including time series of images, current, water depth and temperature. This kind of multimodal time series data represent a new methodological challenge for marine technology and bioinformatics.MethodsIn this work, we continue on the path to computer-aided ocean data analysis and show how, in addition to polyp activity, morphological parameters can be extracted from image data with a new deep learning-based approach. Using customized visualizations we show how this image-derived data together with the environmental data can be analysed and new interpretations are supported.ResultsOur results indicate that the polyp activity and branch diameter variation followed a semi-cyclic pattern, with the polyps changing from fully retracted to fully expanded faster than the image sampling rate, resulting in an almost binary time series that showed positive correlation with the current strength, which may be related to food supply. The periods of full polyp expansion ranged from 1 to 4 h, with the majority around 1 h. The variation in branch diameter and volume of intra-skeletal channels probably relates to the intra-colonial transport of nutrients and waste products. During a 16 day period in March 2019, the polyps remained retracted for an extended period accompanied by little variation in branch diameter. This behaviour was not related to any of the measured environmental conditions. Further studies are required to determine if this could be linked to biological processes, such as reproduction.

A novel solar radiation forecasting model based on time series imaging and bidirectional long short‐term memory network

AbstractThe instability of solar energy is the biggest challenge to its successful integration with modern power grids, and accurate prediction of long‐term solar radiation can effectively solve this problem. In this study, we proposed a novel long‐term solar radiation prediction model based on time series imaging and bidirectional long short‐term memory network. First, inspired by the computer vision algorithm, the recursive graph algorithm is used to transform the one‐dimensional time series into two‐dimensional images, and then convolutional neural network is used to extract the features from the images, thus, the deeper features in the original solar radiation data can be mined. Second, to solve the problem of low accuracy of long‐term solar radiation prediction, a hybrid model BiLSTM‐Transformer is used to predict long‐term solar radiation. The hybrid prediction model can capture the long‐term dependencies, thereby further improving the accuracy of the prediction model. The experimental results show that the hybrid model proposed in this study is superior to other single models and hybrid models in long‐term solar radiation prediction accuracy. The accuracy and stability of the hybrid model are verified by many tests.

Self-Supervised Learning across the Spectrum

Satellite image time series (SITS) segmentation is crucial for many applications, like environmental monitoring, land cover mapping, and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine-grained annotation. We propose S4, a new self-supervised pretraining approach that significantly reduces the requirement for labeled training data by utilizing two key insights of satellite imagery: (a) Satellites capture images in different parts of the spectrum, such as radio frequencies and visible frequencies. (b) Satellite imagery is geo-registered, allowing for fine-grained spatial alignment. We use these insights to formulate pretraining tasks in S4. To the best of our knowledge, S4 is the first multimodal and temporal approach for SITS segmentation. S4’s novelty stems from leveraging multiple properties required for SITS self-supervision: (1) multiple modalities, (2) temporal information, and (3) pixel-level feature extraction. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially aligned, multimodal, and geographic-specific SITS that serves as representative pretraining data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data. Through a series of extensive comparisons and ablation studies, we demonstrate S4’s ability as an effective feature extractor for downstream semantic segmentation.

Investigating the contribution of image time series observations to cauliflower harvest-readiness prediction.

Cauliflower cultivation is subject to high-quality control criteria during sales, which underlines the importance of accurate harvest timing. Using time series data for plant phenotyping can provide insights into the dynamic development of cauliflower and allow more accurate predictions of when the crop is ready for harvest than single-time observations. However, data acquisition on a daily or weekly basis is resource-intensive, making selection of acquisition days highly important. We investigate which data acquisition days and development stages positively affect the model accuracy to get insights into prediction-relevant observation days and aid future data acquisition planning. We analyze harvest-readiness using the cauliflower image time series of the GrowliFlower dataset. We use an adjusted ResNet18 classification model, including positional encoding of the data acquisition dates to add implicit information about development. The explainable machine learning approach GroupSHAP analyzes time points' contributions. Time points with the lowest mean absolute contribution are excluded from the time series to determine their effect on model accuracy. Using image time series rather than single time points, we achieve an increase in accuracy of 4%. GroupSHAP allows the selection of time points that positively affect the model accuracy. By using seven selected time points instead of all 11 ones, the accuracy improves by an additional 4%, resulting in an overall accuracy of 89.3%. The selection of time points may therefore lead to a reduction in data collection in the future.

Measurements of frazil ice flocs in rivers

Abstract. Frazil floc sizes and concentrations have been investigated in a small number of laboratory studies, but no detailed field measurements have been reported previously. In this study, a submersible camera system was deployed a total of 11 times during the principal and residual supercooling phases in the North Saskatchewan, Peace, and Kananaskis rivers to capture time-series images of frazil ice particles and flocs. Images were processed to accurately identify flocs and to calculate their sizes and concentrations. Key hydraulic and meteorological measurements were collected, and air–water heat fluxes were estimated to investigate their influence on floc properties. A lognormal distribution was found to be a good fit for the floc size distribution. The mean floc size ranged from 1.19 to 5.64 mm and the overall mean floc size was 3.80 mm. The mean floc size decreased linearly as the local Reynolds number increased. The average floc number concentration ranged from 1.80×10-4 to 1.15×10-1 cm−3. The average floc volumetric concentration ranged from 2.05×10-7 to 4.56×10-3 and was found to correlate strongly with the fractional height above the river bed. No significant correlations were found between the air–water heat flux and floc properties. Time series analysis showed that during the principal supercooling phase, floc number concentration and mean size increased significantly just prior to peak supercooling and reached a maximum near the end of principal supercooling. During the residual supercooling phase, the mean floc size did not typically vary significantly even 2.5 h after the residual phase had ended and the water temperature had increased above 0 °C.

Studying Drosophila Larval Behavior in Agarose Channels.

Larvae of the fruit fly Drosophila melanogaster are a popular and tractable model system for studying the development and function of sensorimotor circuits, thanks to the relative numerical simplicity of their nervous system and the wealth of available genetic tools to manipulate the anatomy, activity, and function of specific cell types. Researchers studying the role of a particular gene or cell type in sensorimotor circuit activity or function may wish to observe the effects of an experimental manipulation during behavior in the intact animal. Observing these effects, which may include changes in the intracellular calcium concentration or movement of small numbers of neurons, muscles, etc., typically requires high-spatial-resolution imaging, which poses several difficulties in the freely crawling larva. Freely crawling larvae can move quickly and with changeable heading, making manual or automatic tracking challenging; additionally, they may make three-dimensional movements, such as rearing, that can degrade imaging focus. These challenges are potentially solvable using advanced imaging and algorithmic tracking setups, but cost, space, or development time may be prohibitive. This protocol describes a simple and cost-effective method for placing larvae inside agarose channels, thereby restricting larval crawling to a single dimension and enabling higher-magnification time-series imaging of fluorescently labeled structures during many cycles of locomotion. By using larvae that express fluorescent calcium indicators in cells of interest, researchers can apply this method to study the effects of experimental manipulations on neural or muscular activity during behavior in the intact animal.

Mapping Fruit-Tree Plantation Using Sentinel-1/2 Time Series Images with Multi-Index Entropy Weighting Dynamic Time Warping Method

Mapping Fruit-Tree Plantation Using Sentinel-1/2 Time Series Images with Multi-Index Entropy Weighting Dynamic Time Warping Method