Missing Time Series Research Articles

Missing data imputation and classification of small sample missing time series data based on gradient penalized adversarial multi-task learning

Missing data imputation and classification of small sample missing time series data based on gradient penalized adversarial multi-task learning

U-shaped convolutional transformer GAN with multi-resolution consistency loss for restoring brain functional time-series and dementia diagnosis.

The blood oxygen level-dependent (BOLD) signal derived from functional neuroimaging is commonly used in brain network analysis and dementia diagnosis. Missing the BOLD signal may lead to bad performance and misinterpretation of findings when analyzing neurological disease. Few studies have focused on the restoration of brain functional time-series data. In this paper, a novel U-shaped convolutional transformer GAN (UCT-GAN) model is proposed to restore the missing brain functional time-series data. The proposed model leverages the power of generative adversarial networks (GANs) while incorporating a U-shaped architecture to effectively capture hierarchical features in the restoration process. Besides, the multi-level temporal-correlated attention and the convolutional sampling in the transformer-based generator are devised to capture the global and local temporal features for the missing time series and associate their long-range relationship with the other brain regions. Furthermore, by introducing multi-resolution consistency loss, the proposed model can promote the learning of diverse temporal patterns and maintain consistency across different temporal resolutions, thus effectively restoring complex brain functional dynamics. We theoretically tested our model on the public Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, and our experiments demonstrate that the proposed model outperforms existing methods in terms of both quantitative metrics and qualitative assessments. The model's ability to preserve the underlying topological structure of the brain functional networks during restoration is a particularly notable achievement. Overall, the proposed model offers a promising solution for restoring brain functional time-series and contributes to the advancement of neuroscience research by providing enhanced tools for disease analysis and interpretation.

A time series continuous missing values imputation method based on generative adversarial networks

Generative adversarial networks (GANs) have been widely utilized in time series analysis and modeling, wherein generators and discriminators interact to generate realistic data. However, when addressing the challenge of imputing continuous missing values in time series, the generator struggles to learn meaningful features due to the loss of local information, and the discriminator’s loss function exhibits significant deviations in determining the probability, making it difficult to effectively update model parameters during training. In response to these issues, this study presents a novel imputation model called cue Wasserstein generative adversarial network with gradient penalty (CWGAIN-GP). CWGAIN-GP incorporates the contextual cue information matrixs into the generator, enabling obtain and capture the potential data evolution features that hidden beyond the missing positions. This approach constrains the generator’s generation results to be closer to the true values probability distribution. Meanwhile, it optimizes the generator and discriminator neural network structure and loss function computation strategy of the original GAN model, which not only improves the accuracy of continuous missing data interpolation, but also improves the training stability of the model. Finally, We used three imputation accuracies on two real-world datasets, including experiments on imputation accuracy for eight comparison models, ablation study, and experiments on the effect of consecutive missing lengths. The experimental results show that CWGAIN-GP achieves impressive performance in imputing continuous missing time series.

Simulation and Reconstruction of Runoff in the High-Cold Mountains Area Based on Multiple Machine Learning Models

Runoff from the high-cold mountains area (HCMA) is the most important water resource in the arid zone, and its accurate forecasting is key to the scientific management of water resources downstream of the basin. Constrained by the scarcity of meteorological and hydrological stations in the HCMA and the inconsistency of the observed time series, the simulation and reconstruction of mountain runoff have always been a focus of cold region hydrological research. Based on the runoff observations of the Yurungkash and Kalakash Rivers, the upstream tributaries of the Hotan River on the northern slope of the Kunlun Mountains at different time periods, and the meteorological and atmospheric circulation indices, we used feature analysis and machine learning methods to select the input elements, train, simulate, and select the preferences of the machine learning models of the runoffs of the two watersheds, and reconstruct the missing time series runoff of the Kalakash River. The results show the following. (1) Air temperature is the most important driver of runoff variability in mountainous areas upstream of the Hotan River, and had the strongest performance in terms of the Pearson correlation coefficient (ρXY) and random forest feature importance (FI) (ρXY = 0.63, FI = 0.723), followed by soil temperature (ρXY = 0.63, FI = 0.043), precipitation, hours of sunshine, wind speed, relative humidity, and atmospheric circulation were weakly correlated. A total of 12 elements were selected as the machine learning input data. (2) Comparing the results of the Yurungkash River runoff simulated by eight machine learning methods, we found that the gradient boosting and random forest methods performed best, followed by the AdaBoost and Bagging methods, with Nash–Sutcliffe efficiency coefficients (NSE) of 0.84, 0.82, 0.78, and 0.78, while the support vector regression (NSE = 0.68), ridge (NSE = 0.53), K-nearest neighbor (NSE = 0.56), and linear regression (NSE = 0.51) were simulated poorly. (3) The application of four machine learning methods, gradient boosting, random forest, AdaBoost, and bagging, to simulate the runoff of the Kalakash River for 1978–1998 was generally outstanding, with the NSE exceeding 0.75, and the results of reconstructing the runoff data for the missing period (1999–2019) could well reflect the characteristics of the intra-annual and inter-annual changes in runoff.

An Evolutionary Artificial Neural Network approach for spatio-temporal wave height time series reconstruction

This paper proposes a novel methodology for recovering missing time series data, a crucial task for subsequent Machine Learning (ML) analyses. The methodology is specifically applied to Significant Wave Height (SWH) time series in the field of marine engineering. The proposed approach involves two phases. Firstly, the SWH time series for each buoy is independently reconstructed using three transfer function models: regression-based, correlation-based, and distance-based. The distance-based transfer function exhibits the best overall performance. Secondly, Evolutionary Artificial Neural Networks (EANNs) are utilised for the final recovery of each time series, using as inputs highly correlated buoys that have been intermediately recovered. The EANNs are evolved considering two metrics, the novel squared error relevance area, which balances the importance of extreme and around-mean values, and the well-known mean squared error. The study considers SWH time series data from 15 buoys in two coastal zones in the United States. The results demonstrate that the distance-based transfer function is generally the best transfer function, and that EANNs outperform a range of state-of-the-art ML techniques in 12 out of the 15 buoys, with a number of connections comparable to linear models. Furthermore, the proposed methodology outperforms the two most popular approaches for time series reconstruction, BRITS and SAITS, for all buoys except one. Therefore, the proposed methodology provides a promising approach, which may be applied to time series from other fields, such as wind or solar energy farms in the field of green energy.

Correction to: Multi‑head attention‑based model for reconstructing continuous missing time series data

Correction to: Multi‑head attention‑based model for reconstructing continuous missing time series data

Improving traffic time‐series predictability by imputing continuous non‐random missing data

AbstractContinuous non‐random data missing can be a challenging task for model prediction in intelligent transport system (ITS). In ITS, many methods have been proposed to solve this problem. However, the imputation accuracy of them is far from accurate. Thus, the authors propose a novel cross‐modality generative adversarial network, named as cross‐modality GAN, to impute continuous non‐random missing data from the cross‐modality perspective. This model uses the cross‐modality data fusion technique to fuse spatial and temporal modal data into vectorized features, and then imputes the target unseen missing data by a data generation pipeline. Different from the other existing models, this model overcomes the problem of zero observation data, and realizes long‐term missing time series imputation. Many comparative experiments are conducted. The results verify that the cross‐modality GAN achieves better imputation performances on Performance Measurement System (PeMS) dataset, a real public traffic dataset, compared to other baseline models. Furthermore, the results verify that the imputed data of cross‐modality GAN can provide more traffic time‐series predictability information, and improve prediction accuracy of prediction models effectively.

Use of long short-term memory network (LSTM) in the reconstruction of missing water level data in the River Seine

ABSTRACT This paper aims to fill in the missing time series of hourly surface water levels of some stations installed along the River Seine, using the long short-term memory (LSTM) algorithm. In our study, only the water level data from the same station, containing many missing parts, were used as input and output variables, in contrast to other works where several features are available to take advantage of e.g. other station data/physical variables. A sensitive analysis is presented on both the network properties and how the input and output data are reentered to better determine the appropriate strategy. Numerous scenarios are presented, each an updated version of the previous one. Ultimately, the final version of the model can impute missing values of up to one year of hourly data with great flexibility (one-year Root-Mean-Square Error (RMSE) = 0.14 m) regardless of the location of the missing gaps in the series or their size.

Multi-head attention-based model for reconstructing continuous missing time series data

Multi-head attention-based model for reconstructing continuous missing time series data

Analyzing the Performance of Transformers for the Prediction of the Blood Glucose Level Considering Imputation and Smoothing

In this paper we investigate the effect of two preprocessing techniques, data imputation and smoothing, in the prediction of blood glucose level in type 1 diabetes patients, using a novel deep learning model called Transformer. We train three models: XGBoost, a one-dimensional convolutional neural network (1D-CNN), and the Transformer model to predict future blood glucose levels for a 30-min horizon using a 60-min time series history in the OhioT1DM dataset. We also compare four methods of handling missing time series data during the model training: hourly mean, linear interpolation, cubic interpolation, and spline interpolation; and two smoothing techniques: Kalman smoothing and smoothing splines. Our experiments show that the Transformer performs better than XGBoost and 1D-CNN when only continuous glucose monitoring (CGM) is used as a predictor, and that it is very competitive against XGBoost when CGM and carbohydrate intake from the meal are used to predict blood glucose level. Overall, our results are more accurate than those appearing in the literature.

Binned Data Provide Better Imputation of Missing Time Series Data from Wearables.

The presence of missing values in a time-series dataset is a very common and well-known problem. Various statistical and machine learning methods have been developed to overcome this problem, with the aim of filling in the missing values in the data. However, the performances of these methods vary widely, showing a high dependence on the type of data and correlations within the data. In our study, we performed some of the well-known imputation methods, such as expectation maximization, k-nearest neighbor, iterative imputer, random forest, and simple imputer, to impute missing data obtained from smart, wearable health trackers. In this manuscript, we proposed the use of data binning for imputation. We showed that the use of data binned around the missing time interval provides a better imputation than the use of a whole dataset. Imputation was performed for 15 min and 1 h of continuous missing data. We used a dataset with different bin sizes, such as 15 min, 30 min, 45 min, and 1 h, and we carried out evaluations using root mean square error (RMSE) values. We observed that the expectation maximization algorithm worked best for the use of binned data. This was followed by the simple imputer, iterative imputer, and k-nearest neighbor, whereas the random forest method had no effect on data binning during imputation. Moreover, the smallest bin sizes of 15 min and 1 h were observed to provide the lowest RMSE values for the majority of the time frames during the imputation of 15 min and 1 h of missing data, respectively. Although applicable to digital health data, we think that this method will also find applicability in other domains.

Parallel Generative Adversarial Imputation Network for Multivariate Missing Time-Series Reconstruction and Its Application to Aeroengines

Parallel Generative Adversarial Imputation Network for Multivariate Missing Time-Series Reconstruction and Its Application to Aeroengines

Recovery of missing field measured wind pressures on a supertall building based on correlation analysis and machine learning

In long-term field measurements and structural health monitoring (SHM), data loss sometimes occurs due to interruption of power supply, sensor failure, and disruption of data transmission, which inevitably affects subsequent data analysis and structural condition assessment. To address this issue, this paper proposes a novel data recovery method based on correlation analysis and machine learning (ML) for recovery of missing wind pressures on cladding of high-rise buildings in field measurements and SHM. The method first utilizes correlation analysis to select appropriate inputs of the ML model, and subsequently to recover the missing wind pressures using the ML model with the selected appropriate inputs. In this study, four ML models (i.e., data recovery models), including deep neural network (DNN), particle swarm optimization-DNN (PSO-DNN), extreme learning machine (ELM), and support vector regression (SVR), are developed based on the proposed data recovery method. The time series of wind pressures with non-Gaussian features, which were collected by a multi-point synchronous monitoring system on a 600-m-high skyscraper during Super Typhoon Mangkhut, are employed to validate the performance of the developed data recovery models. The results show that all the four ML models perform satisfactorily in the recovery of the missing time series of wind pressures, among which the PSO-DNN model performs best and has good recovery accuracy for data loss that occurs in different areas of building cladding. This demonstrates that the proposed data recovery method based on correlation analysis and machine learning is a reliable and effective tool to recover the missing time series of wind pressures on high-rise buildings under severe wind conditions. Notably, this is the first attempt based on field measurements to recover the missing time series of wind pressures with non-Gaussian characteristics on a skyscraper during an extreme typhoon event, which aims to provide a useful means for the recovery of missing monitoring data in field measurements and SHM.

Uncertain imputation for time-series forecasting: Application to COVID-19 daily mortality prediction.

The object of this study is to put forward uncertainty modeling associated with missing time series data imputation in a predictive context. We propose three imputation methods associated with uncertainty modeling. These methods are evaluated on a COVID-19 dataset out of which some values have been randomly removed. The dataset contains the numbers of daily COVID-19 confirmed diagnoses ("new cases") and daily deaths ("new deaths") recorded since the start of the pandemic up to July 2021. The considered task is to predict the number of new deaths 7 days in advance. The more values are missing, the higher the imputation impact is on the predictive performances. The Evidential K-Nearest Neighbors (EKNN) algorithm is used for its ability to take into account labels uncertainty. Experiments are provided to measure the benefits of the label uncertainty models. Results show the positive impact of uncertainty models on imputation performances, especially in a noisy context where the number of missing values is high.

The Impact of Missing Data and Imputation Methods on the Analysis of 24-Hour Activity Patterns.

The purpose of this study is to characterize the impact of the timing and duration of missing actigraphy data on interdaily stability (IS) and intradaily variability (IV) calculation. The performance of three missing data imputation methods (linear interpolation, mean time of day (ToD), and median ToD imputation) for estimating IV and IS was also tested. Week-long actigraphy records with no non-wear or missing timeseries data were masked with zeros or ‘Not a Number’ (NaN) across a range of timings and durations for single and multiple missing data bouts. IV and IS were calculated for true, masked, and imputed (i.e., linear interpolation, mean ToD and, median ToD imputation) timeseries data and used to generate Bland–Alman plots for each condition. Heatmaps were used to analyze the impact of timings and durations of and between bouts. Simulated missing data produced deviations in IV and IS for longer durations, midday crossings, and during similar timing on consecutive days. Median ToD imputation produced the least deviation among the imputation methods. Median ToD imputation is recommended to recapitulate IV and IS under missing data conditions for less than 24 h.

2D and 3D image reconstruction from slice data based on a constrained bilateral smoothing and dynamic mode decomposition

In this paper, we present an algebraic, graph theoretic and data-based image inpainting algorithm. The algorithm is applied to reconstruct area or volume from one-dimensional and two-dimensional slice data. Given one-dimensional or two-dimensional slice data, our algorithm first performs a simple algebraic presmoothing of the data, e.g., Gaussian filters. The second step is to obtain, from these smoothed data, an operator that reconstructs them via Dynamic Mode Decomposition. Dynamic Mode Decomposition is an equation-free modeler, which is designed to construct the discrete dynamical operator considering slice data as a time series data. We interpret the DMD as a solution to the inverse problem and apply an appropriate continuous extension of DMD to fill the missing time series data. The last step is to smooth the outcome from the second step using a Constraint Bilateral Smoother. The Constraint Bilateral Smoother is designed to smooth the boundary of area or volume, respecting the given slice data. We demonstrate the performance of the proposed algorithm, in several test cases, which include an Magnetic Resonance Imaging of a two-year-old and a Computed Tomography scan of a Covid-19 patient. We also show that a couple of test results are better than those from some competitive alternative techniques up to 50%. Our tests also show that the inpainting computation time increases linearly as a function of the number of pixels or voxels with slope just around 0.5.

Evaluation of multivariate time series clustering for imputation of air pollution data

Abstract. Air pollution is one of the world's leading risk factors for death, with 6.5 million deaths per year worldwide attributed to air-pollution-related diseases. Understanding the behaviour of certain pollutants through air quality assessment can produce improvements in air quality management that will translate to health and economic benefits. However, problems with missing data and uncertainty hinder that assessment. We are motivated by the need to enhance the air pollution data available. We focus on the problem of missing air pollutant concentration data either because a limited set of pollutants is measured at a monitoring site or because an instrument is not operating, so a particular pollutant is not measured for a period of time. In our previous work, we have proposed models which can impute a whole missing time series to enhance air quality monitoring. Some of these models are based on a multivariate time series (MVTS) clustering method. Here, we apply our method to real data and show how different graphical and statistical model evaluation functions enable us to select the imputation model that produces the most plausible imputations. We then compare the Daily Air Quality Index (DAQI) values obtained after imputation with observed values incorporating missing data. Our results show that using an ensemble model that aggregates the spatial similarity obtained by the geographical correlation between monitoring stations and the fused temporal similarity between pollutant concentrations produces very good imputation results. Furthermore, the analysis enhances understanding of the different pollutant behaviours and of the characteristics of different stations according to their environmental type.

A dual-head attention model for time series data imputation

Digital agriculture increasingly relies on the availability and accuracy of measurement data collected from various sensors. Of this data, water quality attracts great attention due to its intended use for crop irrigation, livestock, and other farming activities. Accurate and reliable water quality measurements enable farmers to understand the landscape comprehensively, optimising resource utilisation and reducing the negative impacts of agriculture on the environment. In practice, missing and incomplete data can create biased estimations and reduce the efficiency of many of the valuable applications provided by digital agriculture. The purpose of this paper is to propose a dual-head sequence-to-sequence imputation model (Dual-SSIM) designed to impute missing time series data in sensor networks, therefore reducing the negative consequences of missing and incomplete data. Unlike standard sequence-to-sequence architecture, the Dual-SSIM model features two encoders with gated recurrent units (GRUs) which are used to process temporal information before and after the missing gap separately. Furthermore, the attention mechanism is applied to two encoder outputs concurrently, in order to allow the model to focus on the high relative inputs when estimating missing data. The performance efficacy of Dual-SSIM has been investigated through the monitoring of water quality, sourced from an Australian water quality information system. Experimental results of this investigation indicate that Dual-SSIM outperforms associated alternatives based on the mean absolute error (MAE), root mean square error (RMSE), and dynamic time warping (DTW) scores in imputing two different water quality variables. Therefore, it can be concluded that Dual-SSIM provides an effective and promising approach for water quality data imputation.

ST-FVGAN: filling series traffic missing values with generative adversarial network

ABSTRACT The imputation of time series traffic flow data is of great significance to the intelligent transportation, urban planning, and road emergency handling. This paper proposes a filling missing time series traffic data with Generative Adversarial Network (ST-FVGAN), which not only considers the spatio-temporal correlation and utilizes the idea of data generating of the Generative Adversarial Network, but also considers the external factors and introduces a more comprehensive loss function. Specifically, the model firstly constructs a Generator network which is composed of convolutional layer, residual block, and pixelshuffle block for the better potential distribution of the existing data, and then use the Discriminator network for the input judging. Experiments are conducted on the open-source TaxiBJ GPS dataset, and evaluated by the root mean square error (RMSE) index. The experimental results show that our model has the better accurate and reasonable performance than the traditional imputation methods

Assessment of Fatigue Using Wearable Sensors: A Pilot Study

Background: Fatigue is a broad, multifactorial concept encompassing feelings of reduced physical and mental energy levels. Fatigue strongly impacts patient health-related quality of life across a huge range of conditions, yet, to date, tools available to understand fatigue are severely limited. Methods: After using a recurrent neural network-based algorithm to impute missing time series data form a multisensor wearable device, we compared supervised and unsupervised machine learning approaches to gain insights on the relationship between self-reported non-pathological fatigue and multimodal sensor data. Results: A total of 27 healthy subjects and 405 recording days were analyzed. Recorded data included continuous multimodal wearable sensor time series on physical activity, vital signs, and other physiological parameters, and daily questionnaires on fatigue. The best results were obtained when using the causal convolutional neural network model for unsupervised representation learning of multivariate sensor data, and random forest as a classifier trained on subject-reported physical fatigue labels (weighted precision of 0.70 ± 0.03 and recall of 0.73 ± 0.03). When using manually engineered features on sensor data to train our random forest (weighted precision of 0.70 ± 0.05 and recall of 0.72 ± 0.01), both physical activity (energy expenditure, activity counts, and steps) and vital signs (heart rate, heart rate variability, and respiratory rate) were important parameters to measure. Furthermore, vital signs contributed the most as top features for predicting mental fatigue compared to physical ones. These results support the idea that fatigue is a highly multimodal concept. Analysis of clusters from sensor data highlighted a digital phenotype indicating the presence of fatigue (95% of observations) characterized by a high intensity of physical activity. Mental fatigue followed similar trends but was less predictable. Potential future directions could focus on anomaly detection assuming longer individual monitoring periods. Conclusion: Taken together, these results are the first demonstration that multimodal digital data can be used to inform, quantify, and augment subjectively captured non-pathological fatigue measures.