Time Series Of Values Research Articles

Corrigendum to “A procedure for the detection of undocumented multiple abrupt changes in the mean value of daily temperature time series of a regional network”

Corrigendum to “A procedure for the detection of undocumented multiple abrupt changes in the mean value of daily temperature time series of a regional network”

Metaheuristic-Based Hyperparameter Tuning for Recurrent Deep Learning: Application to the Prediction of Solar Energy Generation

As solar energy generation has become more and more important for the economies of numerous countries in the last couple of decades, it is highly important to build accurate models for forecasting the amount of green energy that will be produced. Numerous recurrent deep learning approaches, mainly based on long short-term memory (LSTM), are proposed for dealing with such problems, but the most accurate models may differ from one test case to another with respect to architecture and hyperparameters. In the current study, the use of an LSTM and a bidirectional LSTM (BiLSTM) is proposed for dealing with a data collection that, besides the time series values denoting the solar energy generation, also comprises corresponding information about the weather. The proposed research additionally endows the models with hyperparameter tuning by means of an enhanced version of a recently proposed metaheuristic, the reptile search algorithm (RSA). The output of the proposed tuned recurrent neural network models is compared to the ones of several other state-of-the-art metaheuristic optimization approaches that are applied for the same task, using the same experimental setup, and the obtained results indicate the proposed approach as the better alternative. Moreover, the best recurrent model achieved the best results with R2 of 0.604, and a normalized MSE value of 0.014, which yields an improvement of around 13% over traditional machine learning models.

Combining embeddings and fuzzy time series for high-dimensional time series forecasting in internet of energy applications

High-dimensional time series increasingly arise in the Internet of Energy (IoE), given the use of multi-sensor environments and the two way communication between energy consumers and the smart grid. Therefore, methods that are capable of computing high-dimensional time series are of great value in smart building and IoE applications. Fuzzy Time Series (FTS) models stand out as data-driven non-parametric models of easy implementation and high accuracy. Unfortunately, the existing FTS models can be unfeasible if all variables were used to train the model. We present a new methodology named Embedding Fuzzy Time Series (EFTS), by applying a combination of data embedding transformation and FTS methods. The EFTS is an explainable and data-driven approach, which is flexible and adaptable for many smart building and IoE applications. The experimental results with three public datasets show that our methodology outperforms several machine learning based forecasting methods (LSTM, GRU, TCN, RNN, MLP and GBM), and demonstrates the accuracy and parsimony of the EFTS in comparison to the baseline methods and the results previously published in the literature, showing an enhancement greater than 80%. Therefore, EFTS has a great value in high-dimensional time series forecasting in IoE applications.

Graph Deep Factors for Probabilistic Time-series Forecasting

Effective time-series forecasting methods are of significant importance to solve a broad spectrum of research problems. Deep probabilistic forecasting techniques have recently been proposed for modeling large collections of time-series. However, these techniques explicitly assume either complete independence (local model) or complete dependence (global model) between time-series in the collection. This corresponds to the two extreme cases where every time-series is disconnected from every other time-series in the collection or likewise, that every time-series is related to every other time-series resulting in a completely connected graph. In this work, we propose a deep hybrid probabilistic graph-based forecasting framework called Graph Deep Factors (GraphDF) that goes beyond these two extremes by allowing nodes and their time-series to be connected to others in an arbitrary fashion. GraphDF is a hybrid forecasting framework that consists of a relational global and relational local model. In particular, a relational global model learns complex non-linear time-series patterns globally using the structure of the graph to improve both forecasting accuracy and computational efficiency. Similarly, instead of modeling every time-series independently, a relational local model not only considers its individual time-series but also the time-series of nodes that are connected in the graph. The experiments demonstrate the effectiveness of the proposed deep hybrid graph-based forecasting model compared to the state-of-the-art methods in terms of its forecasting accuracy, runtime, and scalability. Our case study reveals that GraphDF can successfully generate cloud usage forecasts and opportunistically schedule workloads to increase cloud cluster utilization by 47.5% on average. Furthermore, we target addressing the common nature of many time-series forecasting applications where time-series are provided in a streaming version; however, most methods fail to leverage the newly incoming time-series values and result in worse performance over time. In this article, we propose an online incremental learning framework for probabilistic forecasting. The framework is theoretically proven to have lower time and space complexity. The framework can be universally applied to many other machine learning-based methods.

ClaSP: parameter-free time series segmentation

The study of natural and human-made processes often results in long sequences of temporally-ordered values, aka time series (TS). Such processes often consist of multiple states, e.g. operating modes of a machine, such that state changes in the observed processes result in changes in the distribution of shape of the measured values. Time series segmentation (TSS) tries to find such changes in TS post-hoc to deduce changes in the data-generating process. TSS is typically approached as an unsupervised learning problem aiming at the identification of segments distinguishable by some statistical property. Current algorithms for TSS require domain-dependent hyper-parameters to be set by the user, make assumptions about the TS value distribution or the types of detectable changes which limits their applicability. Common hyper-parameters are the measure of segment homogeneity and the number of change points, which are particularly hard to tune for each data set. We present ClaSP, a novel, highly accurate, hyper-parameter-free and domain-agnostic method for TSS. ClaSP hierarchically splits a TS into two parts. A change point is determined by training a binary TS classifier for each possible split point and selecting the one split that is best at identifying subsequences to be from either of the partitions. ClaSP learns its main two model-parameters from the data using two novel bespoke algorithms. In our experimental evaluation using a benchmark of 107 data sets, we show that ClaSP outperforms the state of the art in terms of accuracy and is fast and scalable. Furthermore, we highlight properties of ClaSP using several real-world case studies.

Grassland Health in Xilin Gol League from the Perspective of Machine Learning—Analysis of Grazing Intensity on Grassland Sustainability

As one of the most widespread and important types of terrestrial vegetation in the world, grasslands play an irreplaceable role in global climate change. The grasslands of Inner Mongolia, represented by the Xilin Gol League, are typical of Eurasian grasslands and have an important ecological status in the world. In this paper, taking the grassland of Xilin Gol League as the research object, based on the machine learning method, we mainly carry out two aspects of work: the prediction of grassland soil health and evaluation of grassland sustainable development. To address the issue of predicting soil health in grasslands, we focus on an important indicator in grasslands: soil moisture. By analyzing the characteristics of soil moisture time series values and related influencing factors, based on a NAR neural network model, three important factors of soil moisture were predicted: soil evaporation data, average air temperature, and precipitation. Subsequently, the corresponding soil moisture calculation model was trained using regression models based on hyperparameter optimization, and the final predicted soil moisture values were obtained for different months and depths in 2023 and 2024. To evaluate the sustainability of grassland development, we developed a model for the degree of grassland desertification based on the kernel principal component analysis, focusing on three dimensions: environmental factors, surface factors, and human factors. Based on this, a quantitative definition of soil denudation is given by analyzing the main influencing factors of grassland soil degradation. At the same time, a prediction model for the evaluation of soil slumping was established based on a fuzzy comprehensive evaluation matrix, and the evaluation weights of each major factor were given and analyzed. Based on the above research, this paper suggests a reasonable grazing strategy for the grassland areas of the Xilin Gol League: when the grazing intensity is medium and the total number of grazing days is [85, 104] days in a year, the degree of soil slumping and soil desertification in the pastures is minimized. The research results of this paper are useful for the future maintenance and management of the grasslands of Xilin Gol League and other similar areas.

Multivariate time series imputation for energy data using neural networks

Multivariate time series with missing values are common in a wide range of applications, including energy data. Existing imputation methods often fail to focus on the temporal dynamics and the cross-dimensional correlation simultaneously. In this paper we propose a two-step method based on an attention model to impute missing values in multivariate energy time series. First, the underlying distribution of the missing values in the data is learned. This information is then further used to train an attention based imputation model. By learning the distribution prior to the imputation process, the model can respond flexibly to the specific characteristics of the underlying data. The developed model is applied to European energy data, obtained from the European Network of Transmission System Operators for Electricity. Using different evaluation metrics and benchmarks, the conducted experiments show that the proposed model is preferable to the benchmarks and is able to accurately impute missing values.

Prediction of Angina Pectoris Events in Middle-Aged and Elderly People Using RR Interval Time Series in the Resting State: A Cohort Study Based on SHHS

Angina pectoris is associated with adverse cardiovascular events. In this study, a Bi-directional Long Short-Term Memory (Bi-LSTM) prediction model with the Attention layer was established to explore the predictive value of the resting-state RR interval time series on the occurrence of angina pectoris. The data of this cohort study were from the Sleep Heart Health Study database, 2,977 people were included with the follow-up of 15 years. We used the RR interval time series of electrocardiogram signals in the resting state. The outcome variables were any angina events during the follow-up. We randomly divided 2,977 participants into training (n = 2680) and testing sets (n = 297) with a partition ratio of 9:1. The prediction model of Bi-LSTM with Attention layer was developed and the predictive performance was assessed. 1,236 had angina pectoris and 1,741 patients did not have angina pectoris during the follow-up period. The predictive performance of the Bi-LSTM model was great with the value of accuracy = 0.913, area under the curve (AUC) = 0.922, precision = 0.913 in the testing set. RR intervals may be the potential predictors of angina events. It is more and more convenient to obtain heart rate with the development of wearable devices; the Bi-LSTM prediction model established in this study is expected to provide support for the intelligent prediction of angina pectoris events.

Data fusion with factored quantization for stock trend prediction using neural networks

As compared to the continuous temporal distributions, discrete data representations may be desired for simplified and faster data analysis and forecasting. Data compression can introduce one of the efficient ways to reduce continuous historical stock market data and present them in discrete forms; while predicting stock trend, a primary concern is towards up and down directions of the price movement and thus, data discretization for a focused approach can be beneficial. In this article, we propose a quantization-based data fusion approach with a primary motivation to reduce data complexity and hence, enhance the prediction ability of a model. Here, the continuous time-series values are transformed into discrete quantum values prior to applying them to a prediction model. We extend the proposed approach and factorize quantization by integrating different quantization step sizes. Such fused data can reduce the data to mainly concentrate on the stock price movement direction. To empirically evaluate the proposed approach for stock trend prediction, we adopt long short-term memory, deep neural network, and backpropagation neural network models and compare our prediction results with five existing approaches on several datasets using ten performance metrics. We analyze the impact of specific quantization factors and determine the individual best as well as overall best factor sizes; the results indicate a consistent performance enhancement in stock trend prediction accuracy as compared to the considered baseline methods with an improvement up to 7%. To evaluate the impact of quantization-based data fusion, we analyze time required to execute the experiments along with percentage reduction in the number of unique numeric terms. Further, these results are statistically evaluated using Wilcoxon signed-rank test. We discuss the superiority and applicability of factored quantization-based data fusion approach and conclude our work with potential future research directions.

Anomaly Detection and Repairing for Improving Air Quality Monitoring.

Clean air in cities improves our health and overall quality of life and helps fight climate change and preserve our environment. High-resolution measures of pollutants' concentrations can support the identification of urban areas with poor air quality and raise citizens' awareness while encouraging more sustainable behaviors. Recent advances in Internet of Things (IoT) technology have led to extensive use of low-cost air quality sensors for hyper-local air quality monitoring. As a result, public administrations and citizens increasingly rely on information obtained from sensors to make decisions in their daily lives and mitigate pollution effects. Unfortunately, in most sensing applications, sensors are known to be error-prone. Thanks to Artificial Intelligence (AI) technologies, it is possible to devise computationally efficient methods that can automatically pinpoint anomalies in those data streams in real time. In order to enhance the reliability of air quality sensing applications, we believe that it is highly important to set up a data-cleaning process. In this work, we propose AIrSense, a novel AI-based framework for obtaining reliable pollutant concentrations from raw data collected by a network of low-cost sensors. It enacts an anomaly detection and repairing procedure on raw measurements before applying the calibration model, which converts raw measurements to concentration measurements of gasses. There are very few studies of anomaly detection in raw air quality sensor data (millivolts). Our approach is the first that proposes to detect and repair anomalies in raw data before they are calibrated by considering the temporal sequence of the measurements and the correlations between different sensor features. If at least some previous measurements are available and not anomalous, it trains a model and uses the prediction to repair the observations; otherwise, it exploits the previous observation. Firstly, a majority voting system based on three different algorithms detects anomalies in raw data. Then, anomalies are repaired to avoid missing values in the measurement time series. In the end, the calibration model provides the pollutant concentrations. Experiments conducted on a real dataset of 12,000 observations produced by 12 low-cost sensors demonstrated the importance of the data-cleaning process in improving calibration algorithms' performances.

Estimation of daily reference evapotranspiration by hybrid singular spectrum analysis-based stochastic gradient boosting

In this study, stochastic gradient boosting (SGB), a commonly-adopted soft computing method, was used to estimate reference evapotranspiration (ETo) for the Adiyaman region of southeastern Türkiye. The FAO-56-Penman-Monteith method was used to calculate ETo, which we then estimated using SGB with maximum temperature, minimum temperature, relative humidity, wind speed, and solar radiation obtained from a meteorological station.•The calculated ETo time series values were decomposed into sub-series using Singular Spectrum Analysis (SSA) to enhance prediction accuracy.•Each sub-series was trained with the first 70% of observations and tested with the remaining 30% via SGB. Final prediction values were obtained by collecting all series predictions.•Three lag times were taken into account during the predictions, and both short-term and long-term ETo values were estimated using the proposed framework. The results were tested with respect to root mean square error (RMSE) and Nash-Sutcliffe efficiency (NSE) indicators for ensuring whether the model produced statically acceptable outcomes.

Experimental System-Dynamic Model of an Influence of a Level of Education on a Spatial Differentiation of a Population of Russian Regions

The study is devoted to the problem of the spatial differentiation of population income in Russia's regions. The objective of the study is the development of a system-dynamic model for the calculation of spatial differentiation trajectories of income parameters due to various scenarios. Regional features of the human capital development level influencing the spatial differentiation of population income in the Russian regions are assumed. The formulation of the mathematical problem is based on the results of regression analysis of the influence of time series values of socio-demographic factors on population incomes differentiation (Gini coefficient). A specialist application, PowerSim Studio Express 10, was used for model design. Rosstat data on households in the Russian regions were used for calculation of the model experimental trajectories. The main research methods are the following: dynamic analysis of time series; econometric and system-dynamic modeling. As a result of the study, a system-dynamics experimental model was proposed. It was tested in application to eight regions with the most reliable statistical relationship between socio-demographic factors and the Gini index. Numerical experiments were used to simulate real economic processes of convergence and divergence in order to identify the main trends and features of territorial income differentiation depending on local priorities of vocational education development. It was shown that an increase in the level of education, both higher professional and secondary vocational, mainly leads to increased income differentiation. The theoretical significance of the results obtained lies in the deepening of the understanding of the regional features of human capital development influencing the spatial differentiation of population income. The practical significance of the study lies in the expansion of instrumental support for decision-making in the implementation of state policy in the field of regulating the population incomes differentiation at the regional level.

A Comparison of a Radial Basis Function Neural Network with other Methods for Estimating Missing Values in Univariate Time Series

Missing values in the time-series data set have an impact on the correct decision-making in the future. Since complete data helps to obtain high accuracy in the estimation process, the reason for missing values is a malfunction of the measuring device or an error in the data entry process by the person. The research aims to compare the radial basis function methods with other methods to estimate missing values in univariate time series data. the simulation method was used to compare the methods to estimate the missing values, and that was used the Box-Jenkins model AR(1) once with the value of and once with the value of and with different sample sizes (60,100,300), assuming four percentages of missing from data values are missing at random MAR (5%,10%,15%,20%). The accuracy of the estimation of the methods was evaluated by using the standard of accuracy, the mean sum of squares error (MSE). from the results obtained using simulation, it was found that the(RBF) method is the best method for estimating the missing values in both values, in all sizes, and all loss ratios because it produces the lowest value of the average square error compared to other methods.
 
 Paper type: Research paper

Recent Trends of Mean Maximum and Minimum Air Temperatures in Bangladesh

The annual, seasonal, and monthly maximum and minimum air temperature time series for 34 stations in Bangladesh have been examined during the period 1981– 2020 in this study. Three statistical tests were used in the analysis namely Mann–Kendall, Sen’s slope estimator, and linear regression, respectively. 47% of stations experienced significantly rising maximum air temperature trends during the pre-monsoon, 100% during the monsoon, 74% during the post-monsoon, and 24% during the winter season, respectively. Similar findings have been made about minimum air temperatures during the pre-monsoon, monsoon, post-monsoon, and winter seasons in 41, 91, 41, and 35% of stations over Bangladesh. According to the linear regression technique, the significant increasing trends of the maximum air temperatures at Patuakhali and Mongla stations are 0.83°C/decade and 0.82°C/decade in February and October, respectively. Furthermore, at Sandwip and M. Court, the significant increasing minimum air temperature trends in July and March are 0.26°C/decade and 0.66°C/decade, respectively. The highest monthly maximum and minimum air temperature time series value falls (decreases) were seen in January over the stations at Mongla (-0.58°C/decade) and Sandwip (-0.63°C/decade), respectively. This information will be useful in developing adaptation plans to mitigate the harmful effects of climate change. Vulnerability assessments, disaster management, enhanced structure design, institutional reform, and antiextreme climate engineering are some of the viable climate change adaptation approaches in Bangladesh due to rising temperatures.

Anomaly Detection of Consumption in Hotel Units: A Case Study Comparing Isolation Forest and Variational Autoencoder Algorithms

Buildings are responsible for a high percentage of global energy consumption, and thus, the improvement of their efficiency can positively impact not only the costs to the companies they house, but also at a global level. One way to reduce that impact is to constantly monitor the consumption levels of these buildings and to quickly act when unjustified levels are detected. Currently, a variety of sensor networks can be deployed to constantly monitor many variables associated with these buildings, including distinct types of meters, air temperature, solar radiation, etc. However, as consumption is highly dependent on occupancy and environmental variables, the identification of anomalous consumption levels is a challenging task. This study focuses on the implementation of an intelligent system, capable of performing the early detection of anomalous sequences of values in consumption time series applied to distinct hotel unit meters. The development of the system was performed in several steps, which resulted in the implementation of several modules. An initial (i) Exploratory Data Analysis (EDA) phase was made to analyze the data, including the consumption datasets of electricity, water, and gas, obtained over several years. The results of the EDA were used to implement a (ii) data correction module, capable of dealing with the transmission losses and erroneous values identified during the EDA’s phase. Then, a (iii) comparative study was performed between a machine learning (ML) algorithm and a deep learning (DL) one, respectively, the isolation forest (IF) and a variational autoencoder (VAE). The study was made, taking into consideration a (iv) proposed performance metric for anomaly detection algorithms in unsupervised time series, also considering computational requirements and adaptability to different types of data. (v) The results show that the IF algorithm is a better solution for the presented problem, since it is easily adaptable to different sources of data, to different combinations of features, and has lower computational complexity. This allows its deployment without major computational requirements, high knowledge, and data history, whilst also being less prone to problems with missing data. As a global outcome, an architecture of a platform is proposed that encompasses the mentioned modules. The platform represents a running system, performing continuous detection and quickly alerting hotel managers about possible anomalous consumption levels, allowing them to take more timely measures to investigate and solve the associated causes.

Development of a simplified technique for gap filling of Normalize Difference Vegetation Index (NDVI) time series data

The presence of gaps or missing values in time series prevents the practical use of such data. The current research aims at developing a simplified, straightforward technique for gap-filling the time series data of the Normalize Difference Vegetation Index (NDVI) generated using Moderate Resolution Imaging Spectroradiometer (MODIS). This research assumes that a relationship exists between the pixel location, date of acquisition and its NDVI value within a defined timeline. Therefore, two relatively simple methods were tested: the Multiple Linear Regression (MLR) analysis and the Artificial Neural Networks (ANN)to fill the NDVI missing values. While MLR is a well-known simple statistical method, the ANN has been successfully applied for the analysis of various scientific data, including the gap-filling of time series data. Nevertheless, ANN proved its supremacy in such approach. The accuracy of estimation utilizing the developed ANN model reached an average of r2 of 0.8, while the average accuracy of MLR was about 0.3. Nevertheless, the developed model could only be applied within the same timeframe of the images used for developing the model. Otherwise, the accuracy of determination was reduced significantly. The results showed that according to its performance, ANN are promising for filling missing data of NDVI time series and could be applied to any other vegetation indices as well.

Attention-Based Sequence-to-Sequence Model for Time Series Imputation.

Time series data are usually characterized by having missing values, high dimensionality, and large data volume. To solve the problem of high-dimensional time series with missing values, this paper proposes an attention-based sequence-to-sequence model to imputation missing values in time series (ASSM), which is a sequence-to-sequence model based on the combination of feature learning and data computation. The model consists of two parts, encoder and decoder. The encoder part is a BIGRU recurrent neural network and incorporates a self-attentive mechanism to make the model more capable of handling long-range time series; The decoder part is a GRU recurrent neural network and incorporates a cross-attentive mechanism into associate with the encoder part. The relationship weights between the generated sequences in the decoder part and the known sequences in the encoder part are calculated to achieve the purpose of focusing on the sequences with a high degree of correlation. In this paper, we conduct comparison experiments with four evaluation metrics and six models on four real datasets. The experimental results show that the model proposed in this paper outperforms the six comparative missing value interpolation algorithms.

A New Interval Type-2 Fuzzy Aggregation Approach for Combining Multiple Neural Networks in Clustering and Prediction of Time Series

Inspired by how some cognitive abilities affect the human decision-making process, the proposed approach combines neural networks with type-2 fuzzy systems. The proposal consists of combining computational models of artificial neural networks and fuzzy systems to perform clustering and prediction of time series corresponding to the population, urban population, particulate matter (PM2.5), carbon dioxide (CO2), registered cases and deaths from COVID-19 for certain countries. The objective is to associate these variables by country based on the identification of similarities in the historical information for each variable. The hybrid approach consists of computationally simulating the behavior of cognitive functions in the human brain in the decision-making process by using different types of neural models and interval type-2 fuzzy logic for combining their outputs. Simulation results show the advantages of the proposed approach, because starting from an input data set, the artificial neural networks are responsible for clustering and predicting values of multiple time series, and later a set of fuzzy inference systems perform the integration of these results, which the user can then utilize as a support tool for decision-making with uncertainty.

Neural ODE Differential Learning and Its Application in Polar Motion Prediction

AbstractThis paper introduces a new learning algorithm for accurate, physically driven time series prediction. The fundamental assumption behind the method is that the phenomena follow Ordinary Differential Equations. We investigate the general case where the time series follows an ODE of degree . The resulting method is a learning algorithm based on the finite differences between the values of time series. We present the application of the method in the field of geodesy for polar motion prediction, the main objective of the present paper. We show that in this application, the linear form of the method is sufficient and offers competitive predictive performance. We present a baseline solution, in which we use historical polar motion time series from 1976 to predict up to the year 2020. The prediction horizon in this case is short‐term (up to 10 days into the future). In addition, we compare the prediction accuracy in the short‐term horizon with the results of the best performing model in the first Earth Orientation Prediction Comparison Campaign. On average, a 53% improvement in prediction performance is achieved. In further analyses, we compare the prediction accuracy for both short‐term and long‐term against the results of state‐of‐the‐art methods, namely Multichannel Singular Spectrum Analysis, and a combination of Singular Spectrum Analysis and Copula sampling. We show that the proposed method in this paper can outperform the mentioned two methods in both short and long‐term horizons, with an average improvement of the prediction performance of 54% and 52%, respectively.

A mixed distribution to fix the threshold for Peak-Over-Threshold wave height estimation

Modelling extreme values distributions, such as wave height time series where the higher waves are much less frequent than the lower ones, has been tackled from the point of view of the Peak-Over-Threshold (POT) methodologies, where modelling is based on those values higher than a threshold. This threshold is usually predefined by the user, while the rest of values are ignored. In this paper, we propose a new method to estimate the distribution of the complete time series, including both extreme and regular values. This methodology assumes that extreme values time series can be modelled by a normal distribution in a combination of a uniform one. The resulting theoretical distribution is then used to fix the threshold for the POT methodology. The methodology is tested in nine real-world time series collected in the Gulf of Alaska, Puerto Rico and Gibraltar (Spain), which are provided by the National Data Buoy Center (USA) and Puertos del Estado (Spain). By using the Kolmogorov-Smirnov statistical test, the results confirm that the time series can be modelled with this type of mixed distribution. Based on this, the return values and the confidence intervals for wave height in different periods of time are also calculated.