Time Series Prediction Method Research Articles

A common feature-driven prediction model for multivariate time series data

Multivariate time series data contain a variety of common features that are difficult to extract, among which the sudden irregular fluctuation trend, the trend feature of large fluctuation amplitude, and the long-term dependency relationship in the time series have an important impact on the accuracy of the prediction model. To predict non-stationary trends in large-scale data accurately using the common characteristics of multivariate time series. A novel generalised forecasting model NeA3L based on common features of time series is developed. The NeA3L model utilizes multiple independent parallel feature extraction modules to obtain the common features of multivariate time series. It utilizes the three-layer iterative structure to deal with sudden irregular fluctuation patterns. NeA3L optimizes the network structure to realize the heterogeneous codec structure. It applies the attention mechanism between the encoder and the decoder to accomplish the multivariate prediction of the multivariate time series, which has good stability for predicting various types of multivariate data. A comparison of the NeA3L model with nine current time series prediction methods on four publicly available datasets shows that the NeA3L model outperforms the current methods in various evaluation metrics. The RMSE is improved by 2.3 %, 26.5 %, 28.9 %, and 20.84 % on average, respectively. The NeA3L model can be universally applicable to the field of multivariate time series prediction, which is important for optimizing the intelligent management and decision-making.

A TCN-Linear Hybrid Model for Chaotic Time Series Forecasting.

The applications of deep learning and artificial intelligence have permeated daily life, with time series prediction emerging as a focal area of research due to its significance in data analysis. The evolution of deep learning methods for time series prediction has progressed from the Convolutional Neural Network (CNN) and the Recurrent Neural Network (RNN) to the recently popularized Transformer network. However, each of these methods has encountered specific issues. Recent studies have questioned the effectiveness of the self-attention mechanism in Transformers for time series prediction, prompting a reevaluation of approaches to LTSF (Long Time Series Forecasting) problems. To circumvent the limitations present in current models, this paper introduces a novel hybrid network, Temporal Convolutional Network-Linear (TCN-Linear), which leverages the temporal prediction capabilities of the Temporal Convolutional Network (TCN) to enhance the capacity of LSTF-Linear. Time series from three classical chaotic systems (Lorenz, Mackey-Glass, and Rossler) and real-world stock data serve as experimental datasets. Numerical simulation results indicate that, compared to classical networks and novel hybrid models, our model achieves the lowest RMSE, MAE, and MSE with the fewest training parameters, and its R2 value is the closest to 1.

Research on Vegetable Pricing and Replenishment Based on LSTM Time Series Prediction Method

Research on Vegetable Pricing and Replenishment Based on LSTM Time Series Prediction Method

Successive model-agnostic meta-learning for few-shot fault time series prognosis

Meta learning is a promising technique for solving few-shot fault prediction problems, which have attracted the attention of many researchers in recent years. Existing meta-learning methods for time series prediction, which predominantly rely on random and similarity matching-based task partitioning, facing three major limitations: (1) feature exploitation inefficiency; (2) suboptimal task data allocation; and (3) limited robustness with small samples. To this end, we introduce a novel‘meta-task’ partitioning scheme, underpinned by a differential autoregressive algorithm, that treats a continuous time period of a time series as a meta-task, composed of multiple successive short time periods. This approach allows us to extract more comprehensive features and relationships from the data, resulting in more accurate predictions. Furthermore, our findings indicate that the differential autoregressive approach significantly bolsters robustness across diverse datasets. Extensive experiments on several fault and time series prediction datasets demonstrate that our approach substantially enhances prediction performance and generalization capability under both few-shot and general conditions.

Prediction of Parking Space Availability Using Improved MAT-LSTM Network

The prediction of parking space availability plays a crucial role in information systems providing parking guidance. However, controversy persists regarding the efficiency and accuracy of mainstream time series prediction methods, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). In this study, a comparison was made between a temporal convolutional network (TCN) based on CNNs and a long short-term memory (LSTM) network based on RNNs to determine an appropriate baseline for predicting parking space availability. Subsequently, a multi-head attention (MAT) mechanism was incorporated into an LSTM network, attempting to improve its accuracy. Experiments were conducted on three real and two synthetic datasets. The results indicated that the TCN achieved the fastest convergence, whereas the MAT-LSTM method provided the highest average accuracy, namely 0.0330 and 1.102 × 10−6, on the real and synthetic datasets, respectively. Furthermore, the improved MAT-LSTM model accomplished an increase of up to 48% in accuracy compared with the classic LSTM model. Consequently, we concluded that RNN-based networks are better suited for predicting long-time series. In particular, the MAT-LSTM method proposed in this study holds higher application value for predicting parking space availability with a higher accuracy.

Sea Surface Temperature Prediction in China Sea Based on SAM-LSTM Approach

Sea Surface Temperature (SST), a critical environmental element in the ocean, significantly impacts the global atmosphere-ocean energy balance and holds the potential to trigger severe weather like droughts, floods, and El Niño events. Therefore, the prediction of future SST dynamics is crucial to identifying these extreme events and mitigating the damage they caused. In this study, we introduce a time series prediction method based on the Self-Attention Mechanism-Long Short-Term Memory (SAM-LSTM) model. In addition, the historical time-series satellite data of SST anomaly (SSTA) is used instead of SST itself considering that the fluctuations of SST are very small compared to their absolute magnitudes. The Seasonal-Trend decomposition using Loess (STL) method is adopted to decompose the complex non-linear SSTA time series into trend components, seasonal components, and residual components. Then, the deseasonalized time series data at 6 locations in the Bohai Sea are used to train and valid the developed SAM-LSTM model. After that, the validated models are applied to the Yellow Sea, East China Sea, and South China Sea. The experimental results show that the combination of STL time series decomposition and SAM-LSTM can achieve high-precision prediction of daily SSTA than LSTM. This suggests that the methodology used in this paper has a good application for short-term daily SST prediction.

Comment on papers using machine learning for significant wave height time series prediction: Complex models do not outperform auto-regression

Significant Wave Height (SWH) is crucial in many aspect of ocean engineering. The accurate prediction of SWH has therefore been of immense practical value. Recently, Artificial Intelligence (AI) time series prediction methods have been widely used for single-point short-term SWH time-series forecasting, resulting in many AI-based models claiming to achieve good results. However, the extent to which these complex AI models can outperform traditional methods has largely been overlooked. This study compared five different models - AutoRegressive (AR), eXtreme Gradient Boosting (XGB), Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), and WaveNet - for their performance on SWH time series prediction at 16 buoy locations. Surprisingly, the results suggest that the differences of performance among different models are negligible, indicating that all these AI models have only “learned” the linear auto-regression from the data. Additionally, we noticed that many recent studies used signal decomposition method for such time series prediction, and most of them decomposed the test sets, which is WRONG.

Learning from Polar Representation: An Extreme-Adaptive Model for Long-Term Time Series Forecasting

In the hydrology field, time series forecasting is crucial for efficient water resource management, improving flood and drought control and increasing the safety and quality of life for the general population. However, predicting long-term streamflow is a complex task due to the presence of extreme events. It requires the capture of long-range dependencies and the modeling of rare but important extreme values. Existing approaches often struggle to tackle these dual challenges simultaneously. In this paper, we specifically delve into these issues and propose Distance-weighted Auto-regularized Neural network (DAN), a novel extreme-adaptive model for long-range forecasting of stremflow enhanced by polar representation learning. DAN utilizes a distance-weighted multi-loss mechanism and stackable blocks to dynamically refine indicator sequences from exogenous data, while also being able to handle uni-variate time-series by employing Gaussian Mixture probability modeling to improve robustness to severe events. We also introduce Kruskal-Wallis sampling and gate control vectors to handle imbalanced extreme data. On four real-life hydrologic streamflow datasets, we demonstrate that DAN significantly outperforms both state-of-the-art hydrologic time series prediction methods and general methods designed for long-term time series prediction.

A process knowledge-based hybrid method for univariate time series prediction with uncertain inputs in process industry

In process industry, accurate and efficient univariate time series prediction (TSP) of key process parameters is crucial for optimizing production processes. However, noisy data and varying working conditions introduce uncertainties that hinder the modeling process. Data-driven methods have been proposed to address input-uncertain TSP models, but they often fail to obtain sufficient and high-quality industrial data for model establishment in complex and dynamic production process. The paper presents a process knowledge-based hybrid method for univariate TSP that handles noisy inputs in different working conditions. Primarily, a knowledge-embedded deep kernel network (KE-DKN) model is proposed that approximates the outputs to the knowledge space in noisy industrial environment, by embedding instantiated process knowledge as soft constraints into the conventional DKN network. Further, a two-stage training schema is designed for KE-DKN to reduce the interference superposition on the predicted variable from random noisy data and operational interferences. Moreover, the pre-trained parameters transferring is conducted from stable working conditions to mitigate the low modeling efficiency in unstable working conditions. In two cases from process industry, the proposed method delivers higher prediction accuracy and robustness than baseline methods in both working conditions, and the knowledge transfer significantly enhances the modeling efficiency in unstable working conditions.

Assessing and predicting indoor environmental quality in 13 naturally ventilated urban residential dwellings

This research presents the assessments of IEQ in 13 naturally ventilated urban residential dwellings based on the long-term monitoring of CO2, PM2.5, formaldehyde, temperature, and relative humidity within a year. In addition to the costly monitoring techniques, we utilized time series prediction methods as an economical and efficient approach to comprehend the quality of indoor environment. Nevertheless, choosing a prediction method proves to be a challenge due to the inconclusive nature of the advantages and disadvantages associated with various methodologies. Thus, we conducted a comprehensive comparison of the LSTM, SVM, ARIMA, and BPNN algorithms for predicting IEQ. The results indicate that for short-term prediction (one day), the SVM algorithm has the shortest fitting time, the program occupies the least memory, and the overfitting problem is not readily apparent, while the prediction accuracy is acceptable. Long-term (one year) predictions can be executed with greater precision using LSTM. In regard to robustness, prediction time and accuracy of SVM in different dwellings varies greatly and the robustness is weak. When considering parameters, the discrepancy between the prediction error of PM2.5 and the sensor error is more pronounced, resulting in an even lower level of predictability. As for the input data, it was observed that the computation time of the four algorithms increased linearly with the sample size. While the accuracy of predictions is predominantly influenced by the sampling frequency of the original data. The investigation results will benefit future decision-making and control strategies regarding the enhancement of thermal comfort and indoor air quality.

Multistep Wind Speed Prediction Based on the Fractional‐Order Longicorn Swarm Algorithm and the Two‐Stage Modal Decomposition Strategy

Least squares support vector machine (LSSVM) and variational mode decomposition are common research methods for wind speed time series prediction. Addressing the challenge of selecting relevant parameters of LSSVM and VMD, a fractional‐order Beetle swarm optimization algorithm is proposed to optimize the relevant parameters. To weaken the negative impact of wind speed volatility on the accuracy of the prediction model, a two‐stage modal decomposition strategy based on extreme‐point symmetric mode decomposition, FO‐BSO algorithm, and VMD is proposed. ESMD is used to decompose the original wind speed time series. The diversity entropy is introduced as the criterion, and the FO‐BSO‐VMD secondary decomposition is applied for the high‐entropy modal components to further weaken the volatility of wind speed. Simulation results show that compared with the original LSSVM model, the ESMD‐FO‐BSO‐VMD‐LSSVM model presented in this study exhibits an average 44.10% increase in the fitting degree of wind speed prediction over three steps, which verifies the superior performance of the model in short‐term multistep wind speed prediction.

A Time Series Prediction Method for the Subsurface Thermal Structure of the South Yellow Sea Cold Water Mass

A Time Series Prediction Method for the Subsurface Thermal Structure of the South Yellow Sea Cold Water Mass

Deep Learning for Time-Series Prediction in IIoT: Progress, Challenges, and Prospects.

Time-series prediction plays a crucial role in the Industrial Internet of Things (IIoT) to enable intelligent process control, analysis, and management, such as complex equipment maintenance, product quality management, and dynamic process monitoring. Traditional methods face challenges in obtaining latent insights due to the growing complexity of IIoT. Recently, the latest development of deep learning provides innovative solutions for IIoT time-series prediction. In this survey, we analyze the existing deep learning-based time-series prediction methods and present the main challenges of time-series prediction in IIoT. Furthermore, we propose a framework of state-of-the-art solutions to overcome the challenges of time-series prediction in IIoT and summarize its application in practical scenarios, such as predictive maintenance, product quality prediction, and supply chain management. Finally, we conclude with comments on possible future directions for the development of time-series prediction to enable extensible knowledge mining for complex tasks in IIoT.

Research on Ensemble Learning-Based Feature Selection Method for Time-Series Prediction

Feature selection has perennially stood as a pivotal concern in the realm of time-series forecasting due to its direct influence on the efficacy of predictive models. Conventional approaches to feature selection predominantly rely on domain knowledge and experiential insights and are, therefore, susceptible to individual subjectivity and the resultant inconsistencies in the outcomes. Particularly in domains such as financial markets, and within datasets comprising time-series information, an abundance of features adds complexity, necessitating adept handling of high-dimensional data. The computational expenses associated with traditional methodologies in managing such data dimensions, coupled with vulnerability to the curse of dimensionality, further compound the challenges at hand. In response to these challenges, this paper advocates for an innovative approach—a feature selection method grounded in ensemble learning. The paper explicitly delineates the formal integration of ensemble learning into feature selection, guided by the overarching principle of “good but different”. To operationalize this concept, five feature selection methods that are well suited to ensemble learning were identified, and their respective weights were determined through K-fold cross-validation when applied to specific datasets. This ensemble method amalgamates the outcomes of diverse feature selection techniques into a numeric composite, thereby mitigating potential biases inherent in traditional methods and elevating the precision and comprehensiveness of feature selection. Consequently, this method improves the performance of time-series prediction models.

An optimization in big data time series prediction method by Parzen estimation with Spark

With the development and change of big data related technologies, more and more large amounts of data need to be analyzed. Now there are companies like Google, Yahoo, etc. Frameworks such as MapReduce, Hadoop, Spark, etc. are developed for processing large amounts of data. In this paper, relevant discussions and researches are carried out on time series forecasting under the new era of big data. Now there are time series forecasting methods based on map reduce, Hadoop, spark data processing framework, including nearest neighbor distribution method, neural network method, etc., which have made quite good achievements in big data time series forecasting. By reading the relevant research literature, it is universally acknowledged that the Sparks framework has good application prospects and potential in predicting big data time series. As a result, this paper is mainly aimed at the optimization and improvement of the big data time series forecasting method on the basis of the spark framework. The author noticed that most of the default configurations of spark clusters are generated by default or automatically, rather than the optimal solution obtained after algorithm optimization, so there is still room for improvement in this regard. In this regard, this paper proposes a kernel method for visual data processing of related configurations and parameters, and then optimizes the default data configuration as much as possible to improve the accuracy and feasibility of the big data time series prediction method on the basis of the spark framework. In this paper, the optimized scheme is used to forecast the domestic electricity consumption in the past five years, and the results show that the optimized scheme has a good improvement performance on the basis of the original method.

A WSFA-based adaptive feature extraction method for multivariate time series prediction

A WSFA-based adaptive feature extraction method for multivariate time series prediction

Modeling of Soil Moisture Data by ARMA Time Series

The use of known data to predict future environmental parameters plays a crucial role in agriculture. In this paper, we propose a novel time series prediction method that combines the Auto-Regressive Moving Average Model (ARMA) and Gradient Boost Regression Tree(GBR) to forecast future soil moisture values. Firstly, the optimal number of decomposition modes for AMRA is determined by using Auto-correlation Function (ACF) and Partial Auto-correlation Function (PACF) plots. Secondly, according to statistics of XIlin Gol grassland offered by the Huawei Cup Mathematical Modeling Contest in 2022, the data including soil evaporation, precipitation, and soil moisture in the past ten years, are used as input parameters of ARMA to predict the precipitation and soil evaporation from 2022 to 2023. Then, the superiority of GBR was verified by comparing algorithms such as Support Vector Regression (SVR) and Random Forest(RF). Finally, GBR was used to realize the prediction for different soil moisture values from 2022 to 2023.

Decomposition strategy and attention-based long short-term memory network for multi-step ultra-short-term agricultural power load forecasting

Accurate forecasting of the agricultural power load is important for rural electricity networks’ safe and stable operation. But it is more uncertain and difficult to forecast than industrial, commercial, and residential loads. Multivariate time series prediction methods have problems such as poor refinement of weather services and difficulty in quantifying the influencing factors. Univariate time series prediction methods should be emphasized, and the characteristic information in the historical load data should be fully exploited. Therefore, this paper systematically investigates the decomposition strategy and attention-based recurrent neural network for univariate time-series multi-step prediction. The decomposition strategy is employed to generate derived variable sequences from the target sequence as inputs for the model, while the dual-stage attention mechanism is used for dynamically selecting features. A one-year real-world dataset of agricultural power loads in southern China is used for validation. The effectiveness of two different decomposition algorithms on dual-stage attention-based recurrent neural network (DARNN) is discussed, then the proposed ensemble empirical mode decomposition and dual-stage attention-based recurrent neural network (EEMD-DARNN) is compared with four univariate time series prediction models and three ablation models. The proposed model is optimal in four evaluation indicators, with R2, RMSE, MAPE, and MAE of 0.970, 12.298, 0.040, and 7.335 respectively in the shortest forecast horizon (15 min), and 0.941, 17.305, 0.065 and 11.347 respectively in the longest forecast horizon (1 h). It is a significant improvement over the existing models. In conclusion, EEMD-DARNN can achieve more accurate ultra-short-term agricultural power load forecasting.

Multiple feature fusion transformer for modeling penicillin fermentation process with unequal sampling intervals.

The quality prediction of batch processes is an important task in the field of biological fermentation. However, dynamic nonlinearity, unequal sampling intervals, uneven duration, and multiple features of a batch process make this task challenging. Thus, the multiple-feature fusion transformer (MFFT) model is proposed for the time series quality prediction of a batch process. First, the application of sequence-to-sequence architecture enables MFFT to perform a wide range of sequence prediction tasks. Second, the transformer parallel operation model imposes no rigid requirement for the order of sequence input, allowing the model to deal with problems of unequal interval sampling and utilize the sequence information. Third, MFFT integrates a pretrained ResNet50 as a mycelium status classifier for fusing image information into the features. Moreover, a multiple-feature encoding structure is proposed to integrate sampling time and mycelium status. Finally, multiple tasks in penicillin fermentation have shown that MFFT significantly outperforms existing methods for time series prediction.

Seepage Prediction Model for Roller-Compacted Concrete Dam Using Support Vector Regression and Hybrid Parameter Optimization

In this study, a seepage prediction model was established for roller-compacted concrete dams using support vector regression (SVR) with hybrid parameter optimization (HPO). The model includes data processing via HPO and machine learning through SVR. HPO benefits from the correlation extraction capability of grey relational analysis and the dimensionality reduction technique of principal component analysis. The proposed model was trained, validated, and tested using 22 years of monitoring data regarding the Shuidong Dam in China. We compared the performance of HPO with other popular methods, while the SVR method was compared with the traditional time-series prediction method of long short-term memory (LSTM). Our findings reveal that the HPO method proves valuable real-time dam safety monitoring during data processing. Meanwhile, the SVR method demonstrates superior robustness in predicting seepage flowrate post-dam reinforcement, compared with LSTM. Thus, the developed model effectively identifies the factors related to seepage and exhibits high accuracy in predicting fluctuation trends regarding the Shuidong Dam, achieving a determination coefficient R2 > 0.9. Further, the model can provide valuable guidance for dam safety monitoring, including diagnosing the efficacy of monitoring parameters or equipment, evaluating equipment monitoring frequency, identifying locations sensitive to dam seepage, and predicting seepage.