Time Series Prediction Model Research Articles

NeuPot: A Neural Network-Based Honeypot for Detecting Cyber Threats in Industrial Control Systems

Honeypots have proven to be an effective defense method for industrial control systems (ICSs). However, as attacker skills become more sophisticated, it becomes increasingly difficult to develop honeypots that can effectively recognize and respond to such attacks. In this paper, we propose a neural network-based ICS honeypot scheme named Neupot that improves security from two aspects: honeypot interaction and cyber threats detection capability. Neupot can respond to attacker requests depending on a specific industrial scenario without constant communication with the ICS and detect malicious traffic. To create this honeypot scheme, a new seq2seq time-series forecast model guided by Huber loss is designed to simulate the long-term changes in actual ICS physical processes. Second, a Modbus honeypot framework is created to react to changes in these ICS physical processes in their interactions with attackers and to capture various cyber threats against the ICS. Further, a novel loss function for industrial protocol-level malicious traffic detection is devised to identify known and unknown threats. According to our experiments, the proposed honeypot scheme is highly effective and outperforms state-of-the-art schemes in terms of interactivity and in detecting cyber threats.

Application of Model-Based Time Series Prediction of Infrared Long-Wave Radiation Data for Exploring the Precursory Patterns Associated with the 2021 Madoi Earthquake

Taking the Madoi MS 7.4 earthquake of 21 May 2021 as an example, this paper proposes using time series prediction models to predict the outgoing long-wave radiation (OLR) anomalies and study short-term pre-earthquake signals. Five time series prediction models, including autoregressive integrated moving average (ARIMA) and long short-term memory (LSTM), were trained with the OLR time series data of the aseismic moments in the 5° × 5° spatial range around the epicenter. The model with the highest prediction accuracy was selected to retrospectively predict the OLR values during the aseismic period and before the earthquake in the area. It was found, by comparing the predicted time series values with the actual time series value, that the similarity indexes of the two time series before the earthquake were lower than the index of the aseismic period, indicating that the predicted time series before the earthquake significantly differed from the actual time series. Meanwhile, the temporal and spatial distribution characteristics of the anomalies in the 90 days before the earthquake were analyzed with a 95% confidence interval as the criterion of the anomalies, and the following was found: out of 25 grids, 18 grids showed anomalies—the anomalies of the different grids appeared on similar dates, and the anomalies of high values appeared centrally at the time of the earthquake, which supports the hypothesis that pre-earthquake signals may be associated with the earthquake.

Forecasting vault cash with an extreme value long short-term memory network

Effective cash management is key in banking operations and has implications for cost control, customer service, and risk management. As transactions become more diverse, manual forecasting methods have become inadequate for accurate vault cash forecasting, which involves extensive data analysis. To address this challenge, the banking industry has adopted FinTech tools based on big data and deep learning for various client services. These methods are generally accurate but perform poorly in cases with extreme events, for which data are scarce. In this study, we propose a time series prediction model with long short-term memory and an attention mechanism that effectively predicts the presence of extreme values. We applied extreme value theory to define the extreme value loss for extreme situations and use a sliding window to process time series data. The enhanced extreme value loss function in our model yields improved prediction accuracy for time series data.We evaluated the proposed model against previous methods in evaluation experiments on data from three branches of a commercial bank in Taiwan, where the vault cash data of each exhibited extreme values. The proposed model was highly accurate: it had a lower mean absolute percentage error and higher trend accuracy than competing methods on a majority of time series, and it was also more accurate in predicting extreme values in time series data.

Impact of Hull Condition and Propeller Surface Maintenance on Fuel Efficiency of Ocean-Going Vessels

The fuel consumption of marine diesel engines holds paramount importance in contemporary maritime transportation and shapes energy efficiency strategies of ocean-going vessels. Nonetheless, a noticeable gap in knowledge prevails concerning the influence of ship hull conditions and propeller roughness on fuel consumption. This study bridges this gap by utilizing artificial intelligence techniques in Matlab, particularly convolutional neural networks (CNNs) to comprehensively investigate these factors. We propose a time-series prediction model that was built on numerical simulations and aimed at forecasting ship hull and propeller conditions. The model's accuracy was validated through a meticulous comparison of predictions with actual ship-hull and propeller conditions. Furthermore, we executed a comparative analysis juxtaposing predictive outcomes with navigational environmental factors encompassing wind speed, wave height, and ship loading conditions by the fuzzy clustering method. This research's significance lies in its pivotal role as a foundation for fostering a more intricate understanding of energy consumption within the realm of maritime transport.

Multivariate multi-step LSTM model for flood runoff prediction: a case study on the Godavari River Basin in India

Abstract Flood is India's most prevalent natural calamity, devastatingly affecting human lives, infrastructure, and agriculture. Predicting floods can help to mitigate the potential damage and conduct timely evacuation drives. This research proposes a deep-learning regression model to forecast flood runoff. Various climatological, hydrological, land, and vegetation-related data have been collected from multiple sources for 18 years (2002–2019) to create a comprehensive dataset for the Godavari River at the Perur water station in India. The relevant attributes identified through feature selection are river water level, precipitation, temperature, surface pressure, evaporation, soil water content, daily runoff, and average river flow. The selected features were fed into various time series prediction models like AutoRegressive Integrated Moving Average (ARIMA), Prophet, Neural Prophet, and Long Short-Term Memory (LSTM). The LSTM model obtained the best results achieving a Root Mean Squared Error (RMSE) value of 0.05, Mean Absolute Error (MAE) value of 0.007, Willmott's Index (WI) of 0.83, Legates-McCabe's Index (LMI) of 0.58, and R2 of 0.67 for a 1-day prediction with a look-back window of 183 days. The model is also trained to predict the flood runoff value for a week ahead. The proposed model can serve as an essential component in flood warning systems.

Assessment of topological boundaries for microgrids in an active power distribution network during emergencies

Microgrid (MG) operation has become an intriguing possibility to enhance the sustainability, reliability, and resilience of electricity distribution systems in emergencies. However, identifying suitable physical boundaries of MGs from an active power delivery infrastructure is a highly challenging task. This study describes a novel approach for determining the topological boundary of multiple MGs to restore power supply in a power distribution system during the unavailability of grid power supply. This method tries to enhance the self-adequacy of the MGs by increasing power flow within the MG boundaries. In this context, possible demand at the nodes of the power delivery infrastructure during an emergency is anticipated by an autoregressive integrated moving average (ARIMA) and long short-term memory (LSTM) network-based time series forecast models. A modified backward-forward sweep (MBFS) power flow analysis, tailored for islanded mode, is employed to assess the power flow status in the MGs during emergencies. The MGs are supplied with locally available dispatchable distributed generators (DDGs). A particle swarm optimization (PSO) algorithm intertwined with the MBFS power flow method has been employed to determine the boundary of MGs by identifying minimum power flow line in the shortest path between the consecutive DDGs. Further, this study demonstrates the evaluation of the shortest path between paths between the consecutive DDGs with the help of a breadth-first search (BFS) algorithm. The proposed strategy has been tested on a modified 33-bus, 69-bus and 118-bus power distribution test systems. The experimented results are verified with the results obtained from similar existing approaches to explore the opportunities in the proposed methodology.

Enhanced machine learning model via twin support vector regression for streamflow time series forecasting of hydropower reservoir

The non-stationary, complex, and non-linear characteristics of streamflow time series have a significant impact on the simulation results of the conventional hydrological forecasting models. To improve the performances, this paper develops an enhanced machine learning model for streamflow time series forecasting, where the twin support vector regression (TSVR) is combined with singular spectrum analysis (SSA) and grey wolf optimizer (GWO). Specially, the SSA method is set as the data preprocessing tool for pattern identification; the TSVR model is set as the basic forecasting module for each pattern and the GWO method is used as the optimizer to select feasible parameter combination. Multi-step-ahead streamflow forecasting tasks are used to examine the feasibility and predictability of the proposed model. The results indicate that the proposed model can yield superior results compared with the traditional forecasting models. Thus, a robust and reliable tool is provided for streamflow time series forecasting under uncertainty.

Spatiotemporal graph neural networks for predicting mid-to-long-term PM2.5 concentrations

Predicting the concentration of PM2.5 particles is of critical importance in public health management because their small size enables them to penetrate deep into the lungs and even enter the bloodstream. However, achieving accurate predictions is challenging due to the complexity of the transport and dispersion processes involved, which are influenced by multiple factors, including atmospheric pollutants, meteorological conditions, and geographic features. To address this challenge, we developed a novel framework for PM2.5 prediction that utilizes multiple edges for feature extraction and employs a multi-gated graph neural network for feature calculations. The model utilizes multiple edges, created by combining an atmospheric diffusion coefficient with a PM2.5 similarity metric, to elucidate the complex characteristics of PM2.5 interactions between different monitoring stations. Node features are computed by aggregating connected nodes' features, weighted by these multi-faceted edges. The experimental results demonstrate that the proposed method outperforms conventional time-series prediction models in the mid-to-long-term prediction of PM2.5 concentrations (96 h in advance), with improvements in Root Mean Square Error (RMSE) by 2.613%, Critical Success Index (CSI) by 3.143%, and R-squared (R2) by 5.263%. Using the proposed model, the learning of global features can be achieved and issues related to local dependency can be mitigated.

Time series prediction of the chemical components of PM2.5 based on a deep learning model

Modeling-based prediction methods enable rapid, reagent-free air pollution detection based on inexpensive multi-source data than traditional chemical reaction-based detection methods in order to quickly understand the air pollution situation. In this study, a convolutional neural network (CNN) and long and short-term memory (LSTM) neural networks are integrated to create a CNN-LSTM time series prediction model to predict the concentration of PM2.5 and its chemical components (i.e., heavy metals, carbon component, and water-soluble ions) using meteorological data and air pollutants (PM2.5, SO2, NO2, CO, and O3). In the integrated CNN-LSTM model, the CNN uses convolutional and pooling layers to extract features from the data, whereas the powerful nonlinear mapping and learning capabilities of LSTM enable the time series prediction of air pollution. The experimental results showed that the CNN-LSTM exhibited good generalization ability in the prediction of As, Cd, Cr, Cu, Ni, and Zn, with a mean R2 above 0.9. Mean R2 predicted for PM2.5, Pb, Ti, EC, OC, SO42−, and NO3− ranged from 0.85 to 0.9. Shapley value showed that PM2.5, NO2, SO2, and CO had a greater influence on the predicted heavy metal results of the model. Regarding water-soluble ions, the predicted results were dominantly influenced by PM2.5, CO, and humidity. The prediction of the carbon fraction was affected mainly by the PM2.5 concentration. Additionally, several input variables for various components were eliminated without affecting the prediction accuracy of the model, with R2 between 0.70 and 0.84, thereby maximizing modeling efficiency and lowering operational costs. The fully trained model prediction results showed that most predicted components of PM2.5 were lower during January to March 2020 than those in 2018 and 2019. This study provides insight into improving the accuracy of modeling-based detection methods and promotes the development of integrated air pollution monitoring toward a more sustainable direction.

Civil gas energy accidents in China from 2012–2021

In this study, civil gas energy accidents reported by the China Gas Network and related organizations from 2012 to 2021 were collected, and a comprehensive multidimensional correlation analysis was conducted considering factors such as accident timing, geography, causes, and casualties. The results identified July and August, Mondays and Sundays, and the morning, mid-day, and evening cooking times as the high-incidence months, days, and times for gas accidents, respectively. Gas accidents were found to occur more frequently in eastern coastal areas, provincial capitals, and larger cities, while residential and construction sites were identified as high-risk areas for gas accidents. Explosions were the most prevalent type of gas accident, followed by leaks, fires, and poisoning. Third-party construction and valve issues were identified as the primary factors contributing to gas leakage, whereas cooking was identified as the most common ignition source. An analysis of the Pearson correlation coefficient indicated a significant correlation among the gas accident factors. Moreover, a time-series prediction model was developed to forecast gas accidents in China, with the results demonstrating fluctuating gas accidents. This study proposes targeted preventive measures in terms of publicity, education, equipment, and facilities to provide scientific support to government units to improve civil gas energy security measures.

Research on the method of digital twin operation and maintenance platform for intelligent early warning of wind turbine tower

Wind power generators have a complex structure and operate in harsh environments, where working conditions are highly variable. As a result, the operation and maintenance of wind turbines face numerous challenges. In response to the need for the development of wind power operation and maintenance informatization, it is necessary to satisfy the requirements for multi-party collaborative monitoring to ensure the long-term safe and reliable operation of wind turbines. In this paper,we proposed a method for building an intelligent early-warning digital twin platform focused on the simulation of wind turbines and tower components. The platform construction method proposed in this article is based on the Web and from the perspective of intelligent operation and maintenance of wind turbines. It establishes a warning model for tower agent simulation and vibration signal time series prediction. The tower mechanism model is established based on the operating data set of a 4MW wind turbine at Shanghai Electric. Different physical responses of the tower under different wind speeds are simulated, and an agent model using LSTM and decision tree models is established for predictive analysis. To account for uncertainty, a Bayesian-LSTM model is established to warn against predictive errors. Finally, a data-driven digital twin wind turbine platform is achieved on the Web.

Time series prediction model based on autoregression weight network

AbstractWe propose an autoregressive weighted network (ARWNet) time series forecasting model inspired by the idea of ensemble learning. The model adopts the classical autoregressive analysis to optimize weak learners. Meanwhile, the combined weight optimization method is used to construct an efficient, strong learner. With these methodological foundations, the scalability of the framework is greatly enhanced by the possibility of experimenting with other learners to assist in decision‐making. Machine learning can provide great utility and acceptable cost in the prediction process of electricity transformers. Over the years, many research papers on time series prediction have been reported. This work will focus on the analysis using the potential properties in the series: long‐term, continuity, periodicity, and delay. In our experiments, the ETT‐small dataset is used to compare the prediction accuracy of ARWNet and other mainstream models. All results suggest that the proposed ARWNet model demonstrates strong generalization ability and high predicting accuracy with delay characteristics, which outperform current popular time‐series prediction methods.

Improving Air Quality Prediction with a Hybrid Bi-LSTM and GAN Model

Air quality is a topic that has been of utmost concern across the globe for the past few decades. Various intelligent monitoring systems are used in diverse scenarios, collecting air quality data that contains missing values. Such missing values in data cause hindrances in forecasting. This time series prediction or forecasting process extracts the necessary information from historical records and predicts future values. To solve the missing values issue in data, Generative Adversarial Networks (GAN) are used to impute the missed data. While the learning of long-term dependencies embedded in the time series poses another threat to the models in the time prediction. To overcome this, Long Short-Term Memory (LSTM) models are used. Yet, most of the neural network-based methods failed to consider the patterns of time series data that varied for each period, and the encoder-decoder performance deteriorated for longer sequences. To combat this, the present study proposes a hybrid probabilistic model to generate parameters for predictive distribution at every step. Hence, an implementation of hierarchical-attention-based BiLSTM with GAN is proposed in the study for effective prediction and minimal error. The proposed model is assessed with the evaluation metrics such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Mean Square Error (MSE). The evaluation metric confirmed the higher accuracy of the proposed model than the existing models in time series prediction.

Gated Spiking Neural P Systems for Time Series Forecasting.

Spiking neural P (SNP) systems are a class of neural-like computing models, abstracted by the mechanism of spiking neurons. This article proposes a new variant of SNP systems, called gated spiking neural P (GSNP) systems, which are composed of gated neurons. Two gated mechanisms are introduced in the nonlinear spiking mechanism of GSNP systems, consisting of a reset gate and a consumption gate. The two gates are used to control the updating of states in neurons. Based on gated neurons, a prediction model for time series is developed, known as the GSNP model. Several benchmark univariate and multivariate time series are used to evaluate the proposed GSNP model and to compare several state-of-the-art prediction models. The comparison results demonstrate the availability and effectiveness of GSNP for time series forecasting.

Soil and Rockfill Dams Safety Assessment for Henan Province: Monitoring, Analysis and Prediction

It is of great significance to explore the spatial and temporal evolution of soil and rockfill dam deformation, ensuring the safety of people’s lives and healthy economic development. The spatial and temporal evolution patterns of deformation of 17 large soil and rockfill dams in Henan Province were analyzed by using the PS-InSAR technique and 55 Sentinel-1A images from March 2017 to September 2021. Based on factors such as reservoir water level and rainfall, a series of analyses were conducted on the Xiaolangdi soil and rockfill dam, which has the highest dam height and the most prominent deformation problem. The monitoring results show that all the soil and rockfill dams in Henan Province have different degrees of deformation, and there is a close relationship between dam height and deformation. In addition, the deformation rate of the Xiaolangdi soil and rockfill dam in the past five years presents a “Stepped Shape” deformation trend from the top to the bottom of the dam. The deformation of the upper, middle, and lower parts of the dam body reaches 80 mm, 40 mm, and 20 mm, respectively, among which the middle part of the dam crest has the largest deformation. Furthermore, the time series prediction model for sparrow search algorithm Long Short-Term Memory considering the moving average filter (MAF-SSA-LSTM) is proposed to predict and accurately analyze the future deformation of Xiaolangdi soil and rockfill dam with RMSE of 1.526 mm, MAE of 1.447 mm, and MAPE of 2.22%, which proved that the model has high prediction accuracy. It can truly reflect the overall deformation trend of the dam body. The results provide a theoretical basis and decision basis for the census of reservoir safety conditions and deformation history retrieval in Henan Province.

Persistence Initialization: a novel adaptation of the Transformer architecture for time series forecasting

AbstractTime series forecasting is an important problem, with many real world applications. Transformer models have been successfully applied to natural language processing tasks, but have received relatively little attention for time series forecasting. Motivated by the differences between classification tasks and forecasting, we propose PI-Transformer, an adaptation of the Transformer architecture designed for time series forecasting, consisting of three parts: First, we propose a novel initialization method called Persistence Initialization, with the goal of increasing training stability of forecasting models by ensuring that the initial outputs of an untrained model are identical to the outputs of a simple baseline model. Second, we use ReZero normalization instead of Layer Normalization, in order to further tackle issues related to training stability. Third, we use Rotary positional encodings to provide a better inductive bias for forecasting. Multiple ablation studies show that the PI-Transformer is more accurate, learns faster, and scales better than regular Transformer models. Finally, PI-Transformer achieves competitive performance on the challenging M4 dataset, both when compared to the current state of the art, and to recently proposed Transformer models for time series forecasting.

Deep learning approach to forecast air pollution based on novel hourly index

Air pollution is a pressing concern that the entire world is striving to combat. Among air pollutants, particulate matter poses a significant threat to human health. The Sustainable Development Goals (SGD3, SGD7 and SGD11) include initiatives to address air pollution. Two innovative methods are proposed in this research to predict the PM2.5 concentration in advance. While multivariate time series prediction models typically employ multiple features as inputs, this research reduces the number of inputs, which makes the proposed combination of approaches simple and effective. The approaches involve the development of two new indexing methods, namely, the Hourly Relative Mean Index and the Hourly Weighted Index. This research offers innovative hybrid deep learning models that incorporate the newly developed indices, Long Short Term Memory (LSTM) models, and robust preprocessing techniques. Multivariate Isolation Forest Relative Index Bidirectional LSTM and Multivariate Isolation Forest Weighted Index LSTM methods are used to forecast PM2.5 concentration for an hourly time frame. Further, Multivariate Isolation Forest Relative Index LSTM and Multivariate Isolation Forest Weighted Index LSTM methods are used to forecast PM2.5 concentration 48 h ahead. The study establishes that the proposed combination of approaches outperform traditional ways to achieve superior performance with reduced complexities requiring fewer inputs for predictions.

Assessing ARIMA-Based Forecasts for the Percentage of Renewables in Germany: Insights and Lessons for the Future

Renewables are the greener substitute for the conventional polluting sources of generating energy. For their successful integration into the power grid, accurate forecasts are required. In this paper, we report the lessons acquired from our previous works on generating time-series ARIMA-based forecasting models for renewables. To this end, we considered a consistent dataset spanning the last four years. Assuming four different performance metrics for each of the best ARIMA-based models of our previous works, we derived a new optimal model for each month of the year, as well as for the two different methodologies suggested in those works. We then evaluated the performance of those models, by comparing the two methodologies: in doing so, we proposed a hybrid methodology that took the best models out of those two methodologies. We show that our proposed hybrid methodology has improved yearly accuracy of about 89.5% averaged over 12 months of the year. Also, we illustrate in detail for the four years under study and each month of the year the observed percentage of renewables and its corresponding accuracy compared to the generated forecasts. Finally, we give the implementation details of our open-source REN4KAST software platform, which provides several services related to renewables in Germany.

Time series analysis of groundwater quality at selected sites of Purba and Paschim Burdwan, West Bengal, India.

The Water Quality Index (WQI) is used to monitor the health and usability of a water body. In this study, we aimed to construct time series prediction models using groundwater WQI (GW-WQI) at four sites: IISCO-Asansol, Durgapur Town, Burdwan University, and Burdwan Station. While statistical spatio-temporal analysis has been reported earlier, no time series analysis of the data or predictive modelling has been done. Pre-monsoon and post-monsoon physico-chemical data from 2010 to 2022 were obtained from the West Bengal Pollution Control Board website to calculate the GW-WQI. Prediction modelling was performed using R 4.1.3 software. Best fit forecast models were selected to predict future trends of GW-WQI with 80% of the data. Subsequently, the models were validated using R-squared, root mean square error (RMSE), mean absolute error (MAE), maximum absolute percentage error (MAPE), and Thiel's U for the model using 20% of the data. Our results show that GW-WQI was good in pre-monsoon but unfit for drinking in post-monsoon in IISCO-Asansol, Durgapur Town, Burdwan University, and Burdwan Station. Arsenic, fluoride, and mercury were the major contaminants resulting in poor GW-WQI. Seasonal ARIMA was the best model for Burdwan University and IISCO-Asansol, ETSfor Durgapur Station, and BaggedARIMA for Burdwan Station. The forecast model for Durgapur and Burdwan Station predicted a sharp increase until 2027 but was fluctuating for IISCO-Asansol and Burdwan University. Thus, GW-WQI is a major problem in the industrial belt of West Bengal that is likely to remain high or worsen in the future.

Fish swarmed Fuzzy Time Series for Photovoltaic’s Forecasting in Microgrid

Forecasting irradiation and temperature is important for designing photovoltaic systems because these two factors have a significant impact on system performance. Irradiation refers to the amount of solar radiation that reaches the earth's surface, and directly affects the amount of energy that can be generated by a photovoltaic system. Therefore, accurate irradiation forecasting is essential for estimating the amount of energy a photovoltaic system can produce, and can assist in determining the appropriate system size, configuration, and orientation to maximize energy output. Temperature also plays an important role in the performance of a photovoltaic system. With increasing temperature, the efficiency of the solar cell decreases, which means that the energy output of the system also decreases. Therefore, accurate temperature forecasts are essential for estimating system energy output, selecting suitable materials, and designing effective cooling systems to prevent overheating. In summary, forecasting irradiation and temperature is important for designing photovoltaic systems as it helps in determining suitable system size, configuration, orientation, material selection, and cooling system, which ultimately results in higher energy output and better system performance. In recent decades, many forecasting models have been built on the idea of fuzzy time series. There are several forecasting models proposed by integrating fuzzy time series with heuristic or evolutionary algorithms such as genetic algorithms, but the results are not satisfactory. To improve forecasting accuracy, a new hybrid forecasting model combines fish swarm optimization algorithm with fuzzy time series. The results of irradiance prediction / forecasting with the smallest error are using the type of Fuzzy Time Series prediction model optimized with FSOA with RMSE is 0.83832.