Step-ahead Prediction Research Articles

TransTLA: A Transfer Learning Approach with TCN-LSTM-Attention for Household Appliance Sales Forecasting in Small Towns

Deep learning (DL) has been widely applied to forecast the sales volume of household appliances with high accuracy. Unfortunately, in small towns, due to the limited amount of historical sales data, it is difficult to forecast household appliance sales accurately. To overcome the above-mentioned challenge, we propose a novel household appliance sales forecasting algorithm based on transfer learning, temporal convolutional network (TCN), long short-term memory (LSTM), and attention mechanism (called “TransTLA”). Firstly, we combine TCN and LSTM to exploit the spatiotemporal correlation of sales data. Secondly, we utilize the attention mechanism to make full use of the features of sales data. Finally, in order to mitigate the impact of data scarcity and regional differences, a transfer learning technique is used to improve the predictive performance in small towns, with the help of the learning experience from the megacity. The experimental outcomes reveal that the proposed TransTLA model significantly outperforms traditional forecasting methods in predicting small town household appliance sales volumes. Specifically, TransTLA achieves an average mean absolute error (MAE) improvement of 27.60% over LSTM, 9.23% over convolutional neural networks (CNN), and 11.00% over the CNN-LSTM-Attention model across one to four step-ahead predictions. This study addresses the data scarcity problem in small town sales forecasting, helping businesses improve inventory management, enhance customer satisfaction, and contribute to a more efficient supply chain, benefiting the overall economy.

A new feature selection method based on importance measures for crude oil return forecasting

This paper introduces a novel feature selection method, called Feature Selection based on Importance Measures (FS-IM), to enhance the forecasting of crude oil returns. FS-IM innovatively combines active learning with the application of Gaussian noise to input features and selects the most relevant features using an optimal threshold value. The paper applies a ridge regression (RR) model based on FS-IM (FS-RR) to identify the factors that have important information for crude oil return forecasting. The paper compares FS-IM with other dimension reduction methods such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Independent Component Analysis (ICA). The results show that FS-IM can significantly improve model accuracy, demonstrating its effectiveness in finding key features. Moreover, FS-IM is more stable and consistent than other dimension reduction methods in enhancing the prediction accuracy in different scenarios, indicating its superior capability in capturing complex relationships between input and output variables. Furthermore, this study compares FS-RR model with other 13 prediction models by conducting experiments using a series of evaluation metrics, different statistical tests, and different step-ahead predictions and training sets. The results confirm that the RR model based on FS-IM can consistently outperform other model in terms of predictive performance and economic value, proving its effectiveness and robustness. This study contributes to the literature on crude oil price forecasting by addressing the challenges of high-dimensional and complex data, and by providing a robust, practical tool for professionals in energy economics and finance.

Online Learning of the Kalman Filter With Logarithmic Regret

In this paper, we consider the problem of predicting observations generated online by an unknown, partially observable linear system, which is driven by Gaussian noise. In the linear Gaussian setting, the optimal predictor in the mean square error sense is the celebrated Kalman filter, which can be explicitly computed when the system model is known. When the system model is unknown, we have to learn how to predict observations online based on finite data, suffering possibly a non-zero regret with respect to the Kalman filter's prediction. We show that it is possible to achieve a regret of the order of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{poly}\log (\mathsf {N})$</tex-math></inline-formula> with high probability, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> is the number of observations collected. This is achieved using an online least-squares algorithm, which exploits the approximately linear relation between future observations and past observations. The regret analysis is based on the stability properties of the Kalman filter, recent statistical tools for finite sample analysis of system identification, and classical results for the analysis of least-squares algorithms for time series. Our regret analysis can also be applied to other predictors, e.g. multiple step-ahead prediction, or prediction under exogenous inputs including closed-loop prediction. A fundamental technical contribution is that our bounds hold even for the class of non-explosive systems (including marginally stable systems), which was not addressed before in the case of online prediction.

Multivariate Time-Series Prediction in Industrial Processes via a Deep Hybrid Network Under Data Uncertainty

With the rapid progress of the industrial Internet of Things (IIoT), reducing data uncertainty has become a critical issue in predicting the development trends of systems and formulating future maintenance strategies. This article proposes an end-to-end, deep hybrid network-based, short-term, multivariate time-series prediction framework for industrial processes. First, the maximal information coefficient is adopted to extract the nonlinear variate correlation features. Second, a convolutional neural network with a residual elimination module is designed to eliminate data uncertainty. Third, a bidirectional gated recurrent unit network is connected in a time-distributed form to achieve step-ahead prediction. Last, an optimized Bayesian optimization method is adopted to optimize the model's learning rate. A comparison with other state-of-the-art, deep learning-based, time-series prediction methods in the case study illustrates the superiority of the proposed framework in noisy IIoT environments.

One Step Ahead Prediction of Ozone Concentration for Determination of Outdoor Air Quality Level

With the rapid spread of urbanization, competent authorities become increasingly anxious from air pollution risks and effect on citizens especially those with respiratory diseases. In this work, performances of six machine learning methods were analyzed for prediction of maximum ozone (O_3) concentration for the next-day. The models make the prediction using concentrations of six atmospheric components (PM2.5, PM10, Ozone (O3), Sulfur Dioxide (SO2), Nitrogen Dioxide (NO2), and Carbon Monoxide (CO)). The utilized machine learning methods are multilayer perception (MLP), Support Vector Regression (SVM), k-Nearest Neighbor (K-NN), Random Forests (RF), Gradient Boosting (GB), and Elastic Net (EN). After the predictions made by these models, the predicted values were further processed to be classified into one of the six air quality levels defined by United States Environmental Protection Agency. The prediction performances of the models as well as their corresponding classification results were analyzed. It was shown that MLP model gives the lowest RMSE of 2246 for prediction step while SVR achieved the highest accuracy score of 0.790.

Development of an Extreme Gradient Boosting Model Integrated With Evolutionary Algorithms for Hourly Water Level Prediction

The establishment of reliable water level prediction models is vital for urban flood control and planning. In this paper, we develop hybrid models (GA-XGBoost and DE-XGBoost) that couple two evolutionary models, a genetic algorithm (GA) and a differential evolution (DE) algorithm, with the extreme gradient boosting (XGBoost) model for hourly water level prediction. The Jungrang urban basin located on the Han River, South Korea, was selected as a case study for the proposed models. Hourly rainfall and water level data were collected between 2003 and 2020 to construct and evaluate the performance of the selected models. To compare the prediction efficiency, two other tree-based models were chosen: classification and registration tree (CART) and random forest (RF) models. A comparison of the results showed that two hybrid models, GA-XGBoost and DE-XGBoost, outperformed RF and CART in the multistep-ahead prediction of water level, and the relative errors of the hybrid model ranged from [2.18%-9.21%], compared to [3.76%-10.41%] and [2.99%-11.88%] for the RF and CART, respectively. Reliable performance was also supported by other measures. In general, the GA-XGBoost and DE-XGBoost models displayed relatively similar performance despite their small differences. The CART model was not preferable for multistep-ahead water level predictions, even though it yielded the lowest Akaike information criterion (AIC) value. This study verifies that despite having some drawbacks when considering long step-ahead prediction and model complexity, hybrid XGBoost models might be superior to many existing models for hourly water level prediction.

Stability control of electric vehicles with in-wheel motors by considering tire slip energy

Abstract Tire wear and overuse are serious issues for four-in-wheel-motor-driven electric vehicles (FIWMD EVs). As a result, this paper proposes a hierarchical control framework to improve the safety of FIWMD EVs while save the tire slip energy (i.e. reduce the tire wear), which includes a linear quadratic regulator (LQR) in the outer layer and a holistic corner controller (HCC) in the inner layer. The LQR can highly improve the lateral stability of the vehicles under extreme conditions by producing an additional yaw moment and a front wheel steering angle. Whereas, the HCC can efficiently distribute the deviation of the longitudinal force and the additional yaw moment by a step-ahead prediction. In this framework, to reduce the tire slip energy, a semi-empirical tire slip energy model is developed and used for tuning the HCC controller. In this way, not only the lateral stability of such EVs can be ensured but also the tire slip energy can be reduced. More importantly, this paper compares the lateral stability and slip energy of the vehicle under lane change condition between four methods – without control, traditional axis distribution, tire workload usage and tire slip energy. The results demonstrate that the proposed controller presents an excellent control capability. In addition, different from the widely used axis load distribution, the longitudinal velocity of the proposed method is more stable. More importantly, the tire slip power and energy are much smaller than others. It achieves 16.62% reduction of the tire slip energy.

Long-run expectations in a learning-to-forecast experiment: a simulation approach

In this paper, we elicit short-run as well as long-run expectations on the evolution of the price of a financial asset in a Learning-to-Forecast Experiment (LtFE). Subjects, in each period, have to forecast the the asset price for each one of the remaining periods. The aim of this paper is twofold: first, we fill the gap in the experimental literature of LtFEs where great effort has been devoted to investigate short-run expectations, i.e. one step-ahead predictions, while there are no contributions that elicit long-run expectations. Second, we propose a new computational algorithm to replicate the main properties of short and long-run expectations observed in the experiment. This learning algorithm, called Exploration-Exploitation Algorithm, is based on the idea that agents anchor their expectations around the last realized price rather than on the fundamental value, with a range proportional to the past observed price volatility. When compared to the Heuristic Switching Model, our algorithm performs equally well in describing the dynamics of short-run expectations and the realized price dynamics. The EEA, additionally, is able to reproduce the dynamics long-run expectations.

Exploring geometrical patterns in streamflow time series: utility for forecasting?

Abstract Streamflow time series often provide valuable insights into the underlying physical processes that govern responses of any watershed to storm events. Patterns derived from time series based on repeated structures within these series can be beneficial for developing new or improved data-driven forecasting models. Data-driven models, artificial neural networks (ANN), are developed in the current study for streamflow prediction using input structures that are classified by geometrically similar patterns. A new modular and integrated ANN architecture that combines multiple ANN models, referred to as pattern-classified neural network (PCNN), is proposed, developed and investigated in this study. The PCNN relies on the development of several independent local models instead of one global data-driven prediction model. The PCNN models are evaluated for one step-ahead prediction of daily streamflows for Reed Creek and Little River, Virginia, and Elkhorn Creek, Kentucky in the United States. Results obtained from this study suggest that the use of these patterns has improved the performance of the neural networks in prediction. The improved performance of the PCNN models can be attributed to prior classification of data benefiting generalization abilities. PCNN model outputs can also provide an ensemble of forecasts that help quantify forecast uncertainty.

Probabilistic forecasting of the disturbance storm time index: An autoregressive Gaussian process approach

AbstractWe present a methodology for generating probabilistic predictions for the Disturbance Storm Time(Dst) geomagnetic activity index. We focus on the One Step Ahead prediction task and use the OMNI hourly resolution data to build our models. Our proposed methodology is based on the technique of Gaussian Process Regression. Within this framework we develop two models; Gaussian Process Autoregressive (GP‐AR) and Gaussian Process Autoregressive with eXogenous inputs (GP‐ARX). We also propose a criterion to aid model selection with respect to the order of autoregressive inputs. Finally, we test the performance of the GP‐AR and GP‐ARX models on a set of 63 geomagnetic storms between 1998 and 2006 and illustrate sample predictions with error bars for some of these events.

Comparison between Cascade Forward and Multi-Layer Perceptron Neural Networks for NARX Functional Electrical Stimulation (FES)-Based Muscle Model

This paper presents the development and comparison of muscle models based on Functional Electrical Stimulation (FES) stimulation parameters using the Nonlinear Auto-Regressive model with Exogenous Inputs (NARX) using Multi-Layer Perceptron and Cascade Forward Neural Network (CFNN). FES stimulations with varying frequency, pulse width and pulse duration were used to estimate the muscle torque. About 722 data points were used to create muscle model. One Step Ahead (OSA) prediction, correlation tests and residual histogram analysis were performed to validate the model. The optimal Multi-Layer Perceptron (MLP) results were obtained from input lag space of 1, output lag space of 43 and hidden units 30. The MLP selected a total of three terms were selected to construct the final model, which producing a final Mean Square Error (MSE) of 1.1299. The optimal CFNN results were obtained from input lag space of 1, output lag space of 5 and hidden units 20 with similar terms selected. The final MSE produced was 1.0320. The proposed approach managed to approximate the behavior of the system well with unbiased residuals, which CFNN showing 8.66% MSE improvement over MLP with 33.33% less hidden units.

Binary Particle Swarm Optimization Structure Selection of Nonlinear Autoregressive Moving Average with Exogenous Inputs (NARMAX) Model of a Flexible Robot Arm

The Nonlinear Auto-Regressive Moving Average with Exogenous Inputs (NARMAX) model is a powerful, efficient and unified representation of a variety of nonlinear models. The model’s construction involves structure selection and parameter estimation, which can be simultaneously performed using the established Orthogonal Least Squares (OLS) algorithm. However, several criticisms have been directed towards OLS for its tendency to select excessive or sub-optimal terms leading to nonparsimonious models. This paper proposes the application of the Binary Particle Swarm Optimization (BPSO) algorithm for structure selection of NARMAX models. The selection process searches for the optimal structure using binary bits to accept or reject the terms to form the reduced regressor matrix. Construction of the model is done by first estimating the NARX model, then continues with the estimation of the MA model based on the residuals produced by NARX. One Step Ahead (OSA) prediction, Mean Squared Error (MSE) and residual histogram analysis were performed to validate the model. The proposed optimization algorithm was tested on the Flexible Robot Arm (FRA) dataset. Results show the success of BPSO structure selection for NARMAX when applied to the FRA dataset. The final NARMAX model combines the NARX and MA models to produce a model with improved predictive ability compared to the NARX model.

Reduction of Delay in Detecting Initial Dips from Functional Near-Infrared Spectroscopy Signals Using Vector-Based Phase Analysis.

In this paper, we present a systematic method to reduce the time lag in detecting initial dips using a vector-based phase diagram and an autoregressive moving average with exogenous signals (ARMAX) model-based q-step-ahead prediction algorithm. With functional near-infrared spectroscopy (fNIRS), signals related to mental arithmetic and right-hand clenching are acquired from the prefrontal and left primary motor cortices, respectively. The interrelationship between oxygenated hemoglobin, deoxygenated hemoglobin, total hemoglobin and cerebral oxygen exchange are related to initial dips. Specifically, a threshold value from the resting state hemodynamics is incorporated, as a decision criterion, into the vector-based phase diagram to determine the occurrence of initial dips. To further reduce the time lag, a [Formula: see text]-step-ahead prediction method is applied to predict the occurrence of the dips. A combination of the threshold criterion and the prediction method resulted in the delay time of about 0.9[Formula: see text]s. The results demonstrate that rapid detection of initial dip is possible and therefore can be used for real-time brain-computer interfacing.

Modelling of a Flexible Single-Link Manipulator Using Metaheuristic Algorithms

This study presents development of approaches with cuckoo search algorithm (CS) and differential evolutionary algorithm (DE) for modelling of flexible single-link manipulator system (FSLMS). Details of simulation study, model selection, optimization and result analyses are given in this study. A mathematical model of a (FSLMS) has been developed and modelled as a pinned-free, an Euler-Bernoulli flexible beam using finite element method (FEM) based on Lagrangian approach analysis. A bang-bang input torque was applied and the dynamic response of the system was investigated. Next, a suitable model structure is selected and optimized using intelligent optimization techniques (CS) and (DE). Mean squared error (MSE), One Step Ahead (OSA) prediction, correlation tests have been performed for validation and verification of the obtained model in characterizing the manipulator system. A comparative assessment of the two models in characterizing the manipulator system is presented in time and frequency domains. It has been demonstrated that the CS and DE modelling techniques have performed well in approximating the system response and the result obtained through CS and DE are nearly same in modelling a single-link flexible manipulator system.

Neuro identification for flexible cantilever beam structure

In this study, system identification of a flexible beam which undergoes a small deflection due to the disturbance force was conducted. Recursive Least Square (RLS) and several intelligent Neuro identifications were established to acquire the dynamic model of the beam based on One Step-Ahead (OSA) prediction. Comparative assessment between conventional and intelligent methods was investigated in this study with the aim to find the best model that can represents the flexible beam. Neural Network (NN) can be classified according to their structures and learning algorithms. For the purpose of this study, Focused Time-Delay Neural Network (FTDNN), Distributed Time-Delay Neural Network (DTDNN), Nonlinear Autoregressive with Exogeneous Variables (NARX), Elman NN and Radial Basis Function (RBF) were compared their effectiveness in emulating the dynamic behaviour of the system. The performance of the network was observed based on its capability to represents the system with the lowest Mean- Squared-Error (MSE) between the actual and predicted output responses of the models.

Intelligence Swarm Model Optimization of Flexible Plate Structure System

This paper presents the performance of modeling the flexible plate structure with free-free-clamped-clamped (FFCC) edges boundary condition using conventional Recursive Least Squares (RLS) and evolutionary algorithm of Genetic Algorithm (GA) and Particle Swarm Optimization (PSO). The auto-regressive with exogenous (ARX) structure was used in this study to obtain the dynamic model of a flexible plate structure. Data acquisition and instrumentation system were designed and integrated with the experimental rig and several experimental procedures were conducted to acquire the input and output of the flexible plate. The input and output data collected from the experimental study were utilized to develop the model of the system. The 4000 data sets collected in the experiment were divided into two parts for training and testing. The first 3000 data sets were used to train the model developed while the last 1000 data sets were used to test the performance of thus developed model. All developed model using RLS, GA and PSO were validated using one step-ahead prediction (OSA), mean squared error (MSE) and correlation tests. Amongst all, it was found that PSO algorithm has performed better in term of lowest mean squared error achieved (0.00032719) as compared to conventional algorithm (RLS) and evolutionary algorithm (GA). However, by comparing in term of estimating the first mode of vibration which the dominant mode of structure, GA has performed better by presented the lowest percentage error (3.63 %). Besides that, it was found that, all estimated models using all methods proposed are comparable, acceptable and possible to be used as a platform of controller development and verification to suppress the vibration of the flexible plate structure.

Non-parametric modelling of a rectangular flexible plate structure

This research investigates the performance of dynamic modelling using non-parametric techniques for identification of a flexible structure system for development of active vibration control. In this paper, the implementation details are described and the experimental studies conducted in this research are analysed. The input–output data of the system were first acquired through the experimental studies using National Instruments (NI) data acquisition system. A sinusoidal force was applied to excite the flexible plate and the dynamic response of the system was then investigated. Non-parametric modelling of the system were developed using several artificial intelligent methodologies namely Adaptive Elman Neural Networks (ENN), Backpropagation Multi-layer Perceptron Neural Networks (MLPNN) and Adaptive Neuro-Fuzzy Inference System (ANFIS). The performance of all these methodologies were compared and discussed. Finally, validation and verification of the obtained model was conducted using One Step Ahead (OSA) prediction, mean squared error (MSE) and correlation tests. The prediction ability of the model was further observed with unseen data. The results verified that the MLPNN converge to an optimum solution faster and the dynamic model obtained described the flexible plate structure very well. The non-parametric models of the flexible plate structure thus developed and validated will be used as the representation of the transfer function of the system in subsequent investigations for the development of active vibration control strategies for vibration suppression in flexible structures.

Prediction of room temperature and relative humidity by autoregressive linear and nonlinear neural network models for an open office

In this study, a linear parametric autoregressive model with external inputs (ARX) and a neural network-based nonlinear autoregressive model with external inputs (NNARX) are developed to predict the thermal behaviour of an open office in a modern building. External and internal climate data recorded over three months were used to build and validate models for predicting dry bulb temperature and relative humidity for different time-scales (30min to 3h ahead). In order to compare the accuracy for different step-ahead predictions, different performance measures, such as goodness of fit, mean squared error, mean absolute error and coefficient of determination between predicted model output and real measurements, were calculated. For the NNARX model, the optimal network structure after training, is subsequently determined by pruning the fully connected network using the optimal brain surgeon strategy. The results demonstrate that both models provide reasonably good predictions but the nonlinear NNARX model outperforms the linear ARX model. These models can both potentially be used for improving indoor air quality by focusing on building intelligence into the controller in HVAC plants, in particular, adaptive control systems.

COMPONENT FAULT DIAGNOSIS USING WAVELET NEURAL NETWORKS WITH LOCAL RECURRENT STRUCTURE

This paper investigates the development of the wavelet neural network with local recurrent structure and its application to fault detection and isolation (FDI) of components of a dynamic process. Hybrid learning based on orthogonal least-squares and the steepest-descent method, is used to train the proposed neural network. The experimental case study concerns the component fault diagnosis of a three-tank system. A neural simplified observer scheme is used to generate the residuals (symptoms) in the form of one step-ahead prediction errors. These are further analysed by a neural classifier in order to take the appropriate decision regarding the actual behaviour of the process.

Hydraulic Looper Fault Detection and Isolation Using Dynamic Neural Networks

The contribution investigates the design of a fault detection and isolation system for a hydraulic looper from a hot rolling mill plant. A dynamic functional-link neural network genetically evolved is used to identify different relationships between process variables. One step-ahead prediction errors, i.e. residuals, are then evaluated by a logic of decision using threshold values and different patterns of residual’s change in case of known faulty behaviours. Experimental results obtained by processing separately simulation and real data characterise the efficiency of the approach as depending on the availability of signals provided from the automation system of considered process.