Recursive Prediction Research Articles

Research on Product Classification and Demand Prediction Based on ALSTM-CNN

The rapid development of e-commerce platforms has prompted a large number of retailers to join and store goods through logistics warehouses provided by the platform, which are then centrally managed by the platform. In this context, the rise of artificial intelligence and machine learning technology has brought revolutionary impact to the e-commerce industry. Under this framework, accurate sales forecasting has become the core link. This article aims to integrate machine learning models and time series analysis, aiming to improve the accuracy of sales data prediction. This article is based on the K-Means algorithm for unsupervised multi-level clustering, determining the optimal K value through elbow rules and contour coefficients. After labeling the classification data with class labels, objectively evaluate the clustering effect by combining the Calinski Harabasz index and visualizing SKUs after each clustering. Finally, using evaluation metric 1-wmape, the performance of ARIMA-ANN, BiLSTM, GRU, CNN-LSTM, Stack LSTM, and three tree models in predicting SKU sales was compared. We compared the accuracy of 15 day recursive prediction and 15 day one-time output, and selected sliding window sizes of 3, 5, 7, and 14. After comparison, it was determined that the BiLSTM model performed best in predicting 15 day sales volume and a 3-day sliding time window under multiple sample categories; ARIMA-ANN performs the best in recursive prediction of 15 day sales volume in a few sample categories. By adjusting the ARIMA parameters through ACF and PACF, the maximum predictive performance is ensured.

A Recursive Prediction-Based Feature Enhancement for Small Object Detection.

Transformer-based methodologies in object detection have recently piqued considerable interest and have produced impressive results. DETR, an end-to-end object detection framework, ingeniously integrates the Transformer architecture, traditionally used in NLP, into computer vision for sequence-to-sequence prediction. Its enhanced variant, DINO, featuring improved denoising anchor boxes, has showcased remarkable performance on the COCO val2017 dataset. However, it often encounters challenges when applied to scenarios involving small object detection. Thus, we propose an innovative method for feature enhancement tailored to recursive prediction tasks, with a particular emphasis on augmenting small object detection performance. It primarily involves three enhancements: refining the backbone to favor feature maps that are more sensitive to small targets, incrementally augmenting the number of queries for small objects, and advancing the loss function for better performance. Specifically, The study incorporated the Switchable Atrous Convolution (SAC) mechanism, which features adaptable dilated convolutions, to increment the receptive field and thus elevate the innate feature extraction capabilities of the primary network concerning diminutive objects. Subsequently, a Recursive Small Object Prediction (RSP) module was designed to enhance the feature extraction of the prediction head for more precise network operations. Finally, the loss function was augmented with the Normalized Wasserstein Distance (NWD) metric, tailoring the loss function to suit small object detection better. The efficacy of the proposed model is empirically confirmed via testing on the VISDRONE2019 dataset. The comprehensive array of experiments indicates that our proposed model outperforms the extant DINO model in terms of average precision (AP) small object detection.

BLSTM convolution and self-attention network enabled recursive and direct prediction for optical chaos.

Chaotic time series prediction has attracted much attention in recent years because of its important applications, such as security analysis for random number generators and chaos synchronization in private communications. Herein, we propose a BLSTM convolution and self-attention network model to predict the optical chaos. We validate the model's capability for direct and recursive prediction, and the model dramatically reduces the accumulation of errors. Moreover, the time duration prediction of optical chaos is increased with comparative accuracy where the predicted sequence length reaches 4 ns with normalized mean squared error (NMSE) of less than 0.01.

Unsupervised incremental transfer learning with knowledge distillation for online remaining useful life prediction of rotating machinery

Online remaining useful life (RUL) prediction has been solved by deep transfer learning, but is still with challenges as follows: (1) Incomplete and unlabeled data under actual operation; (2) Condition monitoring data is streaming with unknown distribution; and (3) The distribution of degradation data is variable. To solve them, an unsupervised incremental transfer learning approach with knowledge distillation (KD) is proposed. First, a time series recursive prediction model is built to generate pseudo labels. Second, an online KD network is constructed to realize unsupervised domain adaptation. Finally, an incremental updating mechanism is designed in the online KD network for online RUL prediction with the pseudo labels. Comparative experiments on the IEEE PHM Challenge 2012 rolling bearing dataset show that the proposed method, being computationally inexpensive, can effectively achieve dynamic prediction only with sequentially-collected data, which can provide an effective RUL prediction solution for rotating machinery under an open environment.

TransforLearn: Interactive Visual Tutorial for the Transformer Model.

The widespread adoption of Transformers in deep learning, serving as the core framework for numerous large-scale language models, has sparked significant interest in understanding their underlying mechanisms. However, beginners face difficulties in comprehending and learning Transformers due to its complex structure and abstract data representation. We present TransforLearn, the first interactive visual tutorial designed for deep learning beginners and non-experts to comprehensively learn about Transformers. TransforLearn supports interactions for architecture-driven exploration and task-driven exploration, providing insight into different levels of model details and their working processes. It accommodates interactive views of each layer's operation and mathematical formula, helping users to understand the data flow of long text sequences. By altering the current decoder-based recursive prediction results and combining the downstream task abstractions, users can deeply explore model processes. Our user study revealed that the interactions of TransforLearn are positively received. We observe that TransforLearn facilitates users' accomplishment of study tasks and a grasp of key concepts in Transformer effectively.

OVERLOAD AND TRAFFIC MANAGEMENT OF MESSAGE SOURCES WITH DIFFERENT PRIORITY OF SERVICE

The scheme of dynamic management of traffic and activity of message sources with different priority of service is considered. The scheme is built on the basis of the neuroprognostic analysis model and the gradient descent method. For prediction and early detection of overload, the apparatus of the general theory of sensitivity with indirect feedback and control of activity of message sources is used. The control algorithm is started at the bottleneck of the network node. It uses a recursive prediction approach where the neural network output is referred to as many steps as defined by a given prediction horizon. Traffic with a higher priority is served without delay using the entire available bandwidth. Low-priority traffic will use the remaining bandwidth not used by higher-priority traffic. An algorithm for estimating the maximum available bandwidth of a communication node for traffic with a low service priority has been developed. This approach makes it possible to improve the efficiency of channel use without affecting the quality of service for high-priority traffic.

, , &

Lassa fever (LF) is endemic in West Africa and Nigeria in particular. Since 1969 when the disease was discovered, a seasonal outbreak is often reported in Nigeria. Many researchers have reported inconsistent or varying numbers of suspected, confirmed and death cases since 2012 to date. To enhance this reportage, and due to the high mortality rate associated with LF, it is pertinent to design a suitable and robust model that could predict or estimate the number of LF cases based on the onset data. To achieve these, we proposed a recursive prediction (RP) model that could do predictions with the onset data. The Pearson correlation coefficient (R) and R2 are applied to determine the performance analysis of the model. The RP model predicted 96.7% confirmed cases and 89.6% death cases for the first three months of 2022 based on the onset data. The model was also applied to predict COVID-19 death cases during the six weeks of the outbreak in India. The result showed a comparable prediction with the regression output for the COVID-19 death cases. This study demonstrated that the proposed model could be applied to perform prediction for any disease of unknown etiology during the onset of the disease outbreak without any treatment similar to the COVID-19 outbreak. The performance analysis of the RP showed that the model is useful to predict the increasing trend of an outbreak of a disease with unknown etiology without prior treatment experience and vaccines.

Attention-Based Wildland Fire Spread Modeling Using Fire-Tracking Satellite Observations

Modeling the spread of wildland fires is essential for assessing and managing fire risks. However, this task remains challenging due to the partially stochastic nature of fire behavior and the limited availability of observational data with high spatial and temporal resolutions. Herein, we propose an attention-based deep learning modeling approach that can be used to learn the complex behaviors of wildfires across different fire-prone regions. We integrate optimized spatial and channel attention modules with a convolutional neural network (CNN) modeling architecture and train the attention-based fire spread models using a recently derived fire-tracking satellite observational dataset in conjunction with corresponding fuel, terrain, and weather conditions. The evaluation results and their comparison with benchmark models, such as a deeper and more complex autoencoder model and the semi-empirical FARSITE fire behavior model, demonstrate the effectiveness of the attention-based models. These new data-driven fire spread models exhibit promising modeling performances in both the next-step prediction (i.e., predicting fire progression from one timestep earlier) and recursive prediction (i.e., recursively predicting final fire perimeters from initial ignition points) of observed large wildfires in California, and they provide a foundation for further practical applications including short-term active fire spread prediction and long-term fire risk assessment.

Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning

Optimising HVAC operations towards human wellness and energy efficiency is a major challenge for smart facilities management, especially amid COVID situations. Although IoT sensors and deep learning were applied to support HVAC operations, the loss of forecasting accuracy in recursive prediction largely hinders their applications. This study presents a data-driven predictive control method with time-series forecasting (TSF) and reinforcement learning (RL), to examine various sensor metadata for HVAC system optimisation. This involves the development and validation of 16 Long Short-Term Memory (LSTM) based architectures with bi-directional processing, convolution, and attention mechanisms. The TSF models are comprehensively evaluated under independent, short-term recursive, and long-term recursive prediction scenarios. The optimal TSF models are integrated with a Soft Actor-Critic RL agent to analyse sensor metadata and optimise HVAC operations, achieving 17.4% energy savings and 16.9% thermal comfort improvement in the surrogate environment. The results show that recursive prediction leads to a significant reduction in model accuracy, and the effect is more pronounced in the temperature-humidity prediction model. The attention mechanism significantly improves prediction performance in both recursive and independent prediction scenarios. This study contributes new data-driven methods for smart HVAC operations in IoT-enabled intelligent buildings towards a human-centric built environment.

Deep Learning Models for Stable Gait Prediction Applied to Exoskeleton Reference Trajectories for Children With Cerebral Palsy

Gait trajectory prediction models have several applications in exoskeleton control; they can be used as feed-forward input to low-level controllers and to generate reference/target trajectories for position-controlled exoskeletons. In our study, we implement four deep learning models (LSTM, FCN, CNN and Transformer) that perform one-step-ahead gait trajectory prediction after training on gait patterns of typically developing children. We propose a methodology that optimises for stability in long-term forecasts, and evaluate the performance of the models on typically developing (TD) and Cerebral Palsy (CP) gait during recursive prediction of 200 time-steps in the future (which may lead to propagation of errors) and in the presence of varying levels of Gaussian noise (1%-5%). Results on TD gait show that the FCN and Transformer, with mean absolute errors (MAEs) for one-step-ahead predictions between 1.17°-1.63°, are the most suitable for the intended application. We also proposed an approach for generating adaptive trajectories that can be used as reference trajectories for position-controlled exoskeletons. Gait patterns from children with Cerebral Palsy were fed into gait trajectory prediction models trained on typically developing gait only, to generate corrective patterns. Preliminary results show that the gait patterns of typically developing children were introduced onto the generated trajectories.

On-line Identification of Delay Attacks in Networked Servo Control

The paper discusses attacks on networked control loops by increased delay, and shows how existing round trip jitter may disguise such attacks. The attackers objective need not be de-stabilization, the paper argues that making settling time requirements fail can be sufficient. To defend against such attacks, the paper proposes the use of joint recursive prediction error identification of the round trip delay and the networked closed loop dynamics. The proposed identification algorithm allows general defense, since it is designed for delayed nonlinear dynamics in state space form. Simulations show that the method is able to detect a delay attack on a printed circuit board component mounting servo loop, long before the attack reaches full effect.

An encoder-decoder fusion battery life prediction method based on Gaussian process regression and improvement

The prediction ability of all traditional machine learning models is limited to a few batteries. When the RUL of more batteries needs to be predicted, the prediction performance of traditional machine learning is not very good. For long-term battery capacity prediction, it is obtained in a recursive way. However, adding the predicted value to the input sequence will cause a large cumulative error after multiple recursive predictions. Based on the codec model, this paper introduces Gaussian process regression, which is used in multi-step prediction to improve the codec fusion method and the Savitzky-Golay method is utilized to smooth the training set. A new kernel function was designed to further improve the accuracy. The method of dynamic weights is adopted to minimize accumulated error. The experimental results show that the fusion prediction method can reduce the cumulative error of recursive prediction without losing the declining trend of the actual capacity of the battery, and can predict the RUL of zinc-ion batteries more precisely than other models.

Recursive identification of a nonlinear state space model

SummaryThe convergence of a recursive prediction error method is analyzed. The algorithm identifies a nonlinear continuous time state space model, parameterized by one right‐hand side component of the differential equation and an output equation with a fixed differential gain, to avoid over‐parametrization. The method minimizes the criterion by simulation using an Euler discretization. A stability analysis of the associated differential equations results in conditions for (local) convergence to a minimum of the criterion function. Simulations verify the theoretical analysis and illustrate the performance in the presence of unmodeled dynamics, by identification of the nonlinear drum boiler dynamics of a power plant model.

An auto-tuned hybrid deep learning approach for predicting fracture evolution

In this study, a novel auto-tuned hybrid deep learning approach composed of three base deep learning models, namely, long short-term memory, gated recurrent unit, and support vector regression, is developed to predict the fracture evolution process. The novelty of this framework lies in the auto-determined hyperparameter configurations for each base model based on the Bayesian optimization technique, which guarantees the fast and easy implementation in various practical applications. Moreover, the ensemble modeling technique auto consolidates the predictive capability of each base model to generate the final optimized hybrid model, which offers a better prediction of the overall fracture pattern evolution, as demonstrated by a case study. The comparison of the different prediction strategies exhibits that the direct prediction is a better option than the recursive prediction, in particular for a longer prediction distance. The proposed approach may be applied in various sequential data predictions by adopting the adaptive prediction scheme.

A novel matrixed nonlinear grey Bernoulli model for interval prediction of power generation

Development trends of power generation sequences have nonlinearity and uncertainty. The nonlinearity can be reflected by the nonlinear grey Bernoulli model. The uncertainty can be reflected by the interval sequence. A matrixed nonlinear grey Bernoulli model based on interval sequences is proposed in this paper. In order to make the model directly applicable to interval sequence, the traditional grey Bernoulli model is linearized by power function transformation, then the parameters are matrixized, and finally the recursive prediction formula is obtained by Cramer's law. Particle swarm optimization algorithm is used to estimate the nonlinear parameter. By adjusting the parameter, the new model can be used to predict exponential, saturated and parabolic interval sequences. The new model is used to predict the intervals of China's thermal power, hydro power, nuclear power, wind power, national power generation and power generation in east China. The prediction results show that the new model has higher prediction accuracy than the competing models. Finally, the power generation of four modes and national power generation from 2022 to 2025 are predicted and analyzed.

Linear Models Applied to Monthly Seasonal Streamflow Series Prediction

Linear models are widely used to perform time series forecasting. The Autoregressive models stand out, due to their simplicity in the parameters adjustment based on close-form solution. The Autoregressive and Moving Average models (ARMA) and Infinite Impulse Response filters (IIR) are also good alternatives, since they are recurrent structures. However, their adjustment is more complex, since the problem has no analytical solution. This investigation performs linear models to predict monthly seasonal streamflow series, from to Brazilian hydroelectric plants. The goal is to reach the best achievable performance addressing linear approaches. We propose the application of recurrent models, estimating their parameters via an immune algorithm. To compare the optimization performance, the Least Mean Square (LMS) and Recursive Prediction Error (RPE) algorithms are utilized. Also, the AR model and the Holt-Winters method were performed. The results showed that the insertion of feedback loops increases the quality of the responses. The ARMA models optimized by the immune algorithms achieved the best overall performance.

Autonomous Air Combat Maneuvering Decision Method of UCAV Based on LSHADE-TSO-MPC under Enemy Trajectory Prediction

In this paper, an autonomous UCAV air combat maneuvering decision method based on LSHADE-TSO optimization in a model predictive control framework is proposed, along with enemy trajectory prediction. First, a sliding window recursive prediction method for multi-step enemy trajectory prediction using a Bi-LSTM network is proposed. Second, Model Predictive Control (MPC) theory is introduced, and when combined with enemy trajectory prediction, a UCAV maneuver decision model based on the MPC framework is proposed. The LSHADE-TSO algorithm is proposed by combining the LSHADE and TSO algorithms, which overcomes the problem of traditional sequential quadratic programming falling into local optimum when solving complex nonlinear models. The LSHADE-TSO-MPC air combat maneuver decision method is then proposed, which combines the LSHADE-TSO algorithm with the MPC framework and employs the LSHADE-TSO algorithm as the optimal control sequence solver. To validate the effectiveness of the maneuvering decision method proposed in this paper, it is tested against the test maneuver and the LSHADE-TSO decision algorithm, respectively, and the experimental results show that the maneuvering decision method proposed in this paper can beat the opponent and win the air combat using the same weapons and flight platform. Finally, to demonstrate that LSHADE-TSO can better exploit the decision-making ability of the MPC model, LSHADE-TSO is compared to various optimization algorithms based on the MPC model, and the results show that LSHADE-TSO-MPC can not only help obtain air combat victory faster but also demonstrates better decision-making ability.

Dealing with prognostics uncertainties: Combination of direct and recursive remaining useful life estimations

Data-driven prognostics and health management is an important part of the future industry. It allows the detection of system faults and estimation of its remaining useful life (RUL) to anticipate failures and schedule appropriate prescriptive maintenance actions. Moreover, with the development of data acquisition tools in such industries, it is possible to collect large amount of data from similar systems, which prompts the use of machine learning algorithms for efficient estimation of the system RUL. However, due to different type of faults and degradation rates that can occur in real processes, there exists high variability of the end-of-life (EoL) time of each system degradation trajectory, making more difficult to fix a failure threshold and consequently generate several uncertainties in the estimation of RUL. To address this situation, this paper proposes a new data-driven approach for estimating the system RUL when dealing with the variability of degradation trends and unknown failure thresholds. Particularly, the proposed approach combines two RUL techniques, recursive and direct RUL estimation. First, the historical collected raw data are fed into a processing algorithm to construct prognostic health indicators (HIs) and choose the one that characterizes well the system’s degradation trajectory. This latter indicator allows identifying the high and low EoL amplitude values from the historical data and is used to build a recursive prediction model that estimates in long-term the degradation trend evolution. After that, the trained model forecasts every degradation trend which EoL amplitude is less than the predefined high value. Thus, a set of possible RULs of each trajectory can be predicted. Finally, the ensemble of the derived RULs and their HI trajectories are fused to directly estimate the final RUL. The proposed approach is applied to a subway door system with multiple degradation scenarios while taking into account different operating conditions.

Forecasting monthly gas field production based on the CNN-LSTM model

Accurate prediction of gas field production is an important task for reservoir engineers, which is challenging due to many unknown reservoir parameters. Aiming to have a low-cost, intelligent, and robust method to predict gas and water production for a given gas reservoir, this paper proposes a CNN-LSTM model to predict gas field production based on a gas field in southwest China. The convolutional neural network (CNN) has a feature extraction ability, and the long short-term memory network (LSTM) can learn sequence dependence. By the combination of the two abilities, the CNN-LSTM model can describe the changing trend of gas field production. A new prediction strategy named partly unknown recursive prediction strategy (PURPS) is proposed that some input features are estimated using the predicted gas and water production according to known equations. The results show that the CNN-LSTM model can effectively predict gas field production. A detailed performance comparison was conducted between CNN-LSTM and other models. The comparison shows that the proposed CNN-LSTM model outperforms the existing methods. The monthly gas production average MAPE errors of the three different stages are CNN-LSTM (7.7%), RNN (18%), Random Forest (23.17%), ARIMA (25.3%), DNN (28.3%), Support Vector Machine (28.3%), CNN (41%), and LSTM (46%).

Long-memory recursive prediction error method for identification of continuous-time fractional models

This paper deals with recursive continuous-time system identification using fractional-order models. Long-memory recursive prediction error method is proposed for recursive estimation of all parameters of fractional-order models. When differentiation orders are assumed known, least squares and prediction error methods, being direct extensions to fractional-order models of the classic methods used for integer-order models, are compared to our new method, the long-memory recursive prediction error method. Given the long-memory property of fractional models, Monte Carlo simulations prove the efficiency of our proposed algorithm. Then, when the differentiation orders are unknown, two-stage algorithms are necessary for both parameter and differentiation-order estimation. The performances of the new proposed recursive algorithm are studied through Monte Carlo simulations. Finally, the proposed algorithm is validated on a biological example where heat transfers in lungs are modeled by using thermal two-port network formalism with fractional models.