Recursive Prediction Research Articles

Surrogate Modeling of Nonlinear Dynamic Systems: A Comparative Study

Abstract Surrogate models play a vital role in overcoming the computational challenge in designing and analyzing nonlinear dynamic systems, especially in the presence of uncertainty. This paper presents a comparative study of different surrogate modeling techniques for nonlinear dynamic systems. Four surrogate modeling methods, namely, Gaussian process (GP) regression, a long short-term memory (LSTM) network, a convolutional neural network (CNN) with LSTM (CNN-LSTM), and a CNN with bidirectional LSTM (CNN-BLSTM), are studied and compared. All these model types can predict the future behavior of dynamic systems over long periods based on training data from relatively short periods. The multi-dimensional inputs of surrogate models are organized in a nonlinear autoregressive exogenous model (NARX) scheme to enable recursive prediction over long periods, where current predictions replace inputs from the previous time window. Three numerical examples, including one mathematical example and two nonlinear engineering analysis models, are used to compare the performance of the four surrogate modeling techniques. The results show that the GP-NARX surrogate model tends to have more stable performance than the other three deep learning (DL)-based methods for the three particular examples studied. The tuning effort of GP-NARX is also much lower than its deep learning-based counterparts.

Data-Driven State Prediction and Analysis of SOFC System Based on Deep Learning Method

A solid oxide fuel cell (SOFC) system is a kind of green chemical-energy–electric-energy conversion equipment with broad application prospects. In order to ensure the long-term stable operation of the SOFC power-generation system, prediction and evaluation of the system’s operating state are required. The mechanism of the SOFC system has not been fully revealed, and data-driven single-step prediction is of little value for practical applications. The state-prediction problem can be regarded as a time series prediction problem. Therefore, an innovative deep learning model for SOFC system state prediction is proposed in this study. The model uses a two-layer LSTM network structure that supports multiple sequence feature inputs and flexible multi-step prediction outputs, which allows multi-step prediction of system states using SOFC system experimental data. Comparing the proposed model with the traditional ARIMA model and LSTM recursive prediction model, it is shown that the multi-step LSTM prediction model performs better than the ARIMA and LSTM recursive prediction models in terms of two evaluation criteria: root mean square error and mean absolute error. Thus, the proposed multi-step LSTM prediction model can effectively and accurately predict and evaluate the SOFC system’s state.

Recursive Predictive Optimal Control Algorithm for Real-Time Hybrid Simulation of Vehicle–Bridge Coupling System

Real-time hybrid simulation (RTHS) is recently applied to study the vibration of the vehicle–bridge coupling system, where bridges and the vehicle are the numerical and experimental substructures respectively. Inevitable time delay of the boundary loading imposed to the experimental substructure will greatly affect the accuracy and stability of RTHS. To minimize the time delay effects, this paper proposes a recursive prediction optimal (RPO) compensator based on optimal control and recursive least squares. The numerical simulation of the RTHS of the vehicle–bridge coupling system is conducted using MATLAB/Simulink. The performance of the RPO method is compared with three traditional time delay compensation methods (i.e. the polynomial extrapolation (NI), the inverse compensation (IC) and the derivative feedforward (DF)) through both open-loop and RTHS simulations. The RPO method shows the best compensation performance in the frequency range up to 20[Formula: see text]Hz.

Recursive Prediction for Lossless Image Compression

This paper introduced an algorithm for lossless image compression to compress natural and medical images. It is based on utilizing various casual fixed predictors of one or two dimension to get rid of the correlation or spatial redundancy embedded between image pixel values then a recursive polynomial model of a linear base is used.
 The experimental results of the proposed compression method are promising in terms of preserving the details and the quality of the reconstructed images as well improving the compression ratio as compared with the extracted results of a traditional linear predicting coding system.

Recursive long short-term memory network for predicting nonlinear structural seismic response

Artificial neural networks have been used to predict nonlinear structural time histories under seismic excitation because they have a significantly lower computational cost than the traditional time-step integration method. However, most existing techniques require simplification procedures such as downsampling to maintain identical length and sampling rates, and they lack sufficient accuracy, generality, or interpretability. In this paper, a recursive long short-term memory (LSTM) network was proposed for predicting nonlinear structural seismic responses for arbitrary lengths and sampling rates. Referring to the traditional integral solution method, the proposed LSTM model uses the recursive prediction principle and is therefore applicable to structures and earthquakes with different spectral characteristics and amplitudes. The measured ground motions and multilayer frame structures were used for model training and validation. The rules of hyperparameter selection for practical applications are herein discussed. The results showed that the proposed recursive LSTM model can adequately reproduce the global and local characteristics of the time history responses on four different structural response datasets, exhibiting good accuracy and generalization capability.

Spatiotemporal and layout-adaptive prediction of leak gas dispersion by encoding-prediction neural network

Gas leak accident has been troublesome issues in the chemical industries. Predicting dispersion boundaries are important to make rapid and proper actions. Currently, computational fluid dynamics (CFD) are used to predict the dispersion boundaries. However, when the facility-layout of a workplace is often modified, using CFD is not desirable since it requires large computational expenses. This study proposes an encoding-prediction neural network to learn representations between dispersion of leak gas, velocity field, and facility-layouts. This network predict volume fraction field of leak gas in t + kΔt timestep by observing that data in t ∼ t + (k-1)Δt timestep. Training and test losses are decreased to 1.04 × 10−5 and 1.46 × 10−5, respectively. The network predicts dispersion of leak gas through recursive prediction scheme, the predicted results shows good agreement with ground truth. Methodology to generated various facility-layouts, and preprocessing methods to deal with skewed data are suggested. The methodology and results proposed in this study would be useful for developing the CFD surrogate model.

Goal-Directed Navigation with A Hybrid Planning Strategy Through Learning from Vision

The existing hippocampal modeling approaches rarely span the wide functionality range from processing raw sensory signals to planning and action. This paper presents a goal-directed navigation system consisting of two planning strategies. The first one is a biologically inspired neural planning and navigation model that is related to learned representations of place and HD cells. It is responsible for generating spatial trajectories leading to the neighboring area of the target. The place and HD cells are trained unsupervisedly from visual images using a modified slow feature analysis (SFA) algorithm. To interpret their functional role in navigation, a planning network is trained to predict the neural activities of place and HD cell representations given selected action signals. Recursive prediction and optimization of the action signals generate goal-directed activation sequences, in which the continuous states and action spaces are represented by the population of place-, HD- and motor neuron activities. Furthermore, a second planning strategy relying on visual recognition is proposed and performs target-driven reaching on a local scale for finer accuracy. Experimental results show the effectiveness of the proposed system.

Collision Prediction and Optimal Trajectory Generation for Collision Avoidance Systems in Trackless Mobile Machines

The continued high number of fatalities associated with Trackless Mobile Machines (TMMs) in South Africa have led to the introduction of Collision Avoidance System (CAS) regulations in the Mine Health and Safety Act in 2015. This has lead to the profusion of technologically-immature CASs from third-party vendors, all of which are centered on automatic stopping and braking systems. These braking systems often result in trivial or ineffective solutions, proving costly to mining operations. The combination of braking and steering control in CASs may substantially increase the solution space and provide far safer and more efficient manoeuvres. A recursive non-linear collision prediction estimator and optimal trajectory generation model was developed to evaluate the potential contribution of the addition of steering to CASs. Three independent optimal trajectory generation models are proposed to compete against one another in an attempt to synthesize the safest, most predictable, and efficient trajectory. A deep reinforcement learning, lattice optimization and Monte Carlo hyper sampling path planning model’s trajectories are evlauated using the Earth Moving Equipment Safety Round Table (EMESRT) interaction scenarios. Initial results indicate increased CAS solution spaces in collision-avoiding scenarios, providing safer and more effective solutions in high velocity vehicle interactions.

A recursive framework for predicting the time-course of drug sensitivity

The biological processes involved in a drug’s mechanisms of action are oftentimes dynamic, complex and difficult to discern. Time-course gene expression data is a rich source of information that can be used to unravel these complex processes, identify biomarkers of drug sensitivity and predict the response to a drug. However, the majority of previous work has not fully utilized this temporal dimension. In these studies, the gene expression data is either considered at one time-point (before the administration of the drug) or two time-points (before and after the administration of the drug). This is clearly inadequate in modeling dynamic gene–drug interactions, especially for applications such as long-term drug therapy. In this work, we present a novel REcursive Prediction (REP) framework for drug response prediction by taking advantage of time-course gene expression data. Our goal is to predict drug response values at every stage of a long-term treatment, given the expression levels of genes collected in the previous time-points. To this end, REP employs a built-in recursive structure that exploits the intrinsic time-course nature of the data and integrates past values of drug responses for subsequent predictions. It also incorporates tensor completion that can not only alleviate the impact of noise and missing data, but also predict unseen gene expression levels (GEXs). These advantages enable REP to estimate drug response at any stage of a given treatment from some GEXs measured in the beginning of the treatment. Extensive experiments on two datasets corresponding to multiple sclerosis patients treated with interferon are included to showcase the effectiveness of REP.

Deeply supervised group recursive saliency prediction

Fully convolutional neural networks (FCNs) have shown great effects in the saliency detection task. In this paper, we present an FCN based model to aggregate multi-level convolutional features for salient object detection. The main contribution of this work is to make full use of the information extracted by the network while preserving the characteristics of each level. We first propose a novel Intra-stage Feature Aggregating (IFA) module to enhance the detailed information by integrating feature maps from different layers within the same stage of the ResNet-50. We then design an Inter-stage Feature Fusion (IFF) module which exploits contextual information by combining the features from neighboring stages to make each feature map more discriminative. Finally, we propose a Deeply Supervised Group Recursive Prediction (DGR) module to refine the details of the predicted saliency maps and generate the final saliency map. Our approach performs favorably against state-of-the-art methods on five popular datasets and achieves a real-time speed of 30 fps.

Prediction of Chlorophyll-a Concentrations in the Nakdong River Using Machine Learning Methods

Many studies have attempted to predict chlorophyll-a concentrations using multiple regression models and validating them with a hold-out technique. In this study commonly used machine learning models, such as Support Vector Regression, Bagging, Random Forest, Extreme Gradient Boosting (XGBoost), Recurrent Neural Network (RNN), and Long–Short-Term Memory (LSTM), are used to build a new model to predict chlorophyll-a concentrations in the Nakdong River, Korea. We employed 1–step ahead recursive prediction to reflect the characteristics of the time series data. In order to increase the prediction accuracy, the model construction was based on forward variable selection. The fitted models were validated by means of cumulative learning and rolling window learning, as opposed to the hold–out technique. The best results were obtained when the chlorophyll-a concentration was predicted by combining the RNN model with the rolling window learning method. The results suggest that the selection of explanatory variables and 1–step ahead recursive prediction in the machine learning model are important processes for improving its prediction performance.

GAN-Poser: an improvised bidirectional GAN model for human motion prediction

A novel method called GAN-Poser has been explored to predict human motion in less time given an input 3D human skeleton sequence based on a generator–discriminator framework. Specifically, rather than using the conventional Euclidean loss, a frame-wise geodesic loss is used for geometrically meaningful and more precise distance measurement. In this paper, we have used a bidirectional GAN framework along with a recursive prediction strategy to avoid mode-collapse and to further regularize the training. To be able to generate multiple probable human-pose sequences conditioned on a given starting sequence, a random extrinsic factor $$\varTheta$$ has also been introduced. The discriminator is trained in order to regress the extrinsic factor $$\varTheta$$ , which is used alongside with the intrinsic factor (encoded starting pose sequence) to generate a particular pose sequence. In spite of being in a probabilistic framework, the modified discriminator architecture allows predictions of an intermediate part of pose sequence to be used as conditioning for prediction of the latter part of the sequence. This adversarial learning-based model takes into consideration of the stochasticity, and the bidirectional setup provides a new direction to evaluate the prediction quality against a given test sequence. Our resulting novel method, GAN-Poser, achieves superior performance over the state-of-the-art deep learning approaches when evaluated on the standard NTU-RGB-D and Human3.6 M dataset.

Linearization of RF Power Amplifiers in Wideband Communication Systems by Adaptive Indirect Learning Using RPEM Algorithm

This paper proposes a new approach of digital predistortion (DPD) technique based on the adaptive indirect learning architecture (ILA) by using a recursive prediction error minimization (RPEM) algorithm for linearizing radio frequency (RF) power amplifiers (PAs) in emerging wideband communication systems. In the proposed RPEM-based linearization approach, the forgetting factor varies with time and is less sensitive to noise. Therefore, the predistorter (PD) parameter estimates become more consistent and accurate in steady state so that the mean square errors can be reduced. Both the error vector magnitude (EVM) and the adjacent channel power ratio (ACPR) are used to evaluate the DPD technique in RF PAs employing the proposed linearization. The efficiency validation of the proposed method is based on a simulated PA Wiener model. The simulation results have clarified the improvement of the proposed adaptive ILA-based DPD with RPEM algorithm in terms of both EVM and ACPR.

Least-squares estimators for systems with stochastic sensor gain degradation, correlated measurement noises and delays in transmission modelled by Markov chains

ABSTRACTThis paper addresses the linear least-squares estimation of a signal from measurements subject to stochastic sensor gain degradation and random delays during the transmission. These uncertainty phenomena, common in network systems, have traditionally been described by independent Bernoulli random variables. We propose a model that is more general and therefore has greater applicability to real-life situations. The model has two particular characteristics: firstly, the sensor gain degradation is represented by a white sequence of random variables with values in [0,1]; in addition, the absence or presence of delays in the transmission is described by a homogeneous three-state Markov chain, which reflects a possible correlation of delays at different sampling times. Furthermore, assuming that the measurement noise is one-step correlated, we obtain recursive prediction, filtering and fixed-point smoothing algorithms using the first and second-order moments of the signal and the processes present in the observation model. Simulation results for a scalar signal are provided to illustrate the feasibility of the proposed algorithms, using the estimation error variances as a measure of the quality of the estimators.

Innovations in Financial Management: Recursive Prediction Model Based on Decision Trees

The issue of enterprise financial distress represents the actual and interdisciplinary topic for the economic community. The bankrupt is thus one of the major externalities of today’s modern economies, which cannot be avoided even with every effort. Where there are investment opportunities, there are individuals and businesses that are willing to assume their financial obligations and the resulting risks to maintain and develop their standard of living or their economic activities. The decision tree algorithm is one of the most intuitive methods of data mining that can be used for financial distress prediction. Systematization literary sources and approaches prove that decision trees represent the part of the innovations in financial management. The main propose of the research is a possibility of application of a decision tree algorithm for the creation of the prediction model, which can be used in economy practice. The Paper's main aim is to create a comprehensive prediction model of enterprise financial distress based on decision trees, under the conditions of emerging markets. Paper methods are based on the decision tree, with emphasis on algorithm CART. Emerging markets included 17 countries: Slovak Republic, Czech Republic, Poland, Hungary, Romania, Bulgaria, Lithuania, Latvia, Estonia, Slovenia, Croatia, Serbia, Russia, Ukraine, Belarus, Montenegro, and Macedonia. Paper research is focused on the possibilities of implementation of a decision tree algorithm for the creation of a prediction model in the condition of emerging markets. Used data contained 2,359,731 enterprises from emerging markets (30% of total amount); divided into prosperous enterprises (1,802,027) and non-prosperous enterprises (557,704); obtained from Amadeus database. Input variables for the model represented 24 financial indicators, 3 dummy variables, and the countries' GDP data, in the years 2015 and 2016. The 80% of enterprises represented the training sample and 20% test sample, for model creation. The model correctly classified 93.2% of enterprises from both the training and test sample. Correctly classification of non-prosperous enterprises was 83.5% in both samples. The result of the research brings a new model for the identification of bankrupt enterprises. The created prediction model can be considered sufficiently suitable for classifying enterprises in emerging markets. Keywords prediction model, decision tree, emerging markets.

A simple parameter‐driven binary time series model

AbstractWe introduce a parameter‐driven, state‐space model for binary time series data. The model is based on a state process with a binomial‐beta dynamics, which has a Markov, endogenous switching regime representation. The model allows for recursive prediction and filtering formulas with extremely low computational cost, and hence avoids the use of computational intensive simulation‐based filtering algorithms. Case studies illustrate the advantage of our model over popular intensity‐based observation‐driven models, both in terms of fit and out‐of‐sample forecast.

Active vibration control for the time-varying systems with a new adaptive algorithm

This paper proposes a new adaptive algorithm for the active vibration control of time-varying systems in the presence of broadband or narrowband disturbances. The new algorithm combines the conventional filtered-x least mean square algorithm with the recursive prediction error (RPE) algorithm after the gradient modification of the RPE algorithm. The modified RPE algorithm is used to estimate the model of the control path online. The well-known filtered-x least mean square (FxLMS) algorithm is effective for the uncertain or time-varying systems, and adopts an auxiliary white noise approach to estimate the model of the control path online. However, the auxiliary excitation will degrade the control performance to some extent. In the new algorithm, the auxiliary excitation is eliminated at the expense of a larger computational burden. The influence of the estimated finite impulse response series on the convergence is also discussed. A propulsion shafting model with the time-varying dynamics is established by frequency response function synthesis. Numerical simulation for the established model is presented to demonstrate the superior performance of the proposed algorithm as compared with the FxLMS algorithm.

Particle filtering‐based recursive identification for controlled auto‐regressive systems with quantised output

Recursive prediction error method is one of the main tools for analysis of controlled auto-regressive systems with quantised output. In this study, a recursive identification algorithm is proposed based on the auxiliary model principle by modifying the standard stochastic gradient algorithm. To improve the convergence performance of the algorithm, a particle filtering technique, which approximates the posterior probability density function with a weighted set of discrete random sampling points is utilised to correct the linear output estimates. It can exclude those invalid particles according to their corresponding weights. The performance of the particle filtering technique-based algorithm is much better than that of the auxiliary model-based one. Finally, results are verified by examples from simulation and engineering.

BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN

Human motion prediction model has applications in various fields of computer vision. Without taking into account the inherent stochasticity in the prediction of future pose dynamics, such methods often converges to a deterministic undesired mean of multiple probable outcomes. Devoid of this, we propose a novel probabilistic generative approach called Bidirectional Human motion prediction GAN, or BiHMP-GAN. To be able to generate multiple probable human-pose sequences, conditioned on a given starting sequence, we introduce a random extrinsic factor r, drawn from a predefined prior distribution. Furthermore, to enforce a direct content loss on the predicted motion sequence and also to avoid mode-collapse, a novel bidirectional framework is incorporated by modifying the usual discriminator architecture. The discriminator is trained also to regress this extrinsic factor r, which is used alongside with the intrinsic factor (encoded starting pose sequence) to generate a particular pose sequence. To further regularize the training, we introduce a novel recursive prediction strategy. In spite of being in a probabilistic framework, the enhanced discriminator architecture allows predictions of an intermediate part of pose sequence to be used as a conditioning for prediction of the latter part of the sequence. The bidirectional setup also provides a new direction to evaluate the prediction quality against a given test sequence. For a fair assessment of BiHMP-GAN, we report performance of the generated motion sequence using (i) a critic model trained to discriminate between real and fake motion sequence, and (ii) an action classifier trained on real human motion dynamics. Outcomes of both qualitative and quantitative evaluations, on the probabilistic generations of the model, demonstrate the superiority of BiHMP-GAN over previously available methods.

Multivariate Shannon's entropy for adaptive IIR filtering via kernel density estimators

In supervised infinite impulse response adaptive filtering, approximate gradient-based approaches are the usual option among optimisation methods. When based on the mean squared error (MSE) criterion, however, these approaches may present biased solutions in noisy scenarios. In that sense, instead of the MSE, the authors propose the use of Shannon's error entropy, an information theoretic learning criterion, which is able to extract higher order statistics from the underlying signals. In particular, a multivariate entropy definition is considered, which is applied to derive a Recursive Prediction Error-based algorithm. The performance analyses are carried out in the context of the supervised channel equalisation problem, with results very favourable to the proposal, in high and low noise level environments.