Nonlinear Benchmark Research Articles

A robust variational mode decomposition based deep random vector functional link network for dynamic system identification

The complexity of system identification problems has been escalated due to their diverse range of applications. In this paper, the non-linear system identification problem is addressed by proposing a deep random vector functional link network (Deep-RVFLN) based on the optimized variational mode decomposition (OVMD). The proposed method has a faster learning speed and trains the network accurately without tuning parameters. Introducing a random link network connecting the input and output layers may lead to reduction in model complexity. To enhance the accuracy and reduce errors, a random vector functional link network (RVFLN) has been implemented with an increased number of hidden layers. The variational mode decomposition (VMD) algorithm is applied to decompose the signal and select optimum modes using an improved particle swarm optimization (IPSO) algorithm. In this method, the data fidelity factor (α) and the number of decomposition modes (k) are chosen by a new discrete Teaser energy operator (DTEO). The DTEO algorithm is utilized to estimate Teaser energy and it serves as a dependable indicator of overall system reliability. To test the efficacy of the model, three complex non-linear benchmark models named autoregressive (AR), moving average (MA), and autoregressive moving average (ARMA) have been considered with examples 1, 2, and 3 respectively. Based on the results and analysis, the proposed method was found to be better than other state-of-the-art methods. Finally, the proposed Deep-RVFLN identifier is implemented on a high-speed reconfigurable field-programmable gate array (FPGA) to validate the efficacy of the proposed method for non-linear system identification in the hardware platform.

Implied volatility is (almost) past-dependent: Linear vs non-linear models

We explore and demonstrate a clear pattern of past-dependence in predicting implied volatility, which extends up to twenty days and is present in both linear and nonlinear models. Empirically, we find that linear models utilizing precedent values up to 20 days explain up to 80% of the variance in the S&P 500 index, while nonlinear models explain 78%. Furthermore, we observe that L2 regularization, as opposed to L1, enhances the performance of linear models and underscores the importance of past-dependence. As a result, a simple linear model with L2 regularization, incorporating precedent values up to the past 20 days, consistently outperforms both linear and nonlinear benchmark models when applied to the S&P 500 index option and SSE 50 ETF option over the past decades. Additionally, we uncover that the impact of all preceding values exists but consistently diminishes over time in all empirical studies involving past-dependence settings. Finally, we demonstrate that these forecasts yield profitable outcomes when implementing a simple long-short portfolio trading strategy. This study primarily investigates the ‘path-dependency’ of implied volatility in financial markets and further confirms that simple models can sometimes surpass complex nonlinear models in predicting volatility.

How nonlinear benchmark in delegation contract can affect asset price and price informativeness

How nonlinear benchmark in delegation contract can affect asset price and price informativeness

Non-linear association and benchmark dose of blood pressure on carotid artery intima-media thickening in a general population of southern China.

This study aimed to evaluate whether there is a J-curve association between blood pressure (BP) and carotid artery intima-media thickening (CAIT) and estimate the effect of the turning point of BP on CAIT. Data from 111,494 regular physical examinations conducted on workers and retirees (aged 18 years or older) between January 2011 and December 2016, exported from the hospital information system, were analyzed. Restricted cubic splines (RCS) logistic regression was employed to access the association of BP with CAIT, and Bayesian benchmark dose methods were used to estimate the benchmark dose as the departure point of BP measurements. All the pnon-linear values of BP measurements were less than 0.05 in the RCS logistic regression models. Both systolic blood pressure (SBP) and diastolic blood pressure (DBP) had J-curve associations with the risk of CAIT at a turning point around 120/70 mmHg in the RCS. The benchmark dose for a 1% change in CAIT risk was estimated to be 120.64 mmHg for SBP and 72.46 mmHg for DBP. The J-curve associations between SBP and DBP and the risk of CAIT were observed in the general population in southern China, and the turning point of blood pressure for significantly reducing the risk of CAIT was estimated to be 120.64/72.46 mmHg for SBP/DBP.

Koopman operator-based multi-model for predictive control

AbstractThis work describes a new model structure developed for prediction in Model Predictive Control (MPC). The model has a multi-model structure in which independent sub-models are employed for the consecutive sampling instants. The model lifts process states into a high-dimensional space in which a linear process description is applied. Depending on the influence of the manipulated variables on lifted states, three general model versions are described and model identification algorithms are derived. As a result of the multi-model structure, model parameters are found analytically from computationally uncomplicated least squares problems using the Extended Dynamic Mode Decomposition algorithm, but the evolution of states over the horizon used in MPC is taken into account. Next, the MPC algorithm for the described model is derived. It requires solving online simple quadratic optimisation tasks. The effectiveness of three considered model configurations and three versions of the lifting functions is examined for a nonlinear DC motor benchmark. Their impact on model accuracy, complexity, possible control accuracy and MPC calculation time is thoroughly discussed. Finally, a more complex polymerisation reactor process is considered to showcase the practical applicability of the presented approach to modelling and MPC.

Hybrid algorithm for global optimization based on periodic selection scheme in engineering computation

PurposeMetaheuristic algorithms based on biology, evolutionary theory and physical principles, have been widely developed for complex global optimization. This paper aims to present a new hybrid optimization algorithm that combines the characteristics of biogeography-based optimization (BBO), invasive weed optimization (IWO) and genetic algorithms (GAs).Design/methodology/approachThe significant difference between the new algorithm and original optimizers is a periodic selection scheme for offspring. The selection criterion is a function of cyclic discharge and the fitness of populations. It differs from traditional optimization methods where the elite always gains advantages. With this method, fitter populations may still be rejected, while poorer ones might be likely retained. The selection scheme is applied to help escape from local optima and maintain solution diversity.FindingsThe efficiency of the proposed method is tested on 13 high-dimensional, nonlinear benchmark functions and a homogenous slope stability problem. The results of the benchmark function show that the new method performs well in terms of accuracy and solution diversity. The algorithm converges with a magnitude of 10-4, compared to 102 in BBO and 10-2 in IWO. In the slope stability problem, the safety factor acquired by the analogy of slope erosion (ASE) is closer to the recommended value.Originality/valueThis paper introduces a periodic selection strategy and constructs a hybrid optimizer, which enhances the global exploration capacity of metaheuristic algorithms.

Double saddle‐point preconditioning for Krylov methods in the inexact sequential homotopy method

AbstractWe derive an extension of the sequential homotopy method that allows for the application of inexact solvers for the linear (double) saddle‐point systems arising in the local semismooth Newton method for the homotopy subproblems. For the class of problems that exhibit (after suitable partitioning of the variables) a zero in the off‐diagonal blocks of the Hessian of the Lagrangian, we propose and analyze an efficient, parallelizable, symmetric positive definite preconditioner based on a double Schur complement approach. For discretized optimal control problems with PDE constraints, this structure is often present with the canonical partitioning of the variables in states and controls. We conclude with numerical results for a badly conditioned and highly nonlinear benchmark optimization problem with elliptic partial differential equations and control bounds. The resulting method allows for the parallel solution of large 3D problems.

Nonlinear system identification using modified variational autoencoders

This research proposes a methodology for identifying nonlinear systems using input/output data and deep learning generative models. Our framework integrates Variational Autoencoders (VAE) with Nonlinear Autoregressive with exogenous input (NARX) in a unified identification structure to address overfitting in nonlinear system identification using NARX structures. Specifically, we modify a variational autoencoder by replacing the decoder module with a NARX model using the latent space information captured from the VAE encoder module as one of the exogenous inputs. Following the training phase, the decoder module can be used as a nonlinear model of the system. We evaluate the efficacy of our approach by performing open-loop prediction tests on data from four nonlinear benchmark systems: Cascaded tanks, Gas furnace, Silverbox, and Wiener-Hammerstein. The proposed VAE-NARX method reported Root Mean Squared Error (RMSE) of 8.23×10−3, 16.69×10−3, 0.002×10−3 and 0.037×10−3 respectively. Our results demonstrate that our proposed method achieves similar and outperforms prediction performances to standard identification techniques and can enhance the performance of traditional nonlinear system identification methods based on multi-layer perceptron models.

A hybrid RBF neural network based model for day-ahead prediction of photovoltaic plant power output

Renewable energy resources like solar power contribute greatly to decreasing emissions of carbon dioxide and substituting generators fueled by fossil fuels. Due to the unpredictable and intermittent nature of solar power production as a result of solar radiance and other weather conditions, it is very difficult to integrate solar power into conventional power systems operation economically in a reliable manner, which would emphasize demand for accurate prediction techniques. The study proposes and applies a revised radial basis function neural network (RBFNN) scheme to predict the short-term power output of photovoltaic plant in a day-ahead prediction manner. In the proposed method, the linear as well as non-linear variables in the RBFNN scheme are efficiently trained using the whale optimization algorithm to speed the convergence of prediction results. A nonlinear benchmark function has also been used to validate the suggested scheme, which was also used in predicting the power output of solar energy for a well-designed experiment. A comparison study case generating different outcomes shows that the suggested approach could provide a higher level of prediction precision than other methods in similar scenarios, which suggests the proposed method can be used as a more suitable tool to deal such solar energy forecasting issues.

Optimal nonlinear system identification using fractional delay second-order volterra system

The aim of this work is to design a fractional delay second order Volterra filter that takes a discrete time sequence as input and its output is as close as possible to the output of a given nonlinear unknown system which may have higher degree nonlinearities in the least square sense. The basic reason for such a design is that rather than including higher than second degree nonlinearities in the designed system, we use the fractional delay degrees of freedom to approximate the given system. The advantage is in terms of obtaining a better approximation of the given nonlinear system than is possible by using only integer delays ( since we are giving more degrees of freedom via the fractional delays ) and simultaneously it does not require to incorporate higher degree nonlinearities than two. This work hinges around the fact that if the input signal is a decimated version of another signal by a factor of M, then fractional delays can be regarded as delays by integers less than M. Using the well known formula for calculating the discrete time Fourier transform ( TFT ) of a decimated signal, we then arrive at an expression for the DTFT of the output of a fractional delay system in terms of the unknown first and second order Volterra system coefficients and the fractional delays. The final energy function to be minimized is the norm square of the difference between the DTFT of the given output and the DTFT of the output of the fractional delay system. Minimization over the filter coefficients is a linear problem and thus the final problem is to minimize a highly nonlinear function of the fractional delays which is accomplished using search techniques like the gradient-search and nature inspired optimization algorithms. The effectiveness of the proposed method is demonstrated using two nonlinear benchmark systems tested with five different input signals. The accuracy of the stated models using the globally convergent metaheuristic, cuckoo-search algorithm ( CSA ) are observed to be superior when compared with other techniques such as real-coded genetic algorithm ( RGA ), particle swarm optimization ( PSO ) and gradient-search ( GS ) methods. Finally, statistical analysis affirms the potential of the proposed designs for its successful implementation.

Optimal nonlinear system identification using fractional delay second-order volterra system

The aim of this work is to design a fractional delay second order Volterra filter that takes a discrete time sequence as input and its output is as close as possible to the output of a given nonlinear unknown system which may have higher degree nonlinearities in the least square sense. The basic reason for such a design is that rather than including higher than second degree nonlinearities in the designed system, we use the fractional delay degrees of freedom to approximate the given system. The advantage is in terms of obtaining a better approximation of the given nonlinear system than is possible by using only integer delays U+0028 since we are giving more degrees of freedom via the fractional delays U+0029 and simultaneously it does not require to incorporate higher degree nonlinearities than two. This work hinges around the fact that if the input signal is a decimated version of another signal by a factor of M, then fractional delays can be regarded as delays by integers less than M. Using the well known formula for calculating the discrete time Fourier transform U+0028 DTFT U+0029 of a decimated signal, we then arrive at an expression for the DTFT of the output of a fractional delay system in terms of the unknown first and second order Volterra system coefficients and the fractional delays. The final energy function to be minimized is the norm square of the difference between the DTFT of the given output and the DTFT of the output of the fractional delay system. Minimization over the filter coefficients is a linear problem and thus the final problem is to minimize a highly nonlinear function of the fractional delays which is accomplished using search techniques like the gradient-search and nature inspired optimization algorithms. The effectiveness of the proposed method is demonstrated using two nonlinear benchmark systems tested with five different input signals. The accuracy of the stated models using the globally convergent metaheuristic, cuckoo-search algorithm U+0028 CSA U+0029 are observed to be superior when compared with other techniques such as real-coded genetic algorithm U+0028 RGA U+0029, particle swarm optimization U+0028 PSO U+0029 and gradient-search U+0028 GS U+0029 methods. Finally, statistical analysis affirms the potential of the proposed designs for its successful implementation.

Linear Quadratic Optimal Robust Multivariable Proportional-Integral Control Design for a Boiler-Turbine Unit

Abstract Boiler-turbine unit is a highly nonlinear, coupled, and ill-conditioned system. Moreover, the presence of physical constraints, such as actuator magnitude and rate limits, makes the system difficult to control. This paper employs a fixed multivariable proportional-integral (PI) controller of the I–P structure for robust linear quadratic (LQ) compensation of a nonlinear boiler-turbine benchmark. In order to ensure that a single PI controller works for the whole operating region of this nonlinear system, the linearized model of the system is represented as a norm-bounded, time-varying uncertain system. The ranges of the uncertain parameters of this linearized model are determined from different operating points of the nonlinear system. To design the PI controller for the uncertain system, first, it is transformed into a state feedback design for an augmented uncertain system and then the state feedback gains satisfying some LQ performance limit are computed by solving a linear matrix inequality (LMI) problem. As the uncertainty in the feedforward matrix of the linearized model cannot be considered in the above design process, an LMI-based method is developed to check if the designed PI controller performance in H∞ sense is close to the one if the neglected uncertainty is included. The performance of the controller is tested on the nonlinear boiler-turbine unit under several operating conditions and physical constraints. Comparisons are also made with some existing PI controllers, to show the superiority of the proposed robust PI controller.

A novel adaptive-weight ensemble surrogate model base on distance and mixture error.

Surrogate models are commonly used as a substitute for the computation-intensive simulations in design optimization. However, building a high-accuracy surrogate model with limited samples remains a challenging task. In this paper, a novel adaptive-weight ensemble surrogate modeling method is proposed to address this challenge. Instead of using a single error metric, the proposed method takes into account the position of the prediction sample, the mixture error metric and the learning characteristics of the component surrogate models. The effectiveness of proposed ensemble models are tested on five highly nonlinear benchmark functions and a finite element model for the analysis of the frequency response of an automotive exhaust pipe. Comparative results demonstrate the effectiveness and promising potential of proposed method in achieving higher accuracy.

DeepBayes—An estimator for parameter estimation in stochastic nonlinear dynamical models

Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks — long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.

The simplified MITC4+ shell element and its performance in linear and nonlinear analysis

The recently developed improved MITC4+ shell element exhibits excellent convergence behavior in both bending- and membrane-dominated shell problems. Compared to the widely recognized MITC4 shell element, the membrane and in-plane shear locking were significantly alleviated. However, it is difficult to implement the assumed membrane strain field because of its complexity. In this study, a simplified MITC4+ shell element is developed. A simpler assumed membrane strain field that facilitates the extension to nonlinear formulations is proposed. To reduce the sensitivity of the element performance to mesh distortion, a geometry-dependent Gauss integration scheme is employed. The proposed element passes the patch, zero-energy mode, and isotropy tests. The performance of the simplified MITC4+ shell element is demonstrated using various linear and nonlinear benchmark problems.

Likelihood-ratio test statistic for the finite-sample case in nonlinear ordinary differential equation models.

Likelihood ratios are frequently utilized as basis for statistical tests, for model selection criteria and for assessing parameter and prediction uncertainties, e.g. using the profile likelihood. However, translating these likelihood ratios into p-values or confidence intervals requires the exact form of the test statistic's distribution. The lack of knowledge about this distribution for nonlinear ordinary differential equation (ODE) models requires an approximation which assumes the so-called asymptotic setting, i.e. a sufficiently large amount of data. Since the amount of data from quantitative molecular biology is typically limited in applications, this finite-sample case regularly occurs for mechanistic models of dynamical systems, e.g. biochemical reaction networks or infectious disease models. Thus, it is unclear whether the standard approach of using statistical thresholds derived for the asymptotic large-sample setting in realistic applications results in valid conclusions. In this study, empirical likelihood ratios for parameters from 19 published nonlinear ODE benchmark models are investigated using a resampling approach for the original data designs. Their distributions are compared to the asymptotic approximation and statistical thresholds are checked for conservativeness. It turns out, that corrections of the likelihood ratios in such finite-sample applications are required in order to avoid anti-conservative results.

Recurrent Neural Network Training With Convex Loss and Regularization Functions by Extended Kalman Filtering

This paper investigates the use of extended Kalman filtering to train recurrent neural networks with rather general convex loss functions and regularization terms on the network parameters, including <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -regularization. We show that the learning method is competitive with respect to stochastic gradient descent in a nonlinear system identification benchmark and in training a linear system with binary outputs. We also explore the use of the algorithm in data-driven nonlinear model predictive control and its relation with disturbance models for offset-free closed-loop tracking.

Nonlinear verification of the resistive-wall boundary modules in the specyl and pixie3d magneto-hydrodynamic codes for fusion plasmas

A nonlinear verification benchmark is reported between the three-dimensional magneto-hydrodynamic (3D MHD) codes specyl [Cappello and Biskamp, Nucl. Fusion 36, 571 (1996)] and pixie3d [Chacón, Phys. Plasmas, 15, 056103 (2008)]. This work substantially extends a former successful verification study between the same two codes [Bonfiglio et al., Phys. Plasmas, 17, 082501 (2010)] and focuses on the verification of thin-shell resistive-wall boundary conditions, recently implemented in both codes. Such boundary conditions feature a thin resistive shell in contact with the plasma and an ideal wall placed at a finite distance, separated from the resistive shell by a vacuum region, along with a 3D boundary flow consistent with Ohm’s law. This setup allows the study of MHD modes that are influenced by the plasma magnetic boundary, such as external kink modes. The linear growth and nonlinear saturation of external kink modes are studied in both the tokamak and reversed-field pinch magnetic configurations, demonstrating excellent agreement between the two codes. For the tokamak, we present a comparison with analytical linear stability results for the external kink mode, demonstrating remarkable agreement between numerical and analytical growth rates.

A fuzzy based intelligent scheme for enhancing the performance of the optimal controllers by online weighting matrix selection in seismically excited nonlinear buildings

In the present work, an effective strategy is proposed to improve the efficiency of the optimal controllers that are based on minimizing a performance index by using a fuzzy controller. In the used fuzzy system, the weighting matrices of the optimal controller are tuned online according to the response of the structural system. In order to assess the effectiveness of the proposed control scheme, its function is investigated in two phases on a 3-story nonlinear benchmark building equipped with MR dampers. In the first phase, a linear quadratic Gaussian (LQG) controller is combined with a fuzzy logic system (Fuzzy-LQG) and the efficiency of the used technique is compared with three different cases of the conventional LQG controllers. The results show that the Fuzzy-LQG, in addition to using an optimal level of demand energy, has a better performance than the conventional LQG methods in controlling the maximum responses of the structure. In the second phase, the performance of a polynomial optimal controller that is a summation of polynomials in nonlinear states in combination with the fuzzy system is investigated. At this stage, the effectiveness of the designed control system (Fuzzy Polynomial) in terms of seismic performance criteria with the results of a conventional LQG active control system, an optimal adaptive neuro-fuzzy inference system (ANFIS) and a neural network predictive control (NNPC) system is evaluated. The results show that this method is successful in improving the structural performance of a nonlinear building subjected to earthquakes.

Kirchhoff–Love shell representation and analysis using triangle configuration B-splines

This paper presents the application of triangle configuration B-splines (TCB-splines) for representing and analyzing the Kirchhoff–Love shell in the context of isogeometric analysis (IGA). TCB-splines offer flexibility in modeling complex geometries with C1 continuity, making them naturally fit into the Kirchhoff–Love shell formulation with complex geometries. We first propose a linear least-squares-based framework to reparametrize the mid-surface of a thin shell, which consists of multiple (trimmed) NURBS patches and is topologically equivalent to an open disk with a finite number of holes, into a single TCB-surface defined over a carefully computed parametric domain. We then utilize TCB-splines for geometric representation and solution approximation in shell analysis. We verify the accuracy and robustness of our method by applying it to linear and nonlinear benchmark shell problems. The applicability of the proposed approach to shell analysis is further exemplified by performing geometrically nonlinear Kirchhoff–Love shell simulations of a pipe junction and a front bumper represented by a single patch of TCB-splines.