Nonlinear Benchmarks Research Articles

Exploiting the capacity of deep networks only at the training stage for nonlinear black-box system identification

To benefit from the modeling capacity of deep models in system identification without worrying about inference time, this study presents a novel training strategy that uses deep models only during the training stage. For this purpose, two separate models with different structures and goals are employed. The first one is a deep generative model aiming at modeling the distribution of system output(s), called the teacher model, and the second one is a shallow basis function model, named the student model, fed by system input(s) to predict the system output(s). That means these isolated paths must reach the same ultimate target. As deep models show a great performance in modeling highly nonlinear systems, aligning the representation space learned by these two models makes the student model inherit the teacher model’s approximation power. The proposed objective function consists of the objective of each student and teacher model, adding up with a distance penalty between the learned latent representations. The simulation results on three nonlinear benchmarks show a comparative performance with examined deep architectures applied on the same benchmarks. Algorithmic transparency and structure efficiency are also achieved as byproducts.

Industry return prediction via interpretable deep learning

We apply an interpretable machine learning model, the LassoNet, to forecast and trade U.S. industry portfolio returns. The model combines a regularization mechanism with a neural network architecture. A cooperative game-theoretic algorithm is also applied to interpret our findings. The latter hierarchizes the covariates based on their contribution to the overall model performance. Our findings reveal that the LassoNet outperforms various linear and nonlinear benchmarks concerning out-of-sample forecasting accuracy and provides economically meaningful and profitable predictions. Valuation ratios are the most crucial covariates, followed by individual and cross-industry lagged returns. The constructed industry ETF portfolios attain positive Sharpe ratios and positive and statistically significant alphas, surviving even transaction costs.

Optimizing quantum noise-induced reservoir computing for nonlinear and chaotic time series prediction

Quantum reservoir computing is strongly emerging for sequential and time series data prediction in quantum machine learning. We make advancements to the quantum noise-induced reservoir, in which reservoir noise is used as a resource to generate expressive, nonlinear signals that are efficiently learned with a single linear output layer. We address the need for quantum reservoir tuning with a novel and generally applicable approach to quantum circuit parameterization, in which tunable noise models are programmed to the quantum reservoir circuit to be fully controlled for effective optimization. Our systematic approach also involves reductions in quantum reservoir circuits in the number of qubits and entanglement scheme complexity. We show that with only a single noise model and small memory capacities, excellent simulation results were obtained on nonlinear benchmarks that include the Mackey-Glass system for 100 steps ahead in the challenging chaotic regime.

Patch-wise quadrature of trimmed surfaces in Isogeometric Analysis

This work presents an efficient quadrature rule for shell analysis fully integrated in CAD by means of Isogeometric Analysis (IGA). General CAD-models may consist of trimmed parts such as holes, intersections, cut-offs etc. Therefore, IGA should be able to deal with these models in order to fulfil its promise of closing the gap between design and analysis. Trimming operations violate the tensor-product structure of the used Non-Uniform Rational B-spline (NURBS) basis functions and of typical quadrature rules. Existing efficient patch-wise quadrature rules consider actual knot vectors and are determined in 1D. They are extended to further dimensions by means of a tensor-product. Therefore, they are not directly applicable to trimmed structures. The herein proposed method extends patch-wise quadrature rules to trimmed surfaces. Thereby, the number of quadrature points can be significantly reduced. Geometrically linear and non-linear benchmarks of plane, plate and shell structures are investigated. The results are compared to a standard trimming procedure and a good performance is observed.

Self-Supervised Primal-Dual Learning for Constrained Optimization

This paper studies how to train machine-learning models that directly approximate the optimal solutions of constrained optimization problems. This is an empirical risk minimization under constraints, which is challenging as training must balance optimality and feasibility conditions. Supervised learning methods often approach this challenge by training the model on a large collection of pre-solved instances. This paper takes a different route and proposes the idea of Primal-Dual Learning (PDL), a self-supervised training method that does not require a set of pre-solved instances or an optimization solver for training and inference. Instead, PDL mimics the trajectory of an Augmented Lagrangian Method (ALM) and jointly trains primal and dual neural networks. Being a primal-dual method, PDL uses instance-specific penalties of the constraint terms in the loss function used to train the primal network. Experiments show that, on a set of nonlinear optimization benchmarks, PDL typically exhibits negligible constraint violations and minor optimality gaps, and is remarkably close to the ALM optimization. PDL also demonstrated improved or similar performance in terms of the optimality gaps, constraint violations, and training times compared to existing approaches.

A robust model predictive control algorithm for input–output LPV systems using parameter extrapolation

In this paper, we introduce a novel robust Model Predictive Control (MPC) algorithm for Linear Parameter Varying (LPV) systems represented in the Input–Output (IO) form. The proposed scheme embeds integral action and ensures output reference tracking for piece-wise constant signals. The algorithm is based on the online extrapolation of the LPV scheduling parameters, which are generated recursively based on a simple Taylor expansion argument. Closed-loop asymptotic stability, recursive feasibility of the online optimisation, as well as robustness towards bounded disturbances and scheduling parameter prediction uncertainties are demonstrated. Two distinct nonlinear multi-input multi-output benchmarks are used to illustrate the effectiveness of the proposed method: a numeric simulation example, used to compare the method to state-of-the-art techniques, and a high-fidelity twin rotor system, for which the method is further validated. Real-time capabilities of the proposed scheme are highlighted, since it only requires one Quadratic Program to be evaluated per discrete-time sample, during the implementation.

Joint Trajectories and Resource Allocation Design for Multi-UAV-Assisted Wireless Power Transfer with Nonlinear Energy Harvesting

In this work, we explore a multi-UAV-assisted wireless power transfer (WPT) network, where multiple UAVs are deployed to provide WPT services to multiple ground devices (GDs) in order to extend their lifespan. To enhance the WPT efficiency while considering fairness, we investigate the joint trajectories and transmit power design. For fairness-aware consideration, our objective is to maximize the harvested energy of the GD with the worst condition, taking into account UAV mobility, anti-collision, and power budget constraints. Unlike previous works that focus on the simplified linear energy harvesting (EH) model, a more accurate multi-source nonlinear EH model is, for the first time, adopted to formulate the problem. Given the highly non-convex nature of the original problem due to the presence of coupled variables, we leverage the convexity of the multi-source nonlinear EH model and introduce a convex approximation method, which enables us to construct a tightly convex problem in each iteration for the original joint design problem, thereby obtaining a high-quality solution. Finally, we present numerical results to showcase the convergence of our algorithm and validate the performance advantages of the proposed multi-UAV WPT scheme with a nonlinear EH model versus benchmarks.

A quasi-brittle fracture investigation of concrete structures integrating random fields with the CSFEM-PFCZM

Design and maintenance of concrete structures require efficient methods for predicting quasi-brittle fracture while incorporating mesoscale-induced structural uncertainties. Toward this aim, a phase field cohesive zone model combined with the cell-based smoothed finite element method (CSFEM-PFCZM) is developed in the study, with mesoscale heterogeneity of concrete characterised by Weibull random fields. An automatic quadtree decomposition technique is presented to adaptively refine meshes around mesoscale areas, in which the hanging-node problem is tackled by treating hierarchical quadtree elements as CSFEM polygons. The crack driving forces are evaluated as history variables at the integration points of CSFEM subcells, and only the boundary geometry and Wachspress shape functions are needed to calculate strain matrices and stiffness matrices without the Jacobian inversion problems. The developed method is validated by extensive Monte Carlo simulations of typical nonlinear fracture benchmarks of concrete structures, demonstrating that the well-captured variability of crack paths and load–displacement curves can supplement the limited statistical results obtained from a small number of experiments. It is also found that larger correlation length and higher variance underline higher heterogeneity that leads to more dispersed responses. The developed method holds promise for the efficient evaluation of structural reliability taking into account stochastic mesoscale effects, due to the fast generation of random field samples in large quantities and flexible simulation of complicated fracture through the CSFEM-PFCZM.

Efficient quasi-brittle fracture simulations of concrete at mesoscale using micro CT images and a localizing gradient damage model

In this work, a localizing gradient damage model (LGDM) based on the generalized micromorphic theory is adopted to investigate the quasi-brittle fracture behaviour of concrete at mesoscale for the first time. Micro Computed Tomography (CT) image-based realistic concrete models are generated including aggregates, mortar, pores, and aggregate-mortar interfacial transition zones (ITZs). In the LGDM, an additional micro-force equilibrium equation is formulated with a damage-dependent interaction function to prevent spurious energetic interactions during the softening stage. The damage driving forces are evaluated as history variables through the maximum nonlocal equivalent strains at the integration points. The stiffness matrices and residual vectors for displacement and nonlocal equivalent strain fields are formulated through the conventional shape function and strain matrices of finite elements, by using the user-defined element (UEL) subroutine implemented with the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm to improve numerical performance. Typical nonlinear fracture benchmarks are presented to investigate the variability of mesoscale fracture evolution, crack trajectories, and load–displacement curves as well as numerical performance. It is found that the LGDM significantly improves the computational efficiency over the conventional phase-field cohesive zone model (PFCZM) with 60% savings of CPU time, and holds great potential for multiscale fracture evaluation of quasi-brittle composite materials, through the high-fidelity representation of real materials’ microstructures, and flexible simulation of the complicated nonlinear fracture without the need for re-meshing or mesh enrichments.

Sparse Bayesian deep learning for dynamic system identification

This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. Although DNNs show impressive approximation ability in various fields, several challenges still exist for system identification problems. First, DNNs are known to be too complex that they can easily overfit the training data. Second, the selection of the input regressors for system identification is nontrivial. Third, uncertainty quantification of the model parameters and predictions are necessary. The proposed Bayesian approach offers a principled way to alleviate the above challenges by marginal likelihood/model evidence approximation and structured group sparsity-inducing priors construction. The identification algorithm is derived as an iterative regularised optimisation procedure that can be solved as efficiently as training typical DNNs. Remarkably, an efficient and recursive Hessian calculation method for each layer of DNNs is developed, turning the intractable training/optimisation process into a tractable one. Furthermore, a practical calculation approach based on the Monte-Carlo integration method is derived to quantify the uncertainty of the parameters and predictions. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear system identification benchmarks by achieving good and competitive simulation accuracy. The code to reproduce the experimental results is open-sourced and available online.

A phase-field cohesive zone model integrated with cell-based smoothed finite element method for quasi-brittle fracture simulations of concrete at mesoscale

In this work, a phase field cohesive zone model (PFCZM) combined with the cell-based smoothed finite element method (CSFEM) is presented to investigate the quasi-brittle fracture behaviour of concrete at mesoscale. An automatic quadtree decomposition technique is employed to discretise the CT image-based meso-structures including aggregates, mortar and pores, with local mesh adaptively refined at material boundaries. The resultant quadtree elements with hanging nodes are naturally treated as CSFEM polygons. In the proposed CSFEM-PFCZM, the stiffness matrices and residual vectors for displacement and damage fields are formulated by the Wachspress shape function and smoothed strain matrices. The crack driving forces are evaluated as history variables at the integration points of CSFEM subcells. Four nonlinear fracture benchmarks are presented to validate the proposed method regarding the variability of mesoscale fracture processes, crack topology and load–displacement curves. It is found that the CSFEM-PFCZM combined with the quadtree decomposition significantly improves the computational efficiency over the traditional FEM counterparts, and shows great potential for multiscale fracture evaluation of quasi-brittle composite materials. This is due to the fast mesh generation and flexible simulation of complicated nonlinear fracture using the CSFEM-PFCZM without the need for remeshing or mesh enrichments.

Linear and nonlinear benchmarks between the CLT code and the M3D-C1 code for the 2/1 resistive tearing mode and the 1/1 resistive kink mode

The linear and nonlinear benchmarks between the CLT code and the M3D-C1 code for the 2/1 resistive tearing mode and the 1/1 resistive kink mode are presented. CLT is an explicit finite difference code, while M3D-C1 is an implicit finite element code. Although the implementations of CLT and M3D-C1 are totally different, we find that the simulation results of the resistive-kink mode and the m/n=2/1 tearing mode from M3D-C1 and CLT are almost the same, including the linear and nonlinear growth rates, the mode structures, the nonlinear saturation levels, the Poincare plots, and the scaling laws. This confirms that the nonlinear results for the 1/1 resistive-kink mode and 2/1 tearing mode are accurate and reliable.

Learning nonlinear state–space models using autoencoders

We propose a methodology for the identification of nonlinear state–space models from input/output data using machine-learning techniques based on autoencoders and neural networks. Our framework simultaneously identifies the nonlinear output and state-update maps of the model. After formulating the approach and providing guidelines for tuning the related hyper-parameters (including the model order), we show its capability in fitting nonlinear models on different nonlinear system identification benchmarks. Performance is assessed in terms of open-loop prediction on test data and of controlling the system via nonlinear model predictive control (MPC) based on the identified nonlinear state–space model.

The Impact of Renewable Energy Forecasts on Intraday Electricity Prices

In this paper we study the impact of errors in wind and solar power forecasts on intraday electricity prices. We develop a novel econometric model which is based on day-ahead wholesale auction curves data and errors in wind and solar power forecasts. The model shifts day-ahead supply curves to calculate intraday prices. We apply our model to the German EPEX SPOT SE data. Our model outperforms both linear and non-linear benchmarks. Our study allows us to conclude that errors in renewable energy forecasts exert a non-linear impact on intraday prices. We demonstrate that additional wind and solar power capacities induce non-linear changes in the intraday price volatility. Finally, we comment on economical and policy implications of our findings.

Deep State Space Models for Nonlinear System Identification

Abstract Deep state space models (SSMs) are an actively researched model class for temporal models developed in the deep learning community which have a close connection to classic SSMs. The use of deep SSMs as a black-box identification model can describe a wide range of dynamics due to the flexibility of deep neural networks. Additionally, the probabilistic nature of the model class allows the uncertainty of the system to be modelled. In this work a deep SSM class and its parameter learning algorithm are explained in an effort to extend the toolbox of nonlinear identification methods with a deep learning based method. Six recent deep SSMs are evaluated in a first unified implementation on nonlinear system identification benchmarks.

A six-node prismatic solid finite element for geometric nonlinear problems in elasticity

The present work deals with an extended six-node prismatic 3D solid finite element to the analysis of nonlinear geometrical problems. The kinematic formulation is based on a virtual Space Fiber Rotation (SFR) concept which conducts to improve the displacement fields with additional displacement terms, presenting rotational degrees of freedom (DOFs). Once the standard and patch tests for linear validation are previously achieved, the present element is assessed again for large displacement and moderate rotation. For this purpose, the total Lagrangian approach is used and the Green–Lagrange strain/Piola–Kirchhoff​ stress tensors are considered. The material behavior considered in this work is restricted to Saint-Venant–Kirchhoff model for 3D large displacements elasticity To demonstrate the efficiency and accuracy of the developed finite element model, extensive and standard nonlinear benchmarks are presented. The obtained results show a good convergence and accuracy compared to similar finite elements and consequently well capability of the present element to deal with geometric nonlinear problems, including prediction of several limit points.

An evaluation of the generalized–[formula omitted] time integration scheme for a transient response of nonlinear structures

The recently developed promising generalized–α time integration scheme for nonlinear structures is evaluated and compared with Bathe scheme. The nonlinear benchmarks are developed and implemented in order to assess reliability and numerical efficiency of the schemes. Both schemes proved to be comparably efficient and accurate while Bathe method has higher frequency resolution. A practical application of the considered schemes to the nonlinear cable structure problem that support conclusions is supplied.

Proxy Functions for Approximate Reinforcement Learning

Abstract Approximate Reinforcement Learning (RL) is a method to solve sequential decisionmaking and dynamic control problems in an optimal way. This paper addresses RL for continuous state spaces which derive the control policy by using an approximate value function (V-function). The standard approach to derive a policy through the V-function is analogous to hill climbing: at each state the RL agent chooses the control input that maximizes the right-hand side of the Bellman equation. Although theoretically optimal, the actual control performance of this method is heavily influenced by the local smoothness of the V-function; a lack of smoothness results in undesired closed-loop behavior with input chattering or limit-cycles. To circumvent these problems, this paper provides a method based on Symbolic Regression to generate a locally smooth proxy to the V-function. The proposed method has been evaluated on two nonlinear control benchmarks: pendulum swing-up and magnetic manipulation. The new method has been compared with the standard policy derivation technique using the approximate V-function and the results show that the proposed approach outperforms the standard one with respect to the cumulative return.

On evolutionary system identification with applications to nonlinear benchmarks

Abstract This paper presents a record of the participation of the authors in a workshop on nonlinear system identification held in 2016. It provides a summary of a keynote lecture by one of the authors and also gives an account of how the authors developed identification strategies and methods for a number of benchmark nonlinear systems presented as challenges, before and during the workshop. It is argued here that more general frameworks are now emerging for nonlinear system identification, which are capable of addressing substantial ranges of problems. One of these frameworks is based on evolutionary optimisation (EO); it is a framework developed by the authors in previous papers and extended here. As one might expect from the ‘no-free-lunch’ theorem for optimisation, the methodology is not particularly sensitive to the particular (EO) algorithm used, and a number of different variants are presented in this paper, some used for the first time in system identification problems, which show equal capability. In fact, the EO approach advocated in this paper succeeded in finding the best solutions to two of the three benchmark problems which motivated the workshop. The paper provides considerable discussion on the approaches used and makes a number of suggestions regarding best practice; one of the major new opportunities identified here concerns the application of grey-box models which combine the insight of any prior physical-law based models (white box) with the power of machine learners with universal approximation properties (black box).

Updated Lagrangian formulations of two hexahedral elements with rotational DOFs

Updated Lagrangian formulations of two eight-node hexahedral solid elements with rotational degrees of freedom (DOFs) are developed to analyse geometrically nonlinear large deflection problems. These two elements are based on the so-called Space Fibre Rotation (SFR) concept that considers fictive small rotations of a nodal fibre within the element to enrich the displacement vector approximation of low-order finite elements. The validity and efficiency of the proposed formulations are demonstrated by solving several nonlinear large deflection benchmarks and the obtained results show a much better accuracy than those based on the total Lagrangian approach.