Sparse Identification Method Research Articles

A Bayesian transfer sparse identification method for nonlinear ARX systems

SummaryIn this paper, we design a transfer sparse identification algorithm under the Bayesian framework through introducing other system knowledge into the system to be identified. This method provides a new identification solution for a nonlinear autoregressive model with exogenous inputs (NARX). The estimates of the transferred parameters are calculated by adding the transfer correction term to the un‐transferred estimates. To achieve this, a joint prior distribution is devised for the parameters, ultimately enhancing the efficient utilization of existing data, reducing the reliance on new data, and achieving more accurate identification. The maximized marginal likelihood method is used to find the transfer gain and the transfer information matrix in the transfer correction term. Meanwhile, in order to make the algorithm automatically adapt to different data, we design an automatic structure detection method based on the transfer framework. The method automatically determines the sparsity threshold based on the maximum inter‐class variance. Two examples are provided to demonstrate the advantages of our algorithm.

A transfer sparse identification method for ARX model

AbstractThe aim of this paper is to improve the parameter estimation accuracy of the system to be identified by using measurements from a known system. By introducing the transfer gain matrix and setting the effective identification criterion, a novel transfer sparse identification method is raised, which deals with the sparse issues more precise. Besides, the unbiased form is given in the parameter analysis and the recursion form can prevent the dimension catastrophe related problems. Moreover, in order to test the effects of the transfer and avoid bad performance, a negative transfer analysis condition is carried out. Finally, the simulation verifies the enhancements and benefits of the proposed transfer sparse identification method, confirming that the transfer performance outperforms better than that of no transfer, especially on the zero parameters identification.

Compressive-sensing model reconstruction of nonlinear systems with multiple attractors

In this study, facing the challenges on model reconstruction for multi-attractor nonlinear systems, the data generation and sparse regression processes in sparse identification method are generalized, referring to compressive sensing, to obtain the accurate description under the least volume of test data set. In the data generation process, we introduce compressive sensing process by an arbitrary initial condition and arbitrary perturbations applied on time domain to transcend the local attractors. In the sparse regression process, the optimum sparse parameter is obtained by bi-optimization criterion according to accuracy and interpretability. Then, accuracy criterion is proposed, and when it discovers the unperceived dynamic behaviors, more dynamic data could be recorded and added into the modeling reconstruction data set by perturbations. It requires less dynamic signals by dynamical compressive sensing with perturbations compared to the previous method with uniform point fetching on the state space. Several numerical cases and two experiments of nonlinear systems with different kinds of multi-attractors are proposed to illustrate the effectiveness of the reconstruction method. In experiment, for the dynamic systems with multi-steady states phenomenon, the most obvious problem is the calibration of the equilibrium at the symmetrical configuration, which cannot be obtained for local dynamic behaviors due to its instability. In applications, this generalized modeling reconstruction method can continuously and compressively sense the dynamic behaviors and the stability of multiple attractors to figure out the accurate governing equations under a small quantity of data. In summary, different from the previous sparse regression algorithm for nonlinear systems with multiple attractors under huge amount of data set assembled offline filling a large enough state space, the proposed model reconstruction process can intelligently sense the dynamic behaviors and give the accurate prediction.

A Levenberg-Marquardt Algorithm for Sparse Identification of Dynamical Systems.

Low complexity of a system model is essential for its use in real-time applications. However, sparse identification methods commonly have stringent requirements that exclude them from being applied in an industrial setting. In this article, we introduce a flexible method for the sparse identification of dynamical systems described by ordinary differential equations. Our method relieves many of the requirements imposed by other methods that relate to the structure of the model and the dataset, such as fixed sampling rates, full state measurements, and linearity of the model. The Levenberg-Marquardt algorithm is used to solve the identification problem. We show that the Levenberg-Marquardt algorithm can be written in a form that enables parallel computing, which greatly diminishes the time required to solve the identification problem. An efficient backward elimination strategy is presented to construct a lean system model.

Research on sparse identification method for aeroelastic dynamic response prediction

Nonlinear aeroelastic system has the characteristics of complex structure, difficult modeling and difficult calculation of dynamic response. For the analysis of nonlinear aeroelastic systems, model identification is a very attractive method. However, the models identified by traditional methods are often relatively complex and limited in scope of use, so it is necessary to develop an interpretable equivalent reduced model. In this paper, sparse regression method and sequential threshold least squares technique are used to establish sparse identification method for complex aeroelastic systems. This method has the ability to identify reduced models containing only required nonlinear terms through measurement data. Then, the sparse identification method is used to identify the binary wing with dead zone nonlinearity and cubic stiffness nonlinearity. The obtained model can provide rapid and accurate prediction of the response of the system according to the sensor measurement, and can also be used as an explicit surrogate model for aeroelastic optimization design, thus verifying the superiority of the proposed method.

A robust sparse identification method for nonlinear dynamic systems affected by non-stationary noise.

In the field of science and engineering, identifying the nonlinear dynamics of systems from data is a significant yet challenging task. In practice, the collected data are often contaminated by noise, which often severely reduce the accuracy of the identification results. To address the issue of inaccurate identification induced by non-stationary noise in data, this paper proposes a method called weighted ℓ1-regularized and insensitive loss function-based sparse identification of dynamics. Specifically, the robust identification problem is formulated using a sparse identification mathematical model that takes into account the presence of non-stationary noise in a quantitative manner. Then, a novel weighted ℓ1-regularized and insensitive loss function is proposed to account for the nature of non-stationary noise. Compared to traditional loss functions like least squares and least absolute deviation, the proposed method can mitigate the adverse effects of non-stationary noise and better promote the sparsity of results, thereby enhancing the accuracy of identification. Third, to overcome the non-smooth nature of the objective function induced by the inclusion of loss and regularization terms, a smooth approximation of the non-smooth objective function is presented, and the alternating direction multiplier method is utilized to develop an efficient optimization algorithm. Finally, the robustness of the proposed method is verified by extensive experiments under different types of nonlinear dynamical systems. Compared to some state-of-the-art methods, the proposed method achieves better identification accuracy.

Sparse identification for ball-screw drives considering position-dependent dynamics and nonlinear friction

Establishing accurate dynamic models in a form that is suitable for integration with model-based control methods, is of great significance for further improving the dynamic motion control precision of ball-screw drives. However, due to the nonlinear time-varying factors such as position-dependent dynamics and nonlinear friction disturbance, it is difficult to model the dynamic characteristics of ball-screw drives accurately, concisely and efficiently. To overcome this challenge, a sparse identification method for ball-screw drives is proposed. Ball-screw drives are modeled as discrete-time linear parameter-varying systems under nonlinear friction disturbance, and two types of dictionary function libraries are designed to represent the position-dependent dynamics and nonlinear friction respectively. After constructing the regression form of the system model, a stepwise sparse regression policy is proposed to solve all the coefficients of dictionary functions. The proposed method is verified in both simulation and real environments. The results both show that by the proposed method, an accurate and linearizable dynamic model of ball-screw drives can be identified only using the data from only one global random excitation experiment covering the working stroke.

Interpretable sparse identification of a bistable nonlinear energy sink

Bistable nonlinear energy sinks have received great interest due to their efficient broad-band targeted energy transfer over a wide range of input energy levels. The precise identification of bistable nonlinear stiffness force is of significance to predict and enhance the system performance of the vibration energy absorption. However, the nonlinear stiffness force in nonlinear energy sink structures with local bistability is difficult to measure and identify because of snap-through characteristics. Inspired by physics-informed data-driven regression in machine learning, an interpretable sparse identification method is proposed to determine the stiffness force of a bistable nonlinear energy sink. The restoring force surface is constructed on bistable nonlinear energy sink equations and the nonlinear stiffness force trajectory is intercepted by assuming two quasi-zero velocity planes. Furthermore, the candidate functions in the sparse regression algorithm can be physically informed by conducting the least-squares parameter fitting of the intercepted nonlinear stiffness force trajectories. Numerical investigations demonstrate that the proposed method not only gives physics information but also improves the accuracy by 0.48%, 3.26% and 22.21% under the noise level of 30 dB, 20 dB, and 10 dB, respectively. Moreover, the reconstructed dynamic response has a good agreement with the theory. Experimental measurements are performed on a magnetically coupled bistable nonlinear energy sink. Results show that the accuracy improves by 4.52% and 11.76% compared to restoring force surface and Hilbert transform-based methods, respectively.

Physics-informed sparse identification of bistable structures

The design of bistable structures is a hot topic in the last decade due to its wide application in smart actuators, energy harvesters, flexible robotics, etc. The characterization of the nonlinear restoring force of bistable structures plays a significant role in modeling and enhancing dynamic performance. However, the traditional nonparametric identification methods may have insufficient accuracy or even be invalid because of numerical differentiation procedures and static fitting. Besides, the modern data-driven sparse regression identification methods rely highly on the assumed nonlinear basis functions and lack interpretability. In this paper, a physics-informed sparse identification method is proposed for the nonlinear restoring force identification of bistable structures. The function of the nonlinear restoring force is physically informed by the derived equation of the Hilbert transform and parameter fitting. Furthermore, sparse identification is conducted based on the free vibration responses of the bistable vibrator. The numerical studies verify the effectiveness of the proposed algorithm under the noise level of 30 dB. Experimental measurement is conducted on a magnetic coupled bistable beam to perform the model identification. It has been demonstrated that the reconstructed dynamic response and nonlinear restoring force both keep in good agreement with the measured ones.

Learning chaotic systems from noisy data via multi-step optimization and adaptive training.

A data-driven sparse identification method is developed to discover the underlying governing equations from noisy measurement data through the minimization of Multi-Step-Accumulation (MSA) in error. The method focuses on the multi-step model, while conventional sparse regression methods, such as the Sparse Identification of Nonlinear Dynamics method (SINDy), are one-step models. We adopt sparse representation and assume that the underlying equations involve only a small number of functions among possible candidates in a library. The new development in MSA is to use a multi-step model, i.e., predictions from an approximate evolution scheme based on initial points. Accordingly, the loss function comprises the total error at all time steps between the measured series and predicted series with the same initial point. This enables MSA to capture the dynamics directly from the noisy measurements, resisting the corruption of noise. By use of several numerical examples, we demonstrate the robustness and accuracy of the proposed MSA method, including a two-dimensional chaotic map, the logistic map, a two-dimensional damped oscillator, the Lorenz system, and a reduced order model of a self-sustaining process in turbulent shear flows. We also perform further studies under challenging conditions, such as noisy measurements, missing data, and large time step sizes. Furthermore, in order to resolve the difficulty of the nonlinear optimization, we suggest an adaptive training strategy, namely, by gradually increasing the length of time series for training. Higher prediction accuracy is achieved in an illustrative example of the chaotic map by the adaptive strategy.

Sparse identification method of extracting hybrid energy harvesting system from observed data

Hybrid energy harvesters under external excitation have complex dynamical behavior and the superiority of promoting energy harvesting efficiency. Sometimes, it is difficult to model the governing equations of the hybrid energy harvesting system precisely, especially under external excitation. Accompanied with machine learning, data-driven methods play an important role in discovering the governing equations from massive datasets. Recently, there are many studies of data-driven models done in aspect of ordinary differential equations and stochastic differential equations (SDEs). However, few studies discover the governing equations for the hybrid energy harvesting system under harmonic excitation and Gaussian white noise (GWN). Thus, in this paper, a data-driven approach, with least square and sparse constraint, is devised to discover the governing equations of the systems from observed data. Firstly, the algorithm processing and pseudo code are given. Then, the effectiveness and accuracy of the method are verified by taking two examples with harmonic excitation and GWN, respectively. For harmonic excitation, all coefficients of the system can be simultaneously learned. For GWN, we approximate the drift term and diffusion term by using the Kramers-Moyal formulas, and separately learn the coefficients of the drift term and diffusion term. Cross-validation (CV) and mean-square error (MSE) are utilized to obtain the optimal number of iterations. Finally, the comparisons between true values and learned values are depicted to demonstrate that the approach is well utilized to obtain the governing equations for the hybrid energy harvester under harmonic excitation and GWN.

A real-time identification method of network structure in complex network systems

This paper develops a real-time identification algorithm for structures of network systems. We have already proposed a structure identification method for nonlinear network systems with nonlinear couplings by combining Koopman operator theory and a sparse identification method. However, the proposed method is only applicable for offline use under the assumption that the state of an isolated system is available. To relax this assumption, this paper shows that the dynamical model of isolated systems can be extracted from measured data of interconnected systems under some weak restrictions on network structures. We also modify the identification algorithm to obtain real-time identification results with fewer calculation resources and analyse the convergence of the proposed methods. Numerical examples demonstrate that the proposed method can identify network structures in real-time and detect changes in networks correctly.

Discovering constitutive equations of crystal structures by sparse identification

In this study, the symbolic constitutive equations of single crystal structures were discovered using the sparse identification (SPID) method with an energy criterion. Six materials belonging to three crystal systems were targeted: cubic structures (Si, Ge), tetragonal crystal structures (rutile TiO2, body-centered-tetragonal with four-atom rings (BCT-4) ZnO), and hexagonal crystal structures (wurtzite GaN, wurtzite ZnO). Using ab initio calculation, the stress and tangent modulus of the six crystal structures were generated as reference data under various deformed states. First, the candidates for constitutive equations were constructed from the form-invariants of crystal, which are based on the generated tangent modulus datasets considering the symmetric conditions of crystal structures. Then, using the SPID method, the insignificant candidate terms were eliminated based on the energy criterion. The coefficients of the remaining terms were then viewed as the discovered material constants of each single crystal structure. Subsequently, the prediction accuracy of the SPID constitutive model based on the discovered material constants was investigated under the test-deformed state, which was not included in the modeling process. Furthermore, the discovered SPID models were embedded in nonlinear finite element analyses and verified.

Learning mean-field equations from particle data using WSINDy

We develop a weak-form sparse identification method for interacting particle systems (IPS) with the primary goals of reducing computational complexity for large particle number N and offering robustness to either intrinsic or extrinsic noise. In particular, we use concepts from mean-field theory of IPS in combination with the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy) to provide a fast and reliable system identification scheme for recovering the governing stochastic differential equations for an IPS when the number of particles per experiment N is on the order of several thousands and the number of experiments M is less than 100. This is in contrast to existing work showing that system identification for N less than 100 and M on the order of several thousand is feasible using strong-form methods. We prove that under some standard regularity assumptions the scheme converges with rate O(N−1/2) in the ordinary least squares setting and we demonstrate the convergence rate numerically on several systems in one and two spatial dimensions. Our examples include a canonical problem from homogenization theory (as a first step towards learning coarse-grained models), the dynamics of an attractive–repulsive swarm, and the IPS description of the parabolic–elliptic Keller–Segel model for chemotaxis. Code is available at https://github.com/MathBioCU/WSINDy_IPS.

A two-stage sparse algorithm for localization and characterization of local nonlinear structures

This paper considers the localization and characterization of nonlinear components in local nonlinear structures with noisy measurements, and a novel two-stage sparse identification method is proposed. In this algorithm, one set of candidate nonlinear basis functions are leveraged to characterize system nonlinearities. To effectively localize and characterize nonlinear components, the algorithm introduces two sparse stages. At the first sparse stage, the adaptive group Lasso is adopted to localize nonlinear components in structures. At the second sparse stage, the adaptive Lasso is used to further characterize the contributing nonlinear basis functions in the identified nonlinear positions. Since the first sparse stage can greatly decrease the number of candidate nonlinear basis functions, the contributing functions can be effectively selected and characterized at the second sparse stage. Compared with the one-stage sparse algorithms, the proposed algorithm is more robust to noise, i.e., the new method can correctly identify the local nonlinear structures with a lower Signal to Noise Ratio (SNR). In addition, many existed nonlinear identification methods, simultaneously require the synchronous data of vibration displacements, velocities and accelerations. If only vibration displacements are measured, the first and second order derivatives of displacement are conducted to estimate velocities and accelerations, respectively. Nevertheless, the noise mixed into the displacement measurement will be severely amplified in the process of derivative calculation, which will make the estimation of velocities and accelerations deviating from the actual value and reduce the identification accuracy. To avoid performing the second order derivative, the Duhamel's integral is adopted to rewrite the dynamic relationship between the system input and output. In addition, a non-parametric de-noising method based on reproducing kernel Hilbert space (RKHS) is further developed to reduce the effect of noise in vibration displacements and obtain vibration velocities, which can effectively reduce the effect of noise and improve the identification accuracy. The numerical and experimental studies verify the potential and effectiveness of the proposed algorithm in localizing and characterizing nonlinear components in local nonlinear structures.

DATA-DRIVEN SPARSE IDENTIFICATION OF GOVERNING EQUATIONS FOR FLUID DYNAMICS

It is a challenging and important issue to establish a nonlinear dynamic model of system by use of limited data. The data-driven sparse identification method is an effective method developed recently to identify the governing equations of the dynamic system from data developed in recent years. In this paper, governing equations for different flows are identified by data-driven sparse identification methods. Partial differential equation functional identification of nonlinear dynamics (PDE-FIND) scheme and least absolute shrinkage and selection operator (LASSO) scheme are used to identify the governing equations of two-dimensional flow past a circular cylinder, liddriven cavity flow, Rayleigh-Benard convection and three-dimensional turbulent channel flow. An over-complete candidate library is constructed by direct numerical simulation flow field data in the process of identification. Variables in the library are retained up to second order, variable derivatives are retained up to second order, and nonlinear terms are retained up to fourth order. By comparing the results from the two methods, we find both methods show good performance in identifying governing equation with no nonlinear terms, i.e., vorticity transport equation, heat transport equation and continuity equation. PDE-FIND scheme correctly identified the governing equations and Rayleigh number and Reynolds number for the flow field. But LASSO scheme failed to identify the governing equations which contain strong nonlinear terms, i.e., Navier-Stokes equations. This is because grouping effect may occur among the items in the candidate library and only one item in the group is chosen in such case in LASSO scheme. So PDE-FIND scheme is more effective than LASSO scheme in sparse identification of strongly nonlinear partial differential equation. It is also found that selecting data from regions with abundant flow structures can improve the accuracy of data-driven sparse identification results.

Data-Driven Discovery of Block-Oriented Nonlinear Models Using Sparse Null-Subspace Methods.

This article develops an identification algorithm for nonlinear systems. Specifically, the nonlinear system identification problem is formulated as a sparse recovery problem of a homogeneous variant searching for the sparsest vector in the null subspace. An augmented Lagrangian function is utilized to relax the nonconvex optimization. Thereafter, an algorithm based on the alternating direction method and a regularization technique is proposed to solve the sparse recovery problem. The convergence of the proposed algorithm can be guaranteed through theoretical analysis. Moreover, by the proposed sparse identification method, redundant terms in nonlinear functional forms are removed and the computational efficiency is thus substantially enhanced. Numerical simulations are presented to verify the effectiveness and superiority of the present algorithm.

Optimization of Galloping Piezoelectric Energy Harvester with V-Shaped Groove in Low Wind Speed

A square cylinder with a V-shaped groove on the windward side in the piezoelectric cantilever flow-induced vibration energy harvester (FIVEH) is presented to improve the output power of the energy harvester and reduce the critical velocity of the system, aiming at the self-powered supply of low energy consumption devices in the natural environment with low wind speed. Seven groups of galloping piezoelectric energy harvesters (GPEHs) were designed and tested in a wind tunnel by gradually changing the angle of two symmetrical sharp angles of the V-groove. The GPEH with a sharp angle of 45° was selected as the optimal energy harvester. Its output power was 61% more than the GPEH without the V-shaped groove. The more accurate mathematical model was made by using the sparse identification method to calculate the empirical parameters of fluid based on the experimental data and the theoretical model. The critical velocity of the galloping system was calculated by analyzing the local Hopf bifurcation of the model. The minimum critical velocity was 2.53 m/s smaller than the maximum critical velocity at 4.69 m/s. These results make the GPEH with a V-shaped groove (GPEH-V) more suitable to harvest wind energy efficiently in a low wind speed environment.

Sparse Identification of Nonlinear Duffing Oscillator From Measurement Data

Abstract In this paper we aim to apply an adaptation of the recently developed technique of sparse identification of nonlinear dynamical systems on a Duffing experimental setup with cubic feedback of the output. The Duffing oscillator described by nonlinear differential equation which demonstrates chaotic behavior and bifurcations, has received considerable attention in recent years as it arises in many real-world engineering applications. Therefore its identification is of interest for numerous practical problems. To adopt the existing identification method to this application, the optimization process which identifies the most important terms of the model has been modified. Selection of the true model is done via balancing complexity and accuracy using Pareto front analysis. This study provides considerable insight into the employment of sparse identification method on the real-world setups and the results show that the developed algorithm is capable of finding the true nonlinear model of the considered application including a nonlinear friction term.

Sparse identification for nonlinear optical communication systems: SINO method.

We introduce low complexity machine learning method method (based on lasso regression, which promotes sparsity, to identify the interaction between symbols in different time slots and to select the minimum number relevant perturbation terms that are employed) for nonlinearity mitigation. The immense intricacy of the problem calls for the development of "smart" methodology, simplifying the analysis without losing the key features that are important for recovery of transmitted data. The proposed sparse identification method for optical systems (SINO) allows to determine the minimal (optimal) number of degrees of freedom required for adaptive mitigation of detrimental nonlinear effects. We demonstrate successful application of the SINO method both for standard fiber communication links (over 3 dB gain) and for few-mode spatial-division-multiplexing systems.