Data-driven Modeling Approach Research Articles

Invertible Koopman Network and its application in data-driven modeling for dynamic systems

Koopman operator, acting on an infinite-dimensional Hilbert space of the observables, provides a global systematic linear representation of nonlinear systems, which is a leading candidate for data-driven modeling. In practical applications, the observable function is required to have an invertible property to predict or control the original system. However, two approaches for constructing the invertible observable functions in current research, introducing the full-state observables and adopting the auto-encoder, have drawbacks such as overly strong assumptions and lossy reconstruction. To address these issues, we propose the Invertible Koopman Network (IKN) to approximate the Koopman operator. With special invertible blocks, IKN achieves structure-level reversibility while retaining sufficient nonlinear mapping capability. This network results in a transformation from the complex dynamic system’s states to the observables with simpler dynamics and affords lossless reconstruction via its invertible process. Then, we use the IKN as a dynamical embedding model and adopt the Transformer based on an autoregressive task to learn the evolution pattern of the observables. Through the greedy decoding strategy, the trained IKN and Transformer model enable data-driven prediction where only initial value is provided. The method is tested on three typical chaotic dynamic systems (Lorenz system, Chen system, and Kuramoto-Sivashinsky equation) and is shown to capture the systems’ intrinsic evolution patterns, such as the structure of the strange attractor. Comparative experiments with alternative data-driven modeling approaches illustrate that our proposed method can model a longer evolutionary process with less error, further demonstrating its effectiveness and superiority.

Quantifying postprandial glucose responses using a hybrid modeling approach: Combining mechanistic and data-driven models in The Maastricht Study.

Computational models of human glucose homeostasis can provide insight into the physiological processes underlying the observed inter-individual variability in glucose regulation. Modelling approaches ranging from "bottom-up" mechanistic models to "top-down" data-driven techniques have been applied to untangle the complex interactions underlying progressive disturbances in glucose homeostasis. While both approaches offer distinct benefits, a combined approach taking the best of both worlds has yet to be explored. Here, we propose a sequential combination of a mechanistic and a data-driven modeling approach to quantify individuals' glucose and insulin responses to an oral glucose tolerance test, using cross sectional data from 2968 individuals from a large observational prospective population-based cohort, the Maastricht Study. The best predictive performance, measured by R2 and mean squared error of prediction, was achieved with personalized mechanistic models alone. The addition of a data-driven model did not improve predictive performance. The personalized mechanistic models consistently outperformed the data-driven and the combined model approaches, demonstrating the strength and suitability of bottom-up mechanistic models in describing the dynamic glucose and insulin response to oral glucose tolerance tests.

Interpretable data-driven modeling of hyperelastic membranes.

We proposed an approach for interpretable data-driven modeling of an isotropic incompressible hyperelastic membrane deformation. The approach is based on the response functions in terms of the Laplace stretch and the finite element method, where response functions are partial derivatives of a hyperelastic potential with respect to the chosen strain measure. The Laplace stretch as the strain measure allows us to recover directly the response functions from experimental data and construct automatically data-driven constitutive relations. All needed formulas were obtained explicitly. We tested our approach for membrane inflation with data-driven constitutive relations based on the perforated membrane extension tests.

On the modeling of continuous kraft pulp digesters: Inclusion of wood characteristics

In order to improve the pulp quality, it is essential to understand the influence of wood characteristics in the Kraft process, highlighting the methylglucuronic (MeGlcA) and hexenuronic (HexA) acids contents, and the syringyl/guaiacyl (S/G) ratio. Since their monitoring is still inefficient, it becomes difficult to use fully data-driven modeling approaches. Therefore, it is important to improve consolidated phenomenological models. In this work, an extended Purdue model was developed to simulate a continuous kraft pulp digester. Unlike most works, we considered the formation of HexA from MeGlcA and the dissolution of both compounds. We also considered the contribution of HexA to the kappa number, an important key performance indicator. As an innovation, we proposed the addition of the S/G ratio influence on the delignification rate of hardwoods by using experimental data taken from literature. The steady-state profiles and dynamic responses were similar to those found in the base work. The blow-line kappa number was 20% higher than that obtained disconsidering HexA. By performing a sensitivity analysis, the steady-state profile of kappa number proved to be very sensitive to variations in the S/G ratio. Thus, the results confirmed the importance of wood characteristics in the kraft pulping.

An application of data-driven modeling for hydroelasticity of an elastically supported semi-circular pipe conveying fluid

PurposeThe paper aims to develop an efficient data-driven modeling approach for the hydroelastic analysis of a semi-circular pipe conveying fluid with elastic end supports. Besides the structural displacement-dependent unsteady fluid force, the steady one related to structural initial configuration and the variable structural parameters (i.e. the variable support stiffness) are considered in the modeling.Design/methodology/approachThe steady fluid force is treated as a pipe preload, and the elastically supported pipe-fluid model is dealt with as a prestressed hydroelastic system with variable parameters. To avoid repeated numerical simulations caused by parameter variation, structural and hydrodynamic reduced-order models (ROMs) instead of conventional computational structural dynamics (CSD) and computational fluid dynamics (CFD) solvers are utilized to produce data for the update of the structural, hydrodynamic and hydroelastic state-space equations. Radial basis function neural network (RBFNN), autoregressive with exogenous input (ARX) model as well as proper orthogonal decomposition (POD) algorithm are applied to modeling these two ROMs, and a hybrid framework is proposed to incorporate them.FindingsThe proposed approach is validated by comparing its predictions with theoretical solutions. When the steady fluid force is absent, the predictions agree well with the “inextensible theory”. The pipe always loses its stability via out-of-plane divergence first, regardless of the support stiffness. However, when steady fluid force is considered, the pipe remains stable throughout as flow speed increases, consistent with the “extensible theory”. These results not only verify the accuracy of the present modeling method but also indicate that the steady fluid force, rather than the extensibility of the pipe, is the leading factor for the differences between the in- and extensible theories.Originality/valueThe steady fluid force and the variable structural parameters are considered in the data-driven modeling of a hydroelastic system. Since there are no special restrictions on structural configuration, steady flow pattern and variable structural parameters, the proposed approach has strong portability and great potential application for other hydroelastic problems.

Deep learning assisted physics-based modeling of aluminum extraction process

Modeling complex physical processes such as the extraction of aluminum is mainly done using pure physics-based models derived from first principles. However, the accuracy of these models can often suffer due to a partial understanding of the process, uncertainty in the input parameters, and numerous modeling assumptions. More recently, with the ever-increasing availability of data, there has been an explosion of interest in applying modern machine learning methods because of their ability to learn complex mappings directly from data. Unfortunately, these models tend to be black boxes, require an enormous amount of data, and do not utilize existing domain knowledge. In this work, we develop a novel approach combining physics-based and data-driven modeling approaches while eliminating some weaknesses. We use a data-driven model to correct a misspecified physics-based model of the Hall–Héroult process in an aluminum electrolysis cell using a corrective source term added to the set of governing ordinary differential equations. Our approach ensures that the existing knowledge is utilized to the maximum extent possible while relying on the data-driven models only to model those aspects which the physics-based model does not represent well. We compare this approach with an end-to-end learning approach and an ablated physics-based model, showing that the proposed hybrid method is more accurate, consistent, and stable for long-term predictions.

Data-driven nonlinear MIMO modeling for turbofan aeroengine control system of autonomous aircraft

The mathematical modeling problem of a general turbofan aeroengine control system used in autonomous aircraft is investigated in this paper. Unlike the thermodynamics or physical mechanism-based modeling methods in the existing literature, a pure data-driven modeling approach is presented with high accuracy via online and offline data only. This is achieved by using the nonlinear autoregressive neural network with exogenous inputs (NARX) neural network. In comparison with the existing NARX-based MIMO modeling methods, the series–parallel structure and the parallel structure are integrated for the neural network training. Faster convergence and stability as well as higher modeling accuracy are thus achieved. Another feature of this approach is that it is independent of all the components’ dynamics in the turbofan aeroengine. Simulation results are given to verify the effectiveness of the proposed modeling approach. This is further validated by experimental testing with the method applied to mini turbofan aeroengine testbed.

Boosting the prediction accuracy of a process-based greenhouse climate-tomato production model by particle filtering and deep learning

By generating high quality data without the big time investment and economic cost of real experiments, dynamic greenhouse climate and crop simulation models can support decisions on greenhouse climate control, crop management and greenhouse design. The reliability of simulation-based decisions depends on both the prediction accuracy and interpretability of simulation models. The prediction accuracy of these simulation models can be increased by: 1) improving mechanisms in process-based models; 2) calibrating process-based model parameters; 3) deriving black-box relationships from data. Considering the descending interpretability from (1) to (3), this study presents a knowledge-based data-driven modelling approach where firstly a process-based model is selected and modified based on domain knowledge, then data-driven improvement is applied including two steps: parameter value estimation by particle filter (PF) and further black-box improvement by deep neural networks (DNN). The approach was tested with an example of greenhouse climate-tomato production system modelling. Modules from GreenLight (Katzin et al., 2020) and TOMSIM (Heuvelink, 1995, 1996) were selected, modified and integrated into a process-based greenhouse climate-tomato model. Validation showed that PF-calibration of five greenhouse parameters decreased the seasonal relative root mean squared error (RRMSE) of indoor air vapor pressure predictions from 40.7% of that before PF-calibration to 16.4%, while it did not decrease the RRMSE of indoor air temperature predictions. Combining the PF-calibrated model with a DNN trained on a season of data decreased the RRMSE of indoor air temperature from 15.0% without DNN to 6.7%, and decreased the RRMSE of indoor air vapor pressure to 12.6%. The knowledge-based data-driven greenhouse climate-tomato model had a relative error of 0.9% for seasonal total fresh yield, and an RRMSE of 6.6% for the cumulative yield throughout the season. If process-based model parameters were not calibrated before combining the model with DNNs, the required amount and diversity of DNN training data increased because more information needed to be learnt from data by the DNNs. Without PF-calibration, combining a DNN trained on 50 days of data with the process-based model resulted in RRMSEs of 44.8% and 31.8% for indoor air temperature and vapor pressure prediction, respectively; with PF-calibration, the RRMSEs were decreased to 13.1% and 17.9%. The proposed three-step knowledge-based data-driven approach can not only improve the model prediction accuracy, but can also help to track and interpret the improvements.

A data-driven modeling approach to quantify morphology effects on transport properties in nanostructured NMC particles

We present a data-driven modeling approach to quantify morphology effects on transport properties in nanostructured materials. Our approach is based on the combination of stochastic modeling of the 3D nanostructure and numerical modeling of effective transport properties, which is used to investigate process-structure–property relationships of hierarchically structured cathode materials for lithium-ion batteries. We focus on nanostructured LiNi1/3Mn1/3Co1/3O2 (NMC) particles, the nanoporous morphology of which has a crucial impact on their effective transport properties (i.e, effective ionic and electric conductivity) and thus on the performance of the cell. First, we develop a parametric stochastic model for the 3D morphology of the nanostructured NMC particles based on excursion sets of so-called χ2-fields. This model, which has only two parameters, is then fitted to FIB-SEM image data of the NMC particles manufactured with different calcination temperatures and different particle sizes. This way it is possible to generate digital twins of the NMC particles. In a second step, measured 3D image data and corresponding digital twins are used as input for the numerical simulation of effective transport properties. Based on the results obtained by these simulations, we can quantify process-structure–property relationships. Overall, we present a methodological framework that allows for an efficient optimization of the fabrication process of nanostructured NMC particles.

Spatiotemporal Coordination of Rac1 and Cdc42 at the Whole Cell Level during Cell Ruffling.

Rho-GTPases are central regulators within a complex signaling network that controls cytoskeletal organization and cell movement. The network includes multiple GTPases, such as the most studied Rac1, Cdc42, and RhoA, along with their numerous effectors that provide mutual regulation through feedback loops. Here we investigate the temporal and spatial relationship between Rac1 and Cdc42 during membrane ruffling, using a simulation model that couples GTPase signaling with cell morphodynamics and captures the GTPase behavior observed with FRET-based biosensors. We show that membrane velocity is regulated by the kinetic rate of GTPase activation rather than the concentration of active GTPase. Our model captures both uniform and polarized ruffling. We also show that cell-type specific time delays between Rac1 and Cdc42 activation can be reproduced with a single signaling motif, in which the delay is controlled by feedback from Cdc42 to Rac1. The resolution of our simulation output matches those of time-lapsed recordings of cell dynamics and GTPase activity. Our data-driven modeling approach allows us to validate simulation results with quantitative precision using the same pipeline for the analysis of simulated and experimental data.

Trustworthiness judgments without the halo effect: A data-driven computational modeling approach.

Trustworthy-looking faces are also perceived as more attractive, but are there other meaningful cues that contribute to perceived trustworthiness? Using data-driven models, we identify these cues after removing attractiveness cues. In Experiment 1, we show that both judgments of trustworthiness and attractiveness of faces manipulated by a model of perceived trustworthiness change in the same direction. To control for the effect of attractiveness, we build two new models of perceived trustworthiness: a subtraction model, which forces the perceived attractiveness and trustworthiness to be negatively correlated (Experiment 2), and an orthogonal model, which reduces their correlation (Experiment 3). In both experiments, faces manipulated to appear more trustworthy were indeed perceived to be more trustworthy, but not more attractive. Importantly, in both experiments, these faces were also perceived as more approachable and with more positive expressions, as indicated by both judgments and machine learning algorithms. The current studies show that the visual cues used for trustworthiness and attractiveness judgments can be separated, and that apparent approachability and facial emotion are driving trustworthiness judgments and possibly general valence evaluation.

Data-Driven Edge Offloading for Wireless Control Systems

As industrial plants embrace modern technologies, such as edge computing and wireless networks, industrial control systems have evolved into multi-tier cyber-physical systems. While traditional local controllers enjoy reliable connectivity to sensors/actuators, they suffer from the limited computation capacity of embedded devices. In contrast, edge servers introduce more computation resources connected to sensors/actuators through wireless networks. Offloading control functions to edge servers presents new opportunities to enhance control performance but also poses critical challenges. As wireless networks have limited bandwidth and varying reliability, it is important to optimize control performance by dynamically offloading a subset of the control functions to edge servers under the bandwidth constraint. Furthermore, the selection of offloaded control functions depends on both the cyber (wireless) and physical states of the wireless control systems. In this paper, we tackle the problem of optimizing the control performance of multiple control loops through dynamic edge offloading. We establish a data-driven model to predict the control performance of each feedback control loop based on its cyber-physical states. We then develop a dynamic edge offloading approach to optimize the overall control performance of a system with multiple feedback control loops while guaranteeing their stability under fluctuating cyber-physical conditions. Finally, we demonstrate the efficacy of the data-driven model and offloading approach in case studies comprising simulations of up to twenty industrial robots.

Dissolved Oxygen Prediction Model for the Yangtze River Estuary Basin Using IPSO-LSSVM

Water ecology has always been key to environmental protection, and the combination of human activities and natural factors has caused eutrophication in the Yangtze estuary and adjacent waters. Among them, dissolved oxygen (DO) concentration is the key indicator to judge the quality of water. Firstly, using principal component analysis (PCA) to determine the number of parameters affecting dissolved oxygen concentration, the least squares support vector machine (LSSVM) prediction model with improved particle swarm optimization (IPSO) is proposed to be applied to the dissolved oxygen prediction in Shanghai’s Yangtze River basin through the data-driven modeling approach and the regression prediction capability of the neural network. Eight parameters of water temperature (WT), pH, potassium permanganate (KMnO4), ammonia nitrogen (NH4+-N), total phosphorus (TP), total nitrogen (TN), conductivity (Cond), and nephelometric turbidity unit (NTU) are selected as model inputs in the published public data, and the output is the dissolved oxygen concentration. The optimal combination of model parameters is found according to the IPSO algorithm, which effectively overcomes the parameter selection problem of regular support vector machines (SVM). The mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), and correlation coefficients of the evaluation indexes of this model (R2) are 0.1702, 0.2221, 0.0267, and 0.9751, respectively. Compared with other similar data driven models, this model has improved model accuracy and stability in predicting DO concentrations in the estuary, and thus it provides technical support for assessing and monitoring offshore water quality.

On the Ca2＋ elevation in vascular endothelial cells due to inositol trisphosphate-sensitive store receptors activation: A data-driven modeling approach

Agonist-induced Ca2+ signaling is essential for the regulation of many vital functions in endothelial cells (ECs). A broad range of stimuli elevate the cytosolic Ca2＋ concentration by promoting a pathway mediated by inositol 1,4,5 trisphosphate (IP3) which causes Ca2＋ release from intracellular stores. Despite its importance, there are very few studies focusing on the quantification of such dynamics in the vascular endothelium. Here, by using data from isolated ECs, we established a minimalistic modeling framework able to quantitatively capture the main features (averaged over a cell population) of the cytosolic Ca2＋ response to different IP3 stimulation levels. A suitable description of Ca2＋-regulatory function of inositol 1,4,5 trisphosphate receptors (IP3Rs) and corresponding parameter space are identified by comparing the different model variants against experimental mean population data. The same approach is used to numerically assess the relevance of cytosolic Ca2＋ buffering, as well as Ca2＋ store IP3-sensitivity in the overall cell dynamics. The variability in the dynamics’ features observed across the population can be explained (at least in part) through variation of certain model parameters (such as buffering capacity or Ca2＋ store sensitivity to IP3). The results, in terms of experimental fitting and validation, support the proposed minimalistic model as a reference framework for the quantification of the EC Ca2＋ dynamics induced by IP3Rs activation.

Interpretable data-driven constitutive modelling of soils with sparse data

There is significant potential for machine learning (ML) to achieve step-change advances in the state-of-the-art for constitutive modelling of geomaterials. However, a common challenge and criticism of ML for geotechnical modelling is a lack of interpretability and onerous dependency on big data. This study proposes an interpretable data-driven constitutive modelling approach for application in the context of sparse geotechnical data. Improved interpretability is achieved by incorporating (a) prior known ‘tried-and-tested’ theoretical knowledge within the ML models and (b) uncertainty within the predictions. Three different theoretical descriptions of soil behaviour including incremental nonlinearity, hyperelasticity and elastoplasticity are considered and incorporated into the data-driven framework. To alleviate dependencies on big data, a ‘multi-fidelity’ modelling framework is adopted to maximise the impact of small ‘high-fidelity’ geotechnical datasets. The ability of pure and physics-constrained data-driven models to predict the response of both synthetic and real elemental data is subsequently compared. The results indicate that data-driven modelling with physical constraints shows more robust performance particularly for extrapolation outside the original calibration space of a small geotechnical dataset. The proposed data-driven modelling is generic and demonstrates exciting potential for modelling complex behaviours of various materials.

Data-Driven Adaptive Modelling and Control for a Class of Discrete-Time Robotic Systems Based on a Generalized Jacobian Matrix Initialization

Data technology advances have increased in recent years, especially for robotic systems, in order to apply data-driven modelling and control computations by only considering the input and output signals’ relationship. For a data-driven modelling and control approach, the system is considered unknown. Thus, the initialization values of the system play an important role to obtain a suitable estimation. This paper presents a methodology to initialize a data-driven model using the pseudo-Jacobian matrix algorithm to estimate the model of a mobile manipulator robot. Once the model is obtained, a control law is proposed for the robot end-effector position tasks. To this end, a novel neuro-fuzzy network is proposed as a control law, which only needs to update one parameter to minimize the control error and avoids the chattering phenomenon. In addition, a general stability analysis guarantees the convergence of the estimation and control errors and the tuning of the closed-loop control design parameters. The simulations results validate the performance of the data-driven model and control.

Modeling Methodology Based on Fast and Refined Neural Networks for Non-Isolated DC–DC Converters With Configurable Parameter Settings

Compared with conventional physics-based methods, e.g., analytical modeling and numerical modeling, data-driven methods can extract input-to-output relationships from the data without much prior knowledge of the physical system, thus showing great potential in modeling power electronics (PE) converters with complex switching behaviors and configurable parameter settings. Previous data-driven PE circuit modeling approaches are mostly based on sequential neural networks, and their execution speed suffers from large sequential lengths due to a high sampling rate for high modeling accuracy. Moreover, modeling of refined singular ripples is missing and configurable parameter settings are not available in these data-driven modeling approaches. To address the above-mentioned issues, this paper proposes a hybrid physics-informed machine learning (ML) method to model the non-isolated DC-DC converters. The approach empirically decomposes the output signals into transient large signals and periodic small signals. For transient large signals, a fully-connected neural network (NN) is used to map circuit parameters with system characteristics, such that configurable circuit parameter settings are allowed. For periodic signals, a long short-time memory (LSTM) network together with convolutional neural network (CNN) is used to accelerate the simulation by predicting signal features in the compressed latent space. A buck converter with configurable parameter settings is modeled by the proposed hybrid physics-informed ML method. Periodic ripples are successfully generated, while execution speed is about 10 times faster than that of conventional numerical methods.

Hamiltonian neural networks with automatic symmetry detection.

Recently, Hamiltonian neural networks (HNNs) have been introduced to incorporate prior physical knowledge when learning the dynamical equations of Hamiltonian systems. Hereby, the symplectic system structure is preserved despite the data-driven modeling approach. However, preserving symmetries requires additional attention. In this research, we enhance HNN with a Lie algebra framework to detect and embed symmetries in the neural network. This approach allows us to simultaneously learn the symmetry group action and the total energy of the system. As illustrating examples, a pendulum on a cart and a two-body problem from astrodynamics are considered.

Deep Koopman Traffic Modeling for Freeway Ramp Metering

Ramp metering has been considered as one of the most effective approaches of dealing with the traffic congestion on the freeways. The modelling of the freeway traffic flow dynamics is challenging because of its non-linearity and uncertainty. Recently, Koopman operator, which transfers a non-linear system to a linear system in an infinite-dimensional space, has been studied for modelling complex dynamics. In this paper, we propose a data-driven modelling approach based on neural networks, denoted by deep Koopman model, to learn a finite-dimensional approximation of the Koopman operator. To consider the sequential relations of the ramps and main roads on the freeway, a long short-term memory network is applied. Furthermore, a model predictive controller with the trained deep Koopman model is proposed for the real-time control of the ramp metering on the freeway. To validate the performance of the proposed approach, experiments based on the simulation in the traffic simulation software Simulation of Urban MObility (SUMO) environment are conducted. The results demonstrate the effectiveness of the proposed approach on both the dynamics prediction and the real-time control of the ramp metering.

A Time-efficient, Data-driven Modeling Approach for Predicting the Geomagnetic Impact of Coronal Mass Ejections

To understand the global-scale physical processes behind coronal mass ejection (CME)–driven geomagnetic storms and predict their intensity as a space weather forecasting measure, we develop an interplanetary CME flux rope–magnetosphere interaction module using 3D magnetohydrodynamics. The simulations adequately describe CME-forced dynamics of the magnetosphere including the imposed magnetotail torsion. These interactions also result in induced currents, which are used to calculate the geomagnetic perturbation. Through a suitable calibration, we estimate a proxy of geoeffectiveness—the Storm Intensity index (STORMI)—that compares well with the Dst/SYM-H index. Simulated impacts of two contrasting CMEs quantified by the STORMI index exhibit a high linear correlation with the corresponding Dst and SYM-H indices. Our approach is relatively simple, has fewer parameters to be fine-tuned, and is time efficient compared to complex fluid-kinetic methods. Furthermore, we demonstrate that flux rope erosion does not significantly affect our results. Thus our method has the potential to significantly extend the time window for predictability—an outstanding challenge in geospace environment forecasting—if early predictions of near-Earth CME flux rope structures based on near-Sun observations are available as inputs. This study paves the way for early warnings based on operational predictions of CME-driven geomagnetic storms.