Data-driven Component Research Articles

Current Status of the MB-pol Data-Driven Many-Body Potential for Predictive Simulations of Water Across Different Phases.

Developing a molecular-level understanding of the properties of water is central to numerous scientific and technological applications. However, accurately modeling water through computer simulations has been a significant challenge due to the complex nature of the hydrogen-bonding network that water molecules form under different thermodynamic conditions. This complexity has led to over five decades of research and many modeling attempts. The introduction of the MB-pol data-driven many-body potential energy function marked a significant advancement toward a universal molecular model capable of predicting the structural, thermodynamic, dynamical, and spectroscopic properties of water across all phases. By integrating physics-based and data-driven (i.e., machine-learned) components, which correctly capture the delicate balance among different many-body interactions, MB-pol achieves chemical and spectroscopic accuracy, enabling realistic molecular simulations of water, from gas-phase clusters to liquid water and ice. In this review, we present a comprehensive overview of the data-driven many-body formalism adopted by MB-pol, highlight the main results and predictions made from computer simulations with MB-pol to date, and discuss the prospects for future extensions to data-driven many-body potentials of generic and reactive molecular systems.

Disentangled representation learning with physics-informed variational autoencoder for structural health monitoring

The maintenance and replacement of aging infrastructure is becoming increasingly costly, as a large number of structures is approaching the end of its design lifetime. Labor intensive manual inspection and assessment on a large scale is often infeasible, while data-driven machine learning techniques can fail to identify relevant failure mechanisms and suffer from poor generalization to previously unseen conditions, particularly when data is sparse. We propose a multi-task learning approach based on a physics-informed variational autoencoder that leverages a dictionary of physics-based models to improve the predictive performance when limited data is available. The latent space of the autoencoder is augmented with a set of physics-based latent variables that are interpretable and allow for prior knowledge on the latent variables to be included in the autoencoder formulation. To prevent the data-driven components of the encoder and decoder from overriding the known physics, a regularization method is used to impose physical constraints on the latent space and the generative model prediction. The feasibility of the proposed approach is evaluated on a case study of quay walls in Amsterdam, demonstrating improved performance over purely data-driven methods.

Towards good modelling practice for parallel hybrid models for wastewater treatment processes.

This study explores various approaches to formulating a parallel hybrid model (HM) for Water and Resource Recovery Facilities (WRRFs) merging a mechanistic and a data-driven model. In the study, the HM is constructed by training a neural network (NN) on the residual of the mechanistic model for effluent nitrate. In an initial experiment using the Benchmark Simulation Model no. 1, a parallel HM effectively addressed limitations in the mechanistic model's representation of autotrophic bacteria growth and the data-driven model's incapability to extrapolate. Next, different versions of a parallel HM of a large pilot-scale WRRF are constructed, using different calibration/training datasets and different versions of the mechanistic model to investigate the balance between the calibration effort for the mechanistic model and the compensation by the NN component. The HM can improve predictions compared to the mechanistic model. Training the NN on an independent validation dataset produced better results than on the calibration dataset. Interestingly, the best performance is achieved for the HM based on a mechanistic model using default (uncalibrated) parameters. Both long short-term memory (LSTM) and convolutional neural network (CNN) are tested as data-driven components, with a CNN HM (root-mean-squared error (RMSE) = 1.58 mg NO3-N/L) outperforming an LSTM HM (RMSE = 4.17 mg NO3-N/L).

Hybrid AI-Analytical Modeling of Droplet Dynamics on Inclined Heterogeneous Surfaces

This work presents a novel approach for the study of the movement of droplets on inclined surfaces under the influence of gravity and chemical heterogeneities. The developed numerical methodology uses data-driven modeling to extend the applicability limits of an analytically derived reduced-order model for the contact line velocity. More specifically, while the reduced-order model is able to capture the effects of the chemical heterogeneities to a satisfactory degree, it does not account for gravity. To alleviate this shortcoming, datasets generated from direct numerical simulations are used to train a data-driven model for the contact line velocity, which is based on the Fourier neural operator and corrects the reduced-order model predictions to match the reference solutions. This hybrid surrogate model, which comprises of both analytical and data-driven components, is then integrated in time to simulate the droplet movement, offering a speedup of five orders of magnitude compared to direct numerical simulations. The performance of this hybrid model is quantified and assessed in different wetting scenarios, by considering various inclination angles and values for the Bond number, demonstrating the accuracy of the predictions as long as the adopted parameters lie within the ranges considered in the training dataset.

Hybrid modeling design patterns

Design patterns provide a systematic way to convey solutions to recurring modeling challenges. This paper introduces design patterns for hybrid modeling, an approach that combines modeling based on first principles with data-driven modeling techniques. While both approaches have complementary advantages there are often multiple ways to combine them into a hybrid model, and the appropriate solution will depend on the problem at hand. In this paper, we provide four base patterns that can serve as blueprints for combining data-driven components with domain knowledge into a hybrid approach. In addition, we also present two composition patterns that govern the combination of the base patterns into more complex hybrid models. Each design pattern is illustrated by typical use cases from application areas such as climate modeling, engineering, and physics.

Implementation support for the data-driven components of care in a multisite network of coordinated specialty care programs.

Beyond training providers prior to the roll-out of coordinated specialty care (CSC) for first-episode psychosis in clinical settings, implementation support for data-informed care remains an area that has received very little attention. The current paper describes the development and refinement of implementation support for the data-driven components of care in the New Journeys network, Washington State's CSC model for psychosis, comprised of 14 CSC programs. Using the Evidence-Based System for Innovation Support Logic Model, this paper outlines the individual components for implementation support, tools, training, technical assistance, and quality improvement/evaluation that have been developed for the New Journeys network. We present examples of modifications that have occurred over nine years to address the needs of clients, providers, and state-level network administrators to facilitate the data-driven components of care. We conclude with recommendations based on lessons learned in Washington State aimed at improving implementation of data-driven care in CSC models throughout the USA.

Physics-Informed Online Learning by Moving Horizon Estimation: Learning Recurrent Neural Networks in Gray-box Models*

In Model Predictive Control (MPC) closed-loop performance heavily depends on the quality of the underlying prediction model, where such a model must be accurate and yet simple. A key feature in modern MPC applications is the potential for online model adaptation to cope with time-varying changes, part-to-part variations, and complex features of the system dynamics not caught by models derived from first principles. In this paper, we propose to use a physics-informed, or gray-box, model that extends the physics-based model with a data-driven component, namely a Recurrent Neural Network (RNN). Relying on physics-informed models allows for a rather limited size of the RNN, thereby enhancing online applicability compared to pure black-box models. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and learning of the RNN sub-model, a potentially challenging issue due to limited information available in noisy input-output data and lack of knowledge of the internal state of the RNN. We provide a case study on a quadruple tank benchmark showing how the method can cope with part-to-part variations.

Special Issue: Machine Learning and Representation Issues in CAD/CAM

Abstract Machine Learning (ML), a sub-field of Artificial Intelligence (AI), is profoundly reshaping various aspects of human life. Its application in engineering systems promises to address long-standing challenges, although it also introduces new questions. While the potential of ML is undeniable, integrating existing ML methods into Computer-Aided Design and Manufacturing (CAD/CAM) presents distinctive challenges. These challenges encompass representation, adaptation, and the development of novel ML techniques to enhance CAD/CAM systems for diverse design and manufacturing solutions. This special issue aims to explore and resolve issues related to effective engineering model representation for ML, the integration of neural networks and deep generative models into CAD/CAM, mathematical frameworks combining ML with CAD/CAM in geometry and topology, data interpretation, and physics-based learning. The issue features ten papers that delve into various topics, including material prediction for assemblies, robotics mechanism design, multi-modal ML in engineering design, data-driven component segmentation in engineering drawings, evaluating assembly-part semantic knowledge in language models, three-dimensional slice reconstruction from high-resolution 2D images, physics-informed neural networks to expedite thermal simulations in additive manufacturing, transfer learning for defect detection in steel strips, AI-aided hull form design for energy-efficient unmanned underwater vehicles, and real-time high-precision calibration of quadruped robots using machine vision and artificial neural networks. Below, we provide concise summaries of each of the ten papers published in this special issue.

Clinical Manifestations.

It is unclear whether primary progressive apraxia of speech (PPAOS) and progressive agrammatic aphasia (PAA) represent the two ends of a clinical continuum or exist as distinct syndromic entities with specific pathologic/prognostic correlates. We aimed to characterize the main clinical features along the spectrum of progressive non-fluent speech-language disorders (nf-SLD) and explore the existence of data-driven symptom clusters with etiological/prognostic value. We included 98 participants presenting with progressive motor speech impairment and/or agrammatism, of which 43 had an autopsy-confirmed neuropathological diagnosis. Speech pathologists rated motor speech features indicative of dysarthria and AOS. Quantitative measures of expressive/receptive agrammatism were obtained and compared with healthy controls. Baseline and longitudinal disease severity was assessed with the Clinical Dementia Rating sum-of-boxes (CDR-SB). We explored the clustering tendency of the data to form robust symptom clusters and used principal component analysis to extract data-driven clinical components. The longitudinal CDR-SB change was estimated with linear mixed-effects models. Ninety-one participants fitted previously reported clinical profiles (69 AOS+agrammatism, 18 PPAOS and 4 PAA), while 7 remained unclassifiable and were characterized by nonfluent speech and dysarthria in the absence of apraxia of speech or agrammatism. None of the baseline clinical features discriminated between FTLD subgroups. The Hopkins statistic ruled out the existence of syndromic clusters in the whole sample. Three data-driven components explained 71% of the variance ([i]severity-agrammatism, [ii]AOS, and [iii]dysarthria). The component characterized by prominent dysarthria was more specific to patients with Progressive Supranuclear Palsy while the component characterized by severity-agrammatism predicted a faster CDR-SB increase. Our results suggest that nf-SLD represents a clinical continuum. Splitting this continuum into different clinical phenotypes based on its major clinical features does not improve clinical-pathological correlations, stressing the need for new clinical and biological markers. The term nf-SLD could be used to group together these speech-language phenotypes strongly associated with FTLD.

Clinical dimensions along the non-fluent variant primary progressive aphasia spectrum.

It is debated whether primary progressive apraxia of speech (PPAOS) and progressive agrammatic aphasia (PAA) belong to the same clinical spectrum, traditionally termed non-fluent/agrammatic variant primary progressive aphasia (nfvPPA), or exist as two completely distinct syndromic entities with specific pathologic/prognostic correlates. We analysed speech, language and disease severity features in a comprehensive cohort of patients with progressive motor speech impairment and/or agrammatism to ascertain evidence of naturally occurring, clinically meaningful non-overlapping syndromic entities (e.g. PPAOS and PAA) in our data. We also assessed if data-driven latent clinical dimensions with aetiologic/prognostic value could be identified. We included 98 participants, 43 of whom had an autopsy-confirmed neuropathological diagnosis. Speech pathologists assessed motor speech features indicative of dysarthria and apraxia of speech (AOS). Quantitative expressive/receptive agrammatism measures were obtained and compared with healthy controls. Baseline and longitudinal disease severity was evaluated using the Clinical Dementia Rating Sum of Boxes (CDR-SB). We investigated the data's clustering tendency and cluster stability to form robust symptom clusters and employed principal component analysis to extract data-driven latent clinical dimensions (LCD). The longitudinal CDR-SB change was estimated using linear mixed-effects models. Of the participants included in this study, 93 conformed to previously reported clinical profiles (75 with AOS and agrammatism, 12 PPAOS and six PAA). The remaining five participants were characterized by non-fluent speech, executive dysfunction and dysarthria without apraxia of speech or frank agrammatism. No baseline clinical features differentiated between frontotemporal lobar degeneration neuropathological subgroups. The Hopkins statistic demonstrated a low cluster tendency in the entire sample (0.45 with values near 0.5 indicating random data). Cluster stability analyses showed that only two robust subgroups (differing in agrammatism, executive dysfunction and overall disease severity) could be identified. Three data-driven components accounted for 71% of the variance [(i) severity-agrammatism; (ii) prominent AOS; and (iii) prominent dysarthria]. None of these data-driven LCDs allowed an accurate prediction of neuropathology. The severity-agrammatism component was an independent predictor of a faster CDR-SB increase in all the participants. Higher dysarthria severity, reduced words per minute and expressive and receptive agrammatism severity at baseline independently predicted accelerated disease progression. Our findings indicate that PPAOS and PAA, rather than exist as completely distinct syndromic entities, constitute a clinical continuum. In our cohort, splitting the nfvPPA spectrum into separate clinical phenotypes did not improve clinical-pathological correlations, stressing the need for new biological markers and consensus regarding updated terminology and clinical classification.

An integrated data- and theory-driven crash severity model

For crash severity modeling, researchers typically view theory-driven models and data-driven models as different or even conflicting approaches. The reason is that the machine-learning models offer good predictability but weak interpretability, while the latter has robust interpretability but moderate predictability. In order to alleviate the tension between them, this study proposes an integrated data- and theory-driven crash-severity model, known as Embedded Fusion model based on Text Vector Representations (TVR-EF), by leveraging the complementary strengths of both. The model specification consists of two parts. (i) the data-driven component not only mitigate the deficiencies of traditional econometric models, where one-hot encoding is frequently used and makes it impossible to observe semantic relatedness between variable categories, but also enhances the interpretability for the relationship between crash severity and potential influencing factors using the learned embedding weight matrix. (ii) In the theory-driven component, the multinomial logit model is implemented as a 2D-Convolutional Neural Network (2D-CNN) to increase flexibility and decrease dependency on prior knowledge for different crash-severity outcomes. A crash dataset from Guangdong Province, China, is utilized to estimate the TVR-EF model, which is then benchmarked against two traditional econometric models and three widely used machine-learning models. Results indicate that TVR-EF model does not only improve the predictive performance but also makes it easier to interpret.

A hybrid approach for fatigue life prediction of in-service asphalt pavement.

Fatigue cracking is one of the main pavement failures, which makes accurate fatigue life prediction forthe design and maintenance of asphalt pavements crucial. The majority of traditional prediction methods are based entirely on the laboratory fatigue test, without considering the field condition and maintenance data. This paper aims to propose a hybrid approach to fill this gap. The key idea is that the damage condition isback-calculated by anartificial intelligence-based finite-element (FE) model updating using field-monitoring information (data-driven component), which is used to update the parameters in themechanistic composition-specific fatigue life prediction equation (model-driven component). The laboratory test of field cores gives the material non-destructive properties. The simulated pavement response subjected to truck loading shows good agreement with measured values, which indicates that the verified constitutive relationship could be used in the data-driven component. Furthermore, in view that the fatigue test is time- and money-consuming, this paper proposes a non-test estimation of thefatigue characteristic curve based on FE simulation of arepeated direct tension test. Three test pavement sections were employed as case studies. Results showed that the predicted fatigue life changes with the service time. At the early age, semi-rigid pavement has alarger fatigue life than flexible and inverted pavements. This article is part of the theme issue 'Artificial intelligence in failure analysis of transportation infrastructure and materials'.

Hybrid modeling in bioprocess dynamics: Structural variabilities, implementation strategies, and practical challenges.

Hybrid modeling, with an appropriate blend of the mechanistic and data-driven framework, is increasingly being adopted in bioprocess modeling, model-based experimental design (digital-twin), identification of critical process parameters, and optimization. However, the development of a hybrid model from experimental data is an inherently complex workflow, involving designed experiments, selection of the data-driven process, identification of model parameters, assessment fitness, and generalization capability. Depending on the complexity of the process system and purpose, each piece of these modules can flexibly be incorporated into the puzzle. However, this extra flexibility can be a cause of concern to trace an "optimal" model structure. In this paper, the development of hybrid models in a common bioprocess system, selection of data-driven components and their mapping to states, choice of parameter identification techniques, and model quality assurance are revisited. The challenges associated with hybrid-model development, and corrective actions have also been reviewed. The review also suggests the lack of data, and code sharing in communal repositories can be a hurdle in the exploration, and expansion of those tools in a bioprocess system.

A Data Augmentation Method for Data-Driven Component Segmentation of Engineering Drawings

Abstract We present a new data generation method to facilitate an automatic machine interpretation of 2D engineering part drawings. While such drawings are a common medium for clients to encode design and manufacturing requirements, a lack of computer support to automatically interpret these drawings necessitates part manufacturers to resort to laborious manual approaches for interpretation which, in turn, severely limits processing capacity. Although recent advances in trainable computer vision methods may enable automatic machine interpretation, it remains challenging to apply such methods to engineering drawings due to a lack of labeled training data. As one step toward this challenge, we propose a constrained data synthesis method to generate an arbitrarily large set of synthetic training drawings using only a handful of labeled examples. Our method is based on the randomization of the dimension sets subject to two major constraints to ensure the validity of the synthetic drawings. The effectiveness of our method is demonstrated in the context of a binary component segmentation task with a proposed list of descriptors. An evaluation of several image segmentation methods trained on our synthetic dataset shows that our approach to new data generation can boost the segmentation accuracy and the generalizability of the machine learning models to unseen drawings.

Encoding nonlinear and unsteady aerodynamics of limit cycle oscillations using nonlinear sparse Bayesian learning

This article investigates the applicability of a recently proposed, nonlinear sparse Bayesian learning (NSBL) algorithm to identify and estimate the complex aerodynamics of limit cycle oscillations. NSBL provides a semi-analytical framework for determining the data-optimal sparse model nested within a (potentially) over-parameterized model. This is particularly relevant to nonlinear dynamical systems where modelling approaches involve the use of physics-based and data-driven components. In such cases, the data-driven components, where analytical descriptions of the physical processes are not readily available, are often prone to overfitting, meaning that the empirical aspects of these models will often involve the calibration of an unnecessarily large number of parameters. While an overparameterized model may fit the observed data well, such models may be inadequate for making predictions in regimes that are different from those wherein the data were recorded. In view of this, it is desirable to not only calibrate the model parameters, but also identify the optimal compromise between data fit and model complexity. In this article, we exhibit the optimal model discovery for an aeroelastic system wherein the structural dynamics are well-known and described by a differential equation model, coupled with a semi-empirical aerodynamic model for laminar separation flutter, resulting in low-amplitude limit cycle oscillations (LCO). To illustrate the performance of the algorithm, in this article, we use synthetic data and demonstrate the ability of the algorithm to correctly rediscover the optimal model and model parameters, given a known data-generating model. The synthetic data are generated from a forward simulation of a known differential equation model with parameters selected so as to mimic the dynamics observed in wind-tunnel experiments. Subsequently, we demonstrate the performance of the algorithm for model selection using noisy LCO data from wind tunnel experiments. As there is no ground truth available for the experimental data case, we provide a comparison between NSBL and Bayesian model selection to validate the results, and demonstrate the use of NSBL as an efficient alternative to traditional methods.

DNN-MLVEM: A Data-Driven Macromodel for RC Shear Walls Based on Deep Neural Networks

This study proposes the DNN-MVLEM, a novel macromodel for the non-linear analysis of RC shear walls based on deep neural networks (DNN); while most RC shear wall macromodeling techniques follow a deterministic approach to find the right configuration and properties of the system, in this study, an alternative data-driven strategy is proposed instead. The proposed DNN-MVLEM is composed of four vertical beam-column elements and one horizontal shear spring. The beam-column elements implement the fiber section formulation with standard non-linear uniaxial material models for concrete and steel, while the horizontal shear spring uses a multi-linear force–displacement relationship. Additionally, three calibration factors are introduced to improve the performance of the macromodel. The data-driven component of the proposed strategy consists of a large DNN that is trained to predict the force–displacement curve of the shear spring and the three calibration factors. The training data is created using a parametric microscopic FEM model based on the multi-layer shell element formulation and a genetic algorithm (GA) that optimizes the response of the macromodel to match the behavior of the microscopic FEM model. The DNN-MVLEM is tested in two types of examples, first as a stand-alone model and then as part of a two-bay multi-story frame structure. The results show that the DNN-MVLEM is capable of reproducing the results obtained with the microscopic FEM model up to 100 times faster and with an estimated error lower than 5%.

Dynamic modeling of dual fluidized bed steam gasification for control design

Dual fluidized bed steam gasification allows the production of high-value product gas from woody biomass or biogenic residuals. Advanced control concepts such as model predictive control are promising approaches to improve the process performance and efficiency. These control techniques require dynamic models of the process that can predict the plant’s behavior as a function of the manipulated variables. This paper presents a gray-box modeling approach based on mass and energy balances to obtain a mathematical description of the temperatures inside the two reactors and the total mass flows leaving the reactors. The model incorporates data-driven components where first-principle modeling is hardly possible with reasonable effort. An artificial neural network is utilized to model the bed material circulation between the two reactors. Experiments were carried out at a 100 kW pilot plant to generate measurement data both for system identification and model validation. Simulations verify that the model achieves reliable predictions for the dual fluidized bed gasification process.

Data-driven component modeling reveals the functional organization of high-level visual cortex

Data-driven component modeling reveals the functional organization of high-level visual cortex

Physics-Informed Neural Networks (PINNs)-Based Traffic State Estimation: An Application to Traffic Network

Traffic state estimation (TSE) is a critical component of the efficient intelligent transportation systems (ITS) operations. In the literature, TSE methods are divided into model-driven methods and data-driven methods. Each approach has its limitations. The physics information-based neural network (PINN) framework emerges to mitigate the limitations of the traditional TSE methods, while the state-of-art of such a framework has focused on single road segments but can hardly deal with traffic networks. This paper introduces a PINN framework that can effectively make use of a small amount of observational speed data to obtain high-quality TSEs for a traffic network. Both model-driven and data-driven components are incorporated into PINNs to combine the advantages of both approaches and to overcome their disadvantages. Simulation data of simple traffic networks are used for studying the highway network TSE. This paper demonstrates how to solve the popular LWR physical traffic flow model with a PINN for a traffic network. Experimental results confirm that the proposed approach is promising for estimating network traffic accurately.

Incorporating Traffic Flow Model into A Deep Learning Method for Traffic State Estimation: A Hybrid Stepwise Modeling Framework

Traffic state estimation (TSE), which reconstructs the traffic variables (e.g., speed, flow) on road segments using partially observed data, plays an essential role in intelligent transportation systems. Generally, traffic estimation problems can be divided into two categories: model-driven approaches and data-driven approaches. The model-driven method is commonly used to solve TSE efficiently and calibrate the parameters of these models. The data-driven method requires a large amount of historical observed traffic data in order to improve performance accurately. In order to combine the advantages of model-driven and data-driven methods, this paper proposed a hybrid framework incorporating the traffic flow model into deep learning (TFMDL) modeling that contains both model-driven and data-driven components. This paper focuses on highway TSE with observed data from loop detectors. We build a hybrid cost function to adjust the weights of model-driven and data-driven proportions. We then evaluate the proposed framework using the open-access performance measurement system (PMS) dataset on a corridor of US I-405 in Los Angeles, California. The experimental results show the advantages of the proposed TFMDL approach in performing better than several benchmark models in terms of estimation accuracy and data efficiency.