Data-driven Modeling Framework Research Articles

Uncertainty Quantification in Metallic Additive Manufacturing Through Physics-Informed Data-Driven Modeling

The complicated metal-based additive manufacturing (AM) process involves various sources of uncertainty, leading to variability in AM products. For comprehensive uncertainty quantification (UQ) of AM processes, we present a physics-informed data-driven modeling framework, in which multilevel data-driven surrogate models are constructed based on extensive computational data obtained by multiscale multiphysics AM models. It starts with computationally inexpensive surrogate models for which the uncertainty can be readily quantified, followed by global sensitivity analysis for comprehensive UQ study. Using AM-fabricated Ti-6Al-4V components as examples, this study demonstrates the capability of the proposed data-driven UQ framework for efficient investigation of uncertainty propagation from process parameters to material microstructures, then to macrolevel mechanical properties through a combination of advanced AM multiphysics simulations and data-driven surrogate modeling. Model correction and parameter calibration for the constructed surrogate models using limited amounts of experimental data are discussed.

Using Multiple-Feature-Spaces-Based Deep Learning for Tool Condition Monitoring in Ultraprecision Manufacturing

Tool condition monitoring is critical in ultraprecision manufacturing in order to optimize the performance of the overall process, while maintaining the desired part quality. Recently, deep learning has been successfully applied to numerous classification tasks in manufacturing, often to forecast part quality. In this paper, a novel deep learning data-driven modeling framework is presented, which includes a fusion of multiple stacked sparse autoencoders for tool condition monitoring in ultraprecision machining. The proposed computational framework consists of two main structures. First, a training model that is designed with the ability to process multiple parallel feature spaces to learn the lower-level features. Second, a feature fusion structure that is used to learn the higher-level features and associations to tool wear. To achieve this learning structure, a modified loss function is utilized that enhances the feature extraction and classification tasks. A dataset from a real manufacturing process is used to demonstrate the performance of the proposed framework. Experimental results and simulations show that the proposed method successfully classifies the ultraprecision machining case study with over 96% accuracy, while also outperforming comparable methodologies.

A distributionally robust optimization approach to reconstructing missing locations and paths using high-frequency trajectory data

Daily high-frequency trajectory data (e.g., 0.1-s connected vehicle data) provide a promising foundation to improve the observability of travel demand dynamics. However, the raw trajectories are not always accurate and complete due to technical and privacy issues. This paper proposes a data-driven optimization modeling framework to reconstruct the location-duration-path choices for the missing observations from the incomplete trajectories. By processing many-day raw trajectories, we observe a set of historical choices of location-duration-path and identify missing observations in space and time dimensions. To improve computational efficiency, we apply data-driven network-time prisms that reduce the search space for the missing choices. Then, we formulate Distributionally Robust Optimization (DRO) models with likelihood bounds, a special case of data-driven optimization models using phi-divergences (i.e., χ2 distance), to reconstruct the missing choices. To solve the minimax programs of the DRO models while maintaining tractability, we reformulate and solve the equivalent dual problems of the DRO models based on the strong duality theory. To demonstrate and validate the proposed models, we use a real-world connected vehicle dataset containing around 2,800 connected vehicles over two separate months in Southeast Michigan from the Safety Pilot Model Deployment (SPMD) project and a transportation network from OpenStreetMap.

Optimizing the prices for airline flight passes

Abstract Flight pass is a new concept in which airline passengers pre-purchase a number of flights for a flat fee. This flat flee can be customized by the passenger and has an expiring date. Being an innovative concept, the industry is still lacking analytical support to define the prices for these passes. This research is aimed to fill this research gap. We propose a data-driven modeling framework that determines the value of each option per flight and that, consequently, estimates the recommended flight pass price. This is the first time in the literature that flight passes are discussed. The framework is divided in two models. First, a random forest regression is used to predict the ticket price of individual flights. Second, the flight pass prices are predicted using a Monte-Carlo simulation. The simulation is used to estimate the potential behavior of a passenger when using the flight pass. Since no reliable flight pass data was available, we make use of historical booking data from revenue management available from a major African airline to design, calibrate, and validate our models. With an average fit of 57 percent, the random forest regression algorithm can adequately predict the flight price, improving the current trial-and-error or linear regression approaches followed by most airlines. Moreover, the Monte-Carlo simulation is fast enough to support an online implementation of the proposed modeling framework to estimate flight passes prices.

Model Error, Information Barriers, State Estimation and Prediction in Complex Multiscale Systems.

Complex multiscale systems are ubiquitous in many areas. This research expository article discusses the development and applications of a recent information-theoretic framework as well as novel reduced-order nonlinear modeling strategies for understanding and predicting complex multiscale systems. The topics include the basic mathematical properties and qualitative features of complex multiscale systems, statistical prediction and uncertainty quantification, state estimation or data assimilation, and coping with the inevitable model errors in approximating such complex systems. Here, the information-theoretic framework is applied to rigorously quantify the model fidelity, model sensitivity and information barriers arising from different approximation strategies. It also succeeds in assessing the skill of filtering and predicting complex dynamical systems and overcomes the shortcomings in traditional path-wise measurements such as the failure in measuring extreme events. In addition, information theory is incorporated into a systematic data-driven nonlinear stochastic modeling framework that allows effective predictions of nonlinear intermittent time series. Finally, new efficient reduced-order nonlinear modeling strategies combined with information theory for model calibration provide skillful predictions of intermittent extreme events in spatially-extended complex dynamical systems. The contents here include the general mathematical theories, effective numerical procedures, instructive qualitative models, and concrete models from climate, atmosphere and ocean science.

A Hierarchical, Data-Driven Approach to Modeling Single-Cell Populations Predicts Latent Causes of Cell-To-Cell Variability.

All biological systems exhibit cell-to-cell variability. Frameworks exist for understanding how stochastic fluctuations and transient differences in cell state contribute to experimentally observable variations in cellular responses. However, current methods do not allow identification of the sources of variability between and within stable subpopulations of cells. We present a data-driven modeling framework for the analysis of populations comprising heterogeneous subpopulations. Our approach combines mixture modeling with frameworks for distribution approximation, facilitating the integration of multiple single-cell datasets and the detection of causal differences between and within subpopulations. The computational efficiency of our framework allows hundreds of competing hypotheses to be compared. We initially validate our method using simulated data with an understood ground truth, then we analyze data collected using quantitative single-cell microscopy of cultured sensory neurons involved in pain initiation. This approach allows us to quantify the relative contribution of neuronal subpopulations, culture conditions, and expression levels of signaling proteins to the observed cell-to-cell variability in NGF/TrkA-initiated Erk1/2 signaling.

Constrained sparse Galerkin regression

The sparse identification of nonlinear dynamics (SINDy) is a recently proposed data-driven modelling framework that uses sparse regression techniques to identify nonlinear low-order models. With the goal of low-order models of a fluid flow, we combine this approach with dimensionality reduction techniques (e.g. proper orthogonal decomposition) and extend it to enforce physical constraints in the regression, e.g. energy-preserving quadratic nonlinearities. The resulting models, hereafter referred to as Galerkin regression models, incorporate many beneficial aspects of Galerkin projection, but without the need for a high-fidelity solver to project the Navier–Stokes equations. Instead, the most parsimonious nonlinear model is determined that is consistent with observed measurement data and satisfies necessary constraints. Galerkin regression models also readily generalize to include higher-order nonlinear terms that model the effect of truncated modes. The effectiveness of such an approach is demonstrated on two canonical flow configurations: the two-dimensional flow past a circular cylinder and the shear-driven cavity flow. For both cases, the accuracy of the identified models compare favourably against reduced-order models obtained from a standard Galerkin projection procedure. Finally, the entire code base for our constrained sparse Galerkin regression algorithm is freely available online.

Data-driven Simulation Modelling for Progressive Care Units in Hospitals

Nowadays, simulation is extensively used in healthcare as a decision-making tool. However, the traditional simulation models made in the past require an operation research analyst or a person who is familiar with the simulation background. To address this shortcoming, the focus of this study is creation of data-driven simulation modeling framework for progressive care units in health care. In data-driven modeling, the simulation model is automatically generated from the data using templates and add-ins. In this research two add-in are designed in Simio: data table generator and model object importer and C# is used as a coding language to create the add-ins. Data table generator is add-in that creates the structure of data files in Simio, while model object importer generates all model objects using the data imported from an external file or database. The data-driven modeling framework is then validated by building the simulation model of PCU which has components as specified in the external files. The proposed simulation model can be reused to create any other PCU model. The test model is then run for a prescribed time period and its results are recorded. The data-driven simulation model would help the inpatient unit managers in determining the number of resources in the unit in a rapid fashion without duplicating the work of model logic building.

In-silico experiments of zebrafish behaviour: modeling swimming in three dimensions

Zebrafish is fast becoming a species of choice in biomedical research for the investigation of functional and dysfunctional processes coupled with their genetic and pharmacological modulation. As with mammals, experimentation with zebrafish constitutes a complicated ethical issue that calls for the exploration of alternative testing methods to reduce the number of subjects, refine experimental designs, and replace live animals. Inspired by the demonstrated advantages of computational studies in other life science domains, we establish an authentic data-driven modelling framework to simulate zebrafish swimming in three dimensions. The model encapsulates burst-and-coast swimming style, speed modulation, and wall interaction, laying the foundations for in-silico experiments of zebrafish behaviour. Through computational studies, we demonstrate the ability of the model to replicate common ethological observables such as speed and spatial preference, and anticipate experimental observations on the correlation between tank dimensions on zebrafish behaviour. Reaching to other experimental paradigms, our framework is expected to contribute to a reduction in animal use and suffering.

Optimized Deep Learning Framework for Water Distribution Data-Driven Modeling

Deep Learning (DL), unlike conventional Artificial Neural network (ANN), is capable of self-learning data features layer by layer in unsupervised manner and creating a data-driven model with the given dataset. DL has been widely applied to big data analytics, graphics object detection, classification, voice recognition and many other problems. This paper presents an integrated data-driven modelling framework that couples DL with the well-developed evolutionary optimization tool in a scalable and heterogeneous high performance computing paradigm. The integrated framework enables modellers to effectively and efficiently construct a model with a given dataset. It is demonstrated that the framework has wide applicability including but not limited to the simulation, optimization and operation decision of water distribution systems. The paper elaborates the development of the deep learning framework with potential applications of facilitating the data fusion, system simulation and predictive analysis, anomaly detection from the time series data (pressures, flows and consumptions etc.), water usage prediction, construction of a meta-model as a surrogate to the physics-based models (hydraulic and water quality) for water distribution management.

Data-Driven Modeling and Analysis Framework for Cockpit Human–Machine Interaction Issues

As the level of automation within an aircraft increases, the interactions between the pilot and autopilot play a crucial role in its proper operation. Issues with human–machine interactions have been cited as one of the main causes behind many aviation accidents. The complex nature of a pilot’s interactions with an autopilot system makes it challenging to identify all possible actions that may lead to an human–machine interaction-related incident. In this paper, a data-driven analysis tool is proposed to detect and categorize potential human–machine interaction issues in large-scale flight operational quality assurance datasets. The proposed tool is developed using a multilevel clustering framework, where a set of basic clustering techniques are combined with a consensus-based approach to group human–machine interaction events and create a data-driven model from the flight operational quality assurance data. The proposed framework is able to effectively compress a large dataset into a small set of representative clusters within a data-driven model, enabling subject matter experts to effectively investigate potential human–machine interaction issues.

Learning behavior patterns from video for agent-based crowd modeling and simulation

This paper proposes a novel data-driven modeling framework to construct agent-based crowd model based on real-world video data. The constructed crowd model can generate crowd behaviors that match those observed in the video and can be used to predict trajectories of pedestrians in the same scenario. In the proposed framework, a dual-layer architecture is proposed to model crowd behaviors. The bottom layer models the microscopic collision avoidance behaviors, while the top layer models the macroscopic crowd behaviors such as the goal selection patterns and the path navigation patterns. An automatic learning algorithm is proposed to learn behavior patterns from video data. The learned behavior patterns are then integrated into the dual-layer architecture to generate realistic crowd behaviors. To validate its effectiveness, the proposed framework is applied to two different real world scenarios. The simulation results demonstrate that the proposed framework can generate crowd behaviors similar to those observed in the videos in terms of crowd density distribution. In addition, the proposed framework can also offer promising performance on predicting the trajectories of pedestrians.

Data-driven agent-based modeling, with application to rooftop solar adoption

Agent-based modeling is commonly used for studying complex system properties emergent from interactions among many agents. We present a novel data-driven agent-based modeling framework applied to forecasting individual and aggregate residential rooftop solar adoption in San Diego county. Our first step is to learn a model of individual agent behavior from combined data of individual adoption characteristics and property assessment. We then construct an agent-based simulation with the learned model embedded in artificial agents, and proceed to validate it using a holdout sequence of collective adoption decisions. We demonstrate that the resulting agent-based model successfully forecasts solar adoption trends and provides a meaningful quantification of uncertainty about its predictions. We utilize our model to optimize two classes of policies aimed at spurring solar adoption: one that subsidizes the cost of adoption, and another that gives away free systems to low-income households. We find that the optimal policies derived for the latter class are significantly more efficacious, whereas the policies similar to the current California Solar Initiative incentive scheme appear to have a limited impact on overall adoption trends.

Data-driven methods to improve baseflow prediction of a regional groundwater model

Physically‐based models of groundwater flow are powerful tools for water resources assessment under varying hydrologic, climate and human development conditions. One of the most important topics of investigation is how these conditions will affect the discharge of groundwater to rivers and streams (i.e. baseflow). Groundwater flow models are based upon discretized solution of mass balance equations, and contain important hydrogeological parameters that vary in space and cannot be measured. Common practice is to use least squares regression to estimate parameters and to infer prediction and associated uncertainty. Nevertheless, the unavoidable uncertainty associated with physically‐based groundwater models often results in both aleatoric and epistemic model calibration errors, thus violating a key assumption for regression-based parameter estimation and uncertainty quantification. We present a complementary data-driven modeling and uncertainty quantification (DDM-UQ) framework to improve predictive accuracy of physically‐based groundwater models and to provide more robust prediction intervals. First, we develop data-driven models (DDMs) based on statistical learning techniques to correct the bias of the calibrated groundwater model. Second, we characterize the aleatoric component of groundwater model residual using both parametric and non-parametric distribution estimation methods. We test the complementary data-driven framework on a real-world case study of the Republican River Basin, where a regional groundwater flow model was developed to assess the impact of groundwater pumping for irrigation. Compared to using only the flow model, DDM-UQ provides more accurate monthly baseflow predictions. In addition, DDM-UQ yields prediction intervals with coverage probability consistent with validation data. The DDM-UQ framework is computationally efficient and is expected to be applicable to many geoscience models for which model structural error is not negligible.

Multi-scale statistical signal processing of cutting force in cutting tool condition monitoring

In a machining system, accurate tool wear condition monitoring is paramount for guaranteeing the quality of the workpiece and tool life. The cutting force signal has been proved to be the most sensitive signal to depict the tool wear variation during the machining process. This paper introduces a data-driven modeling framework for tool wear monitoring in a machining process, which is based on statistical processing of cutting force wavelet transform by a hidden Markov tree. As a kind of data-driven prognostic approach, this method exploits the tool wear states feature from a deeply data mining perspective while the Markov dependence of wavelet transformation at different frequencies or scales is captured. With lathe turning as the research object, a detailed study on the statistical analysis of cutting force in different tool conditions is presented. A two phases monitoring process that assesses the tool wear conditions from generated model by using the statistical features of cutting force wavelet transform is built. Compared to the traditional classifiers, which usually have a difficult condition in distinguishing tool wear states when given a limited amount of samples, the proposed approach make more efficient use of the training data with high sensitivity to the tool wear conditions. Experimental studies of Inconel 718 cutting show that this approach is robust and makes the cutting force information exploited effectively in data mining. Based on the experimental results, the proposed method for tool condition monitoring outperforms the traditional used Hidden Markov model and Gaussian mixture model approach.

Quantifying the cellular uptake of semiconductor quantum dot nanoparticles by analytical electron microscopy.

Semiconductor quantum dot nanoparticles are in demand as optical biomarkers yet the cellular uptake process is not fully understood; quantification of numbers and the fate of internalized particles are still to be achieved. We have focussed on the characterization of cellular uptake of quantum dots using a combination of analytical electron microscopies because of the spatial resolution available to examine uptake at the nanoparticle level, using both imaging to locate particles and spectroscopy to confirm identity. In this study, commercially available quantum dots, CdSe/ZnS core/shell particles coated in peptides to target cellular uptake by endocytosis, have been investigated in terms of the agglomeration state in typical cell culture media, the traverse of particle agglomerates across U-2 OS cell membranes during endocytosis, the merging of endosomal vesicles during incubation of cells and in the correlation of imaging flow cytometry and transmission electron microscopy to measure the final nanoparticle dose internalized by the U-2 OS cells. We show that a combination of analytical transmission electron microscopy and serial block face scanning electron microscopy can provide a comprehensive description of the internalization of an initial exposure dose of nanoparticles by an endocytically active cell population and how the internalized, membrane bound nanoparticle load is processed by the cells. We present a stochastic model of an endosome merging process and show that this provides a data-driven modelling framework for the prediction of cellular uptake of engineered nanoparticles in general.

Data-driven stochastic modelling of zebrafish locomotion.

In this work, we develop a data-driven modelling framework to reproduce the locomotion of fish in a confined environment. Specifically, we highlight the primary characteristics of the motion of individual zebrafish (Danio rerio), and study how these can be suitably encapsulated within a mathematical framework utilising a limited number of calibrated model parameters. Using data captured from individual zebrafish via automated visual tracking, we develop a model using stochastic differential equations and describe fish as a self propelled particle moving in a plane. Based on recent experimental evidence of the importance of speed regulation in social behaviour, we extend stochastic models of fish locomotion by introducing experimentally-derived processes describing dynamic speed regulation. Salient metrics are defined which are then used to calibrate key parameters of coupled stochastic differential equations, describing both speed and angular speed of swimming fish. The effects of external constraints are also included, based on experimentally observed responses. Understanding the spontaneous dynamics of zebrafish using a bottom-up, purely data-driven approach is expected to yield a modelling framework for quantitative investigation of individual behaviour in the presence of various external constraints or biological assays.Electronic supplementary materialThe online version of this article (doi:10.1007/s00285-014-0843-2) contains supplementary material, which is available to authorized users.

Legitimising data-driven models: exemplification of a new data-driven mechanistic modelling framework

Abstract. In this paper the difficult problem of how to legitimise data-driven hydrological models is addressed using an example of a simple artificial neural network modelling problem. Many data-driven models in hydrology have been criticised for their black-box characteristics, which prohibit adequate understanding of their mechanistic behaviour and restrict their wider heuristic value. In response, presented here is a new generic data-driven mechanistic modelling framework. The framework is significant because it incorporates an evaluation of the legitimacy of a data-driven model's internal modelling mechanism as a core element in the modelling process. The framework's value is demonstrated by two simple artificial neural network river forecasting scenarios. We develop a novel adaptation of first-order partial derivative, relative sensitivity analysis to enable each model's mechanistic legitimacy to be evaluated within the framework. The results demonstrate the limitations of standard, goodness-of-fit validation procedures by highlighting how the internal mechanisms of complex models that produce the best fit scores can have lower mechanistic legitimacy than simpler counterparts whose scores are only slightly inferior. Thus, our study directly tackles one of the key debates in data-driven, hydrological modelling: is it acceptable for our ends (i.e. model fit) to justify our means (i.e. the numerical basis by which that fit is achieved)?

Data-driven modeling and simulation framework for material handling systems in coal mines

In coal mining industry, discrete-event simulation has been widely used to support decisions in material handling system (MHS) to achieve premiums on revenues. However, the conventional simulation modeling approach requires extensive expertise of simulation during the modeling phase and lacks flexibility when the MHS structure changes. In this paper, a data-driven modeling and simulation framework is developed for MHS of coal mines to automatically generate a discrete-event simulation model based on current MHS structural and operational data. To this end, a formal information model based on Unified Modeling Language (UML) is first developed to provide MHS structural information for simulation model generation, production information for simulation execution, and output requirement information for defining simulation outputs. Then, Petri net-based model generation procedures are designed and used to automatically generate a simulation model in Arena® based on the simulation inputs conforming to the constructed information model. The proposed framework is demonstrated for one of the largest open-pit coal mines in the USA, and it has been demonstrated that the framework can be used to effectively generate the simulation models that precisely represent MHS of coal mines, and then be used to support various decisions in coal mining such as equipment scheduling.

Knowledge discovery for friction stir welding via data driven approaches

In this final part of this extensive study, a new systematic data driven fuzzy modelling approach has been developed, taking into account both the modelling accuracy and its interpretability (transparency) as attributes. For the first time, a data driven modelling framework has been proposed designed and implemented in order to model the intricate friction stir welding (FSW) behaviours relating to AA 5083 aluminium alloy, consisting of the grain size, mechanical properties, as well as internal process properties. As a result, Pareto-optimal predictive models have been successfully elicited which, through validations on real data for the aluminium alloy AA 5083, have been shown to be accurate, transparent and generic despite the conservative number of data points used for model training and testing. Compared with analytically based methods, the proposed data driven modelling approach provides a more effective way to construct prediction models for FSW when there is an apparent lack of fundamental process knowledge.