Data-driven Modeling Approach Research Articles

A Scalable Geospatial Data-Driven Localization Approach for Modeling of Low Voltage Distribution Networks and Low Carbon Technology Impact Assessment

A Scalable Geospatial Data-Driven Localization Approach for Modeling of Low Voltage Distribution Networks and Low Carbon Technology Impact Assessment

Neural Network Normal Estimation and Bathymetry Reconstruction From Sidescan Sonar

Sidescan sonar intensity encodes information about changes in the surface normal of the seabed. However, other factors such as seabed geometry as well as its material composition also affect the return intensity. One can model these intensity changes in a forward direction from the surface normals from a bathymetric map and physical properties to the measured intensity, or alternatively one can use an inverse model which starts from the intensities and models the surface normals. Here, we use an inverse model which leverages deep learning's ability to learn from data; a convolutional neural network is used to estimate the surface normal from the sidescan. Once this information is estimated, a bathymetric map can be reconstructed through an optimization framework that also includes altimeter readings to provide a sparse depth profile as a constraint. Implicit neural representation learning was recently proposed to represent the bathymetric map in such an optimization framework. In this article, we use a neural network to represent the map and optimize it under constraints of altimeter points and estimated surface normal from sidescan. By fusing multiple observations from different angles from several sidescan lines, the estimated results are improved through optimization. We demonstrate the efficiency and scalability of the approach by reconstructing a high-quality bathymetry using sidescan data from a large sidescan survey. We compare the proposed data-driven inverse model approach of modeling a sidescan with a forward Lambertian model. We assess the quality of each reconstruction by comparing it with data constructed from a multibeam sensor.

Comparison of Data-Driven Modeling and Identification Approaches for a Self-Balancing Vehicle

This paper gives a systematic comparison of different state–of–the–art modeling approaches and the corresponding parameter identification processes for a self–balancing vehicle. In detail, a nonlinear grey box model, its extension to consider friction effects, a parametric black box model based on regression neural networks, and a hybrid approach are presented. The parameters of the models are identified by solving a nonlinear least squares problem. The training, validation, and test datasets are collected in full–scale experiments using a self–balancing vehicle. The performance of the different models used for ego–motion prediction are compared in full–scale scenarios, as well. The investigated model architectures can be used to improve both, simulation environments and model–based controller design. This paper shows the upsides and downsides arising from using the different modeling approaches. Videos showing the self–balancing vehicle in action are available at: https://tinyurl.com/mvn8j7vf

Deep Neural Networks With Koopman Operators for Modeling and Control of Autonomous Vehicles

Autonomous driving technologies have received notable attention in the past decades. In autonomous driving systems, identifying a precise dynamical model for motion control is nontrivial due to the strong nonlinearity and uncertainty in vehicle dynamics. Recent efforts have resorted to machine learning techniques for building vehicle dynamical models, but the generalization ability and interpretability of existing methods still need to be improved. In this paper, we propose a pure data-driven vehicle modeling approach based on deep neural networks with an interpretable Koopman operator. The main advantage of using the Koopman operator is to represent the nonlinear dynamics in a linear lifted feature space. In the proposed approach, a deep learning-based extended dynamic mode decomposition algorithm is presented to learn a finite-dimensional approximation of the Koopman operator. A multi-step prediction loss function is used in the training process, enabling a long-term prediction capability. Furthermore, a data-driven model predictive controller with the learned Koopman model is designed for velocity profile tracking control of autonomous vehicles. Simulation results in a high-fidelity CarSim environment show that our approach outperforms previously developed traditional and advanced modeling methods. Velocity profile tracking tests of the autonomous vehicle are also performed in the CarSim environment. The results show that our approach has better tracking accuracy and higher computational efficiency than the model predictive control algorithms using a nonlinear model and a linear time-varying model.

Selecting slacks-based data envelopment analysis models

Data envelopment analysis (DEA) is a well-known data-driven mathematical modeling approach that aims at evaluating the relative efficiency of a set of comparable decision making units (DMUs) with multiple inputs and multiple outputs. The number of inputs and outputs (performance factors) plays a vital role for successful applications of DEA. There is a statistical and empirical rule in DEA that if the number of performance factors is high in comparison with the number of DMUs, then a large percentage of the units will be determined as efficient, which is questionable and unacceptable in the performance evaluation context. However, in some real-world applications, the number of performance factors is relatively larger than the number of DMUs. To cope with this issue, selecting models have been developed to select a subset of performance factors that lead to acceptable results.In this paper, we extend a pair of optimistic and pessimistic approaches, involving two alternative individual and summative selecting models, based on the slacks-based model. We mathematically validate the proposed models with some theorems and lemmas and illustrate the applicability of our models using 18 active auto part companies in the largest stock exchange in Iran.

Refining acute human exposure assessment to pesticides in surface water: An integrated data-driven modeling approach

The substantial spatial and temporal variability of pesticides has led to large uncertainties when determining their peak aqueous concentrations. There is however a lack of large-scale studies dealing with accurate determination of annual maximum daily concentration (AMDC) across the landscape and over time based on the publicly available monitoring data. We developed a novel data-driven approach that firstly used time series modeling to generate AMDCs for qualified water monitoring sites in the conterminous U.S. With feature variables such as pesticide use and land cover compiled into the dataset, machine learning models using eXtreme Gradient Boosting (XGBoost) and Random Forest Regressor (RF) were then developed to estimate AMDCs in surface waters across the U.S. Both models exhibited significant predictability, while a hybrid model consisting of the average predictions by XGBoost and RF model had the highest prediction accuracy (mean absolute error (MAE): 1.23; R2: 0.61). The analysis of permutation variable importance indicated that pesticide use and drainage area were the two most important drivers. Partial dependence analysis revealed that pesticide use, precipitation, cultivated crop land cover and solubility exhibited concentration-promoting effects, whereas drainage area and molecular weight had concentration-demoting effects. Soil adsorption coefficient (Koc) showed nonmonotonic effects. The hybrid model was used to predict and map AMDCs of four example pesticides, including 2,4-dichlorophenoxyacetic acid (2,4-D), atrazine, glyphosate and imidacloprid during 2016–2019 at national scale. The predictive capability was validated using independent monitoring datasets. The fully evaluated approach significantly reduced the uncertainties in modeling annual peak concentrations and served as a valuable solution for conducting geographically oriented, highly refined exposure assessments for pesticides.

Data-driven modeling of multimode chemical process: Validation with a real-world distillation column

Real-world industrial processes frequently operate in different modes such as start-up, transient, and steady-state operation. Since different operating modes are governed by different process dynamics, deriving a single data-driven model representing the entire operation of such multimode processes is not a viable option. A reasonable solution to this problem is to develop a separate model for each operating mode, which requires the extraction of data for each operating mode from raw data. Based on this viewpoint, this work develops a data-driven modeling approach using clustering and featuring selection techniques to improve the quality of raw data and develop a predictive model for a multimode industrial process. In particular, the developed method focuses on training a steady-state predictive model as monitoring steady-state conditions is crucial for achieving the desired product quality. Firstly, K-means clustering is performed to extract data describing the steady-state operation mode from the available raw data. Next, feature selection is applied to the clustered data using Pearson’s correlation coefficient to identify input features relevant to target features. Finally, an LSTM model is trained using the clustered data and identified features to predict the steady-state operation. The validity and effectiveness of the developed method are demonstrated using a real-world 2,3-Butanediol distillation process dataset.

Data-driven modeling of ultra-supercritical unit coordinated control system by improved transformer network

Ultra-supercritical units are widely employed in peak shaving and fluctuation suppression of power grid. To serve for efficient operation and fast power varying, it is necessary to develop a model for the coordinated control system (CCS) of the unit. The existing model may be limited by operating conditions and assumptions, reflecting partial dynamic characteristics of CCS. In this work, based on an improved transformer neural network and optimization algorithm, a novel data-driven modeling approach is proposed for the boiler-turbine coupled process in the CCS of the ultra-supercritical units. Firstly, the proposed model is composed of transformer network with convolution operation and residual network with self-attention mechanism. Relying on the ability of the network to extract long-term dependencies and local feature maps, an accurate identification model is obtained under the fast power fluctuation operation of ultra-supercritical units. Secondly, some hyper parameters in the network are identified by the proposed heap-based optimization algorithm fused with harris hawks optimization (HHBO). In this stage, the higher accuracy and rapidity are fulfilled with the assistance of two improvements. Then, the convergence of the proposed network is proved using the Lyapunov function. Finally, by employing operational data from 1000 MW ultra-supercritical unit, the superiority of the proposed approach is further validated by extensive simulations and comparative experiments. The mean square errors of the electrical power, the main steam pressure, and the separator temperature are 1.34E−04, 1.54E−04 and 1.71E−04, respectively. The proposed approach can provide reference for further control strategy design.

Data-driven, high resolution ocean wave forecasting and extreme wave predictions

Reliable ocean wave forecasts are critical for reducing the risk posed by extreme waves for ships and offshore infrastructure. Despite insights gained from theoretical investigations and constantly improving operational wave forecasting systems, the emergence of extreme waves remain unpredictable. In this article, the authors propose a data-driven modelling approach to generate high resolution wave forecasts for capturing individual waves, including rogue waves. To this end, a decomposition of the sea surface into a set of rapid oscillations and slowly varying amplitudes is utilized. The slow amplitude variations are subsequently forecasted by using universal, data-driven methods. In this approach, the extrapolation range of the data-driven techniques is extended by the slowness of the amplitude variations. The method’s capabilities are demonstrated by using available measurements from an experimental wave tank and field data from ocean buoys.

Quantifying Spatio-temporal risk of Harmful Algal Blooms and their impacts on bivalve shellfish mariculture using a data-driven modelling approach

Harmful algal blooms (HABs) intoxicate and asphyxiate marine life, causing devastating environmental and socio-economic impacts, costing at least $8bn/yr globally. Accumulation of phycotoxins from HAB phytoplankton in filter-feeding shellfish can poison human consumers, prompting harvesting closures at shellfish production sites. To quantify long-term intoxication risk from Dinophysis HAB species, we used historical HAB monitoring data (2009–2020) to develop a new modelling approach to predict Dinophysis toxin concentrations in a range of bivalve shellfish species at shellfish sites in Western Scotland, South-West England and Northern France. A spatiotemporal statistical modelling framework was developed within the Generalized Additive Model (GAM) framework to quantify long-term HAB risks for different bivalve shellfish species across each region, capturing seasonal variations, and spatiotemporal interactions. In all regions spatial functions were most important for predicting seasonal HAB risk, offering the potential to inform optimal siting of new shellfish operations and safe harvesting periods for businesses. A 10-fold cross-validation experiment was carried out for each region, to test the models’ ability to predict toxin risk at harvesting locations for which data were withheld from the model. Performance was assessed by comparing ranked predicted and observed mean toxin levels at each site within each region: the correlation of ranks was 0.78 for Northern France, 0.64 for Western Scotland, and 0.34 for South-West England, indicating our approach has promise for predicting unknown HAB risk, depending on the region and suitability of training data.

Design assessments of complex systems based on design oriented modelling and uncertainty analysis

In the design of nonlinear complex systems, the output responses of a complex system subject to demanding loading conditions need to be assessed before the system can be used in practice. Current assessment approaches (including physical model based analysis, Finite Element (FE) simulation and similitude experiments) are often time-consuming and problematic. The objective of this study is to develop a novel data-driven approach to predict the output responses, as well as their uncertainties, of a complex system for the aim of design assessment. The results can be used to decide whether redesign of the complex system is required to avoid damage for actual use. To achieve this goal, a novel design-oriented data driven modelling approach has been developed based on the FROLS (Forward Regression Orthogonal Least Squares) algorithm and K-fold cross validation. In this approach, the NARX (Nonlinear AutoRegressive with eXogenous input) model of the system is identified and validated. After that, the Monte Carlo Simulation (MCS) method is applied to quantify the uncertainties of the NARX model coefficients. Subject to specified inputs, the model coefficients’ uncertainties are propagated to the model predicted output responses for design assessments. Finally, the proposed approach is verified by a case study on the design assessment of a Chinese Space Station Scientific Experiment Rack (CSS-SER). The results indicate the efficiency of the proposed approach and its potential to be applied to solve many engineering design problems.

Early detection of violating Mobile Apps: A data-driven predictive model approach

Mobile app stores are the key distributors of mobile applications. They regularly apply vetting processes to the deployed apps. Yet, some of these vetting processes might be inadequate or applied late. The late removal of applications might have unpleasant consequences for developers and users alike. Thus, in this work we propose a data-driven predictive approach that determines whether the respective app will be removed or accepted. It also indicates the features' relevance that help the stakeholders in the interpretation. In turn, our approach can support developers in improving their apps and users in downloading the ones that are less likely to be removed. We focus on the Google App store and we compile a new data set of 870,515 applications, 56% of which have actually been removed from the market. Our proposed approach is a bootstrap aggregating of multiple XGBoost machine learning classifiers. We propose two models: user-centered using 47 features, and developer-centered using 37 features, the ones only available before deployment. We achieve the following Areas Under the ROC Curves (AUCs) on the test set: user-centered = 0.792, developer-centered = 0.762.

Data-Driven Constitutive Modeling via Conjugate Pairs and Response Functions

Response functions completely define the constitutive equations for a hyperelastic material. A strain measure providing an orthogonal stress response, grants response functions directly from experimental curves. One of these strain measures is the Laplace stretch based on QR-decomposition of the deformation gradient. Such a recovery of response functions from experimental data fits the paradigm of data-driven modeling. The set of independent conjugate stress–strain base pairs were proposed as a simple alternative for constitutive modeling and thus might be efficient for data-driven modeling. In the present paper we explore applicability of the conjugate pairs approach for data-driven modeling. The analysis is based on representation of the conjugate pairs in terms of the response functions due to the Laplace stretch. Our analysis shows that one can not guarantee independence of these pairs except in the case of infinitesimal strain.

Hierarchical Coarse-Grained Strategy for Macromolecular Self-Assembly: Application to Hepatitis B Virus-Like Particles.

Macromolecular self-assembly is at the basis of many phenomena in material and life sciences that find diverse applications in technology. One example is the formation of virus-like particles (VLPs) that act as stable empty capsids used for drug delivery or vaccine fabrication. Similarly to the capsid of a virus, VLPs are protein assemblies, but their structural formation, stability, and properties are not fully understood, especially as a function of the protein modifications. In this work, we present a data-driven modeling approach for capturing macromolecular self-assembly on scales beyond traditional molecular dynamics (MD), while preserving the chemical specificity. Each macromolecule is abstracted as an anisotropic object and high-dimensional models are formulated to describe interactions between molecules and with the solvent. For this, data-driven protein-protein interaction potentials are derived using a Kriging-based strategy, built on high-throughput MD simulations. Semi-automatic supervised learning is employed in a high performance computing environment and the resulting specialized force-fields enable a significant speed-up to the micrometer and millisecond scale, while maintaining high intermolecular detail. The reported generic framework is applied for the first time to capture the formation of hepatitis B VLPs from the smallest building unit, i.e., the dimer of the core protein HBcAg. Assembly pathways and kinetics are analyzed and compared to the available experimental observations. We demonstrate that VLP self-assembly phenomena and dependencies are now possible to be simulated. The method developed can be used for the parameterization of other macromolecules, enabling a molecular understanding of processes impossible to be attained with other theoretical models.

A wavelet-based dynamic mode decomposition for modeling mechanical systems from partial observations

Dynamic mode decomposition (DMD) has emerged as a popular data-driven modeling approach to identifying spatio-temporal coherent structures in dynamical systems, owing to its strong relation with the Koopman operator. For dynamical systems with external forcing, the identified model should not only be suitable for a specific forcing function but should generally approximate the input–output behavior of the underlying dynamics. A novel methodology for modeling those classes of dynamical systems is proposed in the present work, using wavelets in conjunction with the input–output dynamic mode decomposition (ioDMD). The wavelet-based dynamic mode decomposition (WDMD) builds on the ioDMD framework without the restrictive assumption of full state measurements. Our non-intrusive approach constructs numerical models directly from trajectories of the full model’s inputs and outputs, without requiring the full-model operators. These trajectories are generated by running a simulation of the full model or observing the original dynamical systems’ response to inputs in an experimental framework. Hence, the present methodology is applicable for dynamical systems whose internal state vector measurements are not available. Instead, data from only a few output locations are only accessible, as often the case in practice. The present methodology’s applicability is explained by modeling the input–output response of an Euler–Bernoulli finite element beam model. The WDMD provides a linear state-space representation of the dynamical system using the response measurements and the corresponding input forcing functions. The developed state-space model can then be used to simulate the beam’s response towards different types of forcing functions. The method is further validated on a real (experimental) data set using modal analysis on a simple free–free beam, demonstrating the efficacy of the proposed methodology as an appropriate candidate for modeling practical dynamical systems despite having no access to internal state measurements and treating the full model as a black-box.

A data-driven modeling approach for the sustainable remediation of persistent arsenic (As) groundwater contamination in a fractured rock aquifer through a groundwater recirculation well (IEG-GCW®)

Persistent arsenic (As) pollution sources from anthropogenic activities pose a serious threat to groundwater quality. This work aims to illustrate the application of an innovative remediation technology to remove As from a heavily contaminated fractured aquifer at a historically polluted industrial site. Groundwater circulation well (GCW) technology was tested to significantly increase and accelerate the mobilization and removal of As in the source area. The GCW extracts and re-injects groundwater at different depths of a vertical circulation well. By pumping out and reinjecting in different screen sections of the well, the resulting vertical hydraulic gradients create recirculation cells and affect and mobilize trapped contaminants that cannot be influenced by traditional pumping systems. The first 45-m deep IEG-GCW® system was installed in 2020, equipped with 4 screen sections at different depths and with an above-ground As removal system by oxidation and filtration on Macrolite (Enki). A geomodeling approach supports both remediation and multi-source data interpretation. The first months of operation demonstrate the hydraulic effectiveness of the IEG-GCW® system in the fractured rock aquifer and the ability to significantly enhance As removal compared to conventional pumping wells currently feeding a centralized treatment system. The recirculation flow rate amounts to about 2 m3/h. Water pumped and treated by the GCW system is reintroduced with As concentrations reduced by an average of 20%–60%. During the pilot test, the recirculating system removed 23 kg As whilst the entire central pump-and-treat (P&T) system removed 129 kg, although it treated 100 times more water volume. The P&T plant removed 259 mg As per m3 of pumped and treated groundwater while the GCW removed 4814 mg As per m3 of the treated groundwater. The results offer the opportunity for a more environmentally sustainable remediation approach by actively attacking the contamination source rather than containing the plume.

NIMG-43. DIFFUSION BASIS SPECTRUM IMAGING DISTINGUISHES HIGH-GRADE GLIOMA TREATMENT EFFECT FROM TUMOR PROGRESSION

Abstract BACKGROUND Following standard-of-care chemoradiation therapy for high-grade glioma, it is often challenging to distinguish treatment changes from true tumor progression using conventional magnetic resonance imaging (MRI). Diffusion basis spectrum imaging (DBSI) uses a data-driven multiple-tensor modeling approach to disentangle specific histologic components and structural features present within individual imaging voxels, offering an opportunity to noninvasively differentiate treatment effect from tumor progression. This represents the first study of in vivo longitudinal DBSI for brain tumors in patients. METHODS Adult patients were prospectively recruited who met the criteria of known pathological diagnosis of high-grade glioma or glioblastoma (WHO grade III-IV) and undergoing or completing standard-of-care radiation therapy with concurrent chemotherapy. DBSI and conventional MRI data were acquired longitudinally beginning with a 4-week post-radiation therapy MRI until progression as defined by standard RANO criteria or biopsy. DBSI analyses were performed with isotropic diffusion profiles defined as restricted fraction (0 ≤ D ≤ 1.0 μm2/ms) and hindered fraction (1.0 < D ≤ 1.5 μm2/ms). RESULTS Eleven patients with contrast-enhancing high-grade gliomas were enrolled between June 2019 and March 2020, with final DBSI data obtained in November 2020. Of these, eight patients had tumor progression, two had treatment effect, and one had stable disease during the study period. Within the contrast-enhancing regions, the DBSI hindered fraction map demonstrated hypointensity for all cases of tumor progression, and hyperintensity for cases of treatment effect. DBSI-based biomarkers were present 0.7 to 12.0 months (median 2.4, IQR 2.8) before definitive diagnosis of treatment effect or tumor progression using conventional methods of MRI and/or biopsy. CONCLUSIONS DBSI hindered fraction can distinguish new contrast-enhancing lesions in the setting of high-grade glioma as treatment effect or tumor progression in advance of conventional MRI. Future work will incorporate machine learning algorithms to further improve DBSI-based predictions of tumor histopathology.

Using the design of dynamic experiments to optimize photosynthetic cyanophycin production by Synechocystis sp.

The production of cyanophycin by photosynthetic microorganisms, as a high-value bio-based compound, is getting increasing interest. The aim of this work is to maximize the production of this compound by the cyanobacterium Synechocystis sp. in semi-batch cultivation systems, by applying a data-driven modeling approach based on the Design of Dynamic Experiments (DoDE) and Response Surface Model (RSM) methodologies. A first set of experiments, carried out inside an initially defined domain, was used to find a preliminary RSM model describing cyanophycin concentration as a function of incident light intensity profile, temperature, and phosphorus supply profile. The model was then improved, according to an evolutionary optimization approach, by carrying out additional experiments in a modified domain, exploiting information derived by the initial model. The updated model was used to identify the optimal operating conditions resulting in maximum cyanophycin concentration at the end of the batch. The cyanophycin concentration found experimentally (228.2 ± 20.0 mg/L) in these conditions fell within the confidence interval of the model prediction. Remarkably, this experimentally obtained value represents a significant (about 20 %) increase in the cyanophycin production with respect to the highest value found in the experiments before the optimization step (184.3 ± 0.8 mg/L).

A Data-Driven Proxy Modeling Approach Adapted to Well Placement Optimization Problem

Abstract Well placement optimization (WPO) plays an essential role in field management and economy. However, it entails massive computational time and demand since hundreds, even thousands, simulation runs are needed. Different types of proxy models have been utilized to address this issue. Among different proxy models, data-driven proxies are preferred as they can determine the combined effect of several parameters without suffering from the type and the number of modeling parameters. This article aims to develop a data-driven proxy model in an artificial intelligence framework adapted to the WPO problem. This proxy estimates and compares the oil recovery for different well configurations. Our contribution is building a dynamic proxy by training a sequence of static proxies in a time-dependent manner to make more benefit from the modeling capability of artificial neural networks (ANNs). The workflow comprises preparing a learning database using experimental design techniques, finding the significant parameters by searching the parameter space, training and validating a series of ANNs to obtain the desired field response, and conducting a blind test to ensure the model performance and generality. This proxy is then coupled with the genetic algorithm to find an optimal well configuration in a test case. Verifying the results obtained by our proxy with those of a commercial simulator shows that the objectives of constructing this proxy for WPO are successfully achieved.

A Review of Data-Driven Approaches for Measurement and Verification Analysis of Building Energy Retrofits

Although the energy and cost benefits for retrofitting existing buildings are promising, several challenges remain for accurate measurement and verification (M&V) analysis to estimate these benefits. Due to the rapid development in advanced metering infrastructure (AMI), data-driven approaches are becoming more effective than deterministic methods in developing baseline energy models for existing buildings using historical energy consumption data. The literature review presented in this paper provides an extensive summary of data-driven approaches suitable for building energy consumption prediction needed for M&V applications. The presented literature review describes commonly used data-driven modeling approaches including linear regressions, decision trees, ensemble methods, support vector machine, deep learning, and kernel regressions. The advantages and limitations of each data-driven modeling approach and its variants are discussed, including their cited applications. Additionally, feature engineering methods used in building energy data-driven modeling are outlined and described based on reported case studies to outline commonly used building features as well as selection and processing techniques of the most relevant features. This review highlights the gap between the listed existing frameworks and recently reported case studies using data-driven models. As a conclusion, this review demonstrates the need for a flexible M&V analysis framework to identify the best data-driven methods and their associated features depending on the building type and retrofit measures.