Data-driven Model Research Articles

TabNet: Locally Interpretable Estimation and Prediction for Advanced Proton Exchange Membrane Fuel Cell Health Management

In the pursuit of advanced Predictive Health Management (PHM) for Proton Exchange Membrane Fuel Cells (PEMFCs), conventional data-driven models encounter considerable barriers due to data reconstruction resulting in poor data quality, and the complexity of models leading to insufficient interpretability. In addressing these challenges, this research introduces TabNet, a model aimed at augmenting predictive interpretability, and integrates it with an innovative data preprocessing technique to enhance the predictive performance of PEMFC health management. In traditional data processing approaches, reconstruction methods are employed on the original dataset, significantly reducing its size and consequently diminishing the accuracy of model predictions. To overcome this challenge, the Segmented Random Sampling Correction (SRSC) methodology proposed herein effectively eliminates noise from the original dataset whilst maintaining its effectiveness. Notably, as the majority of deep learning models operate as black boxes, it becomes challenging to identify the exact factors affecting the Remaining Useful Life (RUL) of PEMFCs, which is clearly disadvantageous for the health management of PEMFCs. Nonetheless, TabNet offers insights into the decision-making process for predicting the RUL of PEMFCs, for instance, identifying which experimental parameters significantly influence the prediction outcomes. Specifically, TabNet’s distinctive design employs sequential attention to choose features for reasoning at each decision-making step, not only enhancing the accuracy of RUL predictions in PEMFC but also offering interpretability of the results. Furthermore, this study utilized Gaussian augmentation techniques to boost the model’s generalization capability across varying operational conditions. Through pertinent case studies, the efficacy of this integrated framework, merging data processing with the TabNet architecture, was validated. This work not only evidences that the effective data processing and strategic deployment of TabNet can markedly elevate model performance but also, via a visual analysis of the parameters’ impact, provides crucial insights for the future health management of PEMFCs.

A review on machine learning approaches in cardiac tissue engineering

Cardiac tissue engineering (CTE) holds promise in addressing the clinical challenges posed by cardiovascular disease, the leading global cause of mortality. Human induced pluripotent stem cells (hiPSCs) are pivotal for cardiac regeneration therapy, offering an immunocompatible, high density cell source. However, hiPSC-derived cardiomyocytes (hiPSC-CMs) exhibit vital functional deficiencies that are not yet well understood, hindering their clinical deployment. We argue that machine learning (ML) can overcome these challenges, by improving the phenotyping and functionality of these cells via robust mathematical models and predictions. This review paper explores the transformative role of ML in advancing CTE, presenting a primer on relevant ML algorithms. We focus on how ML has recently addressed six key address six key challenges in CTE: cell differentiation, morphology, calcium handling and cell-cell coupling, contraction, and tissue assembly. The paper surveys common ML models, from tree-based and probabilistic to neural networks and deep learning, illustrating their applications to better understand hiPSC-CM behavior. While acknowledging the challenges associated with integrating ML, such as limited biomedical datasets, computational costs of learning data, and model interpretability and reliability, we examine suggestions for improvement, emphasizing the necessity for more extensive and diverse datasets that incorporate temporal and imaging data, augmented by synthetic generative models. By integrating ML with mathematical models and existing expert knowledge, we foresee a fruitful collaboration that unites innovative data-driven models with biophysics-informed models, effectively closing the gaps within CTE.

Uncertainty Quantification of Data-driven Quality Prediction Model For Realizing the Active Sampling Inspection of Mechanical Properties in Steel Production

Pre-production quality defect inspection is a crucial step in industrial manufacturing, and many traditional inspection strategies suffer from inefficiency issues. This is especially true for tasks such as mechanical performance testing of steel products, which involve time-consuming processes like offline sampling, specimen preparation, and testing. The inspection volume significantly impacts the production cycle, inventory, yield, and labor costs. Constructing a data-driven model for predicting product quality and implementing proactive sampling inspection based on the prediction results is an appealing solution. However, the prediction uncertainty of data-driven models poses a challenging problem that needs to be addressed. This paper proposes an active quality inspection approach for steel products based on the uncertainty quantification in the predictive model for mechanical performance. The objective is to reduce both the sampling frequency and the omission rate on the production site. First, an ensemble model based on improved lower and upper bound estimation is established for interval prediction of mechanical performance. The uncertainty of the specific value prediction model is quantitatively estimated using interval probability distributions. Then, a predictive model for the mechanical performance failure probability is built based on the prediction interval size and probability distribution. By determining an appropriate probability threshold, the trade-off between prediction accuracy and defect detection accuracy (recall rate) is balanced, enabling the establishment of an active sampling strategy. Finally, this functionality is integrated into the manufacturing execution system of a steel factory, realizing a mechanical performance inspection approach based on proactive sampling. The proposed approach is validated using real production datasets. When the probability threshold is set to 30%, the prediction accuracy and recall rate for failure mechanical performance samples are 75% and 100%, respectively. Meanwhile, the sampling rate is only 5.33%, while controlling the risk of omission. This represents a 50% reduction in sampling rate compared to the inspection rules commonly used in actual production. The overall efficiency of product quality inspection is improved, and inspection costs are reduced.

Application of transfer learning to overcome data imbalance and extrapolation for model predictive control: A real-life case

This paper proposes a transfer learning (TL)-based control-oriented model development framework. In particular, this study examines the transferability from virtual (source) to existing (target) buildings to overcome the data imbalance issue of the data-driven approach. The target system is a cooling system comprising two supply air fans and four condensing units. First, synthetic data rich enough to provide fundamental knowledge about the target system were generated using the EnergyPlus model. A data-driven model was subsequently developed to learn the underlying dynamics of the system. By adopting TL using an imbalanced dataset measured from the target system, the knowledge that the model learned from the virtual data was transferred to the target system of the existing building. The results showed that the transfer learning model could accurately describe the dynamic behavior of the target system and predict the supply air temperature with marginal errors (CVRMSE: 5.4%, MAE: 0.96 ℃). In other words, the TL from virtual to existing buildings can overcome the data imbalance issue for developing a reliable data-driven model.

A data-driven kinetic model for opinion dynamics with social network contacts – ADDENDUM

A data-driven kinetic model for opinion dynamics with social network contacts – ADDENDUM

Tensile damage evolution and mechanical behaviour of SiCf/SiC mini-composites through 4D in-situ micro-CT and data-driven modelling

Damage evolution of SiCf/SiC ceramic matrix mini-composites (CMMCs) was characterised by using a 4D in-situ micro-computed tomography (CT) tensile test and digital volume correlation (DVC) technique. Additionally, a CT damage data-driven shear lag model was developed to predict its tensile stress-strain response. A 4D in-situ X-ray CT tensile test of a unidirectional SiCf/SiC mini-composite was first carried out. Then two ad-hoc deep-learning image segmentation models were developed to automatically identify its microstructure and damages induced by tension, respectively. Damage evolution was quantitively characterised by visualising the initiation and propagation of matrix cracks in three-dimensions (3D). A two-step approach was employed to evaluate its 3D internal strain distributions at various loading levels, which further revealed strain concentrations and helped establishing the tensile stress-strain response of the CMMCs. It was observed that transverse cracking is the predominant damage mode, and the average crack opening displacement increases with loading. A high-fidelity X-ray CT data-driven shear lag model was developed, incorporating inputs of transverse matrix crack spacing calculated by the 4D in-situ CT test data. The predicted stress-strain response showed a good correlation with the experimental results.

Prediction-Based Power Consumption Monitoring of Industrial Equipment Using Interpretable Data-Driven Models

Efficient energy optimization and scheduling in industrial factories depend on accurate, reasonable, and real-time monitoring of equipment power consumption. However, the power prediction of industrial equipment requires a large number of process data, which could be inevitably contaminated by some imperfect data, due to the harsh environments. Monitoring or predicting equipment power consumption is usually not feasible using data-driven black-box models with imperfect data. This paper proposes a prediction-based approach for power consumption monitoring using an interpretable data-driven model. First, a data preprocessing method is used to remove outliers and fill in missing values. Then, a Volterra polynomial basis function (VPBF) model is built to predict equipment power consumption. This model decomposes power values into a series of basis functions consisting of input parameters. Moreover, to compensate for data dropouts during the power consumption monitoring process, a networked predictive monitoring system is also proposed. Finally, this paper presents two case studies based on actual production equipment in an industrial manufacturing factory. The results demonstrate that the proposed approach can achieve satisfactory monitoring accuracy and adaptation ability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by the problem of power consumption monitoring of industrial equipment in harsh production environments. A conventional solution is to predict the power consumption using data-driven black-box models, with limited feasibility and interpretability. This paper proposes a new power consumption monitoring approach, utilizing an interpretable data-driven model and a networked predictive method. This approach accurately reveals the transparent relationships between the output and input parameters. The power prediction result is consequently interpretable and more reasonable. Meanwhile, this approach actively compensates for data dropouts in the network, which can help operators real-time grasp the power consumption of equipment. Furthermore, this approach can be integrated into energy optimization systems as a basis of optimization and decision-making. Practical applications in an aluminum manufacturing company located in Guangdong Province demonstrate that this approach is feasible and applicable. Future research is to expand this approach further for energy optimization and scheduling.

A transient analysis framework for hydropower generating systems under parameter uncertainty by integrating physics-based and data-driven models

It is important to ensure the transient safety and stability of Hydropower generating system (HGS) both in the design and operation stages. Many uncertainties in the Hydropower Generating Units (HGUs) have a strong effect on the transient characteristics. In this study, a transient analysis framework under parameter uncertainty is proposed, which includes: integrating modeling, uncertain transient analysis, and transient calibration. First, not only a physical HGS including a comprehensive model of hydro-turbine but also Polynomial-Chaos-Kriging data-driven models are established for transient considering uncertain HGU parameters. Secondly, based on the evaluation system, the contributions of uncertain HGU parameters on the transient characteristics are quantified objectively. The results show that: (1) in evaluation systems, 6 HGU parameters play a key effect on transient characteristics, and the uncertain properties of 6 indicators can reach moderate variabilities; (2) the effect of HGU uncertainty can be reduced by optimizing the control strategies. Finally, uncertain HGU parameters are calibrated considering 3 measured transient processes. Compared with the conventional HGU model, the max relative absolute error of the volute pressure, the HGU speed, and the servomotor stroke are reduced by 2.14%, 2.70%, and 37.81%. This framework provides new insight into the effect of the uncertain HGU model on transient characteristics, which could be beneficial for the safe and stable operation of HGSs.

Structured barycentric forms for interpolation-based data-driven reduced modeling of second-order systems

An essential tool in data-driven modeling of dynamical systems from frequency response measurements is the barycentric form of the underlying rational transfer function. In this work, we propose structured barycentric forms for modeling dynamical systems with second-order time derivatives using their frequency domain input-output data. By imposing a set of interpolation conditions, the systems’ transfer functions are rewritten in different barycentric forms using different parametrizations. Loewner-like algorithms are developed for the explicit computation of second-order systems from data based on the developed barycentric forms. Numerical experiments show the performance of these new structured data-driven modeling methods compared to other interpolation-based data-driven modeling techniques from the literature.

WSBCV: A data-driven cross-version defect model via multi-objective optimization and incremental representation learning

Cross-version defect prediction (CVDP) refers to training an excellent model on tagged defect data in a previously released project version, and then performing defect prediction on an unlabeled instance module in the current version. Nevertheless, the complicated internal intrinsic construction hidden behind the code defects makes it difficult for the previous cross-version defect models to capture more discriminative software features, and seriously restrains the CVDP performance. In this study, we propose an intelligent data-driven CVDP model named WSBCV based on multi-objective optimization and incremental representation learning. We firstly leverage an advanced deep generation adversarial network – WGAN-GP (Wasserstein GAN with Gradient Penalty) to perform data augmentation, including balancing defect classes and synthesizing abundant training instances. Secondly, a multi-objective SPEA/R (Strength Pareto-based Evolutionary Algorithm / Reference) feature selection optimization method is built to effectively search the fewest representative feature subsets while achieving the minimum error. Finally, a powerful defect predictor for CVDP based on the BLS (Broad Learning System) with incremental learning is built to learn excellent feature representations and achieve incremental online model update quickly. Experimental results across 32 cross-version pairs from 45 version demonstrate that the proposed SPEA/R, BLS and WSBCV all have statistically significant difference advantages compared to ten multi-objective feature selection approaches, six defect predictors and two CVDP models, respectively.

Space Weather with Quantified Uncertainties: Improving Space Weather Predictions with Data-Driven Models of the Solar Atmosphere and Inner Heliosphere

To address Objective II of the National Space Weather Strategy and Action Plan “Develop and Disseminate Accurate and Timely Space Weather Characterization and Forecasts” and US Congress PROSWIFT Act 116–181, our team is developing a new set of open-source software that would ensure substantial improvements of Space Weather (SWx) predictions. On the one hand, our focus is on the development of data-driven solar wind models. On the other hand, each individual component of our software is designed to have accuracy higher than any existing SWx prediction tools with a dramatically improved performance. This is done by the application of new computational technologies and enhanced data sources. The development of such software paves way for improved SWx predictions accompanied with an appropriate uncertainty quantification. This makes it possible to forecast hazardous SWx effects on the space-borne and ground-based technological systems, and on human health. Our models include (1) a new, open-source solar magnetic flux model (OFT), which evolves information to the back side of the Sun and its poles, and updates the model flux with new observations using data assimilation methods; (2) a new potential field solver (POT3D) associated with the Wang–Sheeley–Arge coronal model, and (3) a new adaptive, 4-th order of accuracy solver (HelioCubed) for the Reynolds-averaged MHD equations implemented on mapped multiblock grids (cubed spheres). We describe the software and results obtained with it, including the application of machine learning to modeling coronal mass ejections, which makes it possible to improve SWx predictions by decreasing the time-of-arrival mismatch. The tests show that our software is formally more accurate and performs much faster than its predecessors used for SWx predictions.

A novel data-driven model for prediction and adaptive control of pH in raceway reactor for microalgae cultivation

This work proposes a new data-driven model to estimate and predict pH dynamics in freshwater raceway photobioreactors. The resulting model is based purely on data measured from the reactor and divides the pH dynamics into two different behaviors. One behavior is described by the variation of pH due to the photosynthesis phenomena made by microalgae; and the other comes from the effect of CO$_2$ injections into the medium for control purposes. Moreover, it was observed that the model parameters vary throughout the day depending on the weather conditions and reactor status. Thus, a decision tree algorithm is also developed to capture the parameter variation based on measured variables of the system, such as solar radiation, medium temperature, and medium level. The proposed model has been validated for a data set of more than 100 days during 10 months in a semi-industrial raceway reactor, covering a wide range of weather and system scenarios. Additionally, the proposed model was used to design an adaptive control algorithm which was also experimentally tested and compared with a classical fixed parameter control approach.

Mesh-Agnostic Decoders for Supercritical Airfoil Prediction and Inverse Design

Mesh-agnostic models have advantages in terms of processing unstructured spatial data and incorporating partial differential equations. Recently, they have been widely studied for constructing physics-informed neural networks, but they need to be trained on a case-by-case basis and require long training times. On the other hand, fast prediction and design tools are desired for aerodynamic shape designs, and data-driven mesh-based models have achieved great performance. Therefore, this paper proposes a data-driven mesh-agnostic decoder that combines the fast prediction ability of data-driven models and the flexibility of mesh-agnostic models. The model is denoted by an implicit decoder, which consists of two subnetworks, i.e., ShapeNet and HyperNet. ShapeNet is based on implicit neural representation, and HyperNet is a simple neural network. The implicit decoder is trained for the fast prediction of supercritical airfoils. Different activation functions are compared, and a spatial constraint is proposed to improve the interpretability and generalization ability of the model. Then, the implicit decoder is used together with a mesh-based encoder to build a generative model, which is used for the inverse design of supercritical airfoils with specified physical features.

After-meal blood glucose level prediction for type-2 diabetic patients

Type 2 Diabetes, a metabolic disorder disease, is becoming a fast growing health crisis worldwide. It reduces the quality of life, and increases mortality and health care costs unless managed well. After-meal blood glucose level measure is considered as one of the most fundamental and well-recognized steps in managing Type 2 diabetes as it guides a user to make better plans of their diet and thus control the diabetes well. In this paper, we propose a data-driven approach to predict the 2 h after meal blood glucose level from the previous discrete blood glucose readings, meal, exercise, medication, & profile information of Type 2 diabetes patients. To the best of our knowledge, this is the first attempt to use discrete blood glucose readings for 2 h after meal blood glucose level prediction using data-driven models. In this study, we have collected data from five prediabetic and diabetic patients in free living conditions for six months. We have presented comparative experimental study using different popular machine learning models including support vector regression, random forest, and extreme gradient boosting, and two deep layer techniques: multilayer perceptron, and convolutional neural network. We present also the impact of different features in blood glucose level prediction, where we observe that meal has some modest and medication has a good influence on blood glucose level.

Data-Driven Modeling and Compensation Strategy of PMSM Considering Core Loss and Saturation

Data-Driven Modeling and Compensation Strategy of PMSM Considering Core Loss and Saturation

Three-dimensional finite element-based data-driven thermal model of in-situ automated fiber placement

This study proposes a data-driven model for thermal history prediction during in-situ consolidation of thermoplastic composites using Hot Gas Torch (HGT)-assisted Automated Fiber Placement (AFP). Temperature data was generated using a three-dimensional finite element (FE) model that was fitted to and validated with experimentally measured temperatures. Temperature curves for various combinations of heat source velocity and locations through the thickness and width were extracted from the fitted FE model. Three feedforward neural networks were trained to predict the temperature distribution during processing. The work demonstrates the effectiveness and potential of using neural networks for online prediction and optimization.

Data-Driven Modeling and Operation Optimization With Inherent Feature Extraction for Complex Industrial Processes

In response to the tenets of Industry 4.0, operation optimization in industrial processes has become a significant research topic. However, the uncertainties prevailing in the process pose challenges to production operations, especially the feedstock properties. In this work, the operation optimization study is performed on a distillation unit (DU), a typical plant in the industrial process. To enhance production performance, a modeling and operation optimization strategy based on feedstock property and production features is presented. One of the difficulties is how to uncover features from high-dimensional and imperfect data, where imperfect data refers to product quality data that is unavailable online. In the strategy, we inject the inherent characteristic of the process into the data-driven method to extract the feedstock property in a data-based and knowledge-oriented manner. Further, optimal feature representation and process modeling can be achieved by customizing the network structure. The operation optimization problem is formulated to adjust the top temperature of the distillation column (TTDC) to achieve satisfactory production under varying feedstock properties. Experimental results illustrate that the process model based on feedstock property and production features (PM-FP-PF) can better fit the physical process mechanism even based on incomplete information in industrial data. Industrial experiments have shown the proposed strategy has advanced generalization ability to the different feedstock properties. The proposed operation optimization strategy (OOS) improves the product qualification rate and has broad application prospects in industrial processes with similar features. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Industrial processes suffer from a variety of disturbances that interrupt the smooth operation of the system, such as varying feedstock properties. How to deal with them is the key to improve the product qualification rate. In this work, we propose a data-driven modeling and operation optimization framework to improve product quality under varying feedstock properties. The dynamic variation characteristics of the feedstock properties can be obtained by analyzing the physical properties of the production unit. This is used as a basis for representing and extracting feedstock properties in a data-driven way. Further, a process model based on feedstock property and production features (PM-FP-PF) is built to predict product quality. An operation optimization strategy with production capacity consideration is established. It can determine the optimal operation action required by the current system, mitigating the uncertainty of feedstock property. This operation optimization system has been applied to a distillation unit in the hydrofining process. The application results show that the process model achieves satisfactory estimation accuracy, and the operation optimization strategy has improved production performance.

Data-driven modeling of reconfigurable multi-accelerator systems under dynamic workloads

Reconfigurable multi-accelerator systems used as computing offloading platforms in edge-cloud continuum scenarios usually have to deal with highly dynamic workloads and operating conditions. In order to properly take advantage of their parallel processing capabilities and increase execution performance for a given workload, these systems need to continuously adapt their configuration (i.e., number and type of accelerators) at run time. When working at the edge, additional requirements such as energy efficiency must be also met. In this paper, Machine Learning techniques are applied to extract predictive models of the execution of different combinations of hardware accelerators on a reconfigurable multi-accelerator platform, aiming at satisfying the previously mentioned continuous optimization needs. One of the key benefits of the proposed approach is that its data-driven models can transparently estimate the impact of the complex interactions between hardware accelerators due to run-time resource contention among them and with the rest of the system, as opposed to traditional modeling approaches that cannot include that information in an easy and scalable way (e.g., analytical models). The proposed models are complemented with a complete infrastructure to generate, execute and monitor dynamic workloads in FPGA-based systems. This infrastructure has been used to (i) quantitatively analyze resource contention in reconfigurable multi-accelerator systems and (ii) produce the training and evaluation datasets for the ML-based models using annotated power consumption and execution performance traces. Experimental results obtained with a reconfigurable multi-accelerator platform based on the ARTICo3 framework running the MachSuite benchmarks show that the proposed modeling approach is highly effective, with a relative prediction error of less than 5% on average for both power consumption and execution performance. Result also show that the ML-based models achieve high accuracy levels when predicting the impact of resource contention and accelerator interaction on both metrics, with a mean relative prediction error of less than 0.6% and a standard deviation below 4.7% for the worst case.

AI-Based Design With Data Trimming for Hybrid Phase Shift Modulation for Minimum-Current-Stress Dual Active Bridge Converter

Dual active bridge (DAB) topology has been recognized as the key circuit for the next generation of high-frequency-link power conversion systems. In order to realize desired operating performance, DAB modulation strategies need to be selected carefully. Different modulation strategies have been considered and combined into a hybrid one, which is able to fully optimize performance. However, to develop a hybrid modulation strategy, the conventional methods including harmonic model and piecewise model have difficulty in balancing modeling accuracy and manpower burden. Although recent data-driven modulation approaches can automate the design process, the accuracy of data-driven models decreases nontrivially with the existence of outliers and without sufficient data. Hence, to alleviate human-dependence and improve modeling accuracy, this paper proposes an AI-based design with data trimming (AI-DT) for hybrid phase shift modulation. Two two-degree-of-freedom modulation strategies are considered for the sake of optimal current stress performance. AI-DT firstly adopts the one-class support vector machine (SVM) to exclude outliers. Second, the state-of-the-art extreme gradient boosting (XGBoost), which is insensitive to training data size, is adopted to build data-driven current stress models for the DAB converter. After that, differential evolution (DE) algorithm helps to choose modulation strategy and optimize modulation parameters for optimal current stress. Generally, the proposed AI-DT is developed in an automated fashion, which largely relieves manual computational complexity while exhibiting satisfactory modulation accuracy. The effectiveness of the proposed AI-DT approach has been experimentally verified with a 1kW hardware prototype, realizing optimal current stress under full operating conditions with more than 96.5% as peak efficiency.

Analysis of dissolved oxygen influencing factors and concentration prediction using input variable selection technique: A hybrid machine learning approach

Accurate quantification of dissolved oxygen (DO) is critically important for the protection and management of aquatic ecosystems. Successful applications have utilized mechanistic and data-driven models to simulate DO content in aquatic ecosystems. However, mechanistic models present challenges due to their complex and difficult-to-solve conditions, making them less portable. Additionally, data-driven model predictions are hindered by the challenge of numerous input variables, impacting both the running speed and prediction performance of the model. To address these challenges, water quality data and meteorological data of the Tanjiang River were obtained. The maximum information coefficient (MIC) input variable selection technique was employed to identify primary environmental factors influencing DO changes. Furthermore, coupled with support vector regression (SVR), two models (SVR and MIC-SVR) were employed to estimate the DO concentration of the Tanjiang River, and the optimal model was established. The results indicated a shift in the primary pollution factor from ammonia nitrogen to total phosphorus after recent treatment in the Tanjiang River. In comparison with the SVR model, the root mean square error (RMSE) of the MIC-SVR model was reduced by 4.46%, and the Nash-efficiency coefficient (NSE) was improved by 45.85%. In addition, study of kernel function selection revealed that considering as many kernel functions as possible is necessary for improving the performance of the SVR model. Conclusively, the proposed MIC-SVR model serves as an effective tool to analyze the relationship between DO and environmental factors, identifying the primary causes of low DO, and accurately predict the DO concentration in the Tanjiang River (especially in its middle and lower reaches), thus providing a reference for governmental decision-making on water environmental protection and water resource management.