Development Of Data-driven Models Research Articles

Development of data-driven models to predict seismic drift response of RC wall structures: An application of deep neural networks

This research aimed to develop data-driven models using deep neural networks (DNNs) that can rapidly predict the seismic drift responses of reinforced concrete (RC) wall structures in building frame systems, aiming to overcome the challenges of costly seismic performance evaluation of building structures. A total of 46 RC wall structures ranging from four to 40 stories were analyzed and subjected to 1,000 ground motions including far-field, near-field-pulse, and near-field-no-pulse types. Two input sets were investigated initially. The first input set comprises peak ground acceleration (PGA), spectral accelerations at 1 s–6 s with 1-s intervals, and the first five natural periods of the wall structure. The second input set contains PGA and spectral accelerations at the first five natural periods of the wall structure. The DNN model developed based on the first input set demonstrated superior accuracy achieving an R2 value of 0.880, and slightly outperformed other reputable machine learning models. To enhance the applicability of the developed model, the ground motion type was incorporated as an additional variable to the first input set, which led to an R2 value of 0.869. Ultimately, two DNN models, one based on the first input set and the other considering the ground motion type, were proposed in this study. Finally, the two models were applied to quickly predict seismic drift responses of a new 24-story RC wall structure, and the predicted results were then utilized to efficiently construct the fragility function. The findings highlight the potential of DNNs as an efficient solution to address complex and costly challenges in the earthquake engineering domain.

A combined linear and sinusoidal empirical model for shale gas wellhead pressure and production rates considering fluctuations in gas–water flow

The relationship between wellhead pressure and gas–water flow rate is an important research topic. Nevertheless, a comprehensive theoretical model has not been developed mainly due to complicated field conditions. Finding data-driven models is therefore another crucial direction of research. Despite the development of data-driven models, existing models are not exhaustive. The main disadvantage of the conventional Gilbert model is that it can be used only when both gas and water are produced. Whereas in shale gas production, it is common that only gas or water is produced. To tackle these problems, this study proposes a different type of empirical model considering choke cross-sectional area, volume factors of gas and water, and nonlinearity apart from gas and water rates. In comparison, the Gilbert model only considers choke diameter. The improvements allow the proposed model to only require the existence of one of gas and water, thus expanding the application scope of Gilbert’s model. The applications using field data from 49 shale gas wells indicate the proposed model has the best accuracy among all models compared. The derivation of the proposed model benefits both theoretical and data-driven studies on the relationship between wellhead pressure and gas–water flow rate for hydrocarbon production wells.

On the development of steady-state and dynamic mass-constrained neural networks using noisy transient data

This paper presents the development of algorithms for mass-constrained neural network models that can exactly satisfy mass conservation laws of chemical process systems, even if the training data violates the same. As opposed to approximately satisfying mass balance constraints of a system by considering additional penalty terms in the objective function, algorithms have been developed to solve an equality-constrained optimization problem, thus ensuring the exact satisfaction of the overall mass conservation laws. For developing dynamic mass-constrained networks, hybrid series and parallel all-nonlinear static-dynamic neural network models are leveraged. The proposed algorithms for solving both the inverse and forward problems are tested by considering both steady-state and dynamic data in presence of varieties of noise characterizations. The proposed structures and algorithms are applied to the development of data-driven models of two nonlinear dynamic chemical processes, namely the Van de Vusse reactor system as well as a solvent-based post-combustion CO2 capture process.

Analysis of window opening in arctic community housing and development of data-driven models

Achieving low-energy, comfortable buildings in remote Arctic regions like Nunavik, Canada, presents multiple challenges due to high heating demand and limited energy supply. Understanding human behavior and the impact of practices on energy use is a key element in improving building performance. While previous research on building performance in Arctic regions has often adopted a building-centered approach, this study offers a complementary approach by investigating occupant behavior, focusing on window openings. The sample includes 10 monitored houses in the village of Quaqtaq, Nunavik. Data on weather conditions and window and door status were collected from September 2018 to August 2020. Behavior trends were identified using metrics such as the opening ratio, median duration of an opening, and number of openings. Correlations between opening/closing actions and the conditions of the outdoor environment were analyzed. Findings indicate that occupants interact more with common room windows and tend to leave bedroom windows open for longer periods. Time of the day, temperature, and relative humidity influence both opening and closing actions. Solar irradiance is also an important driver for open actions, while mean wind speed is significant in predicting closing actions. Data-driven models were developed using logistic regression. Comparison between predicted and real data demonstrated a good performance in estimating the opening ratio over the year and seasonal variations but a tendency to overestimate the number of openings and underestimate the duration of an opening.

Acoustic Emission in Ceramic Matrix Composites

Abstract The integration of ceramic matrix composites (CMCs) into safety-critical applications, such as turbine engines and aerospace structures, necessitates a sound understanding of their expected damage evolution under in-service conditions and real-time health-monitoring methods to assess their damage state. The measurement of acoustic emissions (AEs), the transient elastic waves emitted during damage formation, offers an enhanced capability for evaluating damage evolution and structural health in CMCs due to its high sensitivity, accurate temporal resolution, and relative ease of use compared to other nondestructive evaluation (NDE) techniques. Recent advances in numerical simulation methods and data-driven model development, in combination with improved multimodal experimental characterization methods and sensor hardware, are rapidly advancing AE to a mature technique for damage quantification. This review discusses the fundamental principles of acoustic emissions, provides practical guidelines on their experimental characterization and analysis, and offers perspectives on the current state-of-the-art.

Experimental data-driven model development for ESP failure diagnosis based on the principal component analysis

The reliable diagnosis of electrical submersible pump (ESP) failure is a vital process for establishing of the optimal production strategies and achieving minimum development costs. Although traditional ammeter charts and nodal analysis are commonly used for ESP failure diagnosis, the techniques have limitations, as it requires manpower and is difficult to diagnose the failure in real-time. Therefore, in this study, ESP failure diagnosis was performed using the principal component analysis (PCA). First, 11 types of 9,955 pieces of data were acquired from a newly constructed ESP experimental system for 300 days. During the experimental period, ESP failure occurred twice with a significant drop in performance: first on day 112 and second on day 271. The PCA model was constructed with the 8,928 pieces of normal status data and tested with the 1,027 pieces of normal and failure status data. Three principal components were extracted from the measured data to identify the patterns of the normal and failure status. Based on the logistic regression method to analyze the efficiency of the PCA model, it was found out that the developed PCA model showed an accuracy of 93.3%. Therefore, the PCA model was found to be reliable and effective for the ESP failure diagnosis and performance analysis.

MCSSME: Multi-Task Contrastive Learning for Semi-supervised Singing Melody Extraction from Polyphonic Music

Singing melody extraction is an important task in the field of music information retrieval (MIR). The development of data-driven models for this task have achieved great successes. However, the existing models have two major limitations: firstly, most of the existing singing melody extraction models have formulated this task as a pixel-level prediction task. The lack of labeling data has limited the model for further improvements. Secondly, the generalization of the existing models are prone to be disturbed by the music genres. To address the issues mentioned above, in this paper, we propose a multi-Task contrastive learning framework for semi-supervised singing melody extraction, termed as MCSSME. Specifically, to deal with data scarcity limitation, we propose a self-consistency regularization (SCR) method to train the model on the unlabeled data. Transformations are applied to the raw signal of polyphonic music, which makes the network to improve its representation capability via recognizing the transformations. We further propose a novel multi-task learning (MTL) approach to jointly learn singing melody extraction and classification of transformed data. To deal with generalization limitation, we also propose a contrastive embedding learning, which strengthens the intra-class compactness and inter-class separability. To improve the generalization on different music genres, we also propose a domain classification method to learn task-dependent features by mapping data from different music genres to shared subspace. MCSSME evaluates on a set of well-known public melody extraction datasets with promising performances. The experimental results demonstrate the effectiveness of the MCSSME framework for singing melody extraction from polyphonic music using very limited labeled data scenarios.

Local machine learning model-based multi-objective optimization for managing system interdependencies in production: A case study from the ironmaking industry

Modeling interdependencies in a production process is a vital aspect of process engineering. Being built on the fundamental understanding of the process and capturing underlying physical and chemical mechanisms, this task is traditionally carried out through first-principle modeling. However, due to various assumptions and parameter approximation, such approach sometimes struggles to accurately model process dynamics. This is partially due to the requirements for process stability in mission-critical scenarios, where it is not viable to conduct extensive experiments that would enable the development of comprehensive models, and hence, the process control is still partially dependent on the experience of the process operator. In this paper, a data-driven approach based on production data is introduced to address the challenge of discovering, understanding, and modeling interdependencies in a production process by constructing a constrained multi-objective optimization problem. The applicability of the proposed approach is demonstrated through a case study in the ironmaking industry, whereby a set of Machine Learning and causality-based methods is provided while highlighting the significance of transparency and use of domain knowledge in the development of data-driven models. Based on the real data from a Sintering plant, interdependencies are quantified between five key process parameters, which, when supplemented with other control parameters, result in a higher overall process output leading to reductions in operational costs and environmental impact. Such approach shows potential for applications in a broader context across other fields of science and engineering, where it can be used to improve the discovery of interdependencies in complex real-world industrial settings.

A Trip-Based Data-Driven Model for Predicting Battery Energy Consumption of Electric City Buses

The paper presents a novel approach for predicting battery energy consumption in electric city buses (e-buses) by means of a trip-based data-driven regression model. The model was parameterized based on the data collected by running a physical experimentally validated e-bus simulation model, and it consists of powertrain and heating, ventilation, and air conditioning (HVAC) system submodels. The main advantage of the proposed approach is its reliance on readily available trip-related data, such as travel distance, mean velocity, average passenger count, mean and standard deviation of road slope, and mean ambient temperature and solar irradiance, as opposed to the physical model, which requires high-sampling-rate driving cycle data. Additionally, the data-driven model is executed significantly faster than the physical model, thus making it suitable for large-scale city bus electrification planning or online energy consumption prediction applications. The data-driven model development began with applying feature selection techniques to identify the most relevant set of model inputs. Machine learning methods were then employed to achieve a model that effectively balances accuracy, simplicity, and interpretability. The validation results of the final eight-input quadratic-form e-bus model demonstrated its high precision and generalization, which was reflected in the R2 value of 0.981 when tested on unseen data. Owing to the trip-based, mean-value formulation, the model executed six orders of magnitude faster than the physical model.

Over-sampling for data augmentation in data-driven models for the shear strength prediction of RC membranes

Complex reinforced concrete (RC) structures are generally assessed as a group of individual membrane elements subjected to in-plane combined stresses; however, an accurate prediction of the shear strength of such elements is still a complex task. In addition, the limited availability of experimental data of RC panels, which also presents an unbalanced statistical distribution towards lower strength values, limits the development of data-driven models. Thus, it is crucial to explore data augmentation techniques with a view to supporting the development of more accurate and generalizable predictive models in structural engineering. This paper evaluates over-sampling techniques for data augmentation and their use in the creation of an explainable, data-driven model for the shear strength prediction of RC panels. A dataset of 195 experimental tests of RC panels under different loading conditions is initially collected. Five over-sampling techniques are implemented to extend the original dataset and to reduce the imbalance. Three ensemble models (Random Forest, AdaBoost, and XGBoost) are trained with each of the generated datasets. It is observed that all the over-sampling techniques produced predictive models with better performance than the original dataset; however, the results show that by applying the Random Over-Sampling (ROS) the performance metrics of the model can significantly increase (around 39% for some metrics) compared to the model with the original dataset, without any overfitting issues. This strategy allowed to develop an accurate XGBoost model (with a value of R2 = 0.97 for the testing set). The explainability of the final predictive model (XGBoost model obtained from ROS) is evaluated using the SHAP (SHapley Additive exPlanations) analysis. The proposed predictive model outperformed traditional mechanics-based models (improvement of approximately 27% over SMCS and 33% over MCFT for some performance metrics) and with a more controlled error distribution over the range of variables. The proposed model was also more accurate (mean prediction ratio of 0.98) than sophisticated finite element analysis (mean prediction ratio of 0.84) for six specimens of the original dataset.

Labelled dataset for Ultra-Low Temperature Freezer to aid dynamic modelling & fault detection and diagnostics

Ultra-low temperature (ULT) freezers are used to store perishable biological contents and are among the most energy-intensive equipment in laboratory buildings, biobanks, and similar settings. To ensure reliable and efficient operation, it is essential to implement data-driven fault detection and diagnostic algorithms, along with energy optimization techniques. This study presents labelled and long-term ULT-freezer performance dataset, the first of its kind, derived from 53 ULT freezers featuring two different control strategies. The dataset comprises high-resolution historical operation data spanning up to 10 years. More than 10 attributes are recorded from the freezing chamber and critical locations in the refrigeration systems. The dataset is labelled with regular events, such as door openings, as well as fault events obtained from 46 service reports. A scalable data pipeline, consisting of extraction, transformation, and loading processes, is developed to convert the raw data into a format ready for analysis. The dataset can be utilized to support the development of data-driven models and algorithms that advance the intelligent digital operation of ULT freezers.

Synthetic Battery Data Generation and Validation for Capacity Estimation

Simple parameter-based models are typically unable to function in all situations due to the rapidly tightening margins for error in the use of contemporary estimation techniques. The development of data-driven models as a result has made the availability of trustworthy battery data essential. The generation of such data from battery systems necessitates prolonged cycling tests that can last for months, which makes data collection challenging. In this article, a combination of approaches is presented that uses measured operational data from battery packs to generate synthetic data utilizing Markov chains and neural networks in order to ultimately estimate the capacity fade based on operational drive cycle data. The experimental data used for this study are generated using scaled operational cycles with multiple charge/discharge pulses applied repetitively on a commercially available battery pack. The synthetically generated data have the flexibility of matching user-imposed conditions, and have potential for a variety of applications in the analysis and safety of commercial battery systems. Finally, capacity estimation results present the outcome of a comprehensive study into capacity fade estimation in battery packs.

Development of data-driven models for the optimal design of multilayer sand filters for on-site treatment of greywater

Greywater, with limited content of pathogens, makes up more than half of the produced wastewater in urban areas. Given the high cost of wastewater management and treatment, it causes sense to collect greywater separately at the source and employ an on-site treatment system to increase opportunities for on-site water reuse. For this purpose, this paper aims to propose a multilayer granular filter as an inexpensive and simple on-site treatment method for greywater reuse. Furthermore, as determining the optimal structure of multilayer filters is a serious challenge, a simulation-optimization model is developed for determining the best filter configuration. An Artificial Neural Network (ANN) is trained based on experimental results to simulate the filter performance with different combinations of layers and the Genetic Algorithm (GA) is used to find the optimal thickness of different layers based on ANN simulation results. The proposed filter in this paper for greywater treatment consists of silica sand (in three different gradings) and activated carbon (with fixed grading) and treatment measures for evaluation of filter performance are considered as Chemical Oxygen Demand (COD) and Electrical Conductivity (EC). Due to difficulties in collecting, transferring, and storing the real greywater, synthetic greywater was used in this study. 49 experiments with different combinations of filter media thicknesses were performed and the performance of the filter was analyzed. Generally, three-layer filters perform better in COD and EC reduction, however, the average COD and EC elimination equals 36.3% and 15.1%, respectively, which indicates more efficiency of filter in COD reduction in comparison with EC. Based on the optimization-simulation model and experimental results, a filter consisting of 33 cm of fine sand, 20 cm of activated carbon, and 7 cm of medium sand results in the maximum efficiency and can reduce the COD and EC of greywater by 72% and 30%, simultaneously. According to the optimization outputs, the ideal filter can treat greywater up to having EC of 1000 μS/cm and COD of 321 mg/L, which is generally suitable for irrigation purposes.

Infilling of missing data in groundwater pollution prediction models using statistical methods

ABSTRACT Missing data is ubiquitous in hydrology. This phenomenon poses difficulty in the development of data-driven models. Events of missing data in groundwater pollution monitoring networks may occur due to failure of recording devices, malfunctioning of sensors, etc. Handling such missing data implies filling the missing portions of the data structure. Though several studies are available for dealing with missing data in the field of hydrology, literature dealing with such scenarios in groundwater pollution prediction is scarce. This paper assesses four imputation techniques – viz. linear, cubic spline, piece-wise cubic Hermite and modified Akima with cubic Hermite interpolation methods – for developing groundwater pollution prediction models using artificial neural network (ANN). The study employs the development of cascade-forward back-propagation ANN models using missing data ranging from 5% to 75% and evaluating their performance. Results show that imputation techniques can be effective in such circumstances.

Comparative Analysis of Data-Driven Models for Marine Engine In-Cylinder Pressure Prediction

In-cylinder pressure is a key parameter for assessing marine engines health; therefore, its measurement or prediction is paramount for these engines’ diagnosis. Thermodynamic models are typically employed for predicting the in-cylinder pressure, which, however, face challenges pertinent to their calibration and computational time requirements. Recent advances in the field of machine learning have leveraged the development of data-driven models. This study aims to compare two approaches for input features and six regression techniques to select the most effective combination for developing data-driven models to predict the in-cylinder pressure of marine four-stroke engines. Two approaches with different input and output features are initially compared. The first employs regression to directly predict the in-cylinder pressure signal, whereas the second predicts the harmonics coefficients by regression and subsequently estimates the in-cylinder pressure by using a Fourier series function. Typical regression techniques, including linear, elastic, and polynomial regression, support vector machines (SVM), decision trees (DT), and artificial neural networks (ANN), are employed to develop data-driven models based on the second approach. The required datasets for training and testing are derived by using a physical digital twin for the investigated marine engine, which is calibrated against the shop trials and acquired shipboard measurements. The accuracy of the data-driven models are estimated based on the root mean square error considering the testing datasets. For the data-driven model based on the second approach and the ANN regression, a sensitivity study is carried out considering the training datasets and the harmonics number to derive recommendations for these parameters’ values. The results demonstrate that the second approach provides higher accuracy, whereas the ANN regression is the most effective technique for developing data-driven models to estimate the in-cylinder pressure, as the exhibited root mean square error is retained within ±0.2 bar for the ANN trained with 20 samples. This study supports the development and use of data-driven models for marine engines health diagnosis.

Measurement and Verification Building Energy Prediction (MVBEP): An interpretable data-driven model development and analysis framework

Measurement and Verification Building Energy Prediction (MVBEP): An interpretable data-driven model development and analysis framework

Meteorological drought assessment and prediction in association with combination of atmospheric circulations and meteorological parameters via rule based models

The development of data-driven models along with the data that are recorded from satellites has made the study of water engineering more attractive. In this study, SPEI values were calculated with the meteorological variables and then a combination of the meteorological variables as well as atmospheric circulation values was considered as a data mining model input for estimating the droughts. The series of the SPEI values for 3-, 6-, 9-, 12-, 24- and 48-month time scales were obtained during 1988-2016. In this study, both M5 Tree model and Associate Rules were used to predict and analyze the meteorological drought at 6 synoptic stations in the basin considering both the atmospheric circulations (NAO, SO, MOgi, MOac, WeMO, Mediterranean, Red, Black, Caspian, and Persian Gulf SSTs) and the meteorological variables (lagged relative humidity, evapotranspiration, average temperature, minimum-maximum temperature and pressure). The results showed that making use of a combination of the atmospheric circulation indices and meteorological variables in the models increases the accuracy of the model and improves the results in a long term period. The best result in the study of drought in Urmia Lake basin is related to SPEI48 (R = 0.85, RMSE = 0.08, MAE = 0.11) and in the association rules the value of the lift index of best rule is 1.32. Although both approaches provided acceptable results, M5 Tree model had a comparative advantage due to simple and practical linear relationships.

Development of Data-Driven Models for the Prediction of Fuel Effects on Diesel Engine Performance and Emissions

<div>A modelling tool has been developed for the prediction of fuel effects on the performance and exhaust emissions of a heavy-duty diesel engine. Recurrent neural network models with duty-cycle, engine control, and fuel property parameters as inputs were trained with transient test data from a 15-liter heavy-duty diesel engine equipped with a common-rail fuel injection system and a variable geometry turbocharger.</div> <div>The test fuels were formulated by blending market diesel fuels, refinery components, and biodiesel to provide variations in preselected fuel properties, namely, hydrogen-to-carbon (H/C) ratio, oxygen-to-carbon (O/C) ratio, derived cetane number (CN), viscosity, and mid- and end-point distillation parameters. Care was taken to ensure that the correlation between these fuel properties in the test fuel matrix was minimized to avoid confounding model input variables.</div> <div>The test engine was exercised over a wide variety of transient test cycles during which fuel rail pressure, injection timing, airflow, and recirculated exhaust gas flow were systematically varied. The resulting models could predict the transient engine torque and fuel consumption, and nitrogen oxide (NOx), soot, carbon monoxide (CO), total hydrocarbon (THC), and carbon dioxide (CO<sub>2</sub>) exhaust emissions with good accuracy, indicating that the limited number of fuel property parameters selected as model inputs was sufficient to capture the fuel-related effects.</div> <div>The modelling tool can also be used to estimate the relative contributions from changes in the individual fuel inputs to changes in exhaust emissions, and this is illustrated by means of an example blending study with crude-derived diesel fuel, biodiesel, and paraffinic gas-to-liquid (GTL) diesel fuel. This type of novel numerical analysis provides insights into fuel effects which are very difficult to achieve experimentally due to the high degree of intercorrelation between fuel properties that is usually present.</div>

Input-Output Selection for LSTM-Based Reduced-Order State Estimator Design

In this work, we propose a sensitivity-based approach to construct reduced-order state estimators based on recurrent neural networks (RNN). It is assumed that a mechanistic model is available but is too computationally complex for estimator design and that only some target outputs are of interest and should be estimated. A reduced-order estimator that can estimate the target outputs is sufficient to address such a problem. We introduce an approach based on sensitivity analysis to determine how to select the appropriate inputs and outputs for data collection and data-driven model development to estimate the desired outputs accurately. Specifically, we consider the long short-term memory (LSTM) neural network, a type of RNN, as the tool to train the data-driven model. Based on it, an extended Kalman filter, a state estimator, is designed to estimate the target outputs. Simulations are carried out to illustrate the effectiveness and applicability of the proposed approach.

A semi-supervised multi-task learning framework for cancer classification with weak annotation in whole-slide images.

Cancer region detection (CRD) and subtyping are two fundamental tasks in digital pathology image analysis. The development of data-driven models for CRD and subtyping on whole-slide images(WSIs) would mitigate the burden of pathologists and improve their accuracy in diagnosis. However, the existing models are facing two major limitations. Firstly, they typically require large-scale datasets with precise annotations, which contradicts with the original intention of reducing labor effort. Secondly, for the subtyping task, the non-cancerous regions are treated as the same as cancerous regions within a WSI, which confuses a subtyping model in its training process. To tackle the latter limitation, the previous research proposed to perform CRD first for ruling out the non-cancerous region, then train a subtyping model based on the remaining cancerous patches. However, separately training ignores the interaction of these two tasks, also leads to propagating the error of the CRD task to the subtyping task. To address these issues and concurrently improve the performance on both CRD and subtyping tasks, we propose a semi-supervised multi-task learning(MTL) framework for cancer classification. Our framework consists of a backbone feature extractor, two task-specific classifiers, and a weight control mechanism. The backbone feature extractor is shared by two task-specific classifiers, such that the interaction of CRD and subtyping tasks can be captured. The weight control mechanism preserves the sequential relationship of these two tasks and guarantees the error back-propagation from the subtyping task to the CRD task under the MTL framework. We train the overall framework in a semi-supervised setting, where datasets only involve small quantities of annotations produced by our minimal point-based(min-point) annotation strategy. Extensive experiments on four large datasets with different cancer types demonstrate the effectiveness of the proposed framework in both accuracy and generalization.