Empowering Expert Judgment: A Data-Driven Decision Framework for Standard Setting in High-Dimensional and Data-Scarce Assessments.

The aim of this study was to investigate the influence of variable density and data-driven k-space undersampling patterns on reconstruction quality for compressed sensing (CS) magnetic resonance imaging to provide recommendations on how to avoid suboptimal CS reconstructions. First, we investigated the influence of randomness and sampling density on the reconstruction quality when using random variable density and variable density Poisson disk undersampling. Compressed sensing reconstructions on 1 knee and 2 brain data sets were compared with fully sampled data sets and reconstruction errors were measured. Sampling coherence was evaluated on the undersampling patterns to investigate whether there was a relation between this coherence measure and reconstruction error.Second, we investigated whether data-driven undersampling methods could improve reconstruction quality when 1 or more fully sampled scans are available as a training set. We implemented 3 different data-driven undersampling methods: (1) Monte Carlo optimization of variable density and variable density Poisson disk undersampling, (2) calculating sampling probabilities directly from the k-space power spectra of the training data, and (3) iterative design of undersampling patterns based on CS reconstruction errors in k-space.Two cross-validation experiments were set up using retrospective undersampling to evaluate the 3 data-driven methods and the influence of the size of the training set. Furthermore, in an experiment that included prospective under sampling, we show the practical applicability of 2 of the data-driven methods. Compressed sensing reconstruction quality was measured with both the normalized root-mean-square error metric and the mean structural similarity index measure. Different optimal variable sampling densities were found for each of the data sets, showing that the optimal sampling density is data dependent. Choosing a sampling density other than the optimal density decreased reconstruction quality. These results suggest that choosing a sampling density without having any reference scans is likely suboptimal. Furthermore, no meaningful correlation was found between sampling coherence and reconstruction error.For the data-driven methods, the iterative method yielded statistically significantly higher reconstruction quality in both retrospective and prospective experiments. In retrospective experiments, the power spectrum method yielded a reconstruction quality that was comparable with the data-driven variable density method. The size of the training set had only a minor influence on the reconstruction quality. Data-driven undersampling methods can be used to avoid suboptimal reconstruction quality in CS magnetic resonance imaging, provided that at least 1 fully sampled scan is available to train the data-driven method. The iterative design method resulted in the highest reconstruction quality.

An accurate assessment of the state of charge (SOC) of electric vehicle batteries is critical for implementing frequency regulation and peak shaving. This study proposes mechanism- and data-driven SOC fusion calculation methods. First, a second-order Thevenin battery model is developed to obtain the physical parameters of the battery. Second, data from the Thevenin battery model and data from four standard cycling conditions in the electric vehicle industry are added to the dataset of the feed-forward neural network data-driven model to construct the test and training sets of the data-driven model. Finally, the error of the mechanism and data-driven fusion modeling method is quantitatively analyzed by comparing the estimation error of the method for the battery SOC at different temperatures with the accuracy of the data-driven SOC estimation method. The simulation results show that the root mean square error, the mean age absolute error, and the maximum error of mechanism and data-driven method for the estimation error of battery SOC are lower than those of the data-driven method by 0.9%, 0.65%, and 1.3%, respectively. The results show that the mechanism and data-driven fusion SOC estimation method has better generalization performance and higher SOC estimation accuracy.

Accurate prediction of endpoint carbon at the dynamic control stage in the converter is crucial for achieving smelting targets. Currently, there are two main methods for converter endpoint prediction: the data-driven method and the mechanism-based method. Data-driven methods exhibit high accuracy but are vulnerable to data quality variations and lack interpretability. Mechanism-based methods provide great interpretability but face challenges in precisely identifying key parameters in the mechanism formula. Inspired by the design concept of physics-informed neural networks (PINNs), an integrated data-driven and mechanism-based method for endpoint carbon prediction in BOF (dmPINNs, data-driven and mechanism-based physics-informed neural networks) is proposed, which has four parts: feature extraction, mechanism-based calculation, data-driven prediction, and integrated prediction. We identify key parameters of the mechanism formula through the neural network to obtain the specified formula for each heat and supervise the training process of the neural network through the mechanism formula to ensure interpretability. Experimental results show that, within the ±0.012% error range, the hit rate of endpoint carbon content using dmPINNs improved by 5.23% compared with the traditional data-driven method and has greater robustness with the supervision of the mechanism formula.

Due to complex characteristics of shale reservoirs, data-driven techniques offer fast and practical solutions in optimization and better management of shale assets. Developments in data-driven techniques enable robust analysis of not only the primary depletion mechanisms, but also the enhanced oil recovery in unconventionals such as natural gas injection. This study provides a comprehensive background on application of data-driven methods in the O&G industry, the process, methodology and learnings along with examples of data-driven analysis of natural gas injection in shale oil reservoirs through the use of publicly-available data. Data is obtained and organized. Patterns in production data are analyzed using data-driven methods to understand key parameters in the recovery process as well as the optimum operational strategies to improve recovery. The complete process is illustrated step-by-step for clarity and to serve as a practical guide for readers. This study also provides information on what other alternative physics-based evaluation methods will be able to offer in the current conditions of data availability and the understanding of physics of recovery in shale oil assets together with the comparison of outcomes of those methods with respect to the data-driven methods. Thereby, a thorough comparison of physics-based and data-driven methods, their advantages, drawbacks and challenges are provided. It has been observed that data organization and filtering take significant time before application of the actual data-driven method, yet data-driven methods serve as a practical solution in fields that are mature enough to bear data for analysis as long as the methodology is carefully applied. The advantages, challenges and associated risks of using data-driven methods are also included. The results of data-driven methods illustrate the advantages and disadvantages of the methods and a guideline for when to use what kind of strategy and evaluation in an asset. A comprehensive understanding of the interactions between key components of the formation and the way various elements of an EOR process impact these interactions, is of paramount importance. Among the few existing studies on the use of data-driven method for natural gas injection in shale oil, a comparative approach including the physics-based methods is included but they lack the interrelationship between physics-based and data-driven methods as a complementary and a competitor within the era of rise of unconventionals. This study closes the gap and serves as an up-to-date reference for industry professionals.

Lamb-wave-based nondestructive testing and evaluation (NDT/E) methods have drawn much attention due to their potential to inspect plate-like structures in a variety of industrial applications. To estimate and/or predict fatigue crack growth, many research efforts have been made to develop data-driven or physics-based methods. Data-driven methods show high predictive capability without the need for physical domain knowledge; however, fewer data can lead to overfitting in the results. On the other hand, physics-based methods can provide reliable results without the need for measured data; however, small amounts of physical information can worsen their predictive capability. In real applications, both the measurable data and the physical information of systems may be considerably limited; it is thus challenging to estimate and/or predict the crack length using either the data-driven or physics-based method alone. To make use of the advantages and minimize the disadvantages of each method, the work outlined in this paper aims to develop a hybrid approach that combines the data-driven and the physics-based methods for estimation and prediction of fatigue crack growth with and without Lamb wave signals. First, with Lamb wave signals, a data-driven method based on signal processing and the random forest model can be used estimate crack lengths. Second, in the absence of Lamb wave signals, a physics-based method based on an ensemble prognostics approach and Walker’s equation can be used to predict crack lengths with the help of the previously estimated crack lengths. To demonstrate the validity of the proposed approach, a case study is presented using datasets provided in the 2019 PHM Conference Data Challenge by the PHM Society. The case study confirms that the proposed method shows high accuracy; the RMSEs for specimens T7 and T8 are calculated as 0.2021 and 0.551, respectively. A penalty score is calculated as 7.63, this result led to a 2nd place finish in the Data Challenge. To the best of the authors’ knowledge, this is the first attempt to propose a hybrid approach for estimation and prediction of fatigue crack growth.

State of charge (SOC) estimation for lithium- ion battery is an important part of battery management system (BMS). Accurate SOC estimation can extend lifespan of the batteries and ensure safety of the batteries operated in electric vehicles. In this paper, a data-driven SOC estimation method is presented. Various artificial neural networks (ANNs) with different hidden layers are trained by a data set which consists of a series of the battery voltage, current and SOC variables obtained from a dynamic discharge test. The battery SOC is then estimated by the trained ANNs. Comparisons of the SOC estimated by the ANNs and model-based method combining with extended Kalman filter (EKF) are implemented. Root mean squared error (RMSE) and mean absolute error (MAE) of the SOC estimated by the data-driven method are very close to those estimated by the model-based method. The results prove that data-driven methods can accurately estimate the SOC of lithium-ion batteries.

The accuracy of capacity estimation is of great importance to the safe, efficient, and reliable operation of battery systems. In recent years, data-driven methods have emerged as promising alternatives to capacity estimation due to higher estimation accuracy. Despite significant progress, data-driven methods are mainly developed by experimental data under well-controlled charge–discharge processes, which are seldom available for practical battery health monitoring under realistic conditions due to uncertainties in environmental and operational conditions. In this paper, a novel method to estimate the capacity of large-format LiFePO4 batteries based on real data from electric vehicles is proposed. A comprehensive dataset consisting of 85 vehicles that has been running for around one year under diverse nominal conditions derived from a cloud platform is generated. A classification and aggregation capacity prediction method is developed, combining a battery aging experiment with big data analysis on cloud data. Based on degradation mechanisms, IC curve features are extracted, and a linear regression model is established to realize high-precision estimation for slow-charging data with constant-current charging. The selected features are highly correlated with capacity (Pearson correlation coefficient < 0.85 for all vehicles), and the MSE of the capacity estimation results is less than 1 Ah. On the basis of protocol analysis and mechanism studies, a feature set including internal resistance, temperature, and statistical characteristics of the voltage curve is constructed, and a neural network (NN) model is established for multi-stage variable-current fast-charging data. Finally, the above two models are integrated to achieve capacity prediction under complex and changeable realistic working conditions, and the relative error of the capacity estimation method is less than 0.8%. An aging experiment using the battery, which is the same as those equipped in the vehicles in the dataset, is carried out to verify the methods. To the best of the authors’ knowledge, our study is the first to verify a capacity estimation model derived from field data using an aging experiment of the same type of battery.

Place-based interventions may reduce violence, but approaches for capturing nearby incidents using kernel density estimation (KDE) vary. KDE smooths geospatial point data, like crime incidents, using a user-specified bandwidth often selected through data-driven approaches that rely on the underlying point pattern. Because point patterns vary by outcome, time, and context, data-driven methods can produce bandwidth sizes that are misaligned with the spatial extent of a place-based intervention, potentially limiting the ability to detect its effect. To illustrate the inferential challenges associated with data-driven bandwidth selection approaches, this study aimed to (1) quantify variability in bandwidths selected through data-driven methods and (2) examine the impact of bandwidth size on simulated intervention effects. We used violent crime data for Philadelphia (2013-2023). For Aim 1, we calculated bandwidth sizes for each crime-year combination using two default data-driven selection criteria and compared selected sizes across crime types and years. For Aim 2, we used a hypothetical place-based intervention with a known effect (30% reduction in nearby assaults) and ran simulations to examine how the intervention effect, estimated using Poisson regression, changed based on the bandwidth size used to estimate the crime density surface. Bandwidth sizes varied significantly by data-driven selection method, crime type, and year (range: 45.9-48,450 ft). For the simulated intervention, "true effects" (i.e., the reduction of nearby assaults attributed to the intervention) were only detectable at bandwidths between 200 and 2900 ft. Larger bandwidths resulted in estimates that incorrectly suggested the intervention was ineffective or increased crime. Data-driven bandwidth selection can obscure or distort intervention effects. Researchers should be critical and transparent when selecting KDE parameters in place-based violence prevention research.

Optimal sensor placement using data-driven sparse learning method with application to pattern classification of hypersonic inlet

Data-driven control methods for modern controller design are becoming popular recently. However, the traditional Proportional–Integral–Derivative (PID) controller is still the most widely used controller to the industrial preference. To tune the parameters of the PID controller, optimal PID tuning approaches such as solving the Riccati equation of the Linear Quadratic Regulator (LQR) provide the optimal solution. The disadvantages of the LQR are that an accurate model of the system is required, and the high-order system must be reduced to the second-order system so that the Riccati equation can be solved. In this paper, a novel data-driven method is proposed to cope with these problems. For the system which is difficult to be identified accurately, the proposed data-driven method can skip the procedure of system identification and tune the parameters of the PID controller directly with the experimental data instead of solving the Riccati equation. This data-driven tuning method also ensures that the parameters of the PID controller for the high-order system are optimized without using the reduced-order model of the system. Simulations are conducted on a tray indexing system with the second-order model and the full-order model demonstrating high applicability and accuracy of the proposed method.

Travel-time prediction holds significant importance in Intelligent Transportation Systems (ITS), providing essential information for tasks such as accident detection and congestion control. While data-driven methods are commonly used for travel-time prediction, the accuracy of predictions heavily relies on the selection of appropriate features. In this study, a two-stage methodology for travel time prediction is introduced, comprising a novel feature selection method called OA2DD with two layers of optimization and a layer of data-driven predictive methods. In the first stage (offline process), the optimal set of features and architecture for the machine learning model is selected using interconnected optimization algorithms. In the second stage (real-time process), travel time prediction is performed using new data from unseen parts of the dataset. The method is applied to a case study involving the M50 motorway in Dublin. Additionally, several wrapper feature selection methods are employed to assess and validate its performance. Results show that the proposed method has a better convergence curve and reduces the number of selected features by up to half, which reduces the computational cost of prediction process up to 56%. Moreover, employing the selected features from the OA2DD method leads to a reduction in predication error by up to 29% compared to the full set of features and the other feature selection methods.

The study of depth-averaged currents is of great significance for the application of underwater gliders. In order to solve the problem of low prediction accuracy of the time series-based depth-averaged current prediction method, the factors affecting the prediction of depth-averaged currents are analyzed and a data-driven prediction method for depth-averaged currents of an underwater glider with sailing parameters is proposed in this paper. First, depth-averaged currents of the underwater glider’s historical profile period and navigation parameters of the underwater glider are taken as inputs to construct multi-input and double-output characteristics. Then, based on the two sets of the real sea trial data and two groups of the generic set of evaluation criteria, five different data-driven methods are used to predict depth-averaged currents. Experimental results show that the prediction result of depth-averaged currents of an underwater glider driven by data with sailing parameters is better than that based on time series, and the prediction accuracy of depth-averaged currents of a future profile period is improved.

A hybrid framework combining data-driven and model-based methods for system remaining useful life prediction

Patient respiratory motion during PET image acquisition leads to blurring in the reconstructed images and may cause significant artifacts, resulting in decreased lesion detectability, inaccurate standard uptake value calculation and incorrect treatment planning in radiation therapy. To reduce these effects data can be regrouped into (nearly) ‘motion-free’ gates prior to reconstruction by selecting the events with respect to the breathing phase. This gating procedure therefore needs a respiratory signal: on current scanners it is obtained from an external device, whereas with data driven (DD) methods it can be directly obtained from the raw PET data. DD methods thus eliminate the use of external equipment, which is often expensive, needs prior setup and can cause patient discomfort, and they could also potentially provide increased fidelity to the internal movement. DD methods have been recently applied on PET data showing promising results. However, many methods provide signals whose direction with respect to the physical motion is uncertain (i.e. their sign is arbitrary), therefore a maximum in the signal could refer either to the end-inspiration or end-expiration phase, possibly causing inaccurate motion correction. In this work we propose two novel methods, CorrWeights and CorrSino, to detect the correct direction of the motion represented by the DD signal, that is obtained by applying principal component analysis (PCA) on the acquired data. They only require the PET raw data, and they rely on the assumption that one of the major causes of change in the acquired data related to the chest is respiratory motion in the axial direction, that generates a cranio-caudal motion of the internal organs. We also implemented two versions of a published registration-based method, that require image reconstruction. The methods were first applied on XCAT simulations, and later evaluated on cancer patient datasets monitored by the Varian Real-time Position ManagementTM (RPM) device, selecting the lower chest bed positions. For each patient different time intervals were evaluated ranging from 50 to 300 s in duration. The novel methods proved to be generally more accurate than the registration-based ones in detecting the correct sign of the respiratory signal, and their failure rates are lower than 3% when the DD signal is highly correlated with the RPM. They also have the advantage of faster computation time, avoiding reconstruction. Moreover, CorrWeights is not specifically related to PCA and considering its simple implementation, it could easily be applied together with any DD method in clinical practice.

Energy performance certificates (EPC) aim to provide transparency about building energy performance (BEP) and benchmark buildings. Despite having qualified auditors examining buildings through on-site visits, BEP accuracy in EPCs is frequently criticized. Qualified auditors are often bound to engineering-based energy quantification methods. However, recent studies have revealed data-driven methods to be more accurate regarding benchmarking. Unlike engineering methods, data-driven methods can learn from data that non-experts might collect. This raises the question of whether data-driven methods allow for simplified data collection while still achieving the same accuracy as prescribed engineering-based methods. This study presents a method for selecting building variables, which even occupants can reliably collect and which at the same time contribute most to a data-driven method's predictive power. The method is tested and validated in a case study on a real-world data set containing 25,000 German single-family houses. Having all data collected by non-experts, results show that the data-driven method achieves about 35% higher accuracy than the currently used engineering method by qualified auditors. Our study proposes a stepwise method to design data-driven EPCs, outlines design recommendations, and derives policy implications.