Training Data Points Research Articles

A robust approach for bond strength prediction of mortar using machine learning with SHAP interpretability

Application of machine learning (ML) in predicting mortars bond strength contributes to low experimental cost and high accuracy. This study explores the performance of four ML models—Back Propagation Neural Network (BPNN), Support Vector Regression (SVR), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost)—using 304 experimental data points for training and 76 for testing. Among these, the XGBoost model demonstrated the highest predictive accuracy, with R², MAE, and RMSE values of 0.95, 0.06, and 0.13 in the training set, and 0.94, 0.08, and 0.16 in the testing set, respectively. Compared with traditional models, ML approaches, particularly XGBoost, were more effective in providing reliable predictions of bond strength. SHapley Additive exPlanation (SHAP) show that substrate type, ethylene-vinyl acetate (EVA) content and water to binder ratio W/C have the most remarkable effect on bond strength, while the type of cement and admixture shows the least effect. These findings highlight the potential of ML models in optimizing mortar formulations and improving prediction reliability.

Intelligent ultrasonic aspirator: Advancing tissue differentiation through hierarchical classification during hand-held resection

Modern neurosurgery strives to maximize tumor removal while preserving healthy tissue integrity. Accurate intraoperative differentiation between tumor and healthy tissue is crucial yet challenging. Often neurosurgeons rely on their experience and haptic feedback during palpation to distinguish between tumor and healthy tissue. A commonly used hand-held tool for tissue removal during neurosurgery is the ultrasonic aspirator, which changes its electrical properties as it interacts with tissue. The goal is to equip the ultrasonic aspirator with the ability to differentiate between different types of tissue while at the same time not interfering with the surgical workflow and providing comprehensible outcomes. To this end, a hierarchical classification approach is employed as a proof of concept, enabling precise identification of tissue stiffness during resection.The hierarchical approach is compared with the standard flat classification, commonly used in machine learning. Within the hierarchical approach, two strategies are employed: mandatory leaf-node predictions (MLNP) and non-mandatory leaf-node predictions (NMLNP). The NMLNP allows prediction to revert to a parent node when certainty is low. Data are acquired on three artificial tissue models – differing in stiffness – with an ultrasonic aspirator in a hand-held manner. The dataset comprises 1,821 data points for training and 186 for testing after balancing.The results indicate a slight performance advantage for the hierarchical classification MLNP approach over the flat classification approach in the absence of confidence thresholds, with weighted F2-scores of 0.781 and 0.762, respectively. However, the application of confidence thresholds results in both approaches exhibiting comparable performance, with the hierarchical NMLNP approach achieving a weighted F1-score of 0.920, thereby demonstrating superior overall performance. The effects of enforcing these thresholds and excluding data with low certainty are thoroughly investigated. This work emphasizes the feasibility of tissue differentiation using a hand-held ultrasound aspirator while resecting tissue. Moreover, it highlights the capability of hierarchical classification in advancing tissue differentiation accuracy during neurosurgical procedures, which could ultimately aid surgeons and enhance the safety of intraoperative workflows.

Real-time tracking of structural evolution in 2D MXenes using theory-enhanced machine learning

In situ Electron Energy Loss Spectroscopy (EELS) combined with Transmission Electron Microscopy (TEM) has traditionally been pivotal for understanding how material processing choices affect local structure and composition. However, the ability to monitor and respond to ultrafast transient changes, now achievable with EELS and TEM, necessitates innovative analytical frameworks. Here, we introduce a machine learning (ML) framework tailored for the real-time assessment and characterization of in operando EELS Spectrum Images (EELS-SI). We focus on 2D MXenes as the sample material system, specifically targeting the understanding and control of their atomic-scale structural transformations that critically influence their electronic and optical properties. This approach requires fewer labeled training data points than typical deep learning classification methods. By integrating computationally generated structures of MXenes and experimental datasets into a unified latent space using Variational Autoencoders (VAE) in a unique training method, our framework accurately predicts structural evolutions at latencies pertinent to closed-loop processing within the TEM. This study presents a critical advancement in enabling automated, on-the-fly synthesis and characterization, significantly enhancing capabilities for materials discovery and the precision engineering of functional materials at the atomic scale.

Depth estimation of pipe wall thinning using multifrequency reflection coefficients of T(0,1) mode-guided waves with supervised multilayer perceptron

This study entailed the development of a novel method for estimating the depth of wall thinning of pipes using multifrequency (30–65 kHz) reflection coefficients (MRCs) of the T(0,1) mode-guided waves and a multilayer perceptron (MLP). First, this study established why MRCs are a critical feature of the input layer of the MLP for the defect depth estimation of wall thinning. Further, a mathematical model that can quickly collect large amounts of training data was used to calculate the reflection waveforms. The depths of artificial and actual wall thinning were estimated using the MLP based on the MRCs and the mathematical model. Experiments were conducted using the T(0,1) mode-guided waves to obtain the MRCs for 21 artificial and 6 actual wall thinnings to estimate the defect depths. A maximum of 8347 training data points were prepared using the mathematical model. Because the optimization of the MLP strongly depended on the initial weights and biases, 100 random initial values were prepared to evaluate the average estimations and their standard deviations. The classification scheme of the MLP was used, with classification step widths of 0.5 and 0.25 mm. The correct answer rates for the 21 artificial defects were 93%, with a tolerance of ±0.5 mm for the 0.5 mm classification scheme; those for the 0.25 mm classification scheme were 89%. For the six actual defects, the correct answer rates were 100% with a tolerance of ±0.5 mm for both the 0.5 and 0.25 mm classification schemes. Sufficiently high correct answer rates were obtained in all the cases.

Prediksi Kelulusan Mata Kuliah Mahasiswa Teknologi Informasi Menggunakan Algoritma K-Nearest Neighbor

This research aims to develop a predictive model using the K-Nearest Neighbor (KNN) method to forecast the course completion of students in the Information Technology program. The issue at hand is the uncertainty in predicting student success based on historical data and specific attributes. This study focuses on the importance of understanding the factors that influence student success in the Database Management Systems course to provide accurate predictions and help improve student pass rates in this course. The objective of this research is to build a predictive model using the KNN algorithm and to implement this model using the PHP programming language. The study aims to offer valuable insights for educational institutions to enhance teaching and learning processes and to expand understanding of data mining concepts in specific case studies. The prediction aims to determine whether a student will pass or fail the Database Management Systems course based on predetermined training and testing data. By calculating the nearest distance between the training data and the test data. The results showed an accuracy rate of 90% for predicting course completion using k=5, with a dataset consisting of 40 training data points and 20 testing data points.

A Novel Movable Mannequin Platform for Evaluating and Optimising mmWave Radar Sensor for Indoor Crowd Evacuation Monitoring Applications

Developing mmWave radar sensors for indoor crowd motion sensing and tracking faces a critical challenge: the scarcity of large-scale, high-quality training data. Traditional human experiments encounter logistical complexities, ethical considerations, and safety issues. Replicating precise human movements across trials introduces noise and inconsistency into the data. To address this, this study proposes a novel solution: a movable platform equipped with a life-size mannequin to generate realistic and diverse data points for mmWave radar training and testing. Unlike human subjects, the platform allows precise control over movements, optimising sensor placement relative to the target object. Preliminary optimisation results reveal that sensor height impacts tracking performance, with an optimal sensor placement above the test subject yields the best results. The results also reveal that the 3D data format outperforms 2D data in accuracy despite having fewer frames. Additionally, analysing height distribution using 3D data highlights the importance of the sensor angle—15° downwards from the horizontal plane.

A Comparative Study of Students Graduation Analysis Using Classification Methods in Undergraduate Electrical Engineering Tidar University

This research aimed to classify achievement factors for electrical engineering students at Tidar University using K-Means and Agglomerative Clustering classification algorithms. The goal was to understand if any parameters influence high-achieving student performance. The Indonesian government and private sector for university students provide significant education funds. Student scholarships are awarded based primarily on GPA and entry path, overburdening staff and causing confusion during distribution to eligible recipients. A system was needed to accommodate additional eligible criteria. The researcher selected factors to identify engineering student performance, including school origin, entry path, tuition fees, and GPA. These inputs could determine graduation status. The results compared calculation methods based on collected data accuracy, processing times, and characterizing clustered data to determine the best classification method. Agglomerative Hierarchical Clustering performed better. Accuracy testing on 600 training data points yielded 73.94% for improved K-means and 90.42% for AHC. The Average processing time was 674.92 seconds for improved K-means and 554.35 seconds for AHC. Silhouette testing also characterized calculation methods, with improved K-means scoring best at 0.654 and AHC at 0.787 using two clusters.

A Novel Consolidation Analysis Framework: Universal Function Approximators Regularized by Physical Principles

Analytical and numerical techniques are widely used to analyse and interpret soil consolidation problems. An important limitation is the requirement for significant geotechnical knowledge and expertise to derive ‘true’ solutions. This study proposes an alternative: a universal function approximator regularized with known physical principles. The proposed approach here advances previous work to solve one-dimensional (1D) consolidation considering both self-weight and large strains, as well as two-dimensional (2D) consolidation by vertical drains. Initial and boundary conditions of the studied consolidation problems are first strongly and weakly enforced for comparison. A neural network is adopted here as the function approximator. To boost prediction accuracy, a novel strategy is proposed to adaptively sample data points for training. An estimate of epistemic uncertainty is achieved using the confidence interval of ensembled multiple outputs. The results show that the proposed approach accurately predicts the behaviours of complex consolidation processes. Results also indicate that regularization using weak physical constraints can alleviate the imbalance of back-propagated gradients of different loss terms and, in turn, achieve higher accuracy. The proposed method is generic, mesh-free, more robust and can be applied to a wide range of geotechnical problems.

Classifying rock types by geostatistics and random forests in tandem

Rock type classification is crucial for evaluating mineral resources in ore deposits and for rock mechanics. Mineral deposits are formed in a variety of rock bodies and rock types. However, the rock type identification in drill core samples is often complicated by overprinting and weathering processes. An approach to classifying rock types from drill core data relies on whole-rock geochemical assays as features. There are few studies on rock type classification from a limited number of metal grades and dry bulk density as features. The novelty in our approach is the introduction of two sets of feature variables (proxies) at sampled data points, generated by geostatistical leave-one-out cross-validation and by kriging for removing short-scale spatial variation of the measured features. We applied our proposal to a dataset from a porphyry Cu–Au deposit in Mongolia. The model performances on a testing data subset indicate that, when the training dataset is not large, the performance of the classifier (a random forest) substantially improves by incorporating the proxy features as a complement to the original measured features. At each training data point, these proxy features throw light based on the underlying spatial data correlation structure, scales of variations, sampling design, and values of features observed at neighboring points, and show the benefits of combining geostatistics with machine learning.

A Generic Machine Learning Model for Spatial Query Optimization based on Spatial Embeddings

Machine learning (ML) and deep learning (DL) techniques are increasingly applied to produce efficient query optimizers, in particular in regards to big data systems. The optimization of spatial operations is even more challenging due to the inherent complexity of such kind of operations, like spatial join or range query, and the peculiarities of spatial data. Although a few ML-based spatial query optimizers have been proposed in literature, their design limits their use, since each one is tailored for a specific collection of datasets, a specific operation, or a specific hardware setting. Changes to any of these will require building and training a completely new model which entails collecting a new very large training dataset to obtain a good model. This paper proposes a different approach which exploits the use of the novel notion of spatial embedding to overcome these limitations. In particular, a preliminary model is defined which captures the relevant features of spatial datasets, independently from the operation to be optimized and in an unsupervised manner. This model is trained with a large amount of both synthetic and real-world data, with the aim to produce meaningful spatial embeddings. The construction of an embedding model could be intended as a preliminary step for the optimization of many different spatial operations, so the cost of its building can be compensated during the subsequent construction of specific models. Indeed, for each considered spatial operation, a specific tailored model will be trained but by using spatial embeddings as input, so a very little amount of training data points is required for them. Three peculiar operations are considered as proof of concept in this paper: range query, self-join, and binary spatial join. Finally, a comparison with an alternative technique, known as transfer learning, is provided and the advantages of the proposed technique over it are highlighted.

A Python Toolbox for Data-Driven Aerodynamic Modeling Using Sparse Gaussian Processes

In aerodynamics, characterizing the aerodynamic behavior of aircraft typically requires a large number of observation data points. Real experiments can generate thousands of data points with suitable accuracy, but they are time-consuming and resource-intensive. Consequently, conducting real experiments at new input configurations might be impractical. To address this challenge, data-driven surrogate models have emerged as a cost-effective and time-efficient alternative. They provide simplified mathematical representations that approximate the output of interest. Models based on Gaussian Processes (GPs) have gained popularity in aerodynamics due to their ability to provide accurate predictions and quantify uncertainty while maintaining tractable execution times. To handle large datasets, sparse approximations of GPs have been further investigated to reduce the computational complexity of exact inference. In this paper, we revisit and adapt two classic sparse methods for GPs to address the specific requirements frequently encountered in aerodynamic applications. We compare different strategies for choosing the inducing inputs, which significantly impact the complexity reduction. We formally integrate our implementations into the open-source Python toolbox SMT, enabling the use of sparse methods across the GP regression pipeline. We demonstrate the performance of our Sparse GP (SGP) developments in a comprehensive 1D analytic example as well as in a real wind tunnel application with thousands of training data points.

Performance of Fourier-based activation function in physics-informed neural networks for patient-specific cardiovascular flows

Background and ObjectivesPhysics-informed neural networks (PINNs) can be used to inversely model complex physical systems by encoding the governing partial differential equations and training data into the neural network. However, neural networks are known to be biased towards learning less complex functions, called spectral bias. This has important implications in modeling cardiovascular flows, where spatial frequencies can vary substantially across anatomies and pathologies (e.g., aneurysms or stenoses). Recent evidence suggests that Fourier-based activation functions have desirable properties, and can potentially reduce spectral bias; however, the performance and adequacy of such Fourier activation functions have not yet been evaluated in patient-specific cardiovascular flow applications. MethodsThe performance of sine activation function was evaluated against tanh and swish activation functions in a 1D advection-diffusion problem, an eccentric 2D stenosis model (Re=5000), and a patient-specific 3D aortic model (Re=823) under pulsatile flow conditions. CFD simulations were performed at high spatio-temporal resolution and data points were extracted for training the neural network. The number of training data points were normalized by L/D. The performance of the PINNs framework was evaluated with increasing number of training data points and across all three activation functions. ResultsOur results demonstrate that sine activation function presents desirable characteristics, such as monotonic reduction in errors, relatively faster convergence, and accurate eigen spectra at higher modes, compared to tanh and swish activation functions. Interestingly, for all activation functions, the domain-averaged errors tended to asymptote at ≈15−20% despite substantial increase in training point density. For 2D eccentric stenosis, errors asymptoted at a sensor point density of 40L/D. For 3D patient-specific aorta, this asymptote was achieved at 180L/D for all three activation functions with an error of ≈15% although sine activation function demonstrated relatively faster convergence. ConclusionsWe have demonstrated that Fourier-based activation functions have higher performance in terms of accuracy and convergence properties for cardiovascular flow applications; however, inherent challenges of neural networks (e.g., spectral bias) can limit the accuracy to ≈15% under physiological, 3D patient-specific blood flow conditions.

Decentralised, collaborative, and privacy-preserving machine learning for multi-hospital data

BackgroundMachine Learning (ML) has demonstrated its great potential on medical data analysis. Large datasets collected from diverse sources and settings are essential for ML models in healthcare to achieve better accuracy and generalizability. Sharing data across different healthcare institutions or jurisdictions is challenging because of complex and varying privacy and regulatory requirements. Hence, it is hard but crucial to allow multiple parties to collaboratively train an ML model leveraging the private datasets available at each party without the need for direct sharing of those datasets or compromising the privacy of the datasets through collaboration. MethodsIn this paper, we address this challenge by proposing Decentralized, Collaborative, and Privacy-preserving ML for Multi-Hospital Data (DeCaPH). This framework offers the following key benefits: (1) it allows different parties to collaboratively train an ML model without transferring their private datasets (i.e., no data centralization); (2) it safeguards patients’ privacy by limiting the potential privacy leakage arising from any contents shared across the parties during the training process; and (3) it facilitates the ML model training without relying on a centralized party/server. FindingsWe demonstrate the generalizability and power of DeCaPH on three distinct tasks using real-world distributed medical datasets: patient mortality prediction using electronic health records, cell-type classification using single-cell human genomes, and pathology identification using chest radiology images. The ML models trained with DeCaPH framework have less than 3.2% drop in model performance comparing to those trained by the non-privacy-preserving collaborative framework. Meanwhile, the average vulnerability to privacy attacks of the models trained with DeCaPH decreased by up to 16%. In addition, models trained with our DeCaPH framework achieve better performance than those models trained solely with the private datasets from individual parties without collaboration and those trained with the previous privacy-preserving collaborative training framework under the same privacy guarantee by up to 70% and 18.2% respectively. InterpretationWe demonstrate that the ML models trained with DeCaPH framework have an improved utility-privacy trade-off, showing DeCaPH enables the models to have good performance while preserving the privacy of the training data points. In addition, the ML models trained with DeCaPH framework in general outperform those trained solely with the private datasets from individual parties, showing that DeCaPH enhances the model generalizability. FundingThis work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC, RGPIN-2020-06189 and DGECR-2020-00294), Canadian Institute for Advanced Research (CIFAR) AI Catalyst Grants, CIFAR AI Chair programs, Temerty Professor of AI Research and Education in Medicine, University of Toronto, Amazon, Apple, DARPA through the GARD project, Intel, Meta, the Ontario Early Researcher Award, and the Sloan Foundation. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.

Enhancing manufacturing operations with synthetic data: a systematic framework for data generation, accuracy, and utility

Addressing the challenges of data scarcity and privacy, synthetic data generation offers an innovative solution that advances manufacturing assembly operations and data analytics. Serving as a viable alternative, it enables manufacturers to leverage a broader and more diverse range of machine learning models by incorporating the creation of artificial data points for training and evaluation. Current methods lack generalizable framework for researchers to follow and solve these issues. The development of synthetic data sets, however, can make up for missing samples and enable researchers to understand existing issues within the manufacturing process and create data-driven tools for reducing manufacturing costs. This paper systematically reviews both discrete and continuous manufacturing process data types with their applicable synthetic generation techniques. The proposed framework entails four main stages: Data collection, pre-processing, synthetic data generation, and evaluation. To validate the framework’s efficacy, a case study leveraging synthetic data enabled an exploration of complex defect classification challenges in the packaging process. The results show enhanced prediction accuracy and provide a detailed comparative analysis of various synthetic data strategies. This paper concludes by highlighting our framework’s transformative potential for researchers, educators, and practitioners and provides scalable guidance to solve the data challenges in the current manufacturing sector.

Controlling chaotic maps using next-generation reservoir computing.

In this work, we combine nonlinear system control techniques with next-generation reservoir computing, a best-in-class machine learning approach for predicting the behavior of dynamical systems. We demonstrate the performance of the controller in a series of control tasks for the chaotic Hénon map, including controlling the system between unstable fixed points, stabilizing the system to higher order periodic orbits, and to an arbitrary desired state. We show that our controller succeeds in these tasks, requires only ten data points for training, can control the system to a desired trajectory in a single iteration, and is robust to noise and modeling error.

Damage Location Determination with Data Augmentation of Guided Ultrasonic Wave Features and Explainable Neural Network Approach for Integrated Sensor Systems

Machine learning techniques such as deep learning have already been successfully applied in Structural Health Monitoring (SHM) for damage localization using Ultrasonic Guided Waves (UGW) at various temperatures. However, a common issue arises due to the time-consuming nature of collecting guided wave measurements at different temperatures, resulting in an insufficient amount of training data. Since SHM systems are predominantly employed in sensitive structures, there is a significant interest in utilizing methods and algorithms that are transparent and comprehensible. In this study, a method is presented to augment feature data by generating a large number of training features from a relatively limited set of measurements. In addition, robustness to environmental changes, e.g., temperature fluctuations, is improved. This is achieved by utilizing a known temperature compensation method called temperature scaling to determine the function of signal features as a function of temperature. These functions can then be used for data generation. To gain a better understanding of how the damage localization predictions are made, a known explainable neural network (XANN) architecture is employed and trained with the generated data. The trained XANN model was then used to examine and validate the artificially generated signal features and to improve the augmentation process. The presented method demonstrates a significant increase in the number of training data points. Furthermore, the use of the XANN architecture as a predictor model enables a deeper interpretation of the prediction methods employed by the network.

Prediction of CHF location through applied machine learning

The prediction of critical heat flux (CHF) location is a challenging issue in power generation industries. Several factors viz. geometric parameters and operating conditions influence the occurrence of CHF, and many mechanisms have been proposed to better explain the physics behind the occurrence of CHF. As CHF is a complex phenomenon, its dependency on these factors exhibit non-linear relationship. Further, it is challenging to account for all the factors simultaneously in a single model. Moreover, the benchmark experimental data has to be available to validate the constructed model with all the factors viz., operating pressure and power, inlet mass flux and subcooling etc. Due to these reasons, the influence of these factors remain unresolved. Considering the above issues, the present study employs different machine learning models to predict CHF location and identify the accurate model based on the comparisons against the chosen benchmark experimental data. Becker et al. (1983) data bank was selected for machine learning model training as well as the testing due to the wide range of operating conditions covered. Hyperparameter tuning is applied to optimize the performance of the models. Among all the implemented models, the artificial neural network (ANN) model is found to perform well with the training and the test datasets. The mean absolute percentage error (MAPE) of the training and testing data points are 4.01% and 6.04%, respectively. The accuracy score of the ANN model is approximately 96% for the training and 94% for the testing datasets. The proposed model is helpful for design engineers and analysts of power plants for design optimization and to reduce the computational time.

The impact of transitive annotation on the training of taxonomic classifiers.

A common task in the analysis of microbial communities involves assigning taxonomic labels to the sequences derived from organisms found in the communities. Frequently, such labels are assigned using machine learning algorithms that are trained to recognize individual taxonomic groups based on training data sets that comprise sequences with known taxonomic labels. Ideally, the training data should rely on labels that are experimentally verified-formal taxonomic labels require knowledge of physical and biochemical properties of organisms that cannot be directly inferred from sequence alone. However, the labels associated with sequences in biological databases are most commonly computational predictions which themselves may rely on computationally-generated data-a process commonly referred to as "transitive annotation." In this manuscript we explore the implications of training a machine learning classifier (the Ribosomal Database Project's Bayesian classifier in our case) on data that itself has been computationally generated. We generate new training examples based on 16S rRNA data from a metagenomic experiment, and evaluate the extent to which the taxonomic labels predicted by the classifier change after re-training. We demonstrate that even a few computationally-generated training data points can significantly skew the output of the classifier to the point where entire regions of the taxonomic space can be disturbed. We conclude with a discussion of key factors that affect the resilience of classifiers to transitively-annotated training data, and propose best practices to avoid the artifacts described in our paper.

Machine learning of kinetic energy densities with target and feature smoothing: Better results with fewer training data.

Machine learning (ML) of kinetic energy functionals (KEFs), in particular kinetic energy density (KED) functionals, is a promising way to construct KEFs for orbital-free density functional theory (DFT). Neural networks and kernel methods including Gaussian process regression (GPR) have been used to learn Kohn-Sham (KS) KED from density-based descriptors derived from KS DFT calculations. The descriptors are typically expressed as functions of different powers and derivatives of the electron density. This can generate large and extremely unevenly distributed datasets, which complicates effective application of ML techniques. Very uneven data distributions require many training datapoints, can cause overfitting, and can ultimately lower the quality of an ML KED model. We show that one can produce more accurate ML models from fewer data by working with smoothed density-dependent variables and KED. Smoothing palliates the issue of very uneven data distributions and associated difficulties of sampling while retaining enough spatial structure necessary for working within the paradigm of KEDF. We use GPR as a function of smoothed terms of the fourth order gradient expansion and KS effective potential and obtain accurate and stable (with respect to different random choices of training points) kinetic energy models for Al, Mg, and Si simultaneously from as few as 2000 samples (about 0.3% of the total KS DFT data). In particular, accuracies on the order of 1% in a measure of the quality of energy-volume dependence B'=EV0-ΔV-2EV0+E(V0+ΔV)ΔV/V02 (where V0 is the equilibrium volume and ΔV is a deviation from it) are obtained simultaneously for all three materials.

A Proposed Methodology to Evaluate Machine Learning Models at Near-Upper-Bound Predictive Performance—Some Practical Cases from the Steel Industry

The present work aims to answer three essential research questions (RQs) that have previously not been explicitly dealt with in the field of applied machine learning (ML) in steel process engineering. RQ1: How many training data points are needed to create a model with near-upper-bound predictive performance on test data? RQ2: What is the near-upper-bound predictive performance on test data? RQ3: For how long can a model be used before its predictive performance starts to decrease? A methodology to answer these RQs is proposed. The methodology uses a developed sampling algorithm that samples numerous unique training and test datasets. Each sample was used to create one ML model. The predictive performance of the resulting ML models was analyzed using common statistical tools. The proposed methodology was applied to four disparate datasets from the steel industry in order to externally validate the experimental results. It was shown that the proposed methodology can be used to answer each of the three RQs. Furthermore, a few findings that contradict established ML knowledge were also found during the application of the proposed methodology.