Accuracy Of Machine Learning Models Research Articles

Optimizing data acquisition: a Bayesian approach for efficient machine learning model training

Acquiring a substantial number of data points for training accurate machine learning (ML) models is a big challenge in scientific fields where data collection is resource-intensive. Here, we propose a novel approach for constructing a minimal yet highly informative database for training ML models in complex multi-dimensional parameter spaces. To achieve this, we mimic the underlying relation between the output and input parameters using Gaussian process regression (GPR). Using a set of known data, GPR provides predictive means and standard deviation for the unknown data. Given the predicted standard deviation by GPR, we select data points using Bayesian optimization to obtain an efficient database for training ML models. We compare the performance of ML models trained on databases obtained through this method, with databases obtained using traditional approaches. Our results demonstrate that the ML models trained on the database obtained using Bayesian optimization approach consistently outperform the other two databases, achieving high accuracy with a significantly smaller number of data points. Our work contributes to the resource-efficient collection of data in high-dimensional complex parameter spaces, to achieve high precision ML predictions.

TF LITE BASED BLOOD CANCER CLASSIFICATION SYSTEM

Blood cancer, comprising various hematologic malignancies such as leukemia, lymphoma, and myeloma, presents significant diagnostic challenges that require precise and timely identification for effective treatment. The TF-Based Blood Cancer Classification System leverages the capabilities of TensorFlow (TF) to develop a robust, scalable, and accurate machine learning model for classifying different types of blood cancer from medical imaging and genetic data. This system integrates advanced deep learning techniques, including convolutional neural networks (CNNs) for image analysis and recurrent neural networks (RNNs) for genetic sequence analysis, to process and interpret complex biomedical data. The proposed system employs a comprehensive dataset containing labeled images and genetic profiles of various blood cancer types. Data preprocessing steps, such as normalization and augmentation, are applied to enhance model performance and generalization. The classification model is trained and validated using TensorFlow’s high- performance computing capabilities, achieving high accuracy and reliability in distinguishing between different blood cancer subtypes. Initial results demonstrate the system's potential to significantly improve diagnostic accuracy and speed, outperforming traditional diagnostic methods. This advancement not only aids healthcare professionals in making informed decisions but also contributes to personalized treatment strategies. The TF-Based Blood Cancer Classification System represents a significant step towards integrating artificial intelligence in medical diagnostics, offering promising prospects for improving patient outcomes in hematologic oncology. Keywords: Blood cancer, TensorFlow, machine learning, deep learning, convolutional neural networks, recurrent neural networks, medical imaging, genetic data, diagnostics, hematologic malignancies.

Application of Transcriptome-Based Gene Set Featurization for Machine Learning Model to Predict the Origin of Metastatic Cancer

Identifying the primary site of origin of metastatic cancer is vital for guiding treatment decisions, especially for patients with cancer of unknown primary (CUP). Despite advanced diagnostic techniques, CUP remains difficult to pinpoint and is responsible for a considerable number of cancer-related fatalities. Understanding its origin is crucial for effective management and potentially improving patient outcomes. This study introduces a machine learning framework, ONCOfind-AI, that leverages transcriptome-based gene set features to enhance the accuracy of predicting the origin of metastatic cancers. We demonstrate its potential to facilitate the integration of RNA sequencing and microarray data by using gene set scores for characterization of transcriptome profiles generated from different platforms. Integrating data from different platforms resulted in improved accuracy of machine learning models for predicting cancer origins. We validated our method using external data from clinical samples collected through the Kangbuk Samsung Medical Center and Gene Expression Omnibus. The external validation results demonstrate a top-1 accuracy ranging from 0.80 to 0.86, with a top-2 accuracy of 0.90. This study highlights that incorporating biological knowledge through curated gene sets can help to merge gene expression data from different platforms, thereby enhancing the compatibility needed to develop more effective machine learning prediction models.

Basic research for identification and classification of organophosphorus pesticides in water based on ultraviolet-visible spectroscopy information.

In this study, the goal was to develop a method for detecting and classifying organophosphorus pesticides (OPPs) in bodies of water. Sixty-five samples with different concentrations were prepared for each of the organophosphorus pesticides, namely chlorpyrifos, acephate, parathion-methyl, trichlorphon, dichlorvos, profenofos, malathion, dimethoate, fenthion, and phoxim, respectively. Firstly, the spectral data of all the samples was obtained using a UV-visible spectrometer. Secondly, five preprocessing methods, six manifold learning methods, and five machine learning algorithms were utilized to build detection models for identifying OPPs in water bodies. The findings indicate that the accuracy of machine learning models trained on data preprocessed using convolutional smoothing + first-order derivatives (SG + FD) outperforms that of models trained on data preprocessed using other methods. The backpropagation neural network (BPNN) model exhibited the highest accuracy rate at 99.95%, followed by the support vector machine (SVM) and convolutional neural network (CNN) models, both at 99.92%. The extreme learning machine (ELM) and K-nearest neighbors (KNN) models demonstrated accuracy rates of 99.84% and 99.81%, respectively. Following the application of a manifold learning algorithm to the full-wavelength data set for the purpose of dimensionality reduction, the data was then visualized in the first three dimensions. The results demonstrate that the t-distributed domain embedding (t-SNE) algorithm is superior, exhibiting dense clustering of similar clusters and clear classification of dissimilar ones. SG + FD-t-SNE-SVM ranks highest among the feature extraction models in terms of performance. The feature extraction dimension was set to 4, and the average classification accuracy was 99.98%, which slightly improved the prediction performance over the full-wavelength model. As shown in this study, the ultraviolet-visible (UV-visible) spectroscopy system combined with the t-SNE and SVM algorithms can effectively identify and classify OPPs in waterbodies.

Basic research for identification and classification of organophosphorus pesticides in water based on ultraviolet-visible spectroscopy information.

In this study, the goal was to develop a method for detecting and classifying organophosphorus pesticides (OPPs) in bodies of water. Sixty-five samples with different concentrations were prepared for each of the organophosphorus pesticides, namely chlorpyrifos, acephate, parathion-methyl, trichlorphon, dichlorvos, profenofos, malathion, dimethoate, fenthion, and phoxim, respectively. Firstly, the spectral data of all the samples was obtained using a UV-visible spectrometer. Secondly, five preprocessing methods, six manifold learning methods, and five machine learning algorithms were utilized to build detection models for identifying OPPs in water bodies. The findings indicate that the accuracy of machine learning models trained on data preprocessed using convolutional smoothing + first-order derivatives (SG + FD) outperforms that of models trained on data preprocessed using other methods. The backpropagation neural network (BPNN) model exhibited the highest accuracy rate at 99.95%, followed by the support vector machine (SVM) and convolutional neural network (CNN) models, both at 99.92%. The extreme learning machine (ELM) and K-nearest neighbors (KNN) models demonstrated accuracy rates of 99.84% and 99.81%, respectively. Following the application of a manifold learning algorithm to the full-wavelength data set for the purpose of dimensionality reduction, the data was then visualized in the first three dimensions. The results demonstrate that the t-distributed domain embedding (t-SNE) algorithm is superior, exhibiting dense clustering of similar clusters and clear classification of dissimilar ones. SG + FD-t-SNE-SVM ranks highest among the feature extraction models in terms of performance. The feature extraction dimension was set to 4, and the average classification accuracy was 99.98%, which slightly improved the prediction performance over the full-wavelength model. As shown in this study, the ultraviolet-visible (UV-visible) spectroscopy system combined with the t-SNE and SVM algorithms can effectively identify and classify OPPs in waterbodies.

Optimizing data acquisition: a Bayesian approach for efficient machine learning model training

Abstract Acquiring a substantial number of data points for training accurate machine learning (ML) models is a big challenge in scientific fields where data collection is resource-intensive. Here, we propose a novel approach for constructing a minimal yet highly informative database for training ML models in complex multi-dimensional parameter spaces. To achieve this, we mimic the underlying relation between the output and input parameters using Gaussian process regression (GPR). Using a set of known data, GPR provides predictive means and standard deviation for the unknown data. Given the predicted standard deviation by GPR, we select data points using Bayesian optimization to obtain an efficient database for training ML models. We compare the performance of ML models trained on databases obtained through this method, with databases obtained using traditional approaches. Our results demonstrate that the ML models trained on the database obtained using Bayesian optimization approach consistently outperform the other two databases, achieving high accuracy with a significantly smaller number of data points. Our work contributes to the resource-efficient collection of data in high-dimensional complex parameter spaces, to achieve high precision machine learning predictions.

An efficient method for disaster tweets classification using gradient-based optimized convolutional neural networks with BERT embeddings

Event of the disastrous scenarios are actively discussed on microblogging platforms like Twitter which can lead to chaotic situations. In the era of machine learning and deep learning, these chaotic situations can be effectively controlled by developing efficient methods and models that can assist in classifying real and fake tweets. In this research article, an efficient method named BERT Embedding based CNN model with RMSProp Optimizer is proposed to effectively classify the tweets related disastrous scenario. Tweet classification is carried out via some of the popular the machine learning algorithms such as logistic regression and decision tree classifiers. Noting the low accuracy of machine learning models, Convolutional Neural Network (CNN) based deep learning model is selected as the primary classification method. CNNs performance is improved via optimization of the parameters with gradient based optimizers. To further elevate accuracy and to capture contextual semantics from the text data, BERT embeddings are included in the proposed model. The performance of proposed method - BERT Embedding based CNN model with RMSProp Optimizer achieved an F1 score of 0.80 and an Accuracy of 0.83. The methodology presented in this research article is comprised of the following key contributions:•Identification of suitable text classification model that can effectively capture complex patterns when dealing with large vocabularies or nuanced language structures in disaster management scenarios.•The method explores the gradient based optimization techniques such as Adam Optimizer, Stochastic Gradient Descent (SGD) Optimizer, AdaGrad, and RMSprop Optimizer to identify the most appropriate optimizer that meets the characteristics of the dataset and the CNN model architecture.•“BERT Embedding based CNN model with RMSProp Optimizer” – a method to classify the disaster tweets and capture semantic representations by leveraging BERT embeddings with appropriate feature selection is presented and models are validated with appropriate comparative analysis.

Machine learning predicts peak oxygen uptake and peak power output for customizing cardiopulmonary exercise testing using non-exercise features.

Cardiopulmonary exercise testing (CPET) is considered the gold standard for assessing cardiorespiratory fitness. To ensure consistent performance of each test, it is necessary to adapt the power increase of the test protocol to the physical characteristics of each individual. This study aimed to use machine learning models to determine individualized ramp protocols based on non-exercise features. We hypothesized that machine learning models will predict peak oxygen uptake ( O2peak) and peak power output (PPO) more accurately than conventional multiple linear regression (MLR). The cross-sectional study was conducted with 274 (♀168, ♂106) participants who performed CPET on a cycle ergometer. Machine learning models and multiple linear regression were used to predict O2peak and PPO using non-exercise features. The accuracy of the models was compared using criteria such as root mean square error (RMSE). Shapley additive explanation (SHAP) was applied to determine the feature importance. The most accurate machine learning model was the random forest (RMSE: 6.52ml/kg/min [95% CI 5.21-8.17]) for O2peak prediction and the gradient boosting regression (RMSE: 43watts [95% CI 35-52]) for PPO prediction. Compared to the MLR, the machine learning models reduced the RMSE by up to 28% and 22% for prediction of O2peak and PPO, respectively. Furthermore, SHAP ranked body composition data such as skeletal muscle mass and extracellular water as the most impactful features. Machine learning models predict O2peak and PPO more accurately than MLR and can be used to individualize CPET protocols. Features that provide information about the participant's body composition contribute most to the improvement of these predictions. DRKS00031401 (6 March 2023, retrospectively registered).

Evaluating external generalizability of machine learning models for recycled aggregate concrete property prediction

Machine learning (ML) models have gained importance in predicting recycled aggregate concrete (RAC) properties, offering supposed benefits over conventional empirical and statistical techniques. However, whether ML models are externally generalizable for predicting RAC properties remains unanswered. This study addresses this gap by developing a systematic experimental framework for evaluating the external generalizability of ML models for predicting the compressive strength of RAC. Using a literature review, the authors created a primary dataset of 414 data points and sourced a secondary dataset comprising 330 data points from a previous paper. In Phase 1, prominent ML models like Random Forest (RF) and Extreme Gradient Boost (XGB) were tested for high-accuracy prediction on both primary and secondary datasets. A coefficient of determination (R2) as high as 0.76 for the primary dataset and 0.82 for the secondary dataset for testing sets was obtained for XGB. However, when the best-performing models of phase 1 were trained and tested with data sourced from different datasets in varying combinations, the ML model's performance significantly deteriorated (R2 < 0.25), demonstrating that ML models are not externally generalizable. The study's results highlight the trade-off between the ML model's prediction accuracy within the given dataset and its external generalizability. The study reveals that complex ML models, like XGB, may over-fit specific data, reducing their generalizability. These findings call for the intelligent usage of ML tools to identify nuanced hypotheses and promote rigorous science rather than unilaterally working on the accuracy of ML models. The study emphasizes the consideration of ML models' generalizability in the future.

Towards Enhanced Autonomous Driving Takeovers: Fuzzy Logic Perspective for Predicting Situational Awareness

This paper investigates the application of fuzzy logic to enhance situational awareness in Advanced Driver Assistance Systems (ADAS). Situational awareness is critical for drivers to respond appropriately to dynamic driving scenarios. As car automation increases, monitoring situational awareness ensures that drivers can effectively take control of the vehicle when needed. Our study explores whether fuzzy logic can accurately assess situational awareness using a set of 14 critical predictors categorized into time decision, criticality, eye-related metrics, and driver experience. We based our work on prior research that used machine learning (ML) models to achieve high accuracy. Our proposed fuzzy logic system aims to match the predictive accuracy of ML models while providing additional benefits in terms of interpretability and robustness. This approach emphasizes a fresh perspective on situational awareness within ADAS, potentially improving safety and efficiency in real-world driving scenarios.

Prediction and reliability analysis of shear strength of RC deep beams

This study explores machine learning (ML) capabilities for predicting the shear strength of reinforced concrete deep beams (RCDBs). For this purpose, eight typical machine-learning models, i.e., symbolic regression (SR), XGBoost (XGB), CatBoost (CATB), random forest (RF), LightGBM, support vector regression (SVR), artificial neural networks (ANN), and Gaussian process regression (GPR) models, are selected and compared based on a database of 840 samples with 14 input features. The hyperparameter tuning of the introduced ML models is performed using the Bayesian optimization (BO) technique. The comparison results show that the CatBoost model is the most reliable and accurate ML model (R2 = 0.997 and 0.947 in the training and testing sets, respectively). In addition, simple and practical design expressions for RCDBs have been proposed based on the SR model with a physical meaning and acceptable accuracy (an average prediction-to-test ratio of 0.935 and a standard deviation of 0.198). Meanwhile, the shear strength predicted by ML models was then compared with classical mechanics-driven shear models, including two prominent practice codes (i.e., ACI318, EC2) and two previous mechanical models, which indicated that the ML approach is highly reliable and accurate over conventional methods. In addition, a reliability-based design was conducted on two ML models, and their reliability results were compared with those of two code standards. The findings revealed that the ML models demonstrate higher reliability compared to code standards.

Optimization for Best Feature Selection on Microarray Gene Expression Data

Feature selection serves as a crucial technique in data analysis, eliminating unnecessary elements from a dataset to enhance computational efficiency and improve the accuracy of machine learning models. This study introduces a novel method called Rotate Left and Complement (RLC) for feature selection, employing T statistics to identify informative genes. The RLC algorithm, based on the top m informative genes, presents a promising solution to refine feature sets. The accuracy of categorization is evaluated using the KNN method across three diverse datasets, demonstrating the effectiveness of the proposed approach in optimizing feature selection and contributing to the advancement of machine learning methodologies.

Prediction of magnetic nature of oxide compositions by using machine learning models

The magnetic nature of oxide compositions carries most significance in the applications of science and technology, promoting the development of electronic devices, sensors, and data storage technologies. However, the field encounters a remarkable research gap regarding the use of machine learning techniques for accurately predicting the magnetic behavior of compositions. This study tries to fill the gap by looking into the efficacy of machine learning models for predicting the magnetic nature of oxide compositions, which has been relatively not explored compared to traditional analytical methods. The study seeks to identify the most accurate and reliable machine learning model for this prediction task by evaluating five machine learning models, including Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), and Artificial Neural Network (ANN). According to this study, the RF outperforms other models on three distinct evaluation metrics, having accuracy 89.53%, precision 92.354%, F1 score 91.343%, while ANN outperforms on the Recall metric having a score 91.43%. This paper closes a significant research gap and improves the applications of materials science and industry by presenting a successful machine learning method that uses Random Forest to predict the magnetic properties of oxide compositions.

Sensitivity and feature importance of climate factors and evaluation of different machine learning models for predicting fire hotspots in Kalimantan, Indonesia

&lt;p&gt;Hotspots as indicators of forest fires capable of quickly monitoring large areas are often predicted using various machine learning methods. However, there is still little research that analyzes the sensitivity and feature importance of each predictor that forms a machine learning prediction model. This study evaluates and compares several machine learning methods to predict hotspots in Kalimantan. Using the most accurate machine learning model, each climate factor used as a predictor is analyzed for its sensitivity and feature importance. Some of the machine learning methods used include random forest, gradient boosting, Bayesian regression, and artificial neural networks. Meanwhile, several measures of sensitivity and feature importance used are variance-based, density-based, and distribution-based sensitivity indices, as well as permutation and Shapley feature importance. Evaluation of the ML model concluded that the Bayesian linear regression model outperformed other ML models, based on RMSE and explained variance score. Meanwhile, tree-based models, such as random forest and gradient boosting, are indicative of overfit. Based on the results of sensitivity analysis and feature importance, the number of dry days is the most important feature for the Bayesian linear regression model in predicting the number of hotspots in Kalimantan.&lt;/p&gt;

Advancing Objective Mobile Device Use Measurement in Children Ages 6–11 Through Built-In Device Sensors: A Proof-of-Concept Study

Mobile devices (e.g., tablets and smartphones) have been rapidly integrated into the lives of children and have impacted how children engage with digital media. The portability of these devices allows for sporadic, on-demand interaction, reducing the accuracy of self-report estimates of mobile device use. Passive sensing applications objectively monitor time spent on a given device but are unable to identify who is using the device, a significant limitation in child screen time research. Behavioral biometric authentication, using embedded mobile device sensors to continuously authenticate users, could be applied to address this limitation. This study examined the preliminary accuracy of machine learning models trained on iPad sensor data to identify the unique user of the device in a sample of children ages 6 to 11. Data was collected opportunistically from nine participants (8.2 ± 1.75 years, 5 female) in the sedentary portion of two semistructured physical activity protocols. SensorLog was downloaded onto study iPads and collected data from the accelerometer, gyroscope, and magnetometer sensors while the participant interacted with the iPad. Five machine learning models, logistic regression (LR), support vector machine, neural net (NN), k-nearest neighbors (k-NN), and random forest (RF), were trained using 57 features generated from the sensor output to perform multiclass classification. A train-test split of 80%–20% was used for model fitting. Model performance was evaluated using F1 score, accuracy, precision, and recall. Model performance was high, with F1 scores ranging from 0.75 to 0.94. RF and k-NN had the highest performance across metrics, with F1 scores of 0.94 for both models. This study highlights the potential of using existing mobile device sensors to continuously identify the user of a device in the context of screen time measurement. Future research should explore the performance of this technology in larger samples of children and in free-living environments.

Machine learning algorithm predicts urethral stricture following transurethral prostate resection

PurposeTo predict the post transurethral prostate resection(TURP) urethral stricture probability by applying different machine learning algorithms using the data obtained from preoperative blood parameters.MethodsA retrospective analysis of data from patients who underwent bipolar-TURP encompassing patient characteristics, preoperative routine blood test outcomes, and post-surgery uroflowmetry were used to develop and educate machine learning models. Various metrics, such as F1 score, model accuracy, negative predictive value, positive predictive value, sensitivity, specificity, Youden Index, ROC AUC value, and confidence interval for each model, were used to assess the predictive performance of machine learning models for urethral stricture development.ResultsA total of 109 patients’ data (55 patients without urethral stricture and 54 patients with urethral stricture) were included in the study after implementing strict inclusion and exclusion criteria. The preoperative Platelet Distribution Width, Mean Platelet Volume, Plateletcrit, Activated Partial Thromboplastin Time, and Prothrombin Time values were statistically meaningful between the two cohorts. After applying the data to the machine learning systems, the accuracy prediction scores for the diverse algorithms were as follows: decision trees (0.82), logistic regression (0.82), random forests (0.91), support vector machines (0.86), K-nearest neighbors (0.82), and naïve Bayes (0.77).ConclusionOur machine learning models’ accuracy in predicting the post-TURP urethral stricture probability has demonstrated significant success. Exploring prospective studies that integrate supplementary variables has the potential to enhance the precision and accuracy of machine learning models, consequently progressing their ability to predict post-TURP urethral stricture risk.

Concrete Breaking Strength Prediction Using Machine Learning

Concrete Breaking Strength Prediction Using Machine Learning

Federated learning for predicting compound mechanism of action based on image-data from cell painting

Having access to sufficient data is essential in order to train accurate machine learning models, but much data is not publicly available. In drug discovery this is particularly evident, as much data is withheld at pharmaceutical companies for various reasons. Federated Learning (FL) aims at training a joint model between multiple parties but without disclosing data between the parties. In this work, we leverage Federated Learning to predict compound Mechanism of Action (MoA) using fluorescence image data from cell painting. Our study evaluates the effectiveness and efficiency of FL, comparing to non-collaborative and data-sharing collaborative learning in diverse scenarios. Specifically, we investigate the impact of data heterogeneity across participants on MoA prediction, an essential concern in real-life applications of FL, and demonstrate the benefits for all involved parties. This work highlights the potential of federated learning in multi-institutional collaborative machine learning for drug discovery and assessment of chemicals, offering a promising avenue to overcome data-sharing constraints.

SPLINE OPTIMIZATION OF SOFT CONNECTIVES IN MACHINE LEARNING MODELS

In this study, the problem of limited accuracy of machine learning models using soft logical connectives is investigated. Such connectives have shown their effectiveness in models with fuzzy initial data. On the one hand, the fundamental disadvantage of soft connectives is their non-associativity. On the other hand, the disadvantages of the currently used soft connectives include the loss of monotonicity and the inability to control several factors simultaneously. All these problems have been solved by the authors. We have proposed an approximation of the signum function by a ?1-smooth spline. The first part of the spline is responsible for the curvature of the connective at the diagonal and is adjustable. The second part of the spline is the solution to the optimization problem. We minimized the difference between the connective and the associative connective, the latter in our study was the minimum function. In the resulting solution, the rate of deviation reduction is the highest among known connectives. We have achieved not only a small deviation from associativity, but also the presence of a large domain of exact associativity. This area is up to a third of the volume of all triples of arguments. A comparative analysis of the currently used soft connectives with the constructed model was carried out. It was shown that the spline approximation is able to reduce the influence of all negative factors and is more flexible in setting. Moreover, the constructed spline model allows numerous modifications depending on the factor that requires the most attention for different tasks.

An explainable integrated machine learning model for mapping soil erosion by wind and water in a catchment with three desiccated lakes

Soil erosion by water and wind is a critical challenge for sustainable management of catchments in drylands and accurate spatial information can help in mitigation of its destructive consequences. Here, seven machine learning (ML) models were applied to map simultaneously the water and wind erosions in the Bakhtegan catchment, south Iran, with three dried lakes in its southern part and three dams established in upstream parts of the lakes. The analysis identified 10 and 11 effective variables controlling water and wind erosions, among 20 and 17 potential variables, respectively, via the MARS feature selection algorithm. According to the most accurate ML models (artificial neural network for water erosion, and Cubist for wind erosion), an integrated model was developed to map soil erosion by water and wind simultaneously. Permutation feature importance (PFI) and Shapley additive exPlanation (SHAP) interpretation techniques were employed to explain the model outputs, revealing that 19.7 % of the total area belonged to high and very high susceptibility classes to soil erosion by water and wind. The PFI plot revealed that the slope and wind speed were the most influencing factors for water and wind erosion, respectively. According to SHAP decision plot, slope had the highest contribution on the predictive water erosion model’s output, whereas vegetation cover exhibited the highest contribution on the predictive wind erosion model’s output. Due to climate change and intensified drought during the recent years, as well as due to construction of dams upstream of the lakes, the southern part of the study area was converted to a source of sand and dust storms. The hotspots with severe water erosion are distributed all over the study area, rendering it quite vulnerable to adverse climate change projections.