Accuracy Of Machine Learning Models Research Articles

Does noise affect housing prices? A case study in the urban area of Thessaloniki

Real estate markets depend on various methods to predict housing prices, including models that have been trained on datasets of residential or commercial properties. Most studies endeavor to create more accurate machine learning models by utilizing data such as basic property characteristics as well as urban features like distances from amenities and road accessibility. Even though environmental factors like noise pollution can potentially affect prices, the research around this topic is limited. One of the reasons is the lack of data. In this paper, we reconstruct and make publicly available a general purpose noise pollution dataset based on published studies conducted by the Hellenic Ministry of Environment and Energy for the city of Thessaloniki, Greece. Then, we train ensemble machine learning models, like XGBoost, on property data for different areas of Thessaloniki to investigate the way noise influences prices through interpretability evaluation techniques. Our study provides a new noise pollution dataset that not only demonstrates the impact noise has on housing prices, but also indicates that the influence of noise on prices significantly varies among different areas of the same city.

The Diagnosis of Malignant Pleural Effusion Using Tumor-Marker Combinations: A Cost-Effectiveness Analysis Based on a Stacking Model.

By incorporating the cost of multiple tumor-marker tests, this work aims to comprehensively evaluate the financial burden of patients and the accuracy of machine learning models in diagnosing malignant pleural effusion (MPE) using tumor-marker combinations. Carcinoembryonic antigen (CEA), carbohydrate antigen (CA)19-9, CA125, and CA15-3 were collected from pleural effusion (PE) and peripheral blood (PB) of 319 patients with pleural effusion. A stacked ensemble (stacking) model based on five machine learning models was utilized to evaluate the diagnostic accuracy of tumor markers. We evaluated the discriminatory accuracy of various tumor-marker combinations using the area under the curve (AUC), sensitivity, and specificity. To evaluate the cost-effectiveness of different tumor-marker combinations, a comprehensive score (C-score) with a tuning parameter w was proposed. In most scenarios, the stacking model outperformed the five individual machine learning models in terms of AUC. Among the eight tumor markers, the CEA in PE (PE.CEA) showed the best AUC of 0.902. Among all tumor-marker combinations, the PE.CA19-9 + PE.CA15-3 + PE.CEA + PB.CEA combination (C9 combination) achieved the highest AUC of 0.946. When w puts more weight on the cost, the highest C-score was achieved with the single PE.CEA marker. As w puts over 0.8 weight on AUC, the C-score favored diagnostic models with more expensive tumor-marker combinations. Specifically, when w was set to 0.99, the C9 combination achieved the best C-score. The stacking diagnostic model using PE.CEA is a relatively accurate and affordable choice in diagnosing MPE for patients without medical insurance or in a low economic level. The stacking model using the combination PE.CA19-9 + PE.CA15-3 + PE.CEA + PB.CEA is the most accurate diagnostic model and the best choice for patients without an economic burden. From a cost-effectiveness perspective, the stacking diagnostic model with PE.CA19-9 + PE.CA15-3 + PE.CEA combination is particularly recommended, as it gains the best trade-off between the low cost and high effectiveness.

SAT587 Methylation-based Machine Learning Classifiers Discriminate Sellar Tumors By Lobe Origin Using Liquid Biopsy Or Surgical Specimens

Abstract Disclosure: G.A. Herrgott: None. K.P. Asmaro: None. M. Wells: None. K. Nelson: None. B. Thomas: None. L.A. Hasselbach: None. A. Transou: None. S. Cazacu: None. K.M. Tundo: None. S. Nadimidla: None. L. Scarpace: None. J. Barnholtz-Sloan: None. A.E. Sloan: None. W.R. Selman: None. A.C. deCarvalho: None. A. Mukherjee: None. A.M. Robin: None. I.Y. Lee: None. J. Craig: None. S. Kalkanis: None. J. Snyder: None. T. Walbert: None. J. Rock: None. H. Noushmehr: None. A.B. Castro: None. Background: The differential diagnosis of challenging sellar tumor cases can be inconclusive through imaging features and could benefit from noninvasive diagnostic approaches, such as liquid biopsy (LB). Similar to tissue, LB specimens carry tumor-specific DNA methylation signatures amenable to the construction of accurate machine learning models able to discriminate CNS tumors. We aimed to develop methylation-based classifiers which classify sellar tumors by lobe of origin, using either LB or tumor tissue specimens. Methodology: We analyzed the DNA methylome (EPIC array) of tumor tissue (T) and LB specimens from adult patients with tumors representing each of the three pituitary lobes (Anterior: T=177; LB=37; Intermediate: T= 7; LB: 10 and Posterior: T=44, LB=2 cases). Using the most variably methylated CpG probes derived from the unsupervised variance-based analyses across tumors from different lobes, we applied multi-class linear discriminant analysis to construct machine learning models to classify sellar tumor tissue and/or LB specimens. Results: We generated classifiers based on lobe-specific methylation signatures that were able to discriminate across sellar tumors either using tissue and/or LB specimens (500 and 600 CpGs, respectively) with observed accuracies of ∼99% across independent validation. DISCUSSION/CONCLUSION: Our findings suggest that methylation-based classifiers constitute an accurate diagnostic approach to discriminate sellar tumors according to the lobe origin, either pre-surgically through a blood draw or through surgical tumor specimens. These classifiers are objective approaches that could complement imaging and pathology reports for an accurate diagnosis of inconclusive cases, ultimately leading to optimal management of the patients with these diseases. Presentation: Saturday, June 17, 2023

Experimental Uncertainty in Training Data for Protein-Ligand Binding Affinity Prediction Models

The accuracy of machine learning models for protein-ligand binding affinity prediction depends on the quality of the experimental data they are trained on. Most of these models are trained and tested on different subsets of the PDBbind database, which is the main source of protein-ligand complexes with annotated binding affinity in the public domain. However, estimating its experimental uncertainty is not straightforward because just a few protein-ligand complexes have more than one measurement associated. In this work, we analyze bioactivity data from ChEMBL to estimate the experimental uncertainty associated with the three binding affinity measures included in the PDBbind (Ki, Kd, and IC50), as well as the effect of combining them. The experimental uncertainty of combining these three affinity measures was characterized by a mean absolute error of 0.78 logarithmic units, a root mean square error of 1.04 and a Pearson correlation coefficient of 0.76. These estimations were contrasted with the performances obtained by state-of-the-art machine learning models for binding affinity prediction, showing that these models tend to be overoptimistic when evaluated on the core set from PDBbind.

Search of inhibitors of aldose reductase for treatment of diabetic cataracts using machine learning

PurposePatients with diabetes mellitus have an elevated chance of developing cataracts, a degenerative vision-impairing condition often needing surgery. The process of the reduction of glucose to sorbitol in the lens of the human eye that causes cataracts is managed by the Aldose Reductase Enzyme (AR), and it is been found that AR inhibitors may mitigate the onset of diabetic cataracts. There exists a large pool of natural and synthetic AR inhibitors that can prevent diabetic complications, and the development of a machine-learning (ML) prediction model may bring new AR inhibitors with better characteristics into clinical use. MethodsUsing known AR inhibitors and their chemical-physical descriptors we created the ML model for prediction of new AR inhibitors. The predicted inhibitors were tested by computational docking to the binding site of AR. ResultsUsing cross-validation in order to find the most accurate ML model, we ended with final cross-validation accuracy of 90%. Computational docking testing of the predicted inhibitors gave a high level of correlation between the ML prediction score and binding free energy. ConclusionsCurrently known AR inhibitors are not used yet for patients for several reasons. We think that new predicted AR inhibitors have the potential to possess more favorable characteristics to be successfully implemented after clinical testing. Exploring new inhibitors can improve patient well-being and lower surgical complications all while decreasing long-term medical expenses.

An effective correlation-based data modeling framework for automatic diabetes prediction using machine and deep learning techniques

The rising risk of diabetes, particularly in emerging countries, highlights the importance of early detection. Manual prediction can be a challenging task, leading to the need for automatic approaches. The major challenge with biomedical datasets is data scarcity. Biomedical data is often difficult to obtain in large quantities, which can limit the ability to train deep learning models effectively. Biomedical data can be noisy and inconsistent, which can make it difficult to train accurate models. To overcome the above-mentioned challenges, this work presents a new framework for data modeling that is based on correlation measures between features and can be used to process data effectively for predicting diabetes. The standard, publicly available Pima Indians Medical Diabetes (PIMA) dataset is utilized to verify the effectiveness of the proposed techniques. Experiments using the PIMA dataset showed that the proposed data modeling method improved the accuracy of machine learning models by an average of 9%, with deep convolutional neural network models achieving an accuracy of 96.13%. Overall, this study demonstrates the effectiveness of the proposed strategy in the early and reliable prediction of diabetes.

Flexible loss functions for binary classification in gradient-boosted decision trees: An application to credit scoring

This paper introduces new flexible loss functions for binary classification in Gradient-Boosted Decision Trees (GBDT) that combine Dice-based and cross-entropy-based losses and offer link functions from either a generalized extreme value (GEV) or exponentiated exponential logistic (EEL) distribution. Testing 27 different GBDT models using XGBoost on a Freddie Mac mortgage loan database showed that the choice of the loss function is useful. Specifically, when the class imbalance ratio (IR) is less than 99, using a skewed GEV distribution-based link function in XGBoost enhances discriminatory power and classification accuracy while retaining a simple model structure, which is particularly important in credit scoring applications. In cases where class imbalances are severe, typically between IRs of 99 and 200, we found that an advanced loss function, which is composed of a symmetric hybrid loss function and a link derived from a positively skewed EEL distribution, outperforms other XGBoost variants. Based on our findings, the accuracy improvements of these proposed extensions result in lower misclassification costs, which are especially evident when IR is below 99, which results in higher profitability for the business. Furthermore, the study highlights the transparency associated with GBDT, which is also an integral component of financial applications. Researchers and practitioners can use these insights to create more accurate and discriminative machine learning models, with possible extensions to other GBDT implementations and machine learning techniques that take into account loss functions. The source code for the proposed approach is publicly available at https://github.com/jm-ml/flexible-losses-for-binary-classification-with-GBDT.

Efficacy of temporal and spatial abstraction for training accurate machine learning models: A case study in smart thermostats

Efficacy of temporal and spatial abstraction for training accurate machine learning models: A case study in smart thermostats

Effect of Feature Selection on the Accuracy of Machine Learning Model

In real life data science problems, it’s almost rare that all the features in the dataset are useful for building a model. In machine learning, feature selection is the process of selecting a subset of relevant features or attributes for constructing a model. Removing irrelevant and redundant features and, selecting relevant features will improve the accuracy of a machine learning model. Furthermore, adding unnecessary variables to a model increases the overall complexity of the model. Our experiment indicates that the accuracy of a classification model is highly affected by the process of feature selection. We train three algorithms (K-Nearest Neighbors, Decision Tree, Multi-layer Perceptron) by selecting all the features and we got accuracies 49%, 84% and 71% accordingly. After doing some feature selection without any logical changes in models code the accuracy scores jumped to 82%, 86% and 78% accordingly which is quite impressive.

A Novel Interpretable Radiomics Model to Distinguish Nodular Goiter From Malignant Thyroid Nodules.

The purpose of this study is to inquire about the potential association between radiomics features and the pathological nature of thyroid nodules (TNs), and to propose an interpretable radiomics-based model for predicting the risk of malignant TN. In this retrospective study, computed tomography (CT) imaging and pathological data from 141 patients with TN were collected. The data were randomly stratified into a training group (n = 112) and a validation group (n = 29) at a ratio of 4:1. A total of 1316 radiomics features were extracted by using the pyradiomics tool. The redundant features were removed through correlation testing, and the least absolute shrinkage and selection operator (LASSO) or the minimum redundancy maximum relevance standard was used to select features. Finally, 4 different machine learning models (RF Hybrid Feature, SVM Hybrid Feature, RF, and LASSO) were constructed. The performance of the 4 models was evaluated using the receiver operating characteristic curve. The calibration curve, decision curve analysis, and SHapley Additive exPlanations method were used to evaluate or explain the best radiomics machine learning model. The optimal radiomics model (RF Hybrid Feature model) demonstrated a relatively high degree of discrimination with an area under the receiver operating characteristic curve (AUC) of 0.87 (95% CI, 0.70-0.97; P < 0.001) for the validation cohort. Compared with the commonly used LASSO model (AUC, 0.78; 95% CI, 0.60-0.91; P < 0.01), there is a significant improvement in AUC in the validation set, net reclassification improvement, 0.79 (95% CI, 0.13-1.46; P < 0.05), and integrated discrimination improvement, 0. 20 (95% CI, 0.10-0.30; P < 0.001). The interpretable radiomics model based on CT performs well in predicting benign and malignant TNs by using quantitative radiomics features of the unilateral total thyroid. In addition, the data preprocessing method incorporating different layers of features has achieved excellent experimental results. As the detection rate of TNs continues to increase, so does the diagnostic burden on radiologists. This study establishes a noninvasive, interpretable and accurate machine learning model to rapidly identify the nature of TN found in CT.

Long-Short-Term-Memory-Based Deep Stacked Sequence-to-Sequence Autoencoder for Health Prediction of Industrial Workers in Closed Environments Based on Wearable Devices.

To reduce the risks and challenges faced by frontline workers in confined workspaces, accurate real-time health monitoring of their vital signs is essential for improving safety and productivity and preventing accidents. Machine-learning-based data-driven methods have shown promise in extracting valuable information from complex monitoring data. However, practical industrial settings still struggle with the data collection difficulties and low prediction accuracy of machine learning models due to the complex work environment. To tackle these challenges, a novel approach called a long short-term memory (LSTM)-based deep stacked sequence-to-sequence autoencoder is proposed for predicting the health status of workers in confined spaces. The first step involves implementing a wireless data acquisition system using edge-cloud platforms. Smart wearable devices are used to collect data from multiple sources, like temperature, heart rate, and pressure. These comprehensive data provide insights into the workers' health status within the closed space of a manufacturing factory. Next, a hybrid model combining deep learning and support vector machine (SVM) is constructed for anomaly detection. The LSTM-based deep stacked sequence-to-sequence autoencoder is specifically designed to learn deep discriminative features from the time-series data by reconstructing the input data and thus generating fused deep features. These features are then fed into a one-class SVM, enabling accurate recognition of workers' health status. The effectiveness and superiority of the proposed approach are demonstrated through comparisons with other existing approaches.

Efficient Exploration of Adsorption Space for Separations in Metal-Organic Frameworks Combining the Use of Molecular Simulations, Machine Learning, and Ideal Adsorbed Solution Theory.

Adsorption-based separations using metal-organic frameworks (MOFs) are promising candidates for replacing common energy-intensive separation processes. The so-called adsorption space formed by the combination of billions of possible molecules and thousands of reported MOFs is vast. It is very challenging to comprehensively evaluate the performance of MOFs for chemical separation through experiments. Molecular simulations and machine learning (ML) have been widely applied to make predictions for adsorption-based separations. Previous ML approaches to these issues were typically limited to smaller molecules and often had poor accuracy in the dilute limit. To enable exploration of a wider adsorption space, we carefully selected a diverse set of 45 molecules and 335 MOFs and generated single-component isotherms of 15,075 MOF-molecule pairs by grand canonical Monte Carlo. Using this database, we successfully developed accurate (r2 > 0.9) machine learning models predicting adsorption isotherms of diverse molecules in large libraries of MOFs. With this approach, we can efficiently make predictions of large collections of MOFs for arbitrary mixture separations. By combining molecular simulation data and ML predictions with Ideal Adsorbed Solution Theory, we tested the ability of these approaches to make predictions of adsorption selectivity and loading for challenging near-azeotropic mixtures.

Accurate GW frontier orbital energies of 134 kilo molecules

HOMO and LUMO energies are critical molecular properties that typically require high accuracy computations for practical applicability. Until now, a comprehensive dataset containing sufficiently accurate HOMO and LUMO energies has been unavailable. In this study, we introduce a new dataset of HOMO/LUMO energies for QM9 compounds, calculated using the GW method. The GW method offers adequate HOMO/LUMO prediction accuracy for diverse applications, exhibiting mean unsigned errors of 100 meV in the GW100 benchmark dataset. This database may serve as a benchmark of HOMO/LUMO prediction, delta-learning, and transfer learning, particularly for larger molecules where GW is the most accurate but still numerically feasible method. We anticipate that this dataset will enable the development of more accurate machine learning models for predicting molecular properties.

Modeling of Journal Bearings for Wear Diagnosis and Its Verification Using SVM

Recently, deterioration of infrastructure facilities has become problem. To reduce inspection costs, wear diagnosis of journal bearings in operating conditions using machine learning has been studied. However, constructing a highly accurate machine learning model requires training data, and it may be difficult to conduct numerous tests depending on the application. In this study, a mathematical model of a rotor system for wear diagnosis of journal bearings was developed, and the vibration data was obtained when the clearance changed due to wear. Then, changes in the features used for condition monitoring were examined. Furthermore, the important features for wear diagnosis were selected and SVM models were constructed to verify the mathematical model.

Machine learning models for predicting pre-eclampsia: a systematic review protocol

IntroductionPre-eclampsia is one of the most serious clinical problems of pregnancy that contribute significantly to maternal mortality worldwide. This systematic review aims to identify and summarise the predictive factors of...

Comparing the accuracy of machine learning models for accelerometer-based movement measurements

Comparing the accuracy of machine learning models for accelerometer-based movement measurements

ECG and EEG based detection and multilevel classification of stress using machine learning for specified genders: A preliminary study.

Mental health, especially stress, plays a crucial role in the quality of life. During different phases (luteal and follicular phases) of the menstrual cycle, women may exhibit different responses to stress from men. This, therefore, may have an impact on the stress detection and classification accuracy of machine learning models if genders are not taken into account. However, this has never been investigated before. In addition, only a handful of stress detection devices are scientifically validated. To this end, this work proposes stress detection and multilevel stress classification models for unspecified and specified genders through ECG and EEG signals. Models for stress detection are achieved through developing and evaluating multiple individual classifiers. On the other hand, the stacking technique is employed to obtain models for multilevel stress classification. ECG and EEG features extracted from 40 subjects (21 females and 19 males) were used to train and validate the models. In the low&high combined stress conditions, RBF-SVM and kNN yielded the highest average classification accuracy for females (79.81%) and males (73.77%), respectively. Combining ECG and EEG, the average classification accuracy increased to at least 87.58% (male, high stress) and up to 92.70% (female, high stress). For multilevel stress classification from ECG and EEG, the accuracy for females was 62.60% and for males was 71.57%. This study shows that the difference in genders influences the classification performance for both the detection and multilevel classification of stress. The developed models can be used for both personal (through ECG) and clinical (through ECG and EEG) stress monitoring, with and without taking genders into account.

Synthesize high-dimensional longitudinal electronic health records via hierarchical autoregressive language model

Synthetic electronic health records (EHRs) that are both realistic and privacy-preserving offer alternatives to real EHRs for machine learning (ML) and statistical analysis. However, generating high-fidelity EHR data in its original, high-dimensional form poses challenges for existing methods. We propose Hierarchical Autoregressive Language mOdel (HALO) for generating longitudinal, high-dimensional EHR, which preserve the statistical properties of real EHRs and can train accurate ML models without privacy concerns. HALO generates a probability density function over medical codes, clinical visits, and patient records, allowing for generating realistic EHR data without requiring variable selection or aggregation. Extensive experiments demonstrated that HALO can generate high-fidelity data with high-dimensional disease code probabilities closely mirroring (above 0.9 R2 correlation) real EHR data. HALO also enhances the accuracy of predictive modeling and enables downstream ML models to attain similar accuracy as models trained on genuine data.

Insights into the Luminescence Quantum Yields of Cyclometalated Iridium(III) Complexes: A Density Functional Theory and Machine Learning Approach.

Cyclometalated iridium(III) complexes have been used in various optical materials, including organic light-emitting diodes (OLEDs) and photocatalysts, and a deeper understanding and prediction of their luminescence quantum yields (LQYs) greatly aid in accelerating material design. In this study, we integrated density functional theory (DFT) calculations with machine learning (ML) techniques to extract factors controlling LQY. Although a substantial data set of Ir(III) complexes and their LQYs is indispensable for constructing accurate ML models to predict LQYs, generating this type of data set is challenging due to the complexities associated with ab initio calculations of LQYs. To address this issue, we investigated the nonradiative decay process of nine Ir(III) complexes emitting blue to green, each exhibiting varying experimental LQYs, by using DFT calculations. For all nine complexes, the quenching process was induced by the rotation of the single bond in one of the ligands, which converted the six-coordinate structure to the five-coordinate structure. Since the decay mechanism was common for the nine Ir(III) complexes, parameters correlated with LQYs could be used as objective variables instead of LQYs. Based on this idea, we collected a data set featuring Ir(III) complexes and the energy differences between their six- and five-coordinate triplet structures, which correlated with LQYs. We also constructed ML models using the calculated LQYs as the objective variables with the parameters from the ground-state calculations as explanatory variables. The analyses of the constructed model revealed that the LUMO energy of the ligand made the most significant negative contribution to LQY. This suggests that the potential energy surface of the metal-to-ligand charge transfer (MLCT) excited state, which stabilizes the six-coordinate structure, is reduced by decreasing the energy of the unoccupied orbitals.

An Empirical Evaluation of Feature Selection Stability and Classification Accuracy

The performance of inductive learners can be negatively affected by high-dimensional datasets. To address this issue, feature selection methods are used. Selecting relevant features and reducing data dimensions is essential for having accurate machine learning models. Stability is an important criterion in feature selection. Stable feature selection algorithms maintain their feature preferences even when small variations exist in the training set. Studies have emphasized the importance of stable feature selection, particularly in cases where the number of samples is small and the dimensionality is high. In this study, we evaluated the relationship between stability measures, as well as, feature selection stability and classification accuracy, using the Pearson Product-Moment Correlation Coefficient. We conducted an extensive series of experiments using five filter and two wrapper feature selection methods, three classifiers for subset and classification performance evaluation, and eight real-world datasets taken from two different data repositories. We measured the stability of feature selection methods using a total of twelve stability metrics. Based on the results of correlation analyses, we have found that there is a lack of substantial evidence supporting a linear relationship between feature selection stability and classification accuracy. However, a strong positive correlation has been observed among several stability metrics.