Performance Of Prediction Algorithms Research Articles

Data Analytics and Predictive Modeling for Toll Violation Prevention

This study aims to utilize data analytics and predictive modeling techniques to prevent toll violations. By analyzing various sources of data such as travel datasets, unstructured data, and financial data, the study will develop algorithms that can accurately predict toll violations and identify potential fraud patterns. The research, which utilized unstructured data, points to the utility of data mining which would greatly benefit traffic research, particularly African-based studies, that suffer from data inadequacy. The study will also explore the use of machine learning algorithms to decipher the large textual data contained in transportation research papers. Additionally, the study will investigate the accuracy and performance of different predictive algorithms, such as linear regression, analysis of variance, and artificial neural networks, in predicting toll violations based on various factors such as category, purpose, total distance, and speed of travel. Furthermore, the study will examine the forensic aspects of big data in order to enhance the capabilities of forensic investigators in case of a network attack. In today’s rapidly changing world, the significance of accurate toll violation prevention using data analytics and predictive modeling techniques cannot be overstated. In today’s rapidly changing world, the significance of accurate toll violation prevention using data analytics and predictive modeling techniques cannot be overstated. By leveraging data analytics and predictive modeling, this study aims to develop an effective toll violation prevention system

Toward Structural Controllability and Predictability in Directed Networks

The lack of studying the complex organization of directed network usually limits the understanding of the underlying relationship between network structures and functions. Structural controllability and structural predictability, two seemingly unrelated subjects, are revealed in this article to be both highly dependent on the critical links previously thought to only be able to influence the number of driver nodes in controllable directed networks. Here, we show that critical links can not only contribute to structural controllability but can also have a significant impact on the structural predictability of networks, suggesting the universal pattern of structural reciprocity in directed networks. In addition, it is shown that the fraction and location of critical links have a strong influence on the performance of prediction algorithms. Moreover, these empirical results are interpreted by introducing the link centrality based on corresponding line graphs. This work bridges the gap between the two independent research fields, and it provides indications of developing advanced control strategies and prediction algorithms from a microscopic perspective.

A Survey on Classifying Big Data with Label Noise

Class label noise is a critical component of data quality that directly inhibits the predictive performance of machine learning algorithms. While many data-level and algorithm-level methods exist for treating label noise, the challenges associated with big data call for new and improved methods. This survey addresses these concerns by providing an extensive literature review on treating label noise within big data. We begin with an introduction to the class label noise problem and traditional methods for treating label noise. Next, we present 30 methods for treating class label noise in a range of big data contexts, i.e., high-volume, high-variety, and high-velocity problems. The surveyed works include distributed solutions capable of operating on datasets of arbitrary sizes, deep learning techniques for large-scale datasets with limited clean labels, and streaming techniques for detecting class noise in the presence of concept drift. Common trends and best practices are identified in each of these areas, implementation details are reviewed, empirical results are compared across studies when applicable, and references to 17 open-source projects and programming packages are provided. An emphasis on label noise challenges, solutions, and empirical results as they relate to big data distinguishes this work as a unique contribution that will inspire future research and guide machine learning practitioners.

Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic.

Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.

Understanding transit ridership in an equity context through a comparison of statistical and machine learning algorithms

Building an accurate model of travel behaviour based on individuals’ characteristics and built environment attributes is of importance for policy-making and transportation planning. Recent experiments with big data and Machine Learning (ML) algorithms toward a better travel behaviour analysis have mainly overlooked socially disadvantaged groups. Accordingly, in this study, we explore the travel behaviour responses of low-income individuals to transit investments in Greater Toronto and Hamilton Area, Canada, using statistical and ML models. We first investigate how the model choice affects the prediction of transit use by the low-income group. This step includes comparing the predictive performance of traditional and ML algorithms and then evaluating a transit investment policy by contrasting the predicted activities and the spatial distribution of transit trips generated by vulnerable households after improving accessibility. We also empirically investigate the proposed transit investment by each algorithm and compare it with the city of Brampton’s future transportation plan. While, unsurprisingly, the ML algorithms outperform classical models, there are still doubts about using them due to interpretability concerns. Hence, we adopt recent local and global model-agnostic interpretation tools to interpret how the model arrives at its predictions. Our findings reveal the great potential of ML algorithms for enhanced travel behaviour predictions for low-income strata without considerably sacrificing interpretability.

Abstract 14008: Socioeconomic Status and Prediction of Incident Heart Failure in the Atherosclerosis Risk in Communities (ARIC) Study

Introduction: Socioeconomic status (SES) is a potent risk factor for heart failure (HF), but current HF prediction models do not incorporate data on SES. It is not known if the performance of HF risk prediction algorithms differs by SES, and how including SES in risk models impacts risk prediction. Methods: We performed an analysis of participants without HF at Visit 1 (1987-89) of the ARIC study. SES was assessed by household income, educational attainment, Area Deprivation Index, and as a combined cumulative SES score that included all three measures. Incident hospitalizations and deaths related to HF were assessed through 12/31/2019. We compared risk discrimination (c-statistic) of the established ARIC HF risk score across SES categories, and assessed calibration within SES strata by comparing observed versus predicted risk. We examined changes in risk discrimination, net reclassification and calibration (calculating the Hosmer-Lemeshow statistic for the latter) with adding SES to the ARIC HF risk score. Results: We studied 14,468 individuals; 54% were women, 26% were Black adults and 22% had low cumulative SES. There were 3,568 HF events over 33 years of follow-up. The c-statistic of the ARIC HF risk score was lower in those with low versus high cumulative SES (0.726 vs 0.765). There were differences in the ARIC HF risk score’s calibration by SES, with lower predicted than observed HF risk in those with low SES in all quintiles of predicted risk (Figure). Calibration improved with adding SES to the risk model (change in Hosmer-Lemeshow 127.6 to 98.3), especially in those with low SES (Figure). Adding SES to the model modestly improved risk discrimination (change in c-statistic 0.767 to 0.771; p<0.001) and changed net reclassification (0.05 [95% CI 0.02-0.08]). Conclusions: HF prediction algorithms underestimate HF risk in those with low SES. Systematic assessment and incorporation of SES data may enhance HF risk prediction in socioeconomically diverse populations.

Abstract 15358: Comparative Performance of Recurrent Heart Failure Prediction Models: Incorporating Social Determinants and Applying Machine Learning

Introduction: Improving performance of predictive algorithms for 30-d readmission following heart failure (HF) hospitalization may enhance clinical management decisions. Applying novel data (social determinants of health; SDOH) and novel methods (machine learning) have been proposed. We compared performance to predict 30-d HF readmission using models with and without SDOH and under traditional and machine learning techniques. Hypothesis and Methods: We used data on 27,071 admissions within Emory Healthcare (mean age: 66 , 48% female, 51% black). We categorized 69 covariates as patient-level clinical (e.g., diastolic blood pressure), patient-level demographic (e.g. gender, age), and zip code SDOH (e.g. percent unemployed). Logistic regression under generalized linear model (GLM) and gradient boosted model (GBM) frameworks were applied to test all combinations of covariate sets. We evaluated performance using 5-fold cross-validation calculating area under the curve (AUC; discrimination) and mean absolute error (MAE; calibration) statistics. Results: 30-d readmission occurred in 17.7% of hospital admissions. Under GLM, model performance was similar across all covariate sets (AUCs .500 - .502). Under GBM, model performance measures were higher than GLM but similar across covariate sets (.500-.664). Notably, SDOH improved model discrimination (AUC .606 clinical only, AUC .664 clinical+SDOH) in the full dataset (“in-sample”) runs. Conclusions: Patient clinical data provided the richest information for prediction of HF readmission, with incremental gains through the addition of area SDOH data such as zip code level unemployment. Furthermore, these gains were only realized in GBM highlighting the marginal utility in flexible modeling methods. While non-clinical data sources can provide a rich understanding of the social mechanisms producing disease, their utility for clinical prediction may be outcome-specific.

K-RAS Associated Gene-Mutation-Based Algorithm for Prediction of Treatment Response of Patients with Subtypes of Breast Cancer and Especially Triple-Negative Cancer.

There is an urgent need for developing new biomarker tools to accurately predict treatment response of breast cancer, especially the deadly triple-negative breast cancer. We aimed to develop gene-mutation-based machine learning (ML) algorithms as biomarker classifiers to predict treatment response of first-line chemotherapy with high precision. Random Forest ML was applied to screen the algorithms of various combinations of gene mutation profiles of primary tumors at diagnosis using a TCGA Cohort (n = 399) with up to 150 months follow-up as a training set and validated in a MSK Cohort (n = 807) with up to 220 months follow-up. Subtypes of breast cancer including triple-negative and luminal A (ER+, PR+ and HER2-) were also assessed. The predictive performance of the candidate algorithms as classifiers was further assessed using logistic regression, Kaplan-Meier progression-free survival (PFS) plot, and univariate/multivariate Cox proportional hazard regression analyses. A novel algorithm termed the 12-Gene Algorithm based on mutation profiles of KRAS, PIK3CA, MAP3K1, MAP2K4, PTEN, TP53, CDH1, GATA3, KMT2C, ARID1A, RunX1, and ESR1, was identified. The performance of this algorithm to distinguish non-progressed (responder) vs. progressed (non-responder) to treatment in the TCGA Cohort as determined using AUC was 0.96 (95% CI 0.94-0.98). It predicted progression-free survival (PFS) with hazard ratio (HR) of 21.6 (95% CI 11.3-41.5) (p < 0.0001) in all patients. The algorithm predicted PFS in the triple-negative subgroup with HR of 19.3 (95% CI 3.7-101.3) (n = 42, p = 0.000). The 12-Gene Algorithm was validated in the MSK Cohort with a similar AUC of 0.97 (95% CI 0.96-0.98) to distinguish responder vs. non-responder patients, and had a HR of 18.6 (95% CI 4.4-79.2) to predict PFS in the triple-negative subgroup (n = 75, p < 0.0001). The novel 12-Gene algorithm based on multitude gene-mutation profiles identified through ML has a potential to predict breast cancer treatment response to therapies, especially in triple-negative subgroups patients, which may assist personalized therapies and reduce mortality.

Group predictive voltage and frequency regulators for small hydro power plant in the context of low power quality

The study examines the influence of predictive voltage and frequency regulators of generators of a small hydropower plant (HPP) on the quality indices of electric power and those of the control process in the case of disturbances in the system. The study aimed to determine the performance of the group predictive generator control algorithms. The research has revealed that the use of a DC link and a small HPP with group predictive voltage and frequency controllers has a noticeable effect on the improvement of power quality: the voltage distortion factor decreases by an average of 1.4 times and the voltage unbalance factor is reduced by almost ten times. In all considered conditions, the use of group predictive controllers can significantly improve the quality of electricity and the quality of the voltage and frequency control: the voltage dip depth decreases by 1.7 times and the voltage un-balance factor decreases by almost 4 times when an additional load is connected; the voltage dip depth is reduced by 20%, the frequency overshoot value is reduced by a factor of 1.8, and the transient process time for the frequency under the short-term short-circuit conditions is reduced by a factor of 1.5.

Contamination vertical distribution and key factors identification of metal(loid)s in site soil from an abandoned Pb/Zn smelter using machine learning

Soil heterogeneity makes the vertical distribution of metal(loid)s in site soil vary considerably and poses a challenge for identifying the key factors of metal(loid)s migration in site soil profiles. In this study, a machine learning (ML) model was developed to study a typical abandoned Pb/Zn smelter using 267 site soils from 46 drilling points. Results showed that a well-trained ML model could be used to identify the key factors in determining the contamination vertical distribution and predict the metal(loid)s contents in subsurface soil. As, Cd, Pb, and Zn were the primary pollutants and their vertical migration depth arrived to 4–6 m. Based on the predictive performance of different ML algorithms, the extreme gradient boosting (XGB) was selected as the best model to produce accurate predictions for the most metal(loid)s content. Contents of As, Cd, Pb, and Zn in the heavily contaminated zones declined with an increase of soil depth. The metal(loid) contents in surface soil of 0–2 m could be readily used to predict the content of Cd, Cr, Hg, and Zn in subsurface soil from 2 m to 10 m. Based on the metal-specific XGB models, sulfur content, functional area, and soil texture were identified as key factors affecting the vertical distribution of As, Cd, Pb, and Zn in site soil. Results suggested the ML method is helpful to manage the potential environmental risks of metal(loid)s in Pb/Zn smelting site.

Deep neural networks with L1 and L2 regularization for high dimensional corporate credit risk prediction

Accurate credit risk prediction can help companies avoid bankruptcies and make adjustments ahead of time. There is a tendency in corporate credit risk prediction that more and more features are considered in the prediction system. However, this often brings redundant and irrelevant information which greatly impairs the performance of prediction algorithms. Therefore, this study proposes an HDNN algorithm that is an improved deep neural network (DNN) algorithm and can be used for high dimensional prediction of corporate credit risk. We firstly theoretically proved that there was no regularization effect when L1 regularization was added to the batch normalization layer of the DNN, which was a hidden rule in the industrial implementation but never been proved. In addition, we proved that adding L2 constraints on a single L1 regularization can solve the issue. Finally, this study analyzed a case study of credit data with supply chain and network data to show the superiority of the HDNN algorithm in the scenario of a high dimensional dataset.

Using Machine Learning to Predict Treatment Adherence in Patients on Medication for Opioid Use Disorder.

Patients receiving medication for opioid use disorder (MOUD) may continue using nonprescribed drugs or have trouble with medication adherence, and it is difficult to predict which patients will continue to do so. In this study, we develop and validate an automated risk-modeling framework to predict opioid abstinence and medication adherence at a patient's next attended appointment and evaluate the predictive performance of machine-learning algorithms versus logistic regression. Urine drug screen and attendance records from 40,005 appointments drawn from 2742 patients at a multilocation office-based MOUD program were used to train logistic regression, logistic ridge regression, and XGBoost models to predict a composite indicator of treatment adherence (opioid-negative and norbuprenorphine-positive urine, no evidence of urine adulteration) at next attended appointment. The XGBoost model had similar accuracy and discriminative ability (accuracy, 88%; area under the receiver operating curve, 0.87) to the two logistic regression models (accuracy, 88%; area under the receiver operating curve, 0.87). The XGBoost model had nearly perfect calibration in independent validation data; the logistic and ridge regression models slightly overestimated adherence likelihood. Historical treatment adherence, attendance rate, and fentanyl-positive urine at current appointment were the strongest contributors to treatment adherence at next attended appointment. There is a need for risk prediction tools to improve delivery of MOUD. This study presents an automated and portable risk-modeling framework to predict treatment adherence at each patient's next attended appointment. The XGBoost algorithm appears to provide similar classification accuracy to logistic regression models; however, XGBoost may offer improved calibration of risk estimates compared with logistic regression.

On the Identification of Material Constitutive Model Parameters Using Machine Learning Algorithms

This work aims to evaluate the predictive performance of various Machine Learning algorithms when applied to the prediction of material constitutive parameters, particularly the parameters of the Swift hardening law. For this, datasets were generated from the results of the numerical simulations of uniaxial tensile tests. The Machine Learning algorithms considered for this study are: Gaussian Process, Multi-layer Perceptron, Support Vector Regression, Decision Tree and Random Forest. These algorithms were used to train metamodels based on training sets considering different numbers of materials and input parameters, which were then used to predict the hardening law parameters. The Gaussian Process algorithm achieved the overall best predictive performances. The results obtained show the potential of Machine Learning algorithms for application on the identification of material constitutive parameters.

A Comprehensive Evaluation of the Performance of Prediction Algorithms on Clinically Relevant Missense Variants.

The rapid integration of genomic technologies in clinical diagnostics has resulted in the detection of a multitude of missense variants whose clinical significance is often unknown. As a result, a plethora of computational tools have been developed to facilitate variant interpretation. However, choosing an appropriate software from such a broad range of tools can be challenging; therefore, systematic benchmarking with high-quality, independent datasets is critical. Using three independent benchmarking datasets compiled from the ClinVar database, we evaluated the performance of ten widely used prediction algorithms with missense variants from 21 clinically relevant genes, including BRCA1 and BRCA2. A fourth dataset consisting of 1053 missense variants was also used to investigate the impact of type 1 circularity on their performance. The performance of the prediction algorithms varied widely across datasets. Based on Matthews Correlation Coefficient and Area Under the Curve, SNPs&GO and PMut consistently displayed an overall above-average performance across the datasets. Most of the tools demonstrated greater sensitivity and negative predictive values at the expense of lower specificity and positive predictive values. We also demonstrated that type 1 circularity significantly impacts the performance of these tools and, if not accounted for, may confound the selection of the best performing algorithms.

A Comprehensive Evaluation of the Performance of Prediction Algorithms on Clinically Relevant Missense Variants

A Comprehensive Evaluation of the Performance of Prediction Algorithms on Clinically Relevant Missense Variants

Characterization of constrained continuous multiobjective optimization problems: A feature space perspective

Despite the increasing interest in constrained multiobjective optimization in recent years, constrained multiobjective optimization problems (CMOPs) are still insufficiently understood and characterized. For this reason, the selection of appropriate CMOPs for benchmarking is difficult and lacks a formal background. We address this issue by extending landscape analysis to constrained multiobjective optimization. By employing four exploratory landscape analysis techniques, we propose 29 landscape features (of which 19 are novel) to characterize CMOPs. These landscape features are then used to compare eight frequently used artificial test suites against a recently proposed suite consisting of real-world problems based on physical models. The experimental results reveal that the artificial test problems fail to adequately represent some realistic characteristics, such as strong negative correlation between the objectives and the overall constraint violation. Moreover, our findings show that all the studied artificial test suites have advantages and limitations, and that no “perfect” suite exists. Additionally, the effectiveness of the proposed features at predicting algorithm performance is demonstrated for two multiobjective optimization algorithms. Benchmark designers can use the obtained results to select or generate appropriate CMOP instances based on the characteristics they want to explore.

Mortality prediction of patients in intensive care units using machine learning algorithms based on electronic health records

Improving predictive models for intensive care unit (ICU) inpatients requires a new strategy that periodically includes the latest clinical data and can be updated to reflect local characteristics. We extracted data from all adult patients admitted to the ICUs of two university hospitals with different characteristics from 2006 to 2020, and a total of 85,146 patients were included in this study. Machine learning algorithms were trained to predict in-hospital mortality. The predictive performance of conventional scoring models and machine learning algorithms was assessed by the area under the receiver operating characteristic curve (AUROC). The conventional scoring models had various predictive powers, with the SAPS III (AUROC 0.773 [0.766–0.779] for hospital S) and APACHE III (AUROC 0.803 [0.795–0.810] for hospital G) showing the highest AUROC among them. The best performing machine learning models achieved an AUROC of 0.977 (0.973–0.980) in hospital S and 0.955 (0.950–0.961) in hospital G. The use of ML models in conjunction with conventional scoring systems can provide more useful information for predicting the prognosis of critically ill patients. In this study, we suggest that the predictive model can be made more robust by training with the individual data of each hospital.

Reducing the Label Space a Predefined Ratio for a More Efficient Multilabel Classification

The multi-label classification task has been widely used to solve problems where each of the instances may be related not only to one class but to many of them simultaneously. Many of these problems usually comprise a high number of labels in the output space, so learning a predictive model from such datasets may turn into a challenging task since the computational complexity of most algorithms depends on the number of labels. In this paper, we propose a methodology to reduce the label space a predefined ratio of labels, aiming to improve the runtime of the multi-label classification algorithms. Obviously, such reduction should be done without producing a significant drop in their final predictive performance. The experimental analysis carried out over 25 well-known multi-label datasets, demonstrates a drastic reduction in the runtime. Besides, it is statistically proven that reducing 20% the number of labels does not lead to a decrease in the predictive performance of the multi-label algorithms using four well-known evaluation measures. Even more, in many cases, although reductions of up to 50% of the output space are made, the predictive performance of the algorithms is not significantly different from using the whole set of labels.

Randomized Compiling for Scalable Quantum Computing on a Noisy Superconducting Quantum Processor

The successful implementation of algorithms on quantum processors relies on the accurate control of quantum bits (qubits) to perform logic gate operations. In this era of noisy intermediate-scale quantum (NISQ) computing, systematic miscalibrations, drift, and crosstalk in the control of qubits can lead to a coherent form of error that has no classical analog. Coherent errors severely limit the performance of quantum algorithms in an unpredictable manner, and mitigating their impact is necessary for realizing reliable quantum computations. Moreover, the average error rates measured by randomized benchmarking and related protocols are not sensitive to the full impact of coherent errors and therefore do not reliably predict the global performance of quantum algorithms, leaving us unprepared to validate the accuracy of future large-scale quantum computations. Randomized compiling is a protocol designed to overcome these performance limitations by converting coherent errors into stochastic noise, dramatically reducing unpredictable errors in quantum algorithms and enabling accurate predictions of algorithmic performance from error rates measured via cycle benchmarking. In this work, we demonstrate significant performance gains under randomized compiling for the four-qubit quantum Fourier transform algorithm and for random circuits of variable depth on a superconducting quantum processor. Additionally, we accurately predict algorithm performance using experimentally measured error rates. Our results demonstrate that randomized compiling can be utilized to leverage and predict the capabilities of modern-day noisy quantum processors, paving the way forward for scalable quantum computing.Received 13 May 2021Accepted 3 September 2021DOI:https://doi.org/10.1103/PhysRevX.11.041039Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum algorithmsQuantum benchmarkingQuantum controlQuantum information with solid state qubitsQuantum Information

Abstract 13186: A Machine Learning Evaluation of Patient Characteristics Associated With Guideline-Directed Prescribing for Heart Failure and Reduced Ejection Fraction

Introduction: Patients with heart failure and reduced ejection fraction (HFrEF) are consistently under-prescribed guideline-directed medications. Many barriers to prescribing are known, including patient characteristics (e.g., higher potassium values within normal limits predict under-prescribing of some medications). However, identification of these barriers has relied on traditional a priori hypotheses or qualitative methods. To overcome the limitations of traditional methods, we used a supervised machine learning (ML) approach to identify potential new barriers to prescribing. Methods: We used electronic health record data to evaluate characteristics of adults with HFrEF within the UCHealth system. Characteristics evaluated included insurance type, medications, diagnoses and lab values. We evaluated the predictive performance of ML algorithms and statistical models based on area under the receiver operating characteristic curve (AUC) to predict prescription of 4 types of medications: angiotensin converting enzyme inhibitor/angiotensin receptor blocker (ACE/ARB), angiotensin receptor-neprilysin inhibitor (ARNI), beta blocker (BB), or mineralocorticoid receptor antagonist (MRA). We identified top characteristics associated with prescribing each medication type from the models with the best predictive performance. Results: For 3,835 patients meeting inclusion criteria, 70% were prescribed an ACE/ARB, 8% an ARNI, 75% a BB and 40% a MRA. The best-predicting model for each medication type was a random forest. Figure 1 describes the top characteristics associated with each medication type. Conclusions: In this hypothesis generating ML study, we identified additional patient characteristics that may be associated with under-prescribing for HFrEF. These results also align with some patient characteristics that are known barriers to prescribing. Additional research is needed to further elucidate barriers to prescribing for HFrEF.