Evaluate Machine Learning Algorithms Research Articles

Evaluating Machine Learning Algorithms for Fake Currency Detection

Currency is a critical asset in any economy, yet it is vulnerable to counterfeiting, undermining its value and disrupting economic stability. Counterfeit currency is particularly prevalent during economic transition, such as demonetization, as fake notes are circulated to mimic real currency. Due to the subtle similarities between genuine and fake notes, distinguishing between them can be challenging. Consequently, financial institutions like banks and ATMs require robust automated systems to accurately detect counterfeit currency. In this study, we evaluate the effectiveness of six supervised machine learning algorithms—K-Nearest Neighbor, Decision Trees, Support Vector Machine, Random Forests, Logistic Regression, and Naive Bayes—in detecting the authenticity of banknotes. Additionally, we examine the performance of LightGBM, a gradientboosting algorithm, in comparison to these traditional methods. Our findings contribute to developing reliable, automated systems for counterfeit detection, and enhancing financial security.

Improving earthquake prediction accuracy in Los Angeles with machine learning

This research breaks new ground in earthquake prediction for Los Angeles, California, by leveraging advanced machine learning and neural network models. We meticulously constructed a comprehensive feature matrix to maximize predictive accuracy. By synthesizing existing research and integrating novel predictive features, we developed a robust subset capable of estimating the maximum potential earthquake magnitude. Our standout achievement is the creation of a feature set that, when applied with the Random Forest machine learning model, achieves a high accuracy in predicting the maximum earthquake category within the next 30 days. Among sixteen evaluated machine learning algorithms, Random Forest proved to be the most effective. Our findings underscore the transformative potential of machine learning and neural networks in enhancing earthquake prediction accuracy, offering significant advancements in seismic risk management and preparedness for Los Angeles.

Application of Machine Learning Algorithms for Risk Stratification and Efficacy Evaluation in Cervical Cancer Screening Among the ASCUS/LSIL Population: Evidence from the Korean HPV Cohort Study.

We assessed human papillomavirus (HPV) genotype-based risk stratification and the efficacy of cytology testing for cervical cancer screening in patients with atypical squamous cells of undetermined significance (ASCUS)/low-grade squamous intraepithelial lesion (LSIL). Between 2010 and 2021, we monitored 1,237 HPV-positive women with ASCUS/LSIL every 6 months for up to 60 months. HPV infections were categorized as persistent (HPV positivity consistently observed post-enrollment), negative (HPV negativity consistently observed post-enrollment), or non-persistent (neither consistently positive nor negative). HPV genotypes were grouped into high-risk (Hr) groups 1 (types 16, 18, 31, 33, 45, 52, and 58) and 2 (types 35, 39, 51, 56, 59, 66, and 68) and a low-risk group. Hr1 was subdivided into types a) 16 and 18; b) 31, 33, and 45; and c) 52 and 58. Cox regression and machine learning (ML) algorithms were used to analyze progression rates. Among 1,273 participants, 17.6% with persistent HPV infections experienced disease progression versus no progression in the HPV-negative group (p<0.001). Cox analysis revealed the highest hazard ratios (HRs) for Hr1-a (11.6, p<0.001), followed by Hr1-b (9.26, p<0.001) and Hr1-c (7.21, p<0.001). HRs peaked at 12-24 months, with Hr1-a maintaining significance at 24-36 months (10.7, p=0.034). ML analysis identified the final cytology change pattern as the most significant factor, with 14-15 months the optimal time for detecting progression from the first examination. In ASCUS/LSIL cases, follow-up strategies should be based on HPV risk types. Annual follow-up was the most effective monitoring for detecting progression/regression.

Spectral sensor‐based device for real‐time detection and severity estimation of groundnut bud necrosis virus in tomato

AbstractA machine learning‐based approach was utilized to develop a device for groundnut bud necrosis virus (GBNV) disease severity detection and estimation in tomato plants (Solanum lycopersicum L.). The study involved inoculating tomato plants with GBNV, monitoring changes in morphological and spectral characteristics, evaluating machine learning algorithms (decision tree [DT] classifier) for analysis and classification of disease severity, and developing and validating a device for disease detection and severity estimation. Spectral data analysis revealed distinct patterns in reflectance, with notable peaks observed in the 680 and 760 nm bands, while reflectance remained low and constant beyond 900 nm. Machine learning techniques, specifically a DT model, were employed to classify disease severity based on spectral data with high accuracy (95.01% training accuracy and 93.65% testing accuracy). The model identified the near‐infrared band as highly correlated (correlation coefficient of 0.82) with disease severity. Furthermore, a compact handheld device integrating a spectral sensor, organic light‐emitting diode display, and Raspberry Pi 3B was developed for real‐time disease severity estimation. The device demonstrated robust performance, accurately predicting disease severity at different growth stages, even in the absence of visible symptoms. Additionally, disease severity percentages obtained via reverse transcription polymerase chain reaction were used to validate the accuracy of the device's estimations. Its responsive nature, with estimated response times ranging from milliseconds to seconds, facilitates timely interventions in agricultural settings. Overall, this interdisciplinary approach, combining spectral analysis, machine learning, and device development, presents a promising solution for efficient disease monitoring and management in agriculture, contributing to enhanced crop health and food security.

Intelligent condition prediction model for bridge infrastructure based on evaluating machine learning algorithms

PurposePrediction models are essential tools for transportation agencies to forecast the condition of bridge decks based on available data, and artificial intelligence is paramount for this purpose. This study aims at proposing a bridge deck condition prediction model by assessing various classification and regression algorithms.Design/methodology/approachThe 2019 National Bridge Inventory database is considered for model development. Eight different feature selection techniques, along with their mean and frequency, are used to identify the critical features influencing deck condition ratings. Thereafter, four regression and four classification algorithms are applied to predict condition ratings based on the selected features, and their performances are evaluated and compared with respect to the mean absolute error (MAE).FindingsClassification algorithms outperform regression algorithms in predicting deck condition ratings. Due to its minimal MAE (0.369), the random forest classifier with eleven features is recommended as the preferred condition prediction model. The identified dominant features are superstructure condition, age, structural evaluation, substructure condition, inventory rating, maximum span length, deck area, average daily traffic, operating rating, deck width, and the number of spans.Practical implicationsThe proposed bridge deck condition prediction model offers a valuable tool for transportation agencies to plan maintenance and resource allocation efficiently, ultimately improving bridge safety and serviceability.Originality/valueThis study provides a detailed framework for applying machine learning in bridge condition prediction that applies to any bridge inventory database. Moreover, it uses a comprehensive dataset encompassing an entire region, broadening the model’s applicability and representation.

Securing smart cities through machine learning: A honeypot‐driven approach to attack detection in Internet of Things ecosystems

AbstractThe rapid increase and adoption of Internet of Things (IoT) devices have introduced unprecedented conveniences into modern life. However, this growth has also ushered in a wave of cyberattacks targeting these often‐vulnerable systems. Smart cities, relying on interconnected sensors, are particularly susceptible to attacks due to the expanded entry points created by these devices. A security breach in such systems can compromise personal data and disrupt entire ecosystems. Traditional security measures are inadequate against the evolving sophistication of cyberattacks. The authors aim to address these challenges by leveraging honeypot data and machine learning to enhance IoT security. The research focuses on three objectives: identifying datasets from IoT‐targeted honeypots, evaluating machine learning algorithms for threat detection, and proposing comprehensive security solutions. Real‐world cyber‐attack datasets from diverse honeypots simulating IoT devices are analysed using various machine learning and neural network algorithms. Results demonstrate significant improvement in cyber‐attack detection and mitigation when integrating honeypot data into IoT security frameworks. The authors advance knowledge and provides practical insights for implementing robust security measures in diverse IoT applications, filling a crucial research gap.

Reviewing the Impact of Machine Learning on Disease Diagnosis and Prognosis: A Comprehensive Analysis

Aim This study aimed to explore how machine learning algorithms can enhance medical diagnostics through the analysis of illness imagery and patient data, assessing their effectiveness and potential to improve diagnostic accuracy and early disease detection. Background This study highlights the critical role of machine learning in healthcare, particularly in medical diagnostics. By leveraging advanced algorithms to analyse medical data and images, machine learning enhances disease detection and diagnosis, contributing significantly to improved patient outcomes and the advancement of precision medicine. Objective The objective of this study was to thoroughly analyse and evaluate the efficacy of machine learning algorithms in medical diagnostics, focusing on their application in interpreting illness images and patient data. The goal was to ascertain the algorithms' accuracy in disease diagnosis and prognosis, aiming to demonstrate their potential in revolutionizing healthcare through improved diagnostic precision and early disease detection. Methods A systematic approach has been used in this study to evaluate machine learning algorithms' effectiveness in diagnosing diseases from medical images and data. It involved selecting pertinent datasets, applying and comparing models, like SVM and K-nearest neighbors, and assessing their diagnostic accuracy and performance, aiming to identify the most effective methodologies in medical diagnostics. Results The results have highlighted the varying accuracy of machine learning algorithms in medical diagnostics, with a focus on the performance of models, such as SVM and K-nearest neighbors. A comparative analysis has illustrated the differential effectiveness of these algorithms across various diseases and datasets, underscoring their potential to enhance healthcare diagnostics. Conclusion The study has concluded that machine learning algorithms have significantly improved medical diagnostics, offering varied effectiveness across different conditions. Their potential to revolutionize healthcare is evident, with enhanced diagnostic accuracy and efficiency. Ongoing research and clinical application are essential to harness these technologies' full benefits.

Prediction of naloxone dose in opioids toxicity based on machine learning techniques (artificial intelligence).

Treatment management for opioid poisoning is critical and, at the same time, requires specialized knowledge and skills. This study was designed to develop and evaluate machine learning algorithms for predicting the maintenance dose and duration of hospital stay in opioid poisoning, in order to facilitate appropriate clinical decision-making. This study used artificial intelligence technology to predict the maintenance dose and duration of administration by selecting clinical and paraclinical features that were selected by Pearson correlation (filter method) (Stage 1) and then the (wrapper method) Recursive Feature Elimination Cross-Validated (RFECV) (Stage2). The duration of administration was divided into two categories: A (which includes a duration of less than or equal to 24 h of infusion) and B (more than 24 h of naloxone infusion). XGBoost algorithm model with an accuracy rate of 91.04%, a prediction rate of 91.34%, and a sensitivity rate of 91.04% and area under the Curve (AUC) 0.97 was best model for classification patients. Also, the best maintenance dose of naloxone was obtained with XGBoost algorithm with R2 = 0.678. Based on the selected algorithm, the most important features for classifying patients for the duration of treatment were bicarbonate, respiration rate, physical sign, The partial pressure of carbon dioxide (PCO2), diastolic blood pressure, pulse rate, naloxone bolus dose, Blood Creatinine(Cr), Body temperature (T). The most important characteristics for determining the maintenance dose of naloxone were physical signs, bolus dose of 4.5 mg/kg, Glasgow Coma Scale (GCS), Creatine Phosphokinase (CPK) and intensive care unit (ICU) add. A predictive model can significantly enhance the decision-making and clinical care provided by emergency physicians in hospitals and medical settings. XGBoost was found to be the superior model.

A Survey of Learning Criteria Going beyond the Usual Risk (Abstract Reprint)

Virtually all machine learning tasks are characterized using some form of loss function, and "good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs. In this work, we survey and introduce a wide variety of non-traditional criteria used to design and evaluate machine learning algorithms, place the classical paradigm within the proper historical context, and propose a view of learning problems which emphasizes the question of "what makes for a desirable loss distribution?" in place of tacit use of the expected loss.

Enhancing HLA-B27 antigen detection: Leveraging machine learning algorithms for flow cytometric analysis.

As the association of human leukocyte antigen B27 (HLA-B27) with spondylarthropathies is widely known, HLA-B27 antigen expression is frequently identified using flow cytometric or other techniques. Because of the possibility of cross-reaction with off target antigens, such as HLA-B7, each flow cytometric technique applies a "gray zone" reserved for equivocal findings. Our aim was to use machine learning (ML) methods to classify such equivocal data as positive or negative. Equivocal samples (n = 99) were selected from samples submitted to our institution for clinical evaluation by HLA-B27 antigen testing. Samples were analyzed by flow cytometry and polymerase chain reaction. Features of histograms generated by flow cytometry were used to train and validate ML methods for classification as logistic regression (LR), decision tree (DT), random forest (RF) and light gradient boost method (GBM). All evaluated ML algorithms performed well, with high accuracy, sensitivity, specificity, as well as negative and positive predictive values. Although, gradient boost approaches are proposed as high performance methods; nevertheless, their effectiveness may be lower for smaller sample sizes. On our relatively smaller sample set, the random forest algorithm performed best (AUC: 0.92), but there was no statistically significant difference between the ML algorithms used. AUC values for light GBM, DT, and LR were 0.88, 0.89, 0.89, respectively. Implementing these algorithms into the process of HLA-B27 testing can reduce the number of uncertain, false negative or false positive cases, especially in laboratories where no genetic testing is available.

Prediction of the Compressive Strength of Vibrocentrifuged Concrete Using Machine Learning Methods

The determination of mechanical properties for different building materials is a highly relevant and practical field of application for machine learning (ML) techniques within the construction sector. When working with vibrocentrifuged concrete products and structures, it is crucial to consider factors related to the impact of aggressive environments. Artificial intelligence methods can enhance the prediction of vibrocentrifuged concrete properties through the use of specialized machine learning algorithms for materials’ strength determination. The aim of this article is to establish and evaluate machine learning algorithms, specifically Linear Regression (LR), Support Vector Regression (SVR), Random Forest (RF), CatBoost (CB), for the prediction of compressive strength in vibrocentrifuged concrete under diverse aggressive operational conditions. This is achieved by utilizing a comprehensive database of experimental values obtained in laboratory settings. The following metrics were used to analyze the accuracy of the constructed regression models: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root-Mean-Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and coefficient of determination (R2). The average MAPE in the range from 2% (RF, CB) to 7% (LR, SVR) allowed us to draw conclusions about the possibility of using “smart” algorithms in the development of compositions and quality control of vibrocentrifuged concrete, which ultimately entails the improvement and acceleration of the construction and building materials manufacture. The best model, CatBoost, showed MAE = 0.89, MSE = 4.37, RMSE = 2.09, MAPE = 2% and R2 = 0.94.

Evaluating machine learning algorithms for prediction of response to ramelteon for sleep disturbances in patients with schizophrenia

Evaluating machine learning algorithms for prediction of response to ramelteon for sleep disturbances in patients with schizophrenia

Accuracy Comparison between Five Machine Learning Algorithms for Financial Risk Evaluation

An accurate prediction of loan default is crucial in credit risk evaluation. A slight deviation from true accuracy can often cause financial losses to lending institutes. This study describes the non-parametric approach that compares five different machine learning classifiers combined with a focus on sufficiently large datasets. It presents the findings on various standard performance measures such as accuracy, precision, recall and F1 scores in addition to Receiver Operating Curve-Area Under Curve (ROC-AUC). In this study, various data pre-processing techniques including normalization and standardization, imputation of missing values and the handling of imbalanced data using SMOTE will be discussed and implemented. Also, the study examines the use of hyper-parameters in various classifiers. During the model construction phase, various pipelines feed data to the five machine learning classifiers, and the performance results obtained from the five machine learning classifiers are based on sampling with SMOTE or hyper-parameters versus without SMOTE and hyper-parameters. Each classifier is compared to another in terms of accuracy during training and prediction phase based on out-of-sample data. The 2 data sets used for this experiment contain 1000 and 30,000 observations, respectively, of which the training/testing ratio is 80:20. The comparative results show that random forest outperforms the other four classifiers both in training and actual prediction.

Using machine learning algorithms with improved accuracy to analyze and predict employee attrition

Human migration is based on pull factors that individuals evaluate when it comes to moving to a different territory. Likewise, employee attrition is a phenomenon that represents the tendency to a reduction in employees within an organization. This research paper aims to develop and evaluate machine learning algorithms, namely Decision Tree, Random Forest, and Binary Logistic Regression, to predict employee attrition using the IBM dataset available on Kaggle. The objective is to provide organizations with a proactive approach to employee retention and human resource management by creating accurate predictive models. Employee attrition has significant implications for an organization's reputation, profitability, and overall structure. By accurately predicting employee attrition, organizations can identify the factors contributing to it and implement data-driven human resources management practices. This study contributes to improving decision-making processes, including hiring and firing decisions, and ultimately enhances an organization's capital. The IBM dataset used in this study consists of anonymized employee records and their employment outcomes. It provides a comprehensive HR data representation for analysis and prediction. Three machine learning algorithms, Decision Tree, Random Forest, and Binary Logistic Regression, were utilized in this research. These algorithms were selected for their potential to improve accuracy in predicting employee attrition. The Logistic Regression model yielded the highest accuracy of 87.44% among the tested algorithms. By leveraging this study's findings, organizations can develop predictive models to identify factors contributing to employee attrition. These insights can inform strategic decisions and optimize human resource management practices.

Evaluating machine learning algorithms to predict lameness in dairy cattle.

Dairy cattle lameness represents one of the common concerns in intensive and commercial dairy farms. Lameness is characterized by gait-related behavioral changes in cows and multiple approaches are being utilized to associate these changes with lameness conditions including data from accelerometers, and other precision technologies. The objective was to evaluate the use of machine learning algorithms for the identification of lameness conditions in dairy cattle. In this study, 310 multiparous Holstein dairy cows from a herd in Northern Colorado were affixed with a leg-based accelerometer (Icerobotics® Inc, Edinburg, Scotland) to obtain the lying time (min/d), daily steps count (n/d), and daily change (n/d). Subsequently, study cows were monitored for 4 months and cows submitted for claw trimming (CT) were differentiated as receiving corrective claw trimming (CCT) or as being diagnosed with a lameness disorder and consequent therapeutic claw trimming (TCT) by a certified hoof trimmer. Cows not submitted to CT were considered healthy controls. A median filter was applied to smoothen the data by reducing inherent variability. Three different machine learning (ML) models were defined to fit each algorithm which included the conventional features (containing daily lying, daily steps, and daily change derived from the accelerometer), slope features (containing features extracted from each variable in Conventional feature), or all features (3 simple features and 3 slope features). Random forest (RF), Naive Bayes (NB), Logistic Regression (LR), and Time series (ROCKET) were used as ML predictive approaches. For the classification of cows requiring CCT and TCT, ROCKET classifier performed better with accuracy (> 90%), ROC-AUC (> 74%), and F1 score (> 0.61) as compared to other algorithms. Slope features derived in this study increased the efficiency of algorithms as the better-performing models included All features explored. However, further classification of diseases into infectious and non-infectious events was not effective because none of the algorithms presented satisfactory model accuracy parameters. For the classification of observed cow locomotion scores into severely lame and moderately lame conditions, the ROCKET classifier demonstrated satisfactory accuracy (> 0.85), ROC-AUC (> 0.68), and F1 scores (> 0.44). We conclude that ML models using accelerometer data are helpful in the identification of lameness in cows but need further research to increase the granularity and accuracy of classification.

Use of Biochemical Tests and Machine Learning in the Search for Potential Diagnostic Biomarkers of COVID-19, HIV/AIDS, and Pulmonary Tuberculosis

This study aims to develop, validate, and evaluate machine learning algorithms for predicting the diagnosis of coronavirus disease (COVID-19), human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), pulmonary tuberculosis (TB), and HIV/TB co-infection. We also investigated potential biomarkers associated with the diagnosis. Data from biochemical and hematological tests of infected and controls were collected in a single general hospital, totalizing 6,418 patients. The discriminant analysis by partial least squares (PLS-DA) model had the highest performance in predicting the diagnosis of COVID-19, HIV/AIDS, TB, and HIV/TB co-infection with an accuracy of 94, 97, 95, and 96%, respectively. The biomarkers calcium, lactate dehydrogenase, red blood cells (RBC), white blood cells, neutrophils, basophils, eosinophils, hemoglobin, and hematocrit were associated with COVID-19. HIV infection was associated with mean corpuscular volume, platelets, neutrophils, and mean platelet volume. Red blood cell distribution width and urea were associated with infection by Mycobacterium tuberculosis. The following biomarkers were associated with HIV/TB co-infection: lymphocytes, RBC, hematocrit, hemoglobin, aspartate transaminase, alanine transaminase, and glycemia. The PLS-DA model can optimize COVID-19, HIV/AIDS, TB, and HIV/TB co-infection diagnostics. Some biomarkers were potential diagnostic indicators and could be evaluated during the screening of these diseases.

A Survey of Learning Criteria Going Beyond the Usual Risk

Virtually all machine learning tasks are characterized using some form of loss function, and “good performance” is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs. In this work, we survey and introduce a wide variety of non-traditional criteria used to design and evaluate machine learning algorithms, place the classical paradigm within the proper historical context, and propose a view of learning problems which emphasizes the question of “what makes for a desirable loss distribution?” in place of tacit use of the expected loss.

利用人工智能技术改进高校课程评估与个性化学习

With continuous advancements in education and the rapid proliferation of artificial intelligence technology, leveraging AI to improve university course evaluation and personalize learning experiences has garnered significant interest. This article aims to investigate the effective utilization of artificial intelligence technology in enhancing university course evaluation and achieving personalized learning outcomes. Firstly, it provides an overview of the current landscape of university course evaluation and personalized learning while highlighting the limitations of existing methodologies. It then delves into the application of artificial intelligence technology to address these challenges, discussing the utilization of machine learning algorithms for course evaluation, personalized recommendations, and student data analysis. In conclusion, the article summarizes the potential impact of artificial intelligence technology on university course evaluation and personalized learning.

B-146 Discriminative Index: A Novel Indicator for Evaluating Machine Learning Algorithms in Laboratory Medicine

Abstract Background In clinical scenarios, incorrect predictions by machine learning (ML) models are inevitable. One way to reduce misleading predictions is to avoid reporting predictions that fall within a predefined “gray zone”. This method improves predictive performance by reporting only the less uncertain cases. However, the cost-effectiveness of applying the “gray zone” rule in an ML model is unclear without massive computation. Thus, this study aims to propose a novel metric to evaluate the effectiveness of using gray zones and validate the metric in real-world ML models. Methods This study defined a statistical metric called the “discriminative index” (D-index) for evaluating the effectiveness of gray zones. To calculate the D-index, the predictive outcomes of the ML model are first transformed into two probability distributions based on the truth labels (e.g., positive or negative). The D-index is then derived from the kurtosis of these two distributions. To validate the metric, we applied the D-index to three different antibiotic susceptibility-predicting ML models (namely, convolutional neural network (CNN), random forest (RF), and XGBoost (XGB)) based on mass spectrometry data. We assessed the performance and unpredicted case numbers of each model with different gray zones and correlated the results with the proposed D-index. Results The D-index values for the CNN, XGB, and RF models were 4.36, 0.38, and −1.66, respectively. When applying the “gray zone” rule to achieve 90% area under the receiver operating characteristic, the CNN, XGB and RF models retained up to 90%, 68%, and 62% of total cases, respectively. A higher D-index value indicates a more effective application of the gray zone rule. Conclusion The D-index is a simple and statistically insightful metric for evaluating the cost-effectiveness of applying gray zone rule in an ML model. This metric has been validated in three mass spectrometry-based predictive models and has shown promising results. The D-index can be a useful tool for comparing and applying different ML algorithms.

Evaluating machine learning algorithms for predicting compressive strength of concrete with mineral admixture using long short-term memory (LSTM) Technique

Evaluating machine learning algorithms for predicting compressive strength of concrete with mineral admixture using long short-term memory (LSTM) Technique