Performance Evaluation Of Algorithms Research Articles

Performance evaluation of lightweight pattern recognition algorithms for portable environmental monitoring electronic noses

Performance evaluation of lightweight pattern recognition algorithms for portable environmental monitoring electronic noses

An automated algorithm for quantitative morphometry of thoracic and lumbar vertebral bodies in lateral radiographs

Abstract This exploratory study developed and evaluated an AI-based algorithm for quantitative morphometry to assess vertebral body deformities indicative of fractures. To achieve this, 709 radiographs from 355 cases were utilized for algorithm development and performance evaluation. The proposed algorithm integrates a first-stage AI model to identify the positions of thoracic and lumber vertebral bodies in lateral radiographs and a second-stage AI model to annotate six landmarks for calculating vertebral body height ratios (C/A, C/P and A/P). The first-stage AI model achieved a sensitivity of 97.6%, a precision of 95.1%, and an average false-positive ratio of 0.43 per image for vertebral body detection. In the second stage, the algorithm’s performance was evaluated using an independent dataset of vertebrae annotated by two spine surgeons and one radiologist. The average landmark errors ranged from 2.9% to 3.3% on the X-axis and 2.9% to 4.0% on the Y-axis, with errors increasing in more severely collapsed vertebrae, particularly at central landmarks. Spearman’s correlation coefficients were 0.519–0.589 for C/A, 0.558–0.647 for C/P, and 0.735–0.770 for A/P, comparable to correlations observed among human evaluators. Bland–Altman analysis revealed systematic bias in some cases, indicating that the algorithm underestimated anterior and central height collapse in deformed vertebrae. However, the mean differences and limits of agreement between the algorithm and external evaluators were similar to those among the evaluators. Additionally, the algorithm processed each image within 10 seconds. These findings suggest that the algorithm performs comparably to human evaluators, demonstrating sufficient accuracy for clinical use. The proposed approach has the potential to enhance patient care by being widely adopted in clinical settings.

Systematic Literature Review of Optimization Algorithms for P||Cmax Problem

In the era of open data and open science, it is important that, before announcing their new results, authors consider all previous studies and ensure that they have competitive material worth publishing. To save time, it is popular to replace the exhaustive search of online databases with the utilization of generative Artificial Intelligence (AI). However, especially for problems in niche domains, generative AI results may not be precise enough and sometimes can even be misleading. A typical example is P||Cmax, an important scheduling problem studied mainly in a wider context of parallel machine scheduling. As there is an uncovered symmetry between P||Cmax and other similar optimization problems, it is not easy for generative AI tools to include all relevant results into search. Therefore, to provide the necessary background data to support researchers and generative AI learning, we critically discuss comparisons between algorithms for P||Cmax that have been presented in the literature. Thus, we summarize and categorize the “state-of-the-art” methods, benchmark test instances, and compare methodologies, all over a long time period. We aim to establish a framework for fair performance evaluation of algorithms for P||Cmax, and according to the presented systematic literature review, we uncovered that it does not exist. We believe that this framework could be of wider importance, as the identified principles apply to a plethora of combinatorial optimization problems.

Diagnosis of Malignant Endometrial Lesions from Ultrasound Radiomics Features and Clinical Variables Using Machine Learning Methods

Background: The prognosis of patients with early diagnosis of malignant endometrial lesions is good. We aimed to identify benign and malignant lesions in endometrial tissue, explore effective methods for assisting diagnosis, and improve the accuracy and precision of identifying endometrial lesions. Methods: 1142 ultrasound radiomics and 18 clinical features from 1254 patients were analyzed, from which 36 features were selected for machine learning. We sketched the region of interest (ROI) of the abnormalities on the ultrasound images. Then, the radiomics features were extracted. Six common machine learning algorithms, including Support Vector Machine (SVM), Logistic Regression, Decision Tree, Random Forest, Gradient Boosting Tree, and k-Nearest Neighbors, were employed to identify benign and malignant changes in endometrial tissue. Cross-validation and grid search techniques for hyperparameter tuning were utilized to obtain the best model performance. Accuracy, precision, sensitivity, F1-scores, area under the curve (AUC), cross-validation average score and bootstrap average accuracy were also used to evaluate algorithm performance, classification accuracy, and generalization capability. Results: We combined 21 ultrasound characteristics and 15 clinical characteristics to develop and validate six common machine learning algorithms. After internal validation, the best models were the Random Forest models, with accuracy of 89%, precision of 93%, sensitivity of 97%, F1-score of 95%, and AUC of 95%, as well as a 10-fold cross-validation average score of 95% and bootstrap average accuracy of 94%, implying flawless classification in the test set. Conclusions: We identified the clinical and ultrasound features in the early diagnosis of benign or malignant lesions in endometrial tissue. And Random Forest model algorithms have demonstrated excellent performance in identifying benign and malignant changes in endometrial tissue. This is significant for enhancing early diagnostic accuracy and improving treatment outcomes and long-term management.

Empirical Evaluation of Algorithm Performance: Addressing Execution Time Measurement Challenges

Empirical Evaluation of Algorithm Performance: Addressing Execution Time Measurement Challenges

Performance evaluation and comparative analysis of different machine learning algorithms in predicting postnatal care utilization: Evidence from the ethiopian demographic and health survey 2016.

Postnatal care refers to the support provided to mothers and their newborns immediately after childbirth and during the first six weeks of life, a period when most maternal and neonatal deaths occur. In the 30 countries studied, nearly 40 percent of women did not receive a postpartum care check-up. This research aims to evaluate and compare the effectiveness of machine learning algorithms in predicting postnatal care utilization in Ethiopia and to identify the key factors involved. The study employs machine learning techniques to analyse secondary data from the 2016 Ethiopian Demographic and Health Survey. It aims to predict postnatal care utilization and identify key predictors via Python software, applying fifteen machine-learning algorithms to a sample of 7,193 women. Feature importance techniques were used to select the top predictors. The models' effectiveness was evaluated using sensitivity, specificity, F1 score, precision, accuracy, and area under the curve. Among the four experiments, tenfold cross-validation with balancing using Synthetic Minority Over-sampling Technique was outperformed. From fifteen models, the MLP Classifier (f1 score = 0.9548, AUC = 0.99), Random Forest Classifier (f1 score = 0.9543, AUC = 0.98), and Bagging Classifier (f1 score = 0.9498, AUC = 0.98) performed excellently, with a strong ability to differentiate between classes. The Region, residence, maternal education, religion, wealth index, health insurance status, and place of delivery are identified as contributing factors that predict postnatal care utilization. This study assessed machine learning models for forecasting postnatal care usage. Ten-fold cross-validation with Synthetic Minority Oversampling Technique produced the best results, emphasizing the significance of addressing class imbalance in healthcare datasets. This approach enhances the accuracy and dependability of predictive models. Key findings reveal regional and socioeconomic factors influencing PNC utilization, which can guide targeted initiatives to improve postnatal care utilization and ultimately enhance maternal and child health.

Diagnosis of Demagnetization Faults in PMSGs Based on Performance Evaluation of Machine Learning Algorithms through Flux and Current Signals

Diagnosis of Demagnetization Faults in PMSGs Based on Performance Evaluation of Machine Learning Algorithms through Flux and Current Signals

Modelling of Neutrosophic Set-Based k-Nearest Neighbors Classifier for Virus Pneumonia and COVID-19 Recognition

COVID19 otherwise called Severe Acute Respiratory Syndrome Corona virus-2 is an infectious illness. Another transmittable infection called Pneumonia is mainly attributable to infection because of bacteria in the alveoli of the lungs. Once a diseased lung tissue has infection, it elevates excretion in it. Specialists conduct health examinations and identify the patient through ultrasound, biopsy, or Chest X-ray of lungs to identify whether the patient has these diseases. Incorrect treatment, misdiagnosis, and if the disease was disregarded will result in the fatality. The development of Deep Learning and neutrosophic set (NS) supports the decision-making procedure of professionals to identify patients with this disease. NS is a prolongation of the fuzzy set and classical theories. The NS determines three memberships such as T, I and F. T, I, and F display the degree of truth, the false, and the indeterminacy membership, correspondingly. This enables a more nuanced representation of contradiction, uncertainty, and ambiguity within the dataset, allowing superior handling of imprecise and complex data. This study develops a new Deep learning with Neutrosophic Set-Based k-Nearest Neighbors Classifier for disease detection (DLNSKNN-DD) technique. The major purpose of the DLNSKNN-DD method is to identify the existence of virus pneumonia and COVID-19. In the DLNSKNN-DD technique, the feature extraction from the medical images is carried out by residual network (ResNet50v2). Moreover, the parameter tuning of the ResNetv2 model is done using Adadelta optimizer. The DLNSKNN-DD technique exploits NSKNN model for classification purposes. The performance evaluation of the DLNSKNN-DD algorithm can be assessed on medicinal image dataset. The experimental outcomes underlined the effectual recognition results of the DLNSKNN-DD technique on the identification of diseases

Performance Evaluation of Latest Meta-Heuristic Algorithms in Finding Optimum Value of Mathematical Functions and Problems

Performance Evaluation of Latest Meta-Heuristic Algorithms in Finding Optimum Value of Mathematical Functions and Problems

Comprehensive Performance Evaluation of Machine Learning Algorithms for detecting DDoS attacks in SDN

<div align="center">Software-Defined Networking (SDN) revolutionizes networking by separating control logic and data forwarding, enhancing security against threats like Distributed Denial of Service (DDoS) attacks. These attacks flood control plane bandwidth, causing SDN network failures. Recent studies emphasize the efficacy of machine learning and statistical approaches in identifying and mitigating these security risks. However, there has been a lack of focus on employing ensembling techniques, amalgamating diverse machine learning models, selecting pertinent features, and utilizing oversampling techniques to balance categorical data. Our study evaluates 20 machine-learning models, emphasizing feature engineering and addressing class imbalance using Synthetic Minority Oversampling TEchnique (SMOTE). The results indicate that Ensemble methods such as LGBM Classifier, Random Forest Classifier, XGB Classifier, Decision Tree Classifier attained near-perfect scores (almost 100%) across all metrics, suggesting potential overfitting. Conversely, models like AdaBoost Classifier, K-Neighbors Classifier, and SVC exhibited slightly lower (99%) but realistic performance, underscoring the intricacy of accurate prediction in cybersecurity. Simpler models, including Logistic Regression, Linear Discriminant Analysis, and Gaussian Naive Bayes, demonstrated moderate to low accuracy, approximately around 70%. These findings stress the imperative need for a nuanced approach in the selection and fine-tuning of machine learning models to ensure effective DDoS detection in SDN environments.</div>

Techno-Economic and Environmental Analysis of an Off-grid Hybrid Renewable Energy System for Rural Electrification

Rural electrification is key to socio-economic development in developing countries like Nigeria. However, extending the national grid to remote rural areas is expensive and time consuming, and the traditional power supply method also involves utilization of a standalone renewable energy source such as Solar which have their associated drawbacks due to their unreliability in nature. Hence, this research carried out a techno-economic and environmental analysis of an off-grid hybrid renewable energy system for rural electrification using Kepler Optimization Algorithm (KOA). Feasibility study on electricity and hourly load demand assessment of Alayin village was conducted and village load profile was estimated. Mathematical modeling of each hybrid RES was formulated. A KOA technique was employed to carry out optimal sizing and check the cost efficiency of BG/ PHES/ Battery, PV/PHES/Battery, and the hybrid RES (PV/BG/PHES/Battery). Simulation of the model was done using MATLAB R2021a. The value obtained was validated with Gravitational Search Algorithm (GSA) for performance evaluation using Levelized Cost of Energy (LCOE), Loss of Power Supply Probability (LPSP) and CO2 reduction in manure management as performance metrics. The results of the analysis showed an appreciable reduction on the LCOE and CO2 with high reliability using KOA-hybrid RES compared with GSA-hybrid RES.

Bioinspired Algorithm for Performance Evaluation of Biopolymerized Expansive Subgrade Soil Blended with Industrial Waste Additive

Bioinspired Algorithm for Performance Evaluation of Biopolymerized Expansive Subgrade Soil Blended with Industrial Waste Additive

Implementation and Performance Evaluation of Quantum Machine Learning Algorithms for Binary Classification

Implementation and Performance Evaluation of Quantum Machine Learning Algorithms for Binary Classification

Comparison of the performance of classical and quantum machine‐learning methods on the detection of sugar beet Cercospora leaf disease

AbstractImage processing and machine‐learning (ML) techniques are essential for the detection of diseases and pests in plants. This study explored the application of quantum ML (QML) algorithms for the early detection of Cercospora beticola leaf disease in sugar beet, which causes significant impact on global sugar production. Using a dataset of 1065 images (739 diseased and 326 healthy), we extracted 70 ML statistical features, including 10 from the grey‐level co‐occurrence matrix (GLCM) and 60 colour‐related features. Performance evaluations of classical ML algorithms, such as random forest (RF; 91.95% accuracy) and extreme gradient boosting (91.95% accuracy), demonstrated strong results compared to quantum approaches. Notably, the quantum support vector classifier (QSVC) achieved an accuracy of 85% with perfect recall of 1.00, while the variational quantum classifier (VQC) recorded an accuracy of 88.73%. Dimensionality reduction via principal component analysis reduced features from 70 to 5, enabling effective classification with competitive results: ML (RF) 91.41%, VQC with limited‐memory Broyden–Fletcher–Goldfarb–Shanno with box constraints (L_BFGS_B) 88.73% and QSVC 85%. These findings highlight the potential of QML algorithms in improving agricultural disease identification and aiding in the advancement of more efficient, sustainable farming techniques.

Employees’ Reactions to Algorithmic Performance Evaluation: Threat of Evaluation Bias and Objectivity

ABSTRACT Although algorithms are increasingly used for performance evaluation purposes, prior algorithm research shows that individuals tend to trust human decisions more than algorithmic decisions in a subjective domain such as performance evaluation that requires human skills. I conduct an experiment to examine whether the effect of evaluator type (algorithm versus human supervisor) on employees’ perceived trustworthiness of performance evaluations hinges on evaluation bias (positive versus negative) and further investigate whether this interaction effect is mediated by perceived objectivity. I find that employees’ perceived trustworthiness shifts from human evaluations to algorithmic evaluations when negative evaluation bias exists. Further, this reduced algorithm aversion is stronger and significant for people who are vulnerable to evaluation bias because people perceive algorithms as making objective evaluation decisions. Overall, results of this study suggest that the threat of evaluation bias reduces algorithm aversion because of the higher perceived objectivity of algorithmic decisions. Data Availability: Data are available upon request.

Inconsistency among evaluation metrics in link prediction.

Link prediction is a paradigmatic and challenging problem in network science, which aims to predict missing links, future links, and temporal links based on known topology. Along with the increasing number of link prediction algorithms, a critical yet previously ignored risk is that the evaluation metrics for algorithm performance are usually chosen at will. This paper implements extensive experiments on hundreds of real networks and 26 well-known algorithms, revealing significant inconsistency among evaluation metrics, namely different metrics probably produce remarkably different rankings of algorithms. Therefore, we conclude that any single metric cannot comprehensively or credibly evaluate algorithm performance. In terms of information content, we suggest the usage of at least two metrics: one is the area under the receiver operating characteristic curve, and the other is one of the following three candidates, say the area under the precision-recall curve, the area under the precision curve, and the normalized discounted cumulative gain. When the data are imbalanced, say the number of negative samples significantly outweighs the number of positive samples, the area under the generalized Receiver Operating Characteristic curve should also be used. In addition, as we have proved the essential equivalence of threshold-dependent metrics, if in a link prediction task, some specific thresholds are meaningful, we can consider any one threshold-dependent metric with those thresholds. This work completes a missing part in the landscape of link prediction, and provides a starting point toward a well-accepted criterion or standard to select proper evaluation metrics for link prediction.

Development and performance evaluation of an indoor thermal environment control algorithm incorporating MET estimation model with object detection

As people increasingly spend time indoors, the significance of the indoor environment in influencing occupant quality of life is becoming more pronounced. Traditionally, indoor environment control primarily relied on fixed temperature settings, which failed to accommodate the diverse circumstances of occupants. This approach limited the creation of a comfortable indoor environment and the enhancement of energy efficiency. Consequently, there is growing interest in occupant-centric control (OCC), which integrates metabolic rate (MET) information, which is a critical factor in determining the thermal sensation of occupants. Previously, a method was developed to estimate MET by classifying occupant poses and detecting the objects they interact with from indoor images. This study aims to develop and experimentally validate an indoor thermal environment control algorithm (ITEC-algorithm) using the MET estimation model and assess its effectiveness and applicability in real building environments.The performance evaluation revealed that the ITEC-algorithm significantly enhanced the comfort ratios, achieving improvements of up to 59% compared to the fixed temperature control and 28% compared to the control methods that only used the pose classification model for MET estimation. The energy consumption varied depending on the activity and control method, with a reduction of up to 88% compared to fixed temperature control. These results indicate that thermal comfort can be enhanced while minimizing unnecessary energy consumption by incorporating the MET of the occupants. Consequently, it has been confirmed that the ITEC-algorithm effectively improves thermal comfort by managing the MET of various occupants.

A Mountain Gazelle Optimization (MGO) for Enhancing the Deep Learning Performance in Various Operating Systems

This study introduces a novel optimization framework that assesses and enhances deep learning algorithm performance across autonomous car operating systems. The framework generates synthetic performance measures, including inference time, memory use, CPU/GPU utilization, and accuracy, to evaluate algorithm performance. By employing the Mountain Gazelle Optimizer (MGO), the study identifies the best deep learning algorithm and operating system setup for accuracy and resource efficiency. The proposed methodology normalizes performance indicators, defines a fitness function to assist optimization, and then iterates through numerous configurations to discover the optimum option. Extensive trials and scenario comparisons validate the effectiveness of our approach. Computational efficiency and accuracy improved significantly, revealing the ideal autonomous car system performance combinations. This study not only enhances deep learning optimization but also provides practical instructions for building robust autonomous car systems in varied operating contexts, thereby informing the development of future autonomous vehicles and offering actionable insights for developers and researchers in the field.

Using Online Algorithms to Solve NP-Hard Problems

This paper explores the application of online algorithms to tackle NP-hard problems, a class of computational challenges characterized by their intractability and wide-ranging real-world implications. Unlike traditional offline algorithms that have access to complete input data, online algorithms make decisions sequentially, often under constraints of incomplete information. We investigate various strategies, including greedy approaches, randomization, and competitive analysis, to assess their effectiveness in solving NP-hard problems such as the Traveling Salesman Problem, the Knapsack Problem, and the Set Cover Problem. Our analysis highlights the trade-offs between solution quality and computational efficiency, emphasizing the significance of the competitive ratio in evaluating algorithm performance. Additionally, we discuss the practical applications of online algorithms in dynamic environments, such as real-time systems and streaming data processing. Through a comprehensive review of existing literature and novel algorithmic designs, we aim to provide insights into the viability of online algorithms as a robust framework for addressing NP-hard problems in scenarios where immediate decision-making is crucial. The findings underscore the potential for future research to enhance these algorithms, making them increasingly applicable in complex, real-world contexts.

Performance evaluation of machine learning algorithms in predicting machining responses of superalloys

This study explores the application of machine learning algorithms—gene expression programming (GEP), adaptive neuro-fuzzy inference system (ANFIS), and artificial neural networks (ANN)—to predict machining responses during the milling of Inconel 690, a superalloy known for its exceptional mechanical properties and oxidation resistance. Machining Inconel 690 presents significant challenges due to its toughness and work-hardening tendencies, which can lead to rapid tool wear and poor surface finish. Traditional optimization methods often rely on empirical models and trial-and-error approaches, which are time-consuming and costly. In contrast, machine learning techniques can effectively model complex, nonlinear relationships between machining parameters and performance outcomes, such as surface roughness, cutting force, and cutting temperature. This study employs statistical metrics, including Root mean square error (RMSE), coefficient of determination (R2), and mean absolute percentage error (MAPE), to determine the predictive performance of the models. The results show that the GEP model achieved an R2 ranging from 0.944 572 to 0.992 999, with an RMSE between 0.015 527% and 0.694 523% and a MAPE ranging from 1.452 397% to 4.947 892%. ANFIS and ANN also demonstrated strong predictive capabilities, although GEP outperformed them. The importance of this study lies in its demonstration of advanced AI techniques as effective tools for optimizing machining processes, ultimately contributing to improved efficiency and quality in manufacturing superalloys.