Performance Evaluation Of Algorithms Research Articles

Work engagement of online car-hailing drivers: the effects of platforms' algorithmic management

PurposeAlgorithms are widely used to manage various activities in the gig economy. Online car-hailing platforms, such as Uber and Lyft, are exemplary embodiments of such algorithmic management, where drivers are managed by algorithms for task allocation, work monitoring and performance evaluation. Despite employing substantially, the platforms face the challenge of maintaining and fostering drivers' work engagement. Thus, this study aims to examine how the algorithmic management of online car-hailing platforms affects drivers' work engagement.Design/methodology/approachDrawing on the transactional theory of stress, the authors examined the effects of algorithmic monitoring and fairness on online car-hailing drivers' work engagement and revealed the mediation effects of challenge-hindrance appraisals. Based on survey data collected from 364 drivers, the authors' hypotheses were examined using partial least squares structural equation modeling (PLS-SEM). The authors also applied path comparison analyses to further compare the effects of algorithmic monitoring and fairness on the two types of appraisals.FindingsThis study finds that online car-hailing drivers' challenge-hindrance appraisals mediate the relationship between algorithmic management characteristics and work engagement. Algorithmic monitoring positively affects both challenge and hindrance appraisals in online car-hailing drivers. However, algorithmic fairness promotes challenge appraisal and reduces hindrance appraisal. Consequently, challenge and hindrance appraisals lead to higher and lower work engagement, respectively. Further, the additional path comparison analysis showed that the hindering effect of algorithmic monitoring exceeds its challenging effect, and the challenge-promoting effect of algorithmic fairness is greater than the algorithm's hindrance-reducing effect.Originality/valueThis paper reveals the underlying mechanisms concerning how algorithmic monitoring and fairness affect online car-hailing drivers' work engagement and fills the gap in the research on algorithmic management in the context of online car-hailing platforms. The authors' findings also provide practical guidance for online car-hailing platforms on how to improve the platforms' algorithmic management systems.

Performance evaluation of the artificial hummingbird algorithm in the problem of structural damage identification

Recently, Structural Health Monitoring (SHM) has become a critical component of the maintenance and safety of lifeline infrastructures such as dams, skyscrapers, and bridges, thanks to its ability to detect structural failures at the early stages. In this paper, we evaluate the performance of the SHM damage identification tool using a novel metaheuristic algorithm called the Artificial Hummingbird Algorithm (AHA). The proposed approach is evaluated by two case studies of different bridge structures in Vietnam with different simulated damage scenarios. The potency of the AHA is compared against the other well-known metaheuristic algorithms such as Cuckoo Search (CS), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Teaching-Learning Based Optimization (TLBO). The results show that the AHA performs much better than the other algorithms in terms of accuracy and computational cost. The application of AHA can help to reduce the cost and time required for structural maintenance significantly, as well as improve the lifecycle of the structure.

Deep learning algorithm performance evaluation in detection and classification of liver disease using CT images.

To diagnose the liver diseases computed tomography images are used. Most of the time even experienced radiologists find it very tough to note the type, size, and severity of the tumor from computed tomography images due to various complexities involved around the liver. In recent years it is very much essential to develop a computer-assisted imaging technique to diagnose liver disease in turn which improves the diagnosis of a doctor. This paper explains a novel deep learning model for detecting a liver disease tumor and its classification. Tumor from computed tomography images has been classified between Metastasis and Cholangiocarcinoma. We demonstrate that our model predominantly performs very well concerning the accuracy, dice similarity coefficient, and specificity parameters compared to well-known existing algorithms, and adapts very well for different datasets. A dice similarity coefficient value of 98.59% indicates the supremacy of the model.

Deep learning assessment compared to radiologist reporting for metastatic spinal cord compression on CT.

Metastatic spinal cord compression (MSCC) is a disastrous complication of advanced malignancy. A deep learning (DL) algorithm for MSCC classification on CT could expedite timely diagnosis. In this study, we externally test a DL algorithm for MSCC classification on CT and compare with radiologist assessment. Retrospective collection of CT and corresponding MRI from patients with suspected MSCC was conducted from September 2007 to September 2020. Exclusion criteria were scans with instrumentation, no intravenous contrast, motion artefacts and non-thoracic coverage. Internal CT dataset split was 84% for training/validation and 16% for testing. An external test set was also utilised. Internal training/validation sets were labelled by radiologists with spine imaging specialization (6 and 11-years post-board certification) and were used to further develop a DL algorithm for MSCC classification. The spine imaging specialist (11-years expertise) labelled the test sets (reference standard). For evaluation of DL algorithm performance, internal and external test data were independently reviewed by four radiologists: two spine specialists (Rad1 and Rad2, 7 and 5-years post-board certification, respectively) and two oncological imaging specialists (Rad3 and Rad4, 3 and 5-years post-board certification, respectively). DL model performance was also compared against the CT report issued by the radiologist in a real clinical setting. Inter-rater agreement (Gwet's kappa) and sensitivity/specificity/AUCs were calculated. Overall, 420 CT scans were evaluated (225 patients, mean age=60 ± 11.9[SD]); 354(84%) CTs for training/validation and 66(16%) CTs for internal testing. The DL algorithm showed high inter-rater agreement for three-class MSCC grading with kappas of 0.872 (p<0.001) and 0.844 (p<0.001) on internal and external testing, respectively. On internal testing DL algorithm inter-rater agreement (κ=0.872) was superior to Rad 2 (κ=0.795) and Rad 3 (κ=0.724) (both p<0.001). DL algorithm kappa of 0.844 on external testing was superior to Rad 3 (κ=0.721) (p<0.001). CT report classification of high-grade MSCC disease was poor with only slight inter-rater agreement (κ=0.027) and low sensitivity (44.0), relative to the DL algorithm with almost-perfect inter-rater agreement (κ=0.813) and high sensitivity (94.0) (p<0.001). Deep learning algorithm for metastatic spinal cord compression on CT showed superior performance to the CT report issued by experienced radiologists and could aid earlier diagnosis.

Performance Evaluation of Machine Learning-Based Algorithms to Predict the Early Childhood Development Among Under Five Children in Bangladesh

In this research, an effort has been made to apply a number of classifiers to predict Early Child Development (ECD) in the context of Bangladesh using the Bangladesh multiple indicator cluster survey, 2019 data set (i.e., to evaluate which sort of algorithm best identifies ECDI). To predict the ECD, nine well-known machine learning algorithms were applied, including Linear Regression (LR), Random Forest (RF), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Naïve Bayes (NB), Least Absolute Shrinkage and Selection Operation (LASSO), Classification Trees (CT), AdaBoost and Neural Network (NN). Children aged 48-59 months who were female, attending early education, reading three or more children's books, having playthings, having normal nutritional status, and were not disabled had a higher percentage of completing at least three childhood development domains, according to the bivariate analysis results. We found several performance parameters for the classification of early childhood development, including the following: Accuracy (LR) = 67.87%, AUC (LR) = 67.49%; Accuracy (RF) = 67.23%, AUC (RF) = 67.19%; Accuracy (SVM) = 67.37%, AUC (SVM) = 67.64%; Accuracy (NB) = 67.55%, AUC (NB) = 66.80%; Accuracy (LASSO) = 68.04%, AUC (LASSO) = 67.75. Based on the results of this investigation, LASSO regression predicts the ECD in Bangladeshi children moderately better than any other machine learning method utilized in this study.

Performance Evaluation of Machine Learning Algorithms applied in SD-VANET for Efficient Transmission of Multimedia Information

Performance Evaluation of Machine Learning Algorithms applied in SD-VANET for Efficient Transmission of Multimedia Information

Screening for extranodal extension in HPV-associated oropharyngeal carcinoma: evaluation of a CT-based deep learning algorithm in patient data from a multicentre, randomised de-escalation trial

Pretreatment identification of pathological extranodal extension (ENE) would guide therapy de-escalation strategies for in human papillomavirus (HPV)-associated oropharyngeal carcinoma but is diagnostically challenging. ECOG-ACRIN Cancer Research Group E3311 was a multicentre trial wherein patients with HPV-associated oropharyngeal carcinoma were treated surgically and assigned to a pathological risk-based adjuvant strategy of observation, radiation, or concurrent chemoradiation. Despite protocol exclusion of patients with overt radiographic ENE, more than 30% had pathological ENE and required postoperative chemoradiation. We aimed to evaluate a CT-based deep learning algorithm for prediction of ENE in E3311, a diagnostically challenging cohort wherein algorithm use would be impactful in guiding decision-making. For this retrospective evaluation of deep learning algorithm performance, we obtained pretreatment CTs and corresponding surgical pathology reports from the multicentre, randomised de-escalation trial E3311. All enrolled patients on E3311 required pretreatment and diagnostic head and neck imaging; patients with radiographically overt ENE were excluded per study protocol. The lymph node with largest short-axis diameter and up to two additional nodes were segmented on each scan and annotated for ENE per pathology reports. Deep learning algorithm performance for ENE prediction was compared with four board-certified head and neck radiologists. The primary endpoint was the area under the curve (AUC) of the receiver operating characteristic. From 178 collected scans, 313 nodes were annotated: 71 (23%) with ENE in general, 39 (13%) with ENE larger than 1 mm ENE. The deep learning algorithm AUC for ENE classification was 0·86 (95% CI 0·82-0·90), outperforming all readers (p<0·0001 for each). Among radiologists, there was high variability in specificity (43-86%) and sensitivity (45-96%) with poor inter-reader agreement (κ 0·32). Matching the algorithm specificity to that of the reader with highest AUC (R2, false positive rate 22%) yielded improved sensitivity to 75% (+ 13%). Setting the algorithm false positive rate to 30% yielded 90% sensitivity. The algorithm showed improved performance compared with radiologists for ENE larger than 1 mm (p<0·0001) and in nodes with short-axis diameter 1 cm or larger. The deep learning algorithm outperformed experts in predicting pathological ENE on a challenging cohort of patients with HPV-associated oropharyngeal carcinoma from a randomised clinical trial. Deep learning algorithms should be evaluated prospectively as a treatment selection tool. ECOG-ACRIN Cancer Research Group and the National Cancer Institute of the US National Institutes of Health.

RETRACTED ARTICLE: Database task processing optimization based on performance evaluation and machine learning algorithm

RETRACTED ARTICLE: Database task processing optimization based on performance evaluation and machine learning algorithm

An Insight on Optimization Techniques for Uncertain and Reliable Routing in Wireless Body Area Networks

In recent times, Wireless Body Area Networks a subsection of Wireless Sensor Network is a promising technology for future healthcare realm with cutting-edge technologies that can assist healthcare professionals like doctors, nurses and biomedical engineers. Machine Learning and Internet of Things enabled medical big data is the future of healthcare sector and Medical Technology-based industries leading to applications in other sectors such as fitness tracking for commercial purposes, Sportsperson health monitoring to track their day-to-day activities and wearable devices for critical and emergency care. This comprehensive review article addresses Wireless Body Area Network state-of-art and the dependence of Optimization Techniques and Meta-heuristic Algorithms for an efficient routing path between two sensor nodes: source node and destination node and it plays an effective role in optimizing the network parameters such as radio range, energy consumption, throughput, data aggregation, clustering and routing. Designing of energy-efficient routing for wireless body are network is such a challenging task due to uncertainty in dynamic network topology, energy constraints and limited power consumption. Optimization Techniques can help the researchers to achieve the drawbacks mentioned above and energy-efficiency of the network can be improved. In this article, we focus majorly on the efficiency of usage of optimization algorithms for Wireless Body Area Network routing mechanisms and a summary of its earlier studies during 2012-2023 epoch. Genetic Algorithm, Particle Swarm Optimization, Ant Colony Optimization, Artificial Bee Colony and Firefly Optimization algorithms were discussed on achieving local optima for better results through optimization. This article provides an insight into existing gaps and further modifications to the researchers in WBAN that can motivate them to propose new ideas for reliable solutions. Performance comparison and evaluation of different bio-inspired optimization algorithms has been discussed for further improvement in optimized routing algorithms.

Performance Evaluation of Triangular Number Sequence Backoff Algorithm for Constrained Application Protocol

In the of the Internet of Things (IoT) realm, congestion is considered a serious issue affecting network throughput due to the requirement of multiple nodes for message exchange. With free-space optical communications, which can help send a message wirelessly, congestion control mechanisms nowadays depend upon the carrier sense of multiple access with collision avoidance (CSMA/CA), using the so-called Backoff Algorithm (BA). These algorithms,

Engineering applications of multi-objective evolutionary algorithms: A test suite of box-constrained real-world problems

Traditionally, the performance of novel multi-objective optimization algorithms is evaluated on artificial test problems, which are constructed under the consideration of some features observed in real-world problems. However, artificial test problems do not possess all the properties that real-world applications have. Therefore, the evaluation of algorithms on artificial test problems can overestimate the conclusions of an algorithm regarding its performance in real-world applications. In this respect, this paper presents a collection of multi-objective real-world problems taken from different disciplines. The multi-objective problems are suggested to complement the performance evaluation of evolutionary algorithms considering real-life applications without the need to be an expert on the concerned disciplines in which they lie. This study analyzes the conflict between objectives from the correlation point of view of each real-world problem. Notably, this investigation inquires on the real-world multi-objective problems regarding the Pareto front shapes, which have been one of the main difficulties for selection mechanisms in multi-objective evolutionary algorithms. Additionally, the performance of nine state-of-the-art multi-objective evolutionary algorithms based on the main principles of evolutionary computation is analyzed. Although it is impractical to evaluate all the existing algorithms available in the multi-objective evolutionary community, this investigation gives insights into the performance of the main multi-objective approaches considered in this study.

Analisis Sentimen Wisatawan terhadap Kualitas Layanan Hotel dan Resort di Lombok Menggunakan SERVQUAL dan CRISP-DM

The era of digital transformation has sparked innovations in product and service marketing strategies in various sectors, one of which is the tourism sector. In the hospitality industry context, product marketing using website-based digital media allows consumers as hotel guests to review the products and services received. The Tripadvisor website is a digital marketing platform that provides review features for app users, especially consumers, to give ratings and reviews. This study aims to analyze the quality of hotel services using the Service Quality (SERVQUAL) framework based on the results of the classification of hotel guest sentiment data using the Naïve Bayes Classifier (NBC) and Support Vector Machine (SVM) algorithm by the stages of the Cross-Industry Standard Process for Data Mining (CRISP-DM). The CRISP-DM framework consists of six stages, namely: business understanding stage, data understanding stage, data preparation stage, modeling stage, evaluation stage, and deployment stage. The SERVQUAL framework consists of several dimensions: reliability dimension; responsiveness; assurance; empathy; tangibles. The review data that will be processed is the consumer review data of The Oberoi Beach Resort Lombok; Sheraton Senggigi Beach Resort; Sudamala Resort Sengiggi; Holiday Resort Lombok; Aston Sunset Beach Resort. The results of this study show that the SVM algorithm performs better than NBC, where the accuracy value is 98.57%, the precision value is 100%, the recall value is 97.14%, and the f-measure value is 98.54%. The AUC value is 100%, and the t-Test value is 98.6%. Unlike the case with the results of SVM's algorithm performance evaluation without using the SMOTE Upgrading Operator, where the accuracy value is 95.71%, the precision value is 95.71%, the recall value is 100%, and the f-measure value is 97.81%. In addition, the AUC value is 91.1%, and the t-Test value is 95.7%.

Performance Evaluation of Classification Algorithm for Movie Review Sentiment Analysis

The majority of the current research on sentiment analysis, which covers topics like political reviews, movie reviews, and product reviews, has developed quickly. The classification and clustering stage of sentiment analysis research involves a number of subjects. Some of them cover text classification comparison research and algorithm performance optimization. An intricate issue in sentiment analysis research is dealing with unstructured or semi-structured data. The sentiment analysis procedure and improving the efficacy of the classifier’s algorithm are both hampered by unstructured data. In order to manage unstructured data successfully and provide accurate and relevant information, unique strategies are required. The proposed classification model performance evaluation using Support Vector Machine, Naive Bayes, K-Nearest Neighbor, and Decision Tree is specifically covered in this paper. According to the study’s findings, SVM has an accuracy rate of 96% and Naive Bayes is 86%. While the decision tree’s gain accuracy is 78 percent and the kNN classification model’s gain accuracy is 78 percent, respectively. The test results demonstrate that SVM is superior to other classification models in terms of accuracy performance.

Performance evaluation of machine learning algorithms to assess soil erosion in Mediterranean farmland: A case‐study in Syria

AbstractThe development of new techniques, such as machine learning (ML), can provide better insight into the processes and drivers of soil erosion and runoff. However, the performance of these techniques to assess soil erosion in agricultural landscapes is poorly understood. The aim of this study was to evaluate the performance of four machine learning algorithms, generalized linear model (GLM), Random Forest (RF), elastic net regression (EN) and multiple adaptive regression splines (MARS), in predicting soil erosion and runoff in Syria. Soil erosion and runoff were measured on three experimental plots (2.25 m × 1.50 m × 0.50 m, 0.10 m depth in the soil), combined with three different slopes and land use types: RS (8%, olive), SS (12%, citrus), KS (20%, pomegranate). Both erosion and runoff were determined after rainfall events of &gt;10 mm between October 2019 and April 2020. Based on 24 effective rainfall events, the average soil erosion was 0.18 ± 0.14 kg m−2 per event in KS, 0.14 ± 0.11 kg m−2 per event in SS, and 0.12 ± 0.10 kg/m2 per event in RS. Regression analysis indicated strong relationship between the rainfalls and the runoff, the highest connection was recorded in the KS plot (r2 = 0.85; p &lt; 0.05 n = 24). The analysis of covariance indicated that only the runoff had a significant impact on soil erosion (p = 0.02) with a medium effect (ε2p = 0.26). However, the impacts of rainfall events and slope categories on soil erosion were limited (ε2p &lt; 0.01) and not significant (p &gt; 0.05). ML techniques were usually efficient in the prediction, the RF and MARS models were the most accurate: RF had the strongest correlation with the measured values (r = 0.85) with a low estimation error (0.06 kg m−2), but MARS's standard deviation (SD) was closer to the recorded values' SD. GLM and EN were the weakest predictor models. Modeled values of the slightest slope (8%) had the worst accuracies, and the predictions of the 12% slope were the best in all models. This study provides important insights into the usefulness of machine learning techniques and algorithms in predicting the rate of soil erosion and runoff in agricultural dominated landscapes. We highlighted that the RF and MARS algorithms were better predictors of soil erosion and runoff in the coastal region of Syria.

Exploring the impact of design criteria for reference sets on performance evaluation of signal detection algorithms: The case of drug-drug interactions.

To evaluate the impact of multiple design criteria for reference sets that are used to quantitatively assess the performance of pharmacovigilance signal detection algorithms (SDAs) for drug-drug interactions (DDIs). Starting from a large and diversified reference set for two-way DDIs, we generated custom-made reference sets of various sizes considering multiple design criteria (e.g., adverse event background prevalence). We assessed differences observed in the performance metrics of three SDAs when applied to FDA Adverse Event Reporting System (FAERS) data. For some design criteria, the impact on the performance metrics was neglectable for the different SDAs (e.g., theoretical evidence associated with positive controls), while others (e.g., restriction to designated medical events, event background prevalence) seemed to have opposing and effects of different sizes on the Area Under the Curve (AUC) and positive predictive value (PPV) estimates. The relative composition of reference sets can significantly impact the evaluation metrics, potentially altering the conclusions regarding which methodologies are perceived to perform best. We therefore need to carefully consider the selection of controls to avoid misinterpretation of signals triggered by confounding factors rather than true associations as well as adding biases to our evaluation by "favoring" some algorithms while penalizing others.

Performance Evaluation of Quantum-Based Machine Learning Algorithms for Cardiac Arrhythmia Classification.

The electrocardiogram (ECG) is the most common technique used to diagnose heart diseases. The electrical signals produced by the heart are recorded by chest electrodes and by the extremity electrodes placed on the limbs. Many diseases, such as arrhythmia, cardiomyopathy, coronary heart disease, and heart failure, can be diagnosed by examining ECG signals. The interpretation of these signals by experts may take a long time, and there may be differences between expert interpretations. Since technological developments are intertwined with the medical sciences, computer-assisted diagnostic methods have recently come forward. In computer science, machine learning techniques are often preferred for automatic detection. Quantum-based structures have emerged to increase the machine learning algorithm's speed and classification performance. In this study, a quantum-based machine learning algorithm is applied to classify heart rhythms. The ECG properties were converted to qubit structure using principal component analysis (PCA). The resulting qubits are classified using the quantum support vector machine (QSVM) algorithm. Quantum computer simulation over Qiskit was used for classification studies. Within the scope of experimental studies, comparisons between classical SVM and QSVM were made using different data amounts and qubit numbers. In the results of the analysis, classical SVM achieved 86.96% accuracy, and QSVM achieved 84.64% accuracy. Despite the fact that the entire dataset was not used due to various limitations, these successful performances were achieved. Classification of medical data such as that from ECG has shown that quantum-based machine learning frameworks perform well despite current resource constraints. In this respect, the study includes essential contributions to the use of quantum-based machine learning methods on signal data in medicine.

Performance Evaluation of Machine Learning Algorithms in Change Detection and Change Prediction of a Watershed’s Land Use and Land Cover

Performance Evaluation of Machine Learning Algorithms in Change Detection and Change Prediction of a Watershed’s Land Use and Land Cover

Longitudinal and lateral coupling vehicle stability controller designed based on piecewise T-S fuzzy model

Due to tire force coupling and load transfer coupling during the stability control process, the vehicle longitudinal and lateral dynamics shows particularly obvious nonlinear coupling characteristics, which will deteriorate the performance of the stability controller. This paper proposes a Vehicle Stability Controller considering the vehicle longitudinal and lateral coupling characteristics. First, a piecewise guidance method, including the Adaptive Linear Regression Algorithm and Dimensionless Performance Evaluation Weighting Function, is developed to ensure the Takagi-Sugeno fuzzy model piecewise approaches the nonlinear characteristics of the original system and obtains the multiple linear fuzzy subsystems. Then, an H∞ robust controller with dynamic output feedback is used in each fuzzy subsystem based on the identification result of the driving intention, and the Linear Matrix Inequality is applied to obtain the controller gains. Subsequently, the Parallel Distributed Compensation architecture is adopted to integrate the outputs of all fuzzy subsystems, and vehicle stability controller is realized by using a rule-based control distribution strategy. The proposed controller is evaluated by simulations and hardware in the loop tests, and the results demonstrate satisfactory longitudinal following ability and lateral stability.

Performance Evaluation of Classification Algorithms on Academic Performance of Postgraduate Students

Performance Evaluation of Classification Algorithms on Academic Performance of Postgraduate Students

Performance Evaluation of Machine Learning Algorithm in Various Datasets

Machine learning is one of the fast-growing areas of computer science, with far-reaching applications. There are several applications for machine learning. The most significant of which is supervised learning. Supervised learning is common in classification problems. In this study, frequently used twelve machine learning algorithms are considered: NB, LDA, LR, ANN, SVM, K-NN, HT, DT, C4.5, CART, RF and BB. We apply these algorithms on seven datasets. The main goal of this study was to evaluate the performance of the machine learning algorithms on both binary and multiple classification problems using a variety of performance metrics: accuracy, kappa statistic, precision, recall, specificity, F-measure, MAE, RMSE and MCC. Here, we found that RF algorithm proved to have the best performance in three out of seven datasets. But the other four algorithms: NN, NB, BB and LR also performed well.