Performance Evaluation Of Algorithms Research Articles

Performance evaluation of machine learning and Computer Coded Verbal Autopsy (CCVA) algorithms for cause of death determination: A comparative analysis of data from rural South Africa.

Computer Coded Verbal Autopsy (CCVA) algorithms are commonly used to determine the cause of death (CoD) from questionnaire responses extracted from verbal autopsies (VAs). However, they can only operate on structured data and cannot effectively harness information from unstructured VA narratives. Machine Learning (ML) algorithms have also been applied successfully in determining the CoD from VA narratives, allowing the use of auxiliary information that CCVA algorithms cannot directly utilize. However, most ML-based studies only use responses from the structured questionnaire, and the results lack generalisability and comparability across studies. We present a comparative performance evaluation of ML methods and CCVA algorithms on South African VA narratives data, using data from Agincourt Health and Demographic Surveillance Site (HDSS) with physicians' classifications as the gold standard. The data were collected from 1993 to 2015 and have 16,338 cases. The random forest and extreme gradient boosting classifiers outperformed the other classifiers on the combined dataset, attaining accuracy of 96% respectively, with significant statistical differences in algorithmic performance (p < 0.0001). All our models attained Area Under Receiver Operating Characteristics (AUROC) of greater than 0.884. The InterVA CCVA attained 83% Cause Specific Mortality Fraction accuracy and an Overall Chance-Corrected Concordance of 0.36. We demonstrate that ML models could accurately determine the cause of death from VA narratives. Additionally, through mortality trends and pattern analysis, we discovered that in the first decade of the civil registration system in South Africa, the average life expectancy was approximately 50 years. However, in the second decade, life expectancy significantly dropped, and the population was dying at a much younger average age of 40 years, mostly from the leading HIV related causes. Interestingly, in the third decade, we see a gradual improvement in life expectancy, possibly attributed to effective health intervention programmes. Through a structure and semantic analysis of narratives where experts disagree, we also demonstrate the most frequent terms of traditional healer consultations and visits. The comparative approach also makes this study a baseline that can be used for future research enforcing generalization and comparability. Future study will entail exploring deep learning models for CoD classification.

Video Compression Performance Evaluation Method in Transmission via a Low-Speed Channel

The paper presents an overview of the works devoted to video compression algorithms based on artificial intelligence methods. Two main directions for the development of such algorithms are identified - these are the development of modules for post-processing the results of the work of classical algorithms and the development of algorithms that completely replace existing video codecs. The problems of video compression algorithms performance evaluation criteria are considered. It was found that criteria based on the calculation of the standard deviation, such as PSNR, cannot be used to assess the quality of video data compression algorithms when operating under the influence of radio interference, due to the strong influence on the evaluation result of artifacts that inevitably arise as a result of the effects of this kind of interference. An alternative quality assessment method based on the semantic analysis of frames is proposed, which can be used to assess the noise immunity of video data transmission algorithms. The presented estimation method uses a frame-by-frame comparison of the original and restored video sequence, similar to PSNR. However, unlike the latter, it ignores artifacts that occur during exposure to radio interference, due to the use of semantic frame analysis, which consists in searching for objects in the image using approaches based on artificial intelligence. To compare the found objects on the original and reconstructed frames, an estimate is used based on the geometric position of the found objects in the image. After that, the data indicator is averaged for all frames of the video sequence in order to obtain the resulting similarity indicator of the restored video and the original.

Performance Evaluation of Lane Detection and Tracking Algorithm Based on Learning-Based Approach for Autonomous Vehicle

Disruptive technology, especially autonomous vehicles, is predicted to provide higher safety and reduce road traffic emissions. Lane detection and tracking are critical building blocks for developing autonomous or intelligent vehicles. This study presents a lane detecting algorithm for autonomous vehicles on different road pavements (structured and unstructured roads) to overcome challenges such as the low detection accuracy of lane detection and tracking. First, datasets for performance evaluation were created using an interpolation method. Second, a learning-based approach was used to create an algorithm using the steering angle, yaw angle, and sideslip angle as inputs for the adaptive controller. Finally, simulation tests for the lane recognition method were carried out by utilising a road driving video in Melbourne, Australia, and the BDD100K dataset created by the Berkeley DeepDrive Industrial Consortium. The mean detection accuracy ranges from 97% to 99%, and the detection time ranges from 20 to 22 ms under various road conditions with our proposed algorithm. This lane detection algorithm outperformed conventional techniques in terms of accuracy and processing time, as well as efficiency in lane detection and overcoming road interferences. The proposed algorithm will contribute to advancing the lane detection and tracking of intelligent-vehicle driving assistance and help further improve intelligent vehicle driving safety.

Performance Evaluation of Path Planning and Coordination Algorithms for Multiple UGVs in Smart Farm

Performance Evaluation of Path Planning and Coordination Algorithms for Multiple UGVs in Smart Farm

Retraction Note: Model and algorithm of innovation performance evaluation for coordination of supply and demand based on wireless sensor network

Retraction Note: Model and algorithm of innovation performance evaluation for coordination of supply and demand based on wireless sensor network

Spam Profile Detection on Facebook and Performance Evaluation of Machine Learning Algorithms

Spam Profile Detection on Facebook and Performance Evaluation of Machine Learning Algorithms

Performance Evaluation of Pre-Processing and Pre-Treatment Algorithm for Near-Infrared Spectroscopy Signals: Case Study pH of Intact Mango “Arumanis”

pH is one of the important physical parameters to characterize mango damage because it can indicate changes in the structure and chemical content of the fruit. Thus, the present work evaluated the possibility of NIRs as a rapid and non-destructive tool for measuring the pH properties of intact mango from the cultivar "Arumanis" (Mangifera indica L.) using several algorithms for pre-processing, pre-treatment, and prediction. Three different algorithm predictions, namely principal component regression (PCR), partial least squares regression (PLSR), and support vector machine regression (SVMR), were used and compared to predict the pH of mangos. A total of 16 pre-processing and pre-treatment algorithms are used to support algorithm prediction, and the results are also compared with the raw data spectra. The NIR spectral data used range from 1000 to 2500 nm. Algorithm performance will be evaluated using RMSE, error differences and concluded using RPD. The results show that the prediction of the PLSR algorithm can be performed with an RPD of 8.17, which is more significant than the PCR and SVMR algorithms, which are 1.04, and 1.90, respectively. To support this, pre-processing and pretreatment of the second derivative Savitzky–Golay is the best algorithm that can be used to predict the pH of the whole mango cultivar "Arumanis".

A unique method of a PLC controller based performance evaluation of optimization algorithms and power quality methods in a grid-connected hybrid wind-solar system

The electricity demand is growing due to increasing population and fast industrial development . To meet the worldwide energy requirement, the greatest possibilities are the wind-solar energy sources due to excessive accessibility, simplicity in use, and non-polluting in nature . The integration of these resources offers higher advantages, but the quality of the power scheme gets influenced by the different characteristics of wind and solar energy. Thus in the proposed work, an augmented controller and rectifier have been designed to improve the power quality in which the source current gets optimized by using the Hybrid Bat-Dragonfly optimization algorithm. The quality of the optimized power will be enhanced through the Five Legged Power Converter (FLPC) by converting the DC into AC by using a three-phase bridge rectifier without any power loss. The occurrences of harmonics in the current are reduced using a sieve optimized algorithm design.

Study on Employee Performance Evaluation Based on Adaptive Feature Selection Fuzzy Algorithm

In our study, in terms of performance evaluation methods, the performance evaluation algorithm based on genetic algorithm and fuzzy comprehensive performance evaluation algorithm is introduced, their advantages and disadvantages are compared and analyzed, and the design idea of fuzzy performance evaluation algorithm based on compound elements is proposed. It can be divided into seven steps: first, clarify the evaluation purpose and object; second, select the optimal evaluation mode and method; third, compile the evaluation index system; fourth, it is to collect information extensively; fifth, the evaluation adopts a variety of methods, multiple angles, and multiple sides to collect materials so that the conclusion of the evaluation has sufficient factual basis; the sixth is to process the information and make a comprehensive evaluation; the seventh is to analyze the results and write an evaluation report. Based on the existing algorithms, the fuzzy performance evaluation algorithm based on compound elements is studied, and the detailed design of the algorithm is presented. Through the comparison and analysis of the performance evaluation algorithm based on compound elements and the performance evaluation algorithm of specific elements, the superiority of the performance evaluation algorithm based on compound elements is experimentally verified by comparing the operation time and classification accuracy. The performance evaluation system based on the performance evaluation algorithm designed in this study, combined with the official business of the performance evaluation system, modularizes the administrative management activities of the enterprise, digitizes the electronic office information, and conducts in-depth exploration of the unstructured natural language. A real-time performance evaluation system based on the arrangement of corporate administrative activities has been established. By designing and implementing a performance-assisted analysis system based on text content analysis, which is suitable for performance evaluation systems, it solves the imperfect problem of performance evaluation based on electronic enterprise administrative management.

Machine learning algorithm performance evaluation in structural magnetic resonance imaging-based classification of pediatric bipolar disorders type I patients.

The diagnosis based on clinical assessment of pediatric bipolar disorder (PBD) may sometimes lead to misdiagnosis in clinical practice. For the past several years, machine learning (ML) methods were introduced for the classification of bipolar disorder (BD), which were helpful in the diagnosis of BD. In this study, brain cortical thickness and subcortical volume of 33 PBD-I patients and 19 age-sex matched healthy controls (HCs) were extracted from the magnetic resonance imaging (MRI) data and set as features for classification. The dimensionality reduced feature subset, which was filtered by Lasso or f_classif, was sent to the six classifiers (logistic regression (LR), support vector machine (SVM), random forest classifier, naïve Bayes, k-nearest neighbor, and AdaBoost algorithm), and the classifiers were trained and tested. Among all the classifiers, the top two classifiers with the highest accuracy were LR (84.19%) and SVM (82.80%). Feature selection was performed in the six algorithms to obtain the most important variables including the right middle temporal gyrus and bilateral pallidum, which is consistent with structural and functional anomalous changes in these brain regions in PBD patients. These findings take the computer-aided diagnosis of BD a step forward.

Cloud-based large-scale data retrieval, mapping, and analysis for land monitoring applications with Google Earth Engine (GEE)

Cloud-based computing systems are linked with analytical tools for large-scale flood monitoring to solve these problems. Researchers want an exceptionally efficient and resilient geospatial framework with advanced algorithms for immediate results from the examination of large datasets. The study uses web-based analysis to demonstrate the potential of Google Earth Engine (GEE) for geospatial-analytical processes in flood-affected areas and to comprehend the socio-demographic ramifications. Surface water mapping is done using a histogram-based threshold method. The study examines how to analyse Sentinel-1 SAR data for automated flood mapping and how to validate results using data from the optical sensor Sentinel-2. Furthermore, using the Google Earth Engine platform, this study focuses on cloud-based large-scale flood data mapping. The research combines geographic information with advanced data processing techniques, algorithms, and web-based platforms to produce encouraging results and monitor real-time flooding occurrences for significant planning and decision-making. The research effectively assesses the importance of cloud-based data processing for the performance evaluation of algorithms in a cloud-based platform for monitoring real-time issues. The study's findings are useful for analysing surface water mapping applications.

Machine learning approach to estimate soil matric potential in the plant root zone based on remote sensing data.

There is an increasing interest in using the Internet of Things (IoT) in the agriculture sector to acquire soil- and crop-related parameters that provide helpful information to manage farms more efficiently. One example of this technology is using IoT soil moisture sensors for scheduling irrigation. Soil moisture sensors are usually deployed in nodes. A more significant number of sensors/nodes is recommended in larger fields, such as those found in broadacre agriculture, to better account for soil heterogeneity. However, this comes at a higher and often limiting cost for farmers (purchase, labour costs from installation and removal, and maintenance). Methodologies that enable maintaining the monitoring capability/intensity with a reduced number of in-field sensors would be valuable for the sector and of great interest. In this study, sensor data analysis conducted across two irrigation seasons in three cotton fields from two cotton-growing areas of Australia, identified a relationship between soil matric potential and cumulative satellite-derived crop evapotranspiration (ETcn) between irrigation events. A second-degree function represents this relationship, which is affected by the crop development stage, rainfall, irrigation events and the transition between saturated and non-saturated soil. Two machine learning models [a Dense Multilayer Perceptron (DMP) and Support Vector Regression (SVR) algorithms] were studied to explore these second-degree function properties and assess whether the models were capable of learning the pattern of the soil matric potential-ETcn relation to estimate soil moisture from satellite-derived ETc measurements. The algorithms performance evaluation in predicting soil matric potential applied the k-fold method in each farm individually and combining data from all fields and seasons. The latter approach made it possible to avoid the influence of farm consultants’ decisions regarding when to irrigate the crop in the training process. Both algorithms accurately estimated soil matric potential for individual (up to 90% of predicted values within ±10 kPa) and combined datasets (73% of predicted values within ±10 kPa). The technique presented here can accurately monitor soil matric potential in the root zone of cotton plants with reduced in-field sensor equipment and offers promising applications for its use in irrigation-decision systems.

Mayfly Sparrow Search Hybrid Algorithm for RFID Network Planning

The radio frequency identification (RFID) technology is frequently used for large-scale RFID network planning, which brings thought-provoking RFID network planning (RNP) problems such as the uncertain number of readers, high interference rate of readers, poor economic benefits, and weak robustness. In this paper, we propose a mayfly sparrow search hybrid algorithm with Circle chaotic map and Lévy flight, which is named MASSA. MASSA maps out the position of the readers by adding the idea of the mayfly algorithm (MA) into the optimization process of the sparrow search algorithm (SSA). Circle chaos mapping and Lévy flight are added in the SSA optimization process and the nonlinear inertia coefficients to balance the relationship between global search and local search. The MASSA is compared with SSA, Improved sparrow search algorithm (ISSA), MA, particle swarm optimization algorithm (PSO), and gray wolf algorithm (GWO). The first part of the experiment uses the CEC2017 test suite to evaluate algorithm performance and analysis strategy effectiveness. The results encourage the application of MASSA to solve RNP problems. In the second part, experiments were conducted with four objective functions of coverage, interference, power, and load balance. Two different experiments were carried out with fixed number of readers and flexible number of readers under random label conditions. Experimental results show that MASSA has good planning performance, better economy, and robustness.

Groundwater level prediction with machine learning for the Vidisha district, a semi-arid region of Central India

Groundwater is one of the most important freshwater resources, particularly in dry and semi-arid regions with low annual precipitation and frequent droughts. For the planning and the management of water resources, this is very essential to generate a soft computing method that can predict or forecast groundwater table (GWT). The different machine and deep learning algorithms are evaluated in this research work to predict the groundwater table using season, monthly rainfall, and evapotranspiration rate, which are further compared. The performance evaluation of the different algorithms was done using five different statistical parameters, i.e., mean absolute error (MAE), mean squared error (MSE), root-mean-square error (RMSE), R2−score, and Nash-Sutcliffe Efficiency (NSE). From the results of this study, it is concluded that the Multivariate polynomial regression algorithm is the most efficient for predicting the groundwater table with the minimum mean absolute error in the range of 0.63–0.85 and R2-score of 0.82–0.88.

Deep and Machine Learning Image Classification of Coastal Wetlands Using Unpiloted Aircraft System Multispectral Images and Lidar Datasets

The recent developments of new deep learning architectures create opportunities to accurately classify high-resolution unoccupied aerial system (UAS) images of natural coastal systems and mandate continuous evaluation of algorithm performance. We evaluated the performance of the U-Net and DeepLabv3 deep convolutional network architectures and two traditional machine learning techniques (support vector machine (SVM) and random forest (RF)) applied to seventeen coastal land cover types in west Florida using UAS multispectral aerial imagery and canopy height models (CHM). Twelve combinations of spectral bands and CHMs were used. Our results using the spectral bands showed that the U-Net (83.80–85.27% overall accuracy) and the DeepLabV3 (75.20–83.50% overall accuracy) deep learning techniques outperformed the SVM (60.50–71.10% overall accuracy) and the RF (57.40–71.0%) machine learning algorithms. The addition of the CHM to the spectral bands slightly increased the overall accuracy as a whole in the deep learning models, while the addition of a CHM notably improved the SVM and RF results. Similarly, using bands outside the three spectral bands, namely, near-infrared and red edge, increased the performance of the machine learning classifiers but had minimal impact on the deep learning classification results. The difference in the overall accuracies produced by using UAS-based lidar and SfM point clouds, as supplementary geometrical information, in the classification process was minimal across all classification techniques. Our results highlight the advantage of using deep learning networks to classify high-resolution UAS images in highly diverse coastal landscapes. We also found that low-cost, three-visible-band imagery produces results comparable to multispectral imagery that do not risk a significant reduction in classification accuracy when adopting deep learning models.

Development and Performance Verification of Frequency Control Algorithm and Hardware Controller Using Real-Time Cyber Physical System Simulator

Frequency stability is a critical factor in maintaining the quality of the power grid system. A battery energy storage system (BESS) with quick response and flexibility has recently been used as a primary frequency control (PFC) resource, and many studies on its control algorithms have been conducted. The cyber physic system (CPS) simulator, which can perform virtual physical modelling and verification of many hardware systems connected to the network, is an optimal solution for the performance verification of control algorithms and hardware systems. This study introduces a large-scale real-time dynamic simulator that includes the national power system. This simulator comprises a power grid model, an energy management system (EMS) model, a BESS system model, and a communication model. It performs the control algorithm performance evaluation and the hardware controller’s response performance evaluation. The performance of the control algorithm was evaluated by tracking the power system’s characteristic trajectory in the transient state based on the physical response delay time between the output instruction of the frequency regulation controller (FRC), a hardware controller, and the output response of the BESS. Based on this, we examined the response performance evaluation results by linking them to the optimally designed actual FRC. As a result, we present an analysis of the BESS’s characteristic trajectories in the transient state, such as frequency, power system inertia, and power grid constant, and provide FRC response performance evaluation results at a level of 163 ms, by connecting the BESS installed at the actual site with the CPS simulator.

Graph Burning: Mathematical Formulations and Optimal Solutions

The graph burning problem is an NP-hard combinatorial optimization problem that helps quantify how vulnerable a graph is to contagion. This paper introduces three mathematical formulations of the problem: an integer linear program (ILP) and two constraint satisfaction problems (CSP1 and CSP2). Thanks to off-the-shelf optimization software, these formulations can be solved optimally over arbitrary graphs; this is relevant because the only algorithms designed to date for this problem are approximation algorithms and heuristics, which do not guarantee to find optimal solutions. We empirically compared the proposed formulations using random graphs and off-the-shelf optimization software. The results show that CSP1 and CSP2 tend to reach optimal solutions in less time than the ILP. Therefore, we executed them over some benchmark graphs of order at most 5908. The previously best-known solutions for some of these graphs were improved. We draw some empirical observations from the experimental results. For instance, we find the tendency: the larger the graph’s optimal solution, the more difficult it is to find it. Finally, the resulting set of optimal solutions might be helpful as a benchmark dataset for the performance evaluation of non-exact algorithms.

Breast cancer recognition based on performance evaluation of machine learning algorithms

Breast cancer is the one common cause of death in both developed worlds and the most death-causing disease diagnosed among women. &lt;span lang="EN-US"&gt;Early recognition of this condition can help to minimize death rates. The breast problem statement, in brief, is not reliable for accuracy recognition. They have a high degree of classification accuracy as well as diagnostic capabilities. The most common classifications are normal, benign cancer, and malignant cancer. Machine learning (ML) techniques are now widely used in the classification of breast cancer. In this paper, some machine learning technics have been investigated to diagnose breast cancer (BC) on magnetic resonance imaging (MRI) images using multi-step processes. The first step has been to take the MRI image as an input image and have been pre-processing an image, then use feature extraction by using (scale-invariant feature transform (SIFT), histogram of oriented gradient (HOG), local binary patterns (LBP), bag of words (BoW), and edge-oriented histogram (EOH)). Next step we implement the classifying algorithms (KNN, decision tree (DT), naïve Bayes, ANN, SVM, RF, AdaBoost), have been used to detect and classify the normal or breast cancer region for this purpose datasets like ACRIN-Contralateral-Breast-MRI, In and breast cancer MRI dataset) has been collected our breast cancer MRI images from Erbil and Sulaymaniyah hospital the results was 91.9%, the result of ACRIN was 97% and the results Breast Cancer was 92.3%.&lt;/span&gt;

Performance Evaluation of Machine Learning Algorithms on Textual Datasets for Spam Email Classification

Abstract: Email is one of the most popular modes of communication we have today. Billions of emails are sent every day in our world but not every one of them is relevant or of importance. The irrelevant and unwanted emails are termed email spam. These spam emails are sent with many different targets that range from advertisement to data theft. Filtering these spam emails is very essential in order to keep the email space fluent in its functioning. Machine Learning algorithms are being extensively used in the classification of spam emails. This paper showcases the performance evaluation of some selected supervised Machine Learning algorithms namely Naive Bayes Classifier, Support Vector Machine, Random Forest, &amp; XG-Boost for spam email classification on a combination of three different datasets. For feature extraction, both Bag of Words &amp; TF-IDF models were used separately and performance with both of these approaches was also compared. The results showed that SVM performed better than all the other algorithms when trained with TF-IDF feature vectors. The performance metrics used were accuracy, precision, recall, and f1-score, along with the ROC curve.

Droplet Routing Based on Double Deep Q-Network Algorithm for Digital Microfluidic Biochips

Digital microfluidic biochips (DMFBs) are emerging as an alternative to the cumbersome traditional laboratories for biochemical analysis. DMFBs come under micro-electro-mechanical systems and are a class of lab-on-a-chip devices. DMFBs provide automation, miniaturization and software programmability. The droplet routing algorithm determines concurrent routes for a set of droplets from their source cells to individual target cells on a DMFB. In this paper, a double deep Q-network (DDQN)-based droplet routing algorithm has been proposed. DDQN is a temporal difference-based deep reinforcement algorithm that combines Double Q-learning with a deep neural network algorithm. In the proposed work, routes for droplets are determined by DDQN, and later collisions are resolved using stalling and/or detouring. The latest arrival time of droplets arriving last at its target and cell utilization is taken as objectives for routing algorithm performance evaluation. The proposed method is evaluated on two standard benchmark suites. Simulation results show that the proposed DDQN-based droplet routing algorithm produces competitive results compared to state-of-the-art algorithms.