Evaluation Of Machine Learning Techniques Research Articles

Modelling and Performance Evaluation of Machine Learning Techniques in Forecasting Cereal Yield for Shri Ganganagar Region, Rajasthan, India

One of agriculture's most difficult issues is predicting crop yield. Crop yield forecasting enables necessary decisions to be made to guarantee food security. The current study looks into the use of statistical and machine learning approaches for wheat and rice yield prediction using long-term weather and yield data of ShriGanganagar, Rajasthan, India. Weather-based models may give accurate crop production estimates, but choosing the right model for agricultural output projections can be difficult. As a result, different models were compared in this study to determine the best model for rice and wheat yield prediction including Multiple linear regression (SMLR), Artificial neural network (ANN), Least absolute shrinkage and selection operator (LASSO), Elastic net (ELNET), Ridge regression, and K-nearest neighbor (K-NN). K-NN outperformed ANN in rice crops and fared better in wheat crops based on the lowest nRMSE value. Ridge regression based on nRMSE was the next best model for Rice yield prediction in the examined region, and KNN was the second best model for Wheat crop.

Performance Evaluation of Machine Learning Techniques on Resolution Time Prediction in Helpdesk Support System

Performance Evaluation of Machine Learning Techniques on Resolution Time Prediction in Helpdesk Support System

The Evaluation of Machine Learning Techniques for Isotope Identification Contextualized by Training and Testing Spectral Similarity

Precise gamma-ray spectral analysis is crucial in high-stakes applications, such as nuclear security. Research efforts toward implementing machine learning (ML) approaches for accurate analysis are limited by the resemblance of the training data to the testing scenarios. The underlying spectral shape of synthetic data may not perfectly reflect measured configurations, and measurement campaigns may be limited by resource constraints. Consequently, ML algorithms for isotope identification must maintain accurate classification performance under domain shifts between the training and testing data. To this end, four different classifiers (Ridge, Random Forest, Extreme Gradient Boosting, and Multilayer Perceptron) were trained on the same dataset and evaluated on twelve other datasets with varying standoff distances, shielding, and background configurations. A tailored statistical approach was introduced to quantify the similarity between the training and testing configurations, which was then related to the predictive performance. Wilcoxon signed-rank tests revealed that the OVR-wrapped XGB significantly outperformed the other algorithms, with confidence levels of 99.0% or above for the 133Ba, 60Co, 137Cs, and 152Eu sources. The findings from this work are significant as they outline techniques to promote the development of robust ML-based approaches for isotope identification.

Evaluation of Machine Learning Techniques for Enhancing Scholarship Schemes Using Artificial Emotional Intelligence

This paper investigates the sentiment analysis of the” scholarship system” [4], in Odisha, primarily, to identify why some students do not apply for government-sponsored scholarships. Our research focuses on social media platforms, surveys, and machine learning-based analyses to understand the decision-making process and increase awareness about the various scholarship schemes. The goal of our experiment is to determine the efficacy of sentiment analysis in evaluating the effectiveness of different scholarship schemes. A wide variety of techniques based on dictionaries; corpora lexicons are used in different scholarship schemes for sentiment analysis. Our research paper is based on an evaluation process that could have a positive effect on the government by improving scholarship programs and giving financial aid to students from poor families, which would raise the level of education in Odisha. Our research paper concludes with a summary of successful and unsuccessful schemes, as well as their Word frequency counts and Sentiment Polarity scores.

Evaluation of machine learning techniques for heart disease prediction using multi-criteria decision making

Globally, heart disease is the primary cause of death. Early detection of this disease enables cardiologists to make more accurate judgments regarding the health of their patients. Due to machine learning’s ability to identify patterns in data, its use in the medical industry has increased. Many heart disease prediction models have been developed by various researchers utilizing machine learning techniques (MLTs). The performance of MLTs on heart disease prediction may vary for different accuracy measures. Thus, the choice of the appropriate machine-learning technique for heart disease prediction is a challenging task. This paper proposes a multi-criteria decision-making (MCDM)-based method to evaluate the MLTs for heart disease prediction considering various performance measures taken into account altogether. The proposed approach uses the concept of a combined compromise solution (CoCoSo)- an MCDM method. For validation of the proposed approach, an experimental study was conducted to evaluate the performance of fifteen machine learning techniques for predicting heart disease over three heart disease datasets considering six performance measures taken into account altogether. Results show that the logistic regression and support vector machine are recommended as the most suitable MLTs for heart disease prediction modeling with respect to six performance measures considered simultaneously.

Evaluation of Machine Learning Techniques for Motivational Quotes Classification and Categorization

The effective classification and categorization of motivational quotes are critical for various applications in content management and recommendation systems. This paper presents a comprehensive methodology for consolidating and categorizing motivational quotes using machine learning techniques. The dataset, sourced from Kaggle’s Quotes-500K, contains over half a million quotes categorized into various tags such as ”Love,” ”Life,” ”Motivation,” and more. We address category redundancies by consolidating similar tags and selecting appropriate embedding models to capture semantic relationships between quotes. The embedding models are evaluated using t-SNE for finding the most distinctive model for representing motivational quotes. The final categorization focuses on three key categories: ”Love,” ”Inspirational,” and ”Humor”. We then evaluate the performance of several AI models, including Neural Networks, K-Nearest Neighbors (KNN), Euclidean distancebased classifiers, and Cosine Similarity methods. The proposed improved neural network model outperformed other models with a test accuracy of approximately 98%. These results suggest that our approach offers a robust solution for scalable and accurate categorization of motivational content. Future work will explore deeper NLP techniques, fine-tuning strategies for not just increased number of categories but further enhancing the categorization accuracy along with ranking them within their category.

Evaluation of Machine Learning Techniques for Classifying and Balancing Data on an Unbalanced Mini-Mental State Examination Test Data Collection Applied in Chile

Evaluation of Machine Learning Techniques for Classifying and Balancing Data on an Unbalanced Mini-Mental State Examination Test Data Collection Applied in Chile

Comparative evaluation of machine learning techniques in predicting fundamental meteorological factors based on survey data from 1981 to 2021

Comparative evaluation of machine learning techniques in predicting fundamental meteorological factors based on survey data from 1981 to 2021

Evaluation of machine learning techniques for hypertension risk prediction based on medical data in Bangladesh

Hypertension in Bangladesh is a leading cause of cardiovascular diseases, stroke, and kidney failure, resulting in significant morbidity and mortality. Preventive measures and simple health practices can effectively reduce hypertension and its complications. This study utilizes machine learning algorithms (Naive Bayes, support vector machine, logistic regression, random forest) to predict hypertension in high-risk individuals. The proposed hybrid model achieves a prediction accuracy of 78.17%, surpassing other machine learning methods. Random forest has the highest accuracy among the individual algorithms at 73.86%. Classification performance is evaluated using sensitivity, specificity, precision, and F-score, along with receiver operating characteristic analyses and confusion matrices through 10-fold cross-validation. These findings emphasize the importance of managing risk factors for better population health and highlight the efficacy of the hybrid model in hypertension prediction. The study underscores the significance of preventive measures in reducing the burden of hypertension-related diseases and improving overall well-being.

Enhanced wave overtopping simulation at vertical breakwaters using machine learning algorithms.

Accurate prediction of wave overtopping at sea defences remains central to the protection of lives, livelihoods, and infrastructural assets in coastal zones. In addressing the increased risks of rising sea levels and more frequent storm surges, robust assessment and prediction methods for overtopping prediction are increasingly important. Methods for predicting overtopping have typically relied on empirical relations based on physical modelling and numerical simulation data. In recent years, with advances in computational efficiency, data-driven techniques including advanced Machine Learning (ML) methods have become more readily applicable. However, the methodological appropriateness and performance evaluation of ML techniques for predicting wave overtopping at vertical seawalls has not been extensively studied. This study examines the predictive performance of four ML techniques, namely Random Forest (RF), Gradient Boosted Decision Trees (GBDT), Support Vector Machines-Regression (SVR), and Artificial Neural Network (ANN) for overtopping discharge at vertical seawalls. The ML models are developed using data from the EurOtop (2018) database. Hyperparameter tuning is performed to curtail algorithms to the intrinsic features of the dataset. Feature Transformation and advanced Feature Selection methods are adopted to reduce data redundancy and overfitting. Comprehensive statistical analysis shows superior performance of the RF method, followed in turn by the GBDT, SVR, and ANN models, respectively. In addition to this, Decision Tree (DT) based methods such as GBDT and RF are shown to be more computationally efficient than SVR and ANN, with GBDT performing simulations more rapidly that other methods. This study shows that ML approaches can be adopted as a reliable and computationally effective method for evaluating wave overtopping at vertical seawalls across a wide range of hydrodynamic and structural conditions.

Performance evaluation of machine learning techniques in predicting cumulative absolute velocity

Cumulative absolute velocity (CAV) is a powerful intensity measure for quantifying potential earthquake damage to structures. Machine learning (ML) methods can provide more accurate and reliable predictions of cumulative absolute velocity due to handling nonlinear relationships, adaptability to changing conditions, automation, efficiency, and the potential for real-time predictions. This study aims to identify machine learning regressions with the highest accuracy for CAV prediction. Several supervised machine learning algorithms were applied and comprehensively compared for performance and accuracy in CAV prediction using the recently compiled Turkish strong-motion database. Support Vector Machine, Linear Regression, Random Forest, Artificial Neural Network, Bayesian Ridge Regression, and Gradient Boosting algorithms were evaluated and compared with traditional Ground Motion Models (GMMs).Two new datasets including 24,667 strong-motion recordings from Turkiye along with global strong-motion recordings are used to build machine learning models. The first dataset contains all recordings of events with 3.5≤MomentMagnitude(Mw)≤7.6, while the second dataset contains only recordings with Mw≥5.5. Moreover, feature selection and outlier detection were performed as preprocessing steps to choose the best of seven CAV estimator parameters in order to boost the ML model performance. To measure the performance of ML methods five evaluation metrics were utilized which are mean square error (MSE), root mean squared error (RMSE), mean absolute error (MAE), mean absolute percent error (MAPE), and correlation coefficient (R).Comparative assessment of the machine learning algorithms suggests that models trained by Mw≥5.5 dataset are quite successful in CAV prediction compared to predictive models trained by the 3.5≤Mw≤7.6 dataset. The result proves that the Gradient Boosting models significantly outperform the other machine learning algorithms in terms of R and RMSE. Machine learning techniques are more successful with user-selected estimators representing the key components that make up earthquake recordings. Finally, the machine learning-based CAV prediction models for Turkiye are compared with available CAV GMMs, and it is observed that the machine learning-based models can predict CAV as successfully as GMMs if there are a sufficient number of recordings for training machine learning algorithms.

Evaluation of Machine Learning Techniques Towards Early Detection of Cardiovascular Diseases

The effectiveness of three Machine Learning (ML) algorithms: Support Vector Machine (SVM), Random Forest (RF) and K-Nearest Neighbour (KNN) techniques for the early diagnosis of heart diseases were evaluated. Heart disease’ dataset collected from kaggle.com data repository, which comprised of 303 data points with 13 features and a target variable were used and data preprocessing by data shuffling and dimension reduction were performed. The new dimension of the dataset was chosen such that 85.03% of the original information is retained. The preprocessed dataset was partitioned into 70% of the training set and 30% of the testing set. The ML algorithms were trained and tested for the diagnosis of cardiovascular diseases (CVD). The training performances of these models were evaluated with a k-fold cross-validation algorithm using 10 folds. The k-fold accuracy shows KNN with an accuracy of 0.837662, RF with an accuracy of 0.834091, and SVM with an accuracy of 0.814935. The test results also show KNN with an accuracy of 0.8, SVM with an accuracy of 0.7889, and RF with an accuracy of 0.7667. KNN emerged the best model both in training and test’s performances and is recommended for the early diagnosis of CVD.

Discussion of “Comprehensive Evaluation of Machine Learning Techniques for Hydrological Drought Forecasting” by Muhammad Jehanzaib, Muhammad Bilal Idrees, Dongkyun Kim, and Tae-Woong Kim

Discussion of “Comprehensive Evaluation of Machine Learning Techniques for Hydrological Drought Forecasting” by Muhammad Jehanzaib, Muhammad Bilal Idrees, Dongkyun Kim, and Tae-Woong Kim

Evaluation of Machine Learning Techniques for Traffic Flow-Based Intrusion Detection

Cybersecurity is one of the great challenges of today's world. Rapid technological development has allowed society to prosper and improve the quality of life and the world is more dependent on new technologies. Managing security risks quickly and effectively, preventing, identifying, or mitigating them is a great challenge. The appearance of new attacks, and with more frequency, requires a constant update of threat detection methods. Traditional signature-based techniques are effective for known attacks, but they are not able to detect a new attack. For this reason, intrusion detection systems (IDS) that apply machine learning (ML) techniques represent an alternative that is gaining importance today. In this work, we have analyzed different machine learning techniques to determine which ones permit to obtain the best traffic classification results based on classification performance measurements and execution times, which is decisive for further real-time deployments. The CICIDS2017 dataset was selected in this work since it contains bidirectional traffic flows (derived from traffic captures) that include benign traffic and different types of up-to-date attacks. Each traffic flow is characterized by a set of connection-related attributes that can be used to model the traffic and distinguish between attacks and normal flows. The CICIDS2017 also contains the raw network traffic captures collected during the dataset creation in a packet-based format, thus permitting to extract the traffic flows from them. Various classification techniques have been evaluated using the Weka software: naive Bayes, logistic, multilayer perceptron, sequential minimal optimization, k-nearest neighbors, adaptive boosting, OneR, J48, PART, and random forest. As a general result, methods based on decision trees (PART, J48, and random forest) have turned out to be the most efficient with F1 values above 0.999 (average obtained in the complete dataset). Moreover, multiclass classification (distinguishing between different types of attack) and binary classification (distinguishing only between normal traffic and attack) have been compared, and the effect of reducing the number of attributes using the correlation-based feature selection (CFS) technique has been evaluated. By reducing the complexity in binary classification, better results can be obtained, and by selecting a reduced set of the most relevant attributes, less time is required (above 30% of decrease in the time required to test the model) at the cost of a small performance loss. The tree-based techniques with CFS attribute selection (six attributes selected) reached F1 values above 0.990 in the complete dataset. Finally, a conventional tool like Zeek has been used to process the raw traffic captures to identify the traffic flows and to obtain a reduced set of attributes from these flows. The classification results obtained using tree-based techniques (with 14 Zeek-based attributes) were also very high, with F1 above 0.997 (average obtained in the complete dataset) and low execution times (allowing several hundred thousand flows/s to be processed). These classification results obtained on the CICIDS2017 dataset allow us to affirm that the tree-based machine learning techniques may be appropriate in the flow-based intrusion detection problem and that algorithms, such as PART or J48, may offer a faster alternative solution to the RF technique.

A Systematic Review of Localization in WSN: Machine Learning and Optimization‐Based approaches

SummaryIn recent years, wireless sensor networks (WSNs) have been widely used in various applications. The localization problem has been identified as one of the biggest problems faced by WSNs. The traditional localization techniques may not be able to handle the issues during the scenario to estimate the location of sensor nodes due to anchor mobility, mobile WSNs, latency, energy harvesting, unfavorable environmental states, and many more issues. However, these issues open the door for the amalgamation of machine learning (ML) and optimization techniques with localization techniques. Motivated by the earlier discussion, we explored various ML and optimization techniques to estimate the location coordinates in a sensor network in this paper. Finally, a comparison of existing ML algorithms concerning optimization techniques has been presented, highlighting their improved outcomes. This research offers a detailed survey by exploring the various parameters for location estimation through tabular forms by incorporating ML and optimized localization techniques. A survey of surveys is also presented to identify the key limitations of existing surveys and to introduce the novelty in the comprehensive study done in this paper. A year‐wise evaluation of ML Techniques with localization (2011–2022) is also discussed and presented over various performance parameters, including energy‐efficiency, accuracy, error, and complexity. This discussion concluded that Hybrid Techniques are least explored for using optimized localization machine learning. Further, a summarized discussion of the various comparison tables paves the path for future research in the area of localization in WSN.

Evaluation of machine learning techniques for forecast uncertainty quantification

AbstractEnsemble forecasting is, so far, the most successful approach to produce relevant forecasts with an estimation of their uncertainty. The main limitations of ensemble forecasting are the high computational cost and the difficulty to capture and quantify different sources of uncertainty, particularly those associated with model errors. In this article we perform toy‐model and state‐of‐the‐art model experiments to analyze to what extent artificial neural networks (ANNs) are able to model the different sources of uncertainty present in a forecast. In particular, those associated with the accuracy of the initial conditions and those introduced by the model error. We also compare different training strategies: one based on a direct training using the mean and spread of an ensemble forecast as target, and the other ones rely on an indirect training strategy using an analyzed state as target in which the uncertainty is implicitly learned from the data. Experiments using the Lorenz'96 model show that the ANNs are able to emulate some of the properties of ensemble forecasts like the filtering of the most unpredictable modes and a state‐dependent quantification of the forecast uncertainty. Moreover, ANNs provide a reliable estimation of the forecast uncertainty in the presence of model error. Preliminary experiments conducted with a state‐of‐the‐art forecasting system also confirm the ability of ANNs to produce a reliable quantification of the forecast uncertainty.

Evaluation of machine learning techniques with AVIRIS-NG dataset in the identification and mapping of minerals

Identifying valuable mineral resources is obligatory for human development and survival due to their scientific and economic implications. Hyperspectral Remote Sensing (HRS) is an efficient technique that empowers us to precisely identify and map altered minerals based on spectral absorption curves in the VNIR and SWIR range of the electromagnetic spectrum. Machine Learning Algorithms (MLAs) are a subclass of Artificial Intelligence (AI) that improves and automate HRS based lithological mapping using spectra based classification approach. The present study evaluates various MLAs in identifying and mapping hydrothermally altered and weathered minerals such as kaolinite, talc, kaosmec, and montmorillonite. The study was performed with the Airborne Visible Infrared Imaging Spectrometer- Next Generation (AVIRIS-NG) hyperspectral dataset of Jahazpur town of Bhilwara district in the state of Rajasthan, India (75° 9′ 57′' E, 25° 32′ 24″ N). The Spectral Angle Mapper (SAM) algorithm was applied to create a reference mineral distribution map for the target mineral classes. Further, the reference map has been verified with the field validation survey. A total of 828 pixels were identified as end member pixels. The obtained endmember pixels were further used for developing predictive machine learning based models for mapping the mineral resources using eight supervised classifiers, namely Support Vector Machine, k-Nearest Neighbor, Decision Tree, Random Forest, Logistic Regression, Artificial Neural Network, Linear Discriminant Analysis, and Naïve Bayes. The performance of the classifiers was measured using the kappa coefficient, overall accuracy, precision, recall, F1-score, Matthews’s correlation coefficient, and ROC curve. It was observed in the study that the Support Vector Machine outperformed all the classifiers in terms of overall accuracy and AVIRIS-NG data poses an excellent potential for mineral mapping using machine learning based models in a diverse mountainous area.

Evaluation of machine learning techniques for structural health monitoring using ultrasonic guided waves under varying temperature conditions

The implementation of machine learning methods for structural health monitoring applications has proven to be very effective, especially in detecting damage and compensating for environmental and operational conditions. The use of guided waves in this area has also shown to be a powerful tool due to their sensitivity to structural changes in the propagation medium. In this work, two strategies for detecting damage and distinguishing their positions and for dealing with temperature variations without an additional classical temperature compensation technique are investigated. For this purpose, four unsupervised dimensionality reduction learning methods were used and compared: Principal Component Analysis, Kernel Principal Component Analysis, t-distributed stochastic neighbour embedding and Autoencoder. The first strategy (score plot) consists of using the latent dimensions directly to distinguish the data points of different states of the structure, and the second (DI plot) proposes a method to use Q- and T2-statistics, which have been proposed in previous work for PCA, computed using the compressed representation of the monitoring data. To this end, monitoring data from intact and damaged states of a 500x500x2 mm plate of carbon-fibre–reinforced polymer recorded by 12 piezoelectric transducers at different temperatures are examined. As a reversible damage model, a 10 mm thick aluminium disc is placed at four different locations on the plate. The results primarily show the success of the methods used with DI plot in detecting damage regardless of varying temperature. The autoencoder in the first strategy also demonstrates promising performance in detecting and distinguishing the position of the damage, even in the presence of varying temperature conditions.

Evaluation of machine learning techniques to select marine oil spill response methods under small-sized dataset conditions

Oil spill incidents can significantly impact marine ecosystems in Arctic/subarctic areas. Low biodegradation rate, harsh environments, remoteness, and lack of sufficient response infrastructure make those cold waters more susceptible to the impacts of oil spills. A major challenge in Arctic/subarctic areas is to timely select suitable oil spill response methods (OSRMs), concerning the process complexity and insufficient data for decision analysis. In this study, we used various regression-based machine learning techniques, including artificial neural networks (ANNs), Gaussian process regression (GPR), and support vector regression, to develop decision-support models for OSRM selection. Using a small hypothetical oil spill dataset, the modelling performance was thoroughly compared to find techniques working well under data constraints. The regression-based machine learning models were also compared with integrated and optimized fuzzy decision trees models (OFDTs) previously developed by the authors. OFDTs and GPR outperformed other techniques considering prediction power (> 30 % accuracy enhancement). Also, the use of the Bayesian regularization algorithm enhanced the performance of ANNs by reducing their sensitivity to the size of the training dataset (e.g., 29 % accuracy enhancement compared to an unregularized ANN).

An empirical evaluation of machine learning techniques to classify code comprehension based on EEG data

Context:Code comprehension consists of a cognitive process in which developers invest mental effort to grasp source code snippets. The literature evidence that electroencephalogram (EEG) signals correlate with developers’ code comprehension. An effective machine learning technique that classifies code comprehension based on EEG would be a means for recommendation systems to suggest an appropriate task for the developer. Problem:However, there is a lack of empirical studies to investigate the effectiveness of machine learning techniques for classifying developers’ code comprehension based on their EEG data. Objective:This study conducts an empirical study to analyze the effectiveness of machine learning techniques to classify developers’ code comprehension trained with EEG Data. Method:A wireless EEG device was used to collect brainwaves from software developers while performing code comprehension tasks. The generated data set was used to train a K-Nearest Neighbor (KNN), Neural Network (NN), a Naïve Bayes (NB), a Random Forest (RF), and a Support Vector Machine (SVM) classifier. We measured the effectiveness of these techniques using precision, recall, and f-measure. Furthermore, we analyzed the mean differences of f-measure between classifiers to check if the classifiers’ effectiveness is significantly higher than the f-measure mean of a random guessing classifier and higher than a high effectiveness threshold of 80%. Results:After we trained the classifiers using 10-fold cross-validation, the K-Nearest Neighbor classifier obtained an f-measure mean of 86% to classify code comprehension. Moreover, the results also show that KNN, NN, and RF techniques were able to classify code comprehension based on developers’ EEG data higher with an f-measure above 80%.