Multi-class Classification Algorithm Research Articles

SAMME.C2 algorithm for imbalanced multi-class classification

SAMME.C2 algorithm for imbalanced multi-class classification

Comparing Regression and Classification Models to Estimate Leaf Spot Disease in Peanut (Arachis hypogaea L.) for Implementation in Breeding Selection

Late leaf spot (LLS) is an important disease of peanut, causing global yield losses. Developing resistant varieties through breeding is crucial for yield stability, especially for smallholder farmers. However, traditional phenotyping methods used for resistance selection are laborious and subjective. Remote sensing offers an accurate, objective, and efficient alternative for phenotyping for resistance. The objectives of this study were to compare between regression and classification for breeding, and to identify the best models and indices to be used for selection. We evaluated 223 genotypes in three environments: Serere in 2020, and Nakabango and Nyankpala in 2021. Phenotypic data were collected using visual scores and two handheld sensors: a red–green–blue (RGB) camera and GreenSeeker. RGB indices derived from the images, along with the normalized difference vegetation index (NDVI), were used to model LLS resistance using statistical and machine learning methods. Both regression and classification methods were also evaluated for selection. Random Forest (RF), the artificial neural network (ANN), and k-nearest neighbors (KNNs) were the top-performing algorithms for both regression and classification. The ANN (R2: 0.81, RMSE: 22%) was the best regression algorithm, while the RF was the best classification algorithm for both binary (90%) and multiclass (78% and 73% accuracy) classification. The classification accuracy of the models decreased with the increase in classification classes. NDVI, crop senescence index (CSI), hue, and greenness index were strongly associated with LLS and useful for selection. Our study demonstrates that the integration of remote sensing and machine learning can enhance selection for LLS-resistant genotypes, aiding plant breeders in managing large populations effectively.

SecureNet: Network Intrusion Detection using Machine Learning and Deep Learning Techniques

Abstract: In the ever-evolving landscape of cybersecurity, the need for robust intrusion detection systems has become paramount. This paper introduces a cutting-edge intrusion detection algorithm designed to enhance network security through the integration of advanced machine learning and deep learning methodologies. The proposed algorithm capitalizes on the strengths of both paradigms to achieve a comprehensive and adaptive approach to identifying malicious activities within a network. This research focuses on enhancing network security through the development and evaluation of a novel intrusion detection system leveraging both deep learning and traditional machine learning approaches. Utilizing the NSL-KDD dataset, we employ the Long Short-Term Memory (LSTM) model, a superior version of Recurrent Neural Networks (RNNs), and the K-Nearest Neighbors (KNN) algorithm for binary and multi-class classification of network intrusion anomalies. The LSTM model excels in capturing temporal dependencies, enabling the detection of nuanced sequential patterns, while the KNN algorithm contributes to a comprehensive classification framework. Experimental results demonstrate the effectiveness of the hybrid methodology, showcasing improved accuracy, precision, and recall compared to traditional methods. This research underscores the potential of integrating deep learning and classical machine learning techniques to bolster the capabilities of intrusion detection systems in safeguarding against evolving cyber threats

Text Classification and Cluster Analysis based on Deep Learning and Natural Language Processing

At present, the commonly used Bag of Words (BOW) expression ignores the semantic information of text and the problems of high dimension and high sparsity of feature extraction. This paper presents a multi-class text representation and classification algorithm. This project is based on the vector expression of keywords and takes the multi-category classification problem as the research object. Then, a hybrid Deep Location network (HDBN) is constructed by combining DBN with Boltzmann (DBM). Then, this paper does a lot of tests on the algorithm and proves the effectiveness of the algorithm. In addition, the 2D visual experiment is carried out with HDBN, and then the high-level text expression based on HDBN is obtained. The expression has strong cohesion and weak coupling.

Sentiment Analysis of Short Texts Using SVMs and VSMs-Based Multiclass Semantic Classification

ABSTRACT In our approach, a hybrid machine learning model is proposed which uses Enhanced Vector Space Model (EVSM) along with Hybrid Support Vector Machine (HSVM) classifier. Initially the social media-based information is retrieved using Enhanced Vector Space Model (EVSM). EVSMs are employed in order to characterize the text content by mapping them into high-dimensional vector spaces, capturing the relationships between words and their contextual meanings. Rigorous feature selection methods are employed to designate texts for review, and a multiclass semantic classification algorithm, specifically the HSVM classifier, is utilized for categorization. Decision tree algorithm is used along with SVM to refine the selection process. To enhance sentiment analysis accuracy, sentiment dictionaries are not only presented but also extended through the expansion of Stanford’s GloVE tool. To enhance precision, the proposed work introduces weight-enhancing methods for processing renowned text weights. Sentiments are classified into positive, negative, and neutral categories. Notably, the achieved results demonstrate improved accuracy, attributed to the incorporation of an emotional sentiment enhancement factor for determining weights and leveraging sentiment dictionaries for word availability. The accuracy is obtained to be 92.78% with 91.33% positive sentiment rate and 97.32% negative sentiment rate.

Interval-based sparse ensemble multi-class classification algorithm for terahertz data

Terahertz time-domain spectroscopy (THz-TDS) has been widely used for food and drug identification. The classification information of a THz spectrum usually does not exist in the whole spectral band but exists only in one or several small intervals. Therefore, feature selection is indispensable in THz-based substance identification. However, most THz-based identification methods empirically intercept the low-frequency band of the THz absorption coefficients for analysis. In order to adaptively find out important intervals of the THz spectra, an interval-based sparse ensemble multi-class classifier (ISEMCC) for THz spectral data classification is proposed. In ISEMCC, the THz spectra are first divided into several small intervals through window sliding. Then the data of training samples in each interval are extracted to train some base classifiers. Finally, a final robust classifier is obtained through a nonnegative sparse combination of these trained base classifiers. With l1 -norm, two objective functions that based on Mean Square Error (MSE) and Cross Entropy (CE) are established. For these two objective functions, two iterative algorithms based on the Alternating Direction Method of Multipliers (ADMM) and Gradient Descent (GD) are built respectively. ISEMCC transforms the problem of interval feature selection and decision-level fusion into a nonnegative sparse optimization problem. The sparse constraint ensures only a few important spectral segments are selected. In order to verify the performance of the proposed algorithm, comparative experiments on identifying the origin of Bupleurum and the harvesting year of Tangerine peel are carried out. The base classifiers used by ISEMCC are Support Vector Machine (SVM) and Decision Tree (DT). The experimental results demonstrate that the proposed algorithm outperforms six typical classifiers, including Random Forest (RF), AdaBoost, RUSBoost, ExtraTree, and the two base classifiers, in terms of classification accuracy.

Genetic Clustering Algorithm-Based Feature Selection and Divergent Random Forest for Multiclass Cancer Classification Using Gene Expression Data

AbstractComputational identification and classification of clinical disorders gather major importance due to the effective improvement of machine learning methodologies. Cancer identification and classification are essential clinical areas to address, where accurate classification for multiple types of cancer is still in a progressive stage. In this article, we propose a multiclass cancer classification model that categorizes the five different types of cancers using gene expression data. To perform efficient analysis of the available clinical data, we propose feature selection and classification methods. We propose a genetic clustering algorithm (GCA) for optimal feature selection from the RNA-gene expression data, consisting of 801 samples belonging to the five major classes of cancer. The proposed feature selection method reduces the 1621 gene expressions into a cluster of 21 features. The optimum feature set acts as input data to the proposed divergent random forest. Based on the features computed, the proposed classifier categorizes the data samples into 5 different classes of cancers, including breast cancer, colon cancer, kidney cancer, lung cancer, and prostate cancer. The proposed divergent random forest provided performance improvisation in terms of accuracy with 95.21%, specificity with 93%, and sensitivity with 94.29% which outperformed all the other existing multiclass classification algorithms.

A Hazard-Aware Metric for Ordinal Multi-Class Classification in Pathology

Artificial Intelligence (AI) for decision support and diagnosis in pathology could provide immense value to society, improving patient outcomes and alleviating workload demands on pathologists. However, this potential cannot be realized until sufficient methods for testing and evaluation of such AI systems are developed and adopted. We present a novel metric for evaluation of multi-class classification algorithms for pathology, Error Severity Index (ESI), to address the needs of pathologists and pathology lab managers in evaluating AI systems.

Discovery of astronomical objects in galaxies by means of deep learning

The study of space exploration has been studied for a very long time, and as technology has developed, so too have the methods and techniques employed, along with the quantity and type of data acquired. We now receive so much astronomical data, and so much brand new data is being generated every day, that it is physically impracticable to examine it all only by human work. In our study, we look at a number of astronomers face while working with this massive amount of data, and they use deep learning techniques to discover the best data for each objective. [1]previously SVM, KNN, the random forest approach, decision trees, and other multi-class classification algorithms are all used in the methodology. Galaxies' propensity to belong to specific classes is forecasted using regression. even if the findings from the random forest method were the best it was unable to effectively divide galaxies into the five groups. This approach does not explain real-time categorization and does not take outliers into consideration. The model's adaptability is constrained. This categorization scheme is unable to account for the modelling of galaxies as well as their evolution. Here, we suggest using Inception v3 for categorization and VGG-19 for image analysis. Segmentation is a method for discovering and classifying galaxies. These techniques greatly advance certain fields of study where there are enormous volumes of duplicate data that must be deleted in accordance with the demands of the study, thanks to Python's high performance in the investigation of image processing and computer vision.. As a result, there is less of a need for researchers to carefully sort through all of the data that has been collected from satellite telescopes, sky surveys, etc. [2]

Vocal music curriculum design and teaching quality evaluation in colleges and universities based on interactive structure model

Abstract Aiming at the current phenomenon of low teacher-student interaction in vocal music teaching, this paper combines the interaction structure model for the innovation and practice of college vocal music courses. Then, this paper uses Euclidean distance to improve the binary tree SVM multi-class classification algorithm, calculates the distance between the teaching samples of each vocal music course, classifies the samples for processing, and obtains the DBT-SVM multi-class classification algorithm. Finally, the DBT-SVM algorithm was used to establish an evaluation model for vocal course teaching quality in colleges and universities to assess the quality of vocal course teaching models. Each three-level index of teaching motivationality has an evaluation score above 80, which is a good level. The evaluation score for the pedagogical index is 89.44, which is almost excellent. The average score of the second-level index of interpersonal interaction is 74.52, which needs to be further improved. The teaching model of the college vocal music course proposed in this paper lays the foundation for the interactive transformation of vocal music teaching and provides a reference model for the improvement of the learning effect of college students’ vocal music courses.

MK-MSMCR: An efficient multiple kernel approach to multi-class classification

This paper introduces a novel multi-class support vector classification and regression (MSVCR) algorithm with multiple kernel learning (MK-MSVCR). We present a new MK-MSVCR algorithm based on two-stage learning (MK-MSVCRTSL). The two-stage learning aims to make classification algorithms better when dealing with complex data by using the first stage of learning to generate ?representative? or ?important? samples. We first establish the fast learning rate of MKMSVCR algorithm for multi-class classification with independent and identically distributed (i.i.d.) samples amd uniformly ergodic Markov chain (u.e.M.c.) smaples, and prove that MK-MSVCR algorithm is consistent. We show the numerical investigation on the learning performance of MK-MSVCR-TSL algorithm. The experimental studies indicate that the proposed MK-MSVCR-TSL algorithm has better learning performance in terms of prediction accuracy, sampling and training total time than other multi-class classification algorithms.

Research on objective evaluation methods of labor education teaching for college students in higher education

Abstract Labor education in colleges and universities in the new era is the fundamental task of implementing the principle of establishing morality and educating people, as well as an essential hand in cultivating college students with all-round development of morality, intelligence, physicality, aesthetics, and labor. This paper focuses on the objective evaluation method of college students’ labor education to conduct research. Using the improved binary tree support vector machine multi-class classification algorithm, an evaluation model for college labor education based on a binary tree support vector machine has been constructed, and an evaluation system for labor education has been developed. The results show that the majors that have compulsory labor education courses, elective labor education courses, and have added labor education modules in their major courses are all art and sports, accounting for more than 50% on average. The percentages of liberal arts, science, economics and management, art and sports, medicine and science, and agriculture, which did not offer labor education-related courses, were 26.35%, 22.16%, 19.91%, 13.59%, 11.03%, and 13.19%, respectively. It is evident that labor education courses are not distributed equally in colleges and universities, and there are still some majors that do not offer labor education courses. In terms of labor education, most of the labor education through Civics and Political Science classes is provided to students. Managers are those who choose the most specialized courses. Students also tend to participate in bicultural education and social practice activities, accounting for 29.02% and 43.01%, respectively. It shows that the algorithm in this paper can objectively and effectively evaluate labor education in colleges and universities and identify the deficiencies in education.

Enhancing safety of construction workers in Korea: an integrated text mining and machine learning framework for predicting accident types

Construction workers face a high risk of various occupational accidents, many of which can result in fatalities. This study aims to develop a prediction model for nine prevalent types of construction accidents, utilizing construction tasks, activities, and tools/materials as input features, through the application of machine learning-based multi-class classification algorithms. 152,867 construction accident summary reports, composed of both structured (construction task, construction activity, accident type) and unstructured data (tools/materials) were used for the study. The study employed several data processing techniques, including keyword extraction through text mining, Boruta feature selection, and SMOTE data resampling enhance model accuracy. Three performance metrics (Multi-class area under the receiver operating characteristic curve (MAUC), Multi-class Matthews Correlation Coefficient (MMCC), Geometric-mean (G-mean)) were used to compare the predictive performance of four machine learning algorithms, including Decision tree, Random forest, Naïve bayes, and XGBoost. Of the four algorithms, XGBoost showed the highest performance in predicting accident type (MAUC: 0.8603, MMCC: 0.3523, G-mean: 0.5009). Furthermore, a Shapley additive explanation (SHAP) analysis was conducted to visualize feature importance. The findings of this study make a valuable contribution to improving construction safety by presenting a prediction model for accident types derived from real-world big data.

XGBoost machine learning algorithm for differential diagnosis of pediatric syncope

Abstract. The search for new methods of differential diagnosis of syncope types will allow to improve the diagnosis of vasovagal syncope (VVS), syncope due to orthostatic hypotension (OH) and cardiac syncope (CS) in childhood in order to make timely adequate diagnostic and therapeutic decisions. The aim of the study was to develop an effective machine learning model for the differential diagnosis of VVS, syncope due to OH and CS in children. Materials and Methods. 140 patients with syncope, aged 8-17 years, were examined: 92 children with a diagnosis of VVS, 28 children with syncope due to OH and 20 children with CS. A machine learning model was built using XGBoost algorithm for multiclass classification based on input clinical, laboratory and instrumental patient data. Results. The developed machine learning model based on the XGBoost algorithm is effective in the differential diagnosis of VVS, syncope due to OH and CS, which is confirmed by the metrics of accuracy (0.93), precision (0.93 for VVS; 1.00 for syncope due to OH; 0.80 for CS), recall (0.96 for VVS; 1.00 for syncope due to OH; 0.67 for CS), f1 (0.95 for VVS; 1.00 for syncope due to OH; 0.73 for CS), ROC AUC (0.95 for VVS; 1.00 for syncope due to OH; 0.89 for CS), PR AUC (0.96 for VVS; 1.00 for syncope due to OH; 0.79 for CS),Cohen’s Kappa (0.85), and Matthews correlation coefficient (0.85). The most informative parameters of the syncope types differential diagnosis model are OH, paroxysmal supraventricular tachycardia, Hildebrandt coefficient, Calgary Syncope Seizure Score, vitamin B6, average duration of the P-Q interval during 24 hours, duration of tachycardia during 24 hours, stroke index, homocysteine, heart volume, and systolic blood volume. Conclusions. The proposed machine learning model has sufficient efficiency and can be used by pediatricians and pediatric cardiologists for the differential diagnosis of VS, syncope due to OH, and CS in childhood.

Variational quantum state discriminator for supervised machine learning

Abstract Quantum state discrimination (QSD) is a fundamental task in quantum information processing with numerous applications. We present a variational quantum algorithm that performs the minimum-error QSD, called the variational quantum state discriminator (VQSD). The VQSD uses a parameterized quantum circuit that is trained by minimizing a cost function derived from the QSD, and finds the optimal positive-operator valued measure (POVM) for distinguishing target quantum states. The VQSD is capable of discriminating even unknown states, eliminating the need for expensive quantum state tomography. Our numerical simulations and comparisons with semidefinite programming demonstrate the effectiveness of the VQSD in finding optimal POVMs for minimum-error QSD of both pure and mixed states. In addition, the VQSD can be utilized as a supervised machine learning algorithm for multi-class classification. The area under the receiver operating characteristic curve obtained in numerical simulations with the Iris flower dataset ranges from 0.97 to 1 with an average of 0.985, demonstrating excellent performance of the VQSD classifier.

Smartphone-based fluorescence spectroscopic device for cervical precancer diagnosis: a random forest classification of in vitro data.

Cervical cancer can be treated and cured if diagnosed at an early stage. Optical devices, developed on smartphone-based platforms, are being tested for this purpose as they are cost-effective, robust, and field portable, showing good efficiency compared to the existing commercial devices. This study reports on the applicability of a 3D printed smartphone-based spectroscopic device (3D-SSD) for the early diagnosis of cervical cancer. The proposed device has the ability to evaluate intrinsic fluorescence (IF) from the collected polarized fluorescence (PF) and elastic-scattering (ES) spectra from cervical tissue samples of different grades. IF spectra of 30 cervical tissue samples have been analyzed and classified using a combination of principal component analysis (PCA) and random forest (RF)-based multi-class classification algorithm with an overall accuracy above 90%. The usage of smartphone for image collection, spectral data analysis, and display makes this device a potential contender for use in clinics as a regular screening tool.

Discriminative Metric Learning for Partial Label Learning.

One simple strategy to deal with ambiguity in partial label learning (PLL) is to regard all candidate labels equally as the ground-truth label, and then solve the PLL problem using existing multiclass classification algorithms. However, due to the noisy false-positive labels in the candidate set, these approaches are readily mislead and do not generalize well in testing. Consequently, the method of identifying the ground-truth label straight from the candidate label set has grown popular and effective. When the labeling information in PLL is ambiguous, we ought to take advantage of the data's underlying structure, such as label and feature interdependencies, to conduct disambiguation. Furthermore, while metric learning is an excellent method for supervised learning classification that takes feature and label interdependencies into account, it cannot be used to solve the weekly supervised learning PLL problem directly due to the ambiguity of labeling information in the candidate label set. In this article, we propose an effective PLL paradigm called discriminative metric learning for partial label learning (DML-PLL), which aims to learn a Mahanalobis distance metric discriminatively while identifying the ground-truth label iteratively for PLL. We also design an efficient algorithm to alternatively optimize the metric parameter and the latent ground-truth label in an iterative way. Besides, we prove the convergence of the designed algorithms by two proposed lemmas. We additionally study the computational complexity of the proposed DML-PLL in terms of training and testing time for each iteration. Extensive experiments on both controlled UCI datasets and real-world PLL datasets from diverse domains demonstrate that the proposed DML-PLL regularly outperforms the compared approaches in terms of prediction accuracy.

DETECTING HOMONIMY BY MEANS OF THE NAIVE BAYES CLASSIFIER

One of the relevant issues of a natural language processing is word sense disambiguation. Homonyms are considered as an important element of determining the meaning of a word. Methods based on machine learning play a special role in solving this problem. Naive Bayes classifier is one of the important machine learning methods. When eliminating homonymy between different and grammatically similar groups of words in the Uzbek language, the Naive Bayes classifier differs from other methods in its simplicity and speed. This classifier is one of the most popular multi-class classification algorithms, and depending on the data in question, any of the 3 types of Naive Bayes algorithms (Gaussian, Polynomial, Bernoulli) can be used. This article scrutinizes the processes of using the classifier to identify homonymy between grammatically similar groups of words in the Uzbek language.

A multi-class classification algorithm based on hematoxylin-eosin staining for neoadjuvant therapy in rectal cancer: a retrospective study.

Neoadjuvant therapy (NAT) is a major treatment option for locally advanced rectal cancer. With recent advancement of machine/deep learning algorithms, predicting the treatment response of NAT has become possible using radiological and/or pathological images. However, programs reported thus far are limited to binary classifications, and they can only distinguish the pathological complete response (pCR). In the clinical setting, the pathological NAT responses are classified as four classes: (TRG0-3), with 0 as pCR, 1 as moderate response, 2 as minimal response and 3 as poor response. Therefore, the actual clinical need for risk stratification remains unmet. By using ResNet (Residual Neural Network), we developed a multi-class classifier based on Hematoxylin-Eosin (HE) images to divide the response to three groups (TRG0, TRG1/2, and TRG3). Overall, the model achieved the AUC 0.97 at 40×magnification and AUC 0.89 at 10×magnification. For TRG0, the model under 40×magnification achieved a precision of 0.67, a sensitivity of 0.67, and a specificity of 0.95. For TRG1/2, a precision of 0.92, a sensitivity of 0.86, and a specificity of 0.89 were achieved. For TRG3, the model obtained a precision of 0.71, a sensitivity of 0.83, and a specificity of 0.88. To find the relationship between the treatment response and pathological images, we constructed a visual heat map of tiles using Class Activation Mapping (CAM). Notably, we found that tumor nuclei and tumor-infiltrating lymphocytes appeared to be potential features of the algorithm. Taken together, this multi-class classifier represents the first of its kind to predict different NAT responses in rectal cancer.

Clustered ensemble feature selection with M-GRU classification for efficient intrusion detection system of industrial systems

Industrial Control Systems (ICS) are susceptible to threats or attacks, and even minor changes or manipulation could cause major damage to industrial operations. Industrial control system cybersecurity is vital owing to the severe negative effects it could have on the economy, the environment, people, and politics. Therefore, it’s also crucial to design intrusion detection systems for industrial control systems. In this paper, an efficient intrusion detection system with clustered ensemble feature selection and a Multi-Level Modified Gated Recurrent Unit (M-GRU) classification model is proposed. This intrusion detection system with a general framework for clustered ensemble feature ranking approach is proposed to effectively find the best feature subset in network packet traffic data. The features designated are fed into a multi class classification algorithm Multi-Level Modified Gated Recurrent Unit (M-GRU) to efficiently detect the cyberattacks. Evaluation criteria including precision, accuracy, recall and F1 score are assessed and compared to other cutting-edge algorithms to assess the performance of the proposed model. The proposed model attained an average accuracy of 98.21 %. Results show that the suggested model increased the attack detection accuracy by an average of 5.935% and 0.116% when compared to the Gated Recurrent Unit, Long Short Term Memory, random forest and naïve bayes models.