Multi-class Classification Problem Research Articles

Automatic evaluation of the Nine-Hole Peg Test in multiple sclerosis patients using machine learning models

Neurological damage in Multiple Sclerosis (MS) affects the motor, sensory, visual, cognitive and sphincteric systems, resulting in different degrees of disability. Upper limb dysfunction is a core deficit affecting MS patients. However, since upper limb function impairment is not properly recorded in the Expanded Disability Status Scale (EDSS) for MS, other tests have been developed in the last decades. The Nine Hole Peg Test (9HPT) is widely considered a gold standard metric for manual dexterity. Unfortunately, its quantification strongly depends on the subjectivity of each clinician.The aim of this work is to standardize the measurement process of the 9HPT. Instead of a human interpretation of the time needed to complete the exercise, this paper aims to objectively extract multiple kinematic information of the subject’s hand (period, velocity, time). This information is used to detect the presence and the state of the disease more accurately and classify the patients automatically. The manuscript describes a developed framework to automate and objectivize the 9HPT. The framework evaluates the performance of different Machine Learning models and Artificial Neural Networks on MS diagnosis. The built system analyzes videos of subjects performing the 9HPT, extracts the hand’s pose using Convolutional Neural Networks, acquires the hand’s kinematic features and classifies the severity of the disease using Machine Learning algorithms. For the framework validation, two classification approaches are proposed. The first one is a binary classification problem, separating the subjects between control and patients. The second one is a multiclass classification problem, where the patients are separated into three different classes based on the severity assessed by the clinicians. A technical validation has been implemented with an accuracy of up 96.67% for the binary classification and 92.22% for the multiclass classification.

Ensemble methods of rank-based trees for single sample classification with gene expression profiles

Building Single Sample Predictors (SSPs) from gene expression profiles presents challenges, notably due to the lack of calibration across diverse gene expression measurement technologies. However, recent research indicates the viability of classifying phenotypes based on the order of expression of multiple genes. Existing SSP methods often rely on Top Scoring Pairs (TSP), which are platform-independent and easy to interpret through the concept of “relative expression reversals”. Nevertheless, TSP methods face limitations in classifying complex patterns involving comparisons of more than two gene expressions. To overcome these constraints, we introduce a novel approach that extends TSP rules by constructing rank-based trees capable of encompassing extensive gene-gene comparisons. This method is bolstered by incorporating two ensemble strategies, boosting and random forest, to mitigate the risk of overfitting. Our implementation of ensemble rank-based trees employs boosting with LogitBoost cost and random forests, addressing both binary and multi-class classification problems. In a comparative analysis across 12 cancer gene expression datasets, our proposed methods demonstrate superior performance over both the k-TSP classifier and nearest template prediction methods. We have further refined our approach to facilitate variable selection and the generation of clear, precise decision rules from rank-based trees, enhancing interpretability. The cumulative evidence from our research underscores the significant potential of ensemble rank-based trees in advancing disease classification via gene expression data, offering a robust, interpretable, and scalable solution. Our software is available at https://CRAN.R-project.org/package=ranktreeEnsemble.

A Transformer Approach for Cognitive Impairment Classification and Prediction.

Early classification and prediction of Alzheimer disease (AD) and amnestic mild cognitive impairment (aMCI) with noninvasive approaches is a long-standing challenge. This challenge is further exacerbated by the sparsity of data needed for modeling. Deep learning methods offer a novel method to help address these challenging multiclass classification and prediction problems. We analyzed 3 target feature-sets from the National Alzheimer Coordinating Center (NACC) dataset: (1) neuropsychological (cognitive) data; (2) patient health history data; and (3) the combination of both sets. We used a masked Transformer-encoder without further feature selection to classify the samples on cognitive status (no cognitive impairment, aMCI, AD)-dynamically ignoring unavailable features. We then fine-tuned the model to predict the participants' future diagnosis in 1 to 3 years. We analyzed the sensitivity of the model to input features via Feature Permutation Importance. We demonstrated (1) the masked Transformer-encoder was able to perform prediction with sparse input data; (2) high multiclass current cognitive status classification accuracy (87% control, 79% aMCI, 89% AD); (3) acceptable results for 1- to 3-year multiclass future cognitive status prediction (83% control, 77% aMCI, 91% AD). The flexibility of our methods in handling inconsistent data provides a new venue for the analysis of cognitive status data.

Multiclass Classification With Fuzzy-Feature Observations: Theory and Algorithms.

The theoretical analysis of multiclass classification has proved that the existing multiclass classification methods can train a classifier with high classification accuracy on the test set, when the instances are precise in the training and test sets with same distribution and enough instances can be collected in the training set. However, one limitation with multiclass classification has not been solved: how to improve the classification accuracy of multiclass classification problems when only imprecise observations are available. Hence, in this article, we propose a novel framework to address a new realistic problem called multiclass classification with imprecise observations (MCIMO), where we need to train a classifier with fuzzy-feature observations. First, we give the theoretical analysis of the MCIMO problem based on fuzzy Rademacher complexity. Then, two practical algorithms based on support vector machine and neural networks are constructed to solve the proposed new problem. The experiments on both synthetic and real-world datasets verify the rationality of our theoretical analysis and the efficacy of the proposed algorithms.

Nonparametric plug‐in classifier for multiclass classification of S.D.E. paths

AbstractWe study the multiclass classification problem where the features come from a mixture of time‐homogeneous diffusions. Specifically, the classes are discriminated by their drift functions while the diffusion coefficient is common to all classes and unknown. In this framework, we build a plug‐in classifier which relies on nonparametric estimators of the drift and diffusion functions. We first establish the consistency of our classification procedure under mild assumptions and then provide rates of convergence under different set of assumptions. Finally, a numerical study supports our theoretical findings.

A novel redistribution-based feature selection for text classification

Text classification is the process of automatically categorizing text documents into predefined labels, gaining increasing importance with the growing volume of data. In the vector space model, terms within documents must be quantified to be used with classifiers. The abundance of unique terms associated with documents can lead to unwieldy vectors. To address this issue, feature selection, reducing the number of terms by removing irrelevant terms, is a common solution. In this paper, a novel feature selection method called Amount of ReDistribution to Establish Neutrality, ARDEN, is proposed for the task of text classification. ARDEN, is designed with statistical distance perspective by measuring the distance of a term to its neutral counterpart, which represents the least distinguishing term having uniform document frequencies across all classes. ARDEN introduces a method to measure the related distance, providing insights into the degree of deviation of the term from neutrality, thus capturing its distinguishing power. ARDEN is experimentally tested against state-of-the-art feature selection methods. The results suggest that it is a competitive feature selection method, exhibiting superior performance in terms of summary statistics, μ and δ′. Furthermore, ARDEN is designed for multi-class text classification problems, which does not require a globalization function. Additionally, the proposed method is compared against Wasserstein statistical distance metric, and clearly outperforms it. Finally, we also propose δ′ evaluation criterion in order to facilitate the interpretation of experimental outcomes across multiple feature sizes.

Analyzing and assessing explainable AI models for smart agriculture environments

We analyze a case study in the field of smart agriculture exploiting Explainable AI (XAI) approach, a field of study that aims to provide interpretations and explanations to the behaviour of AI systems. The study regards a multiclass classification problem on the Crop Recommendation dataset. The original task is the prediction of the most adequate crop, according to seven features. In addition to the predictions, two of the most well-known XAI approaches have been used in order to obtain explanations and interpretations of the behaviour of the models: SHAP (SHapley Additive ExPlanations), and LIME (Local Interpretable Model-Agnostic Explanations). Both packages provide easy-to-understand visualizations that allow common users to understand explanations of single predictions even without going into the mathematical details of the algorithms. Within the scientific community criticisms have been raised against these approaches, and recently some papers brought to light some weaknesses. However, the two algorithms are among the most popular in XAI and are still considered points of reference for this field of study.

A novel classification method for GPR B-scan images based on weak-shot learning

Ground Penetrating Radar (GPR) is extensively employed for underground target detection, and the advancements in this domain have been hastened by the utilization of deep learning methodologies. Despite advancements in deep learning, practical engineering applications often face challenges in meeting the demanding requirements of large-scale data and high data quality. In this study, based on cross-domain transfer learning, we proposed a method for underground target classification of weak labeled GPR data. It combined strongly annotated source domain data with weakly annotated target domain data through adversarial learning to leverage the prior knowledge from existing large datasets and addressed the classification problem of small-sample GPR data. By employing adversarial learning to integrate strongly labeled source domain data with weakly labeled target domain data, we leverage prior knowledge from existing large datasets, thereby enhancing classification accuracy on a small GPR dataset. This approach reduces the requirements for annotation quality and sample quantity in the GPR B-scan dataset of underground targets. Additionally, it exhibits robustness to label noise, making it applicable to real-world GPR data classification tasks. Compared to some well-known networks, the proposed model achieved the best classification performance in multiclass classification problems, with an accuracy of 87.62% and an F1macro score of 87.60% on the test set. These results represent an improvement of 17.14 and 17.20 percentage points, respectively, over the best-performing classical classification networks. Additionally, the proposed model's reliability in cross-domain transfer learning, even with limited data, was substantiated through a comparison with other few-shot learning networks.

CPS Attack Detection under Limited Local Information in Cyber Security: An Ensemble Multi-Node Multi-Class Classification Approach

Cybersecurity breaches are common anomalies for distributed cyber-physical systems (CPS). However, the cyber security breach classification is still a difficult problem, even using cutting-edge artificial intelligence (AI) approaches. In this article, we study a multi-class classification problem in cyber security for attack detection. A challenging multi-node data-censoring case is considered. In such a case, data within each data center/node cannot be shared while the local data is incomplete. Particularly, local nodes contain only a part of the multiple classes. In order to train a global multi-class classifier without sharing the raw data across all nodes, we design a multi-node multi-class classification ensemble approach which is the main result of our study. By gathering the estimated parameters of the binary classifiers and data densities from each local node, the missing information for each local node is completed to build the global multi-class classifier. Numerical experiments are given to validate the effectiveness of the proposed approach under the multi-node data-censoring case. Under such a case, we even show the out-performance of the proposed approach over the full-data approach.

Milling cutter fault diagnosis using unsupervised learning on small data: A robust and autonomous framework

Tool condition affects the tolerances and the energy consumption and hence needs to be monitored. Artificial intelligence based data-driven techniques for tool condition determination are proposed. Unfortunately, the data-driven techniques are data-hungry. This paper proposes a methodology for classification based on unsupervised learning using limited unlabeled training data. The work presents a multi-class classification problem for the tool condition monitoring. The principal component analysis (PCA) is employed for dimensionality reduction and the principal components (PCs) are used as input for classification using k-means clustering. New collected data is then projected on the PC space, and classified using the clusters from the training. The methodology has been applied for classification of tool faults in 6 classes. The use of limited input parameters from the user makes the method ideal for monitoring a large number of machines with minimal intervention. Furthermore, due to the small amount of data needed for the training, the method has the potential to be transferable.

Developing a web application for Chinese calligraphy learners using convolutional neural network and scale invariant feature transform

Chinese Calligraphy is an ancient art form with practical benefits such as improving elders' cognition and helping diagnose Parkinson's disease. Learning Chinese calligraphy requires a lot of time and effort. This is due to the different styles and the vast number of Chinese characters used daily. Chinese character and style recognition are multi-class classification problems. In multi-class classification problems, the accuracy decreases as the number of classes increases. Previous research on Chinese calligraphy was limited to either style recognition or character recognition. More research that combines style and character recognition is needed. Furthermore, the models in previous research were limited to recognizing small sets of 300 characters. These small sets cover only 12% of the Chinese characters used daily. The small size is a severe limitation for creating applications for Chinese calligraphy learning. Thus, new research is needed on AI models that combine style and character recognition to recognize a much larger dataset, while maintaining reasonable accuracy rates. Besides, there are no applications that use such AI models to help learners of Chinese calligraphy compare their works with those created by skillful calligraphers. This research proposed enhancing Chinese calligraphy recognitions by combining the VGG (Visual Geometry Group)-net architecture, data augmentation techniques, and dropout in the ANN (artificial neural network) hidden layers. The research produced two AI models that can recognize 962 Chinese characters in five calligraphic styles. We proved that using dropout increased the accuracy convergence during training epochs. In the proposed models, style recognition achieved up to 88.3% accuracy, while character recognition achieved 95.6% accuracy. Finally, we built an online learning application by combining the proposed models with Scale Invariant Feature Transform (SIFT). The application could compare the learner's work with skillful calligraphers' and provide a similarity score to help Chinese calligraphy learners track their progress. Five university students tested the application. They provided positive feedback and some suggestions for improvement.

New Bounds on the Accuracy of Majority Voting for Multiclass Classification.

Majority voting is a simple mathematical function that returns the most frequently occurring value within a given set. As a popular decision fusion technique (DFT), the majority voting function (MVF) finds applications in resolving conflicts, where several independent voters report their opinions on a classification problem. Despite its importance and its various applications in ensemble learning, data crowdsourcing, remote sensing, and data oracles for blockchains, the accuracy of the MVF for the general multiclass classification problem has remained unknown. In this article, we derive a new upper bound on the accuracy of the MVF for the multiclass classification problem. More specifically, we show that under certain conditions, the error rate of the MVF exponentially decays toward 0 as the number of independent voters increases. Conversely, the error rate of the MVF exponentially grows with the number of voters if these conditions are not met. We first explore the problem for voters with independent and identically distributed (i.i.d.) outputs, where we assume that, given the true classification of the data point, every voter follows the same conditional probability distribution of voting for different classes. Next, we extend our results to encompass independent but nonidentically distributed votes. Using the derived results, we then provide a discussion on the accuracy of the truth discovery algorithms. We show that in the best-case scenarios, truth discovery algorithms operate as an amplified MVF and thereby achieve a small error rate only when the MVF achieves a small error rate too, and vice versa, achieve a large error rate when the MVF also achieves a large error rate. However, in the worst case scenario, the truth discovery algorithms may exhibit a significantly higher error rate than the MVF. Finally, we confirm our theoretical results using numerical simulations.

Classifying Risks On Motor Insurance Policies For IFRS 17 Implementation In General Insurance Companies

IFRS 17 is a financial accounting standard issued by the International Financial Reporting System that regulates internationally agreed accounting treatment for insurance contracts. In an effort to increase the accuracy of risk assessment for IFRS 17 adaptation, a good way is needed to classify risks from the insured. Therefore, it is necessary to determine the risk group. Because data from an insurance company is large, the CLARA method is suitable for dealing with the problem. CLARA has a more robust nature of outliers and can be used to handle large amounts of data. After grouping, it is important to know what factors cause a person to enter a certain group. For this, classification analysis is needed. Some classification analysis methods are XGBoost, SVM, and AdaBoost. Extreme Gradient Boosting and Adaptive Boosting is a technique in machine learning for binary or multiclass regression and classification problems that results in predictive models in the form of weak predictive models. Support Vector Machine (SVM) is a technique for making predictions, both in the case of regression and binary or multiclass classification. SVM has the basic principle of linear classifier.

Merge Loss Calculation Method for Highly Imbalanced Data Multiclass Classification.

In real classification scenarios, the number distribution of modeling samples is usually out of proportion. Most of the existing classification methods still face challenges in comprehensive model performance for imbalanced data. In this article, a novel theoretical framework is proposed that establishes a proportion coefficient independent of the number distribution of modeling samples and a general merge loss calculation method independent of class distribution. The loss calculation method of the imbalanced problem focuses on both the global and batch sample levels. Specifically, the loss function calculation introduces the true-positive rate (TPR) and the false-positive rate (FPR) to ensure the independence and balance of loss calculation for each class. Based on this, global and local loss weight coefficients are generated from the entire dataset and batch dataset for the multiclass classification problem, and a merge weight loss function is calculated after unifying the weight coefficient scale. Furthermore, the designed loss function is applied to different neural network models and datasets. The method shows better performance on imbalanced datasets than state-of-the-art methods.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Development of a Novel Transformation of Spiking Neural Classifier to an Interpretable Classifier.

This article presents a new approach for providing an interpretation for a spiking neural network classifier by transforming it to a multiclass additive model. The spiking classifier is a multiclass synaptic efficacy function-based leaky-integrate-fire neuron (Mc-SEFRON) classifier. As a first step, the SEFRON classifier for binary classification is extended to handle multiclass classification problems. Next, a new method is presented to transform the temporally distributed weights in a fully trained Mc-SEFRON classifier to shape functions in the feature space. A composite of these shape functions results in an interpretable classifier, namely, a directly interpretable multiclass additive model (DIMA). The interpretations of DIMA are also demonstrated using the multiclass Iris dataset. Further, the performances of both the Mc-SEFRON and DIMA classifiers are evaluated on ten benchmark datasets from the UCI machine learning repository and compared with the other state-of-the-art spiking neural classifiers. The performance study results show that Mc-SEFRON produces similar or better performances than other spiking neural classifiers with an added benefit of interpretability through DIMA. Furthermore, the minor differences in accuracies between Mc-SEFRON and DIMA indicate the reliability of the DIMA classifier. Finally, the Mc-SEFRON and DIMA are tested on three real-world credit scoring problems, and their performances are compared with state-of-the-art results using machine learning methods. The results clearly indicate that DIMA improves the classification accuracy by up to 12% over other interpretable classifiers indicating a better quality of interpretations on the highly imbalanced credit scoring datasets.

Ordinal Regression With Pinball Loss.

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

A new computer‐aided diagnostic method for classifying anaemia disease: Hybrid use of Tree Bagger and metaheuristics

AbstractAnaemia occurs when the haemoglobin (Hgb) value falls below a certain reference range. It requires many blood tests, radiological images, and tests for diagnosis and treatment. By processing medical data from patients with artificial intelligence and machine learning methods, disease predictions can be made for newly ill individuals and decision‐support mechanisms can be created for physicians with these predictions. Thanks to these methods, which are very important in reducing the margin of error in the diagnoses made by doctors, the evaluation of data records in health institutions is also important for patients and hospitals. In this study, six hybrid models are proposed to classify non‐anaemia records, Hgb‐anaemia, folate deficiency anaemia (FDA), iron deficiency anaemia (IDA), and B12 deficiency anaemia by combining artificial intelligence and machine learning methods TreeBagger, Crow Search Algorithm (CSA), Chicken Swarm Optimization Algorithm (CSO) and JAYA methods. The proposed hybrid models are analysed with two different approaches, with/without applying the SMOTE technique to achieve high performance by better emphasizing the importance of parameters. To solve the multiclass anaemia classification problem, fuzzy logic‐based parameter optimization is applied to improve the class‐based accuracy as well as the overall accuracy in the dataset. The proposed methods are evaluated using ROC criteria to build a prediction model to determine the anaemia type of anaemic patients. As a result of the study on the dataset taken from the Kaggle database, it is observed that the six proposed hybrid methods outperformed other studies using the same dataset and similar studies in the literature.

A meta-learning network method for few-shot multi-class classification problems with numerical data

The support vector machine (SVM) method is an important basis of the current popular multi-class classification (MCC) methods and requires a sufficient number of samples. In the case of a limited number of samples, the problem of over-learning easily occurs, resulting in unsatisfactory classification. Therefore, this work investigates an MCC method that requires only a small number of samples. During model construction, raw data are converted into two-dimensional form via preprocessing. Via feature extraction, the learning network is measured and the loss function minimization principle is considered to better solve the problem of learning based on a small sample. Finally, three examples are provided to illustrate the feasibility and effectiveness of the proposed method.

Glove-Based Hand Gesture Recognition for Diver Communication.

We have developed a smart dive glove that recognizes 13 static hand gestures used in diving communication. The smart glove employs five dielectric elastomer sensors to capture finger motion and implements a machine learning classifier in the onboard electronics to recognize gestures. Five basic classification algorithms are trained and assessed: the decision tree, support vector machine (SVM), logistic regression, Gaussian naïve Bayes, and multilayer perceptron. These basic classifiers were selected as they perform well in multiclass classification problems, can be trained using supervised learning, and are model-based algorithms that can be implemented on a microprocessor. The training dataset was collected from 24 participants providing for a range of different hand sizes. After training, the algorithms were evaluated in a dry environment using data collected from ten new participants to test how well they cope with new information. Furthermore, an underwater experiment was conducted to assess any impact of the underwater environment on each algorithm's classification. The results show all classifiers performed well in a dry environment. The accuracies and F1-scores range between 0.95 and 0.98, where the logistic regressor and SVM have the highest scores for both the accuracy and F1-score (0.98). The underwater results showed that all algorithms work underwater; however, the performance drops when divers must focus on buoyancy control, breathing, and diver trim.