Multi-class Classification Problem Research Articles

Automatic classification of the severity level of Parkinson’s disease: A comparison of speaking tasks, features, and classifiers

Automatic speech-based severity level classification of Parkinson’s disease (PD) enables objective assessment and earlier diagnosis. While many studies have been conducted on the binary classification task to distinguish speakers in PD from healthy controls (HCs), clearly fewer studies have addressed multi-class PD severity level classification problems. Furthermore, in studying the three main issues of speech-based classification systems – speaking tasks, features, and classifiers – previous investigations on the severity level classification have yielded inconclusive results due to the use of only a few, and sometimes just one, type of speaking task, feature, or classifier in each study. Hence, a systematic comparison is conducted in this study between different speaking tasks, features, and classifiers. Five speaking tasks (vowel task, sentence task, diadochokinetic (DDK) task, read text task, and monologue task), four features (phonation, articulation, prosody, and their fusion), and four classifier architectures (support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), and AdaBoost) were compared. The classification task studied was a 3-class problem to classify PD severity level as healthy vs. mild vs. severe. Two MDS-UPDRS scales (MDS-UPDRS-III and MDS-UPDRS-S) were used for the ground truth severity level labels. The results showed that the use of the monologue task and the articulation and fusion of features improved classification accuracy significantly compared to the use of the other speaking tasks and features. The best classification systems resulted in a rate of accuracy of 58% (using the monologue task with the articulation features) for the MDS-UPDR-III scale and 56% (using the monologue task with fusion of features) for the MDS-UPDRS-S scale.

Multiclass Laplacian support vector machine with functional analysis of variance decomposition

In classification problems, acquiring a sufficient amount of labeled samples sometimes proves expensive and time-consuming, while unlabeled samples are relatively easier to obtain. The Laplacian Support Vector Machine (LapSVM) is one of the successful methods that learn better classification functions by incorporating unlabeled samples. However, since LapSVM was originally designed for binary classification, it can not be applied directly to multiclass classification problems commonly encountered in practice. Thus we derive an extension of LapSVM to multiclass classification problems using an appropriate multiclass formulation. Another problem with LapSVM is that irrelevant variables easily degrade classification performance. The irrelevant variables can increase the variance of predicted values and make the model difficult to interpret. Therefore, this paper also proposes the multiclass LapSVM with functional analysis of variance decomposition to identify relevant variables. Through comprehensive simulations and real-world datasets, we demonstrate the efficiency and improved classification performance of the proposed methods.

Decision Tree Learning Based Feature Selection and Evaluation for Image Classification

Abstract: The problem statement focuses on feature evaluation and selection for image classification using decision tree learning. The objective is to identify the most significant features in an image dataset and train a decision tree classifier using these selected features. The accuracy of an image classifier heavily relies on the quality and relevance of the features used to represent the images. Hence, it is crucial to identify the most important features and eliminate the irrelevant ones to enhance the classifier's accuracy. To implement this approach, we can utilize scikit-learn, a popular machine learning library in Python. The solution must involve training a decision tree classifier on the dataset and extracting feature importances, selecting the top features using modules from sklearn like “SelectFromModel”, and also performing hyperparameter tuning using “GridSearchCV” and training a new decision tree classifier on the selected features with the best hyperparameters. Decision trees are a popular machine learning algorithm that uses a tree-like model of decisions and their possible consequences. By training a decision tree classifier on an image dataset and extracting feature importances, it is possible to identify the most important features and select them for use in a new decision tree classifier that can improve classification accuracy. It is important to note that decision tree learning is a versatile machine learning algorithm that can handle both binary and multiclass classification problems. Additionally, it is advantageous for feature evaluation and selection in image classification tasks. By identifying the most relevant features, this approach can enhance the accuracy of the classifier and reduce computational complexity, making it suitable for large datasets. By following this outlined approach, you can create a project that addresses feature evaluation, selection, and classification accuracy improvement using decision tree learning in the context of image classification

Graph modeling for vocal melody extraction from polyphonic music

In this paper, a vocal melody extraction method based on graph modeling is proposed. First, constant-Q transform of mixed audio signal is applied. Then, amplitude spectra of several adjacent frames are concatenated together to construct the input feature. Afterwards, an undirected graph is constructed to model the melody extraction issue, and the frame-wise melodic pitches are estimated by a graph convolutional network (GCN), where the pitch estimation issue is regarded as a multi-class classification problem. The frequency bins are viewed as nodes and the underlying connection relationships of the frequency bins are defined as edges. Finally, the quantized frame-wise pitches are fine-tuned according to the salience function defined at a certain range of the smoothed melody trajectory based on the pitches estimated by GCN. The proposed method addresses the vocal melody extraction issue in an explainable way where the edges are defined according to the underlying connection relationships of different frequency bins. Experimental results demonstrate that the proposed method achieves good performance with light weight parameters.

STAGE: Span Tagging and Greedy Inference Scheme for Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) has become an emerging task in sentiment analysis research, aiming to extract triplets of the aspect term, its corresponding opinion term, and its associated sentiment polarity from a given sentence. Recently, many neural networks based models with different tagging schemes have been proposed, but almost all of them have their limitations: heavily relying on 1) prior assumption that each word is only associated with a single role (e.g., aspect term, or opinion term, etc. ) and 2) word-level interactions and treating each opinion/aspect as a set of independent words. Hence, they perform poorly on the complex ASTE task, such as a word associated with multiple roles or an aspect/opinion term with multiple words. Hence, we propose a novel approach, Span TAgging and Greedy infErence (STAGE), to extract sentiment triplets in span-level, where each span may consist of multiple words and play different roles simultaneously. To this end, this paper formulates the ASTE task as a multi-class span classification problem. Specifically, STAGE generates more accurate aspect sentiment triplet extractions via exploring span-level information and constraints, which consists of two components, namely, span tagging scheme and greedy inference strategy. The former tag all possible candidate spans based on a newly-defined tagging set. The latter retrieves the aspect/opinion term with the maximum length from the candidate sentiment snippet to output sentiment triplets. Furthermore, we propose a simple but effective model based on the STAGE, which outperforms the state-of-the-arts by a large margin on four widely-used datasets. Moreover, our STAGE can be easily generalized to other pair/triplet extraction tasks, which also demonstrates the superiority of the proposed scheme STAGE.

Machine Learning Algorithms for Raw and Unbalanced Intrusion Detection Data in a Multi-Class Classification Problem

Various machine learning algorithms have been applied to network intrusion classification problems, including both binary and multi-class classifications. Despite the existence of numerous studies involving unbalanced network intrusion datasets, such as CIC-IDS2017, a prevalent approach is to address the issue by either merging the classes to optimize their numbers or retaining only the most dominant ones. However, there is no consistent trend showing that accuracy always decreases as the number of classes increases. Furthermore, it is essential for cybersecurity practitioners to recognize the specific type of attack and comprehend the causal factors that contribute to the resulting outcomes. This study focuses on tackling the challenges associated with evaluating the performance of multi-class classification for network intrusions using highly imbalanced raw data that encompasses the CIC-IDS2017 and CSE-CIC-IDS2018 datasets. The research concentrates on investigating diverse machine learning (ML) models, including Logistic Regression, Random Forest, Decision Trees, CNNs, and Artificial Neural Networks. Additionally, it explores the utilization of explainable AI (XAI) methods to interpret the obtained results. The results obtained indicated that decision trees using the CART algorithm performed best on the 28-class classification task, with an average macro F1-score of 0.96878.

Affective state estimation based on Russell’s model and physiological measurements

Affective states are psycho-physiological constructs connecting mental and physiological processes. They can be represented in terms of arousal and valence according to the Russel’s model and can be extracted from physiological changes in human body. However, a well-established optimal feature set and a classification method effective in terms of accuracy and estimation time are not present in the literature. This paper aims at defining a reliable and efficient approach for real-time affective state estimation. To obtain this, the optimal physiological feature set and the most effective machine learning algorithm, to cope with binary as well as multi-class classification problems, were identified. ReliefF feature selection algorithm was implemented to define a reduced optimal feature set. Supervised learning algorithms, such as K-Nearest Neighbors (KNN), cubic and gaussian Support Vector Machine, and Linear Discriminant Analysis, were implemented to compare their effectiveness in affective state estimation. The developed approach was tested on physiological signals acquired on 20 healthy volunteers during the administration of images, belonging to the International Affective Picture System, conceived for inducing different affective states. ReliefF algorithm reduced the number of physiological features from 23 to 13. The performances of machine learning algorithms were compared and the experimental results showed that both accuracy and estimation time benefited from the optimal feature set use. Furthermore, the KNN algorithm resulted to be the most suitable for affective state estimation. The results of the assessment of arousal and valence states on 20 participants indicate that KNN classifier, adopted with the 13 identified optimal features, is the most effective approach for real-time affective state estimation.

Deep-discovery: Anomaly discovery in software-defined networks using artificial neural networks

This paper proposes Deep-Discovery, an Intrusion Detection System (IDS), to perform Anomaly Discovery in Software-Defined Networking (SDN) using Artificial Neural Network (ANN). The proposed IDS framework utilizes the Multi-Layer Perceptron (MLP), a Feedforward (FF) ANN, to detect volume-based and protocol-based Distributed Denial of Service (DDoS) attacks on the data plane of SDN. The proposed model considers the attack detection a multi-class classification problem and classifies the network traffic into six attack classes with an accuracy of 98.81% and a minimal False Alarm Rate (FAR) of 0.002. The proposed classification model addresses the binary classification problem to compare and analyze the classification performance metrics. The Deep-Discovery that deals with the binary classification problem categorizes the network traffic into anomalous and normal traffic with 99.79% accuracy and a nominal FAR of 0.0001. The novelty of this work is its emphasis on obtaining the optimal performance metrics with a simple neural network with minimal computational overhead rather than an intricate and complex model.

Subgraph Learning for Topological Geolocalization with Graph Neural Networks.

One of the challenges of spatial cognition, such as self-localization and navigation, is to develop an efficient learning approach capable of mimicking human ability. This paper proposes a novel approach for topological geolocalization on the map using motion trajectory and graph neural networks. Specifically, our learning method learns an embedding of the motion trajectory encoded as a path subgraph where the node and edge represent turning direction and relative distance information by training a graph neural network. We formulate the subgraph learning as a multi-class classification problem in which the output node IDs are interpreted as the object's location on the map. After training using three map datasets with small, medium, and large sizes, the node localization tests on simulated trajectories generated from the map show 93.61%, 95.33%, and 87.50% accuracy, respectively. We also demonstrate similar accuracy for our approach on actual trajectories generated by visual-inertial odometry. The key benefits of our approach are as follows: (1) we take advantage of the powerful graph-modeling ability of neural graph networks, (2) it only requires a map in the form of a 2D graph, and (3) it only requires an affordable sensor that generates relative motion trajectory.

MarshCover: A Web-based Tool for Estimating Vegetation Coverage in Marsh Images Using Convolutional Neural Networks

Marsh ecosystems are some of our most important, serving many crucial ecological functions. They are also rapidly changing, and it is vital for scientists to track these changes. This includes monitoring the health of marshes via estimating ground coverage by various grass species, a task that requires human labor to look at marsh images and manually estimate the coverage. Clearly, this task can be quite formidable. To automate this standard yet laborsome process, we developa web-based system, called MarshCover, that automates the process of estimating vegetation density in marsh images using convolutional neural networks (CNNs). MarshCover, to the best of our knowledge, is the first such tool available to biologists that uses CNNs for marsh vegetation estimations. In order to select effective CNN models for our MarshCover server, we conduct extensive empirical analyses of three distinct CNNs, i.e., LeNet-5, AlexNet and VGG-16, to compare their performances on a public marsh image dataset. To this end, we address two classification problems for this paper: a binary classification problem classifying points as vegetated and unvegetated, and a multiclass classification problem that classifies points into either an unvegetated class or one of five different species classes. Our experiments identify the VGG16 model as the best classifier to embed in MarshCover for both the binary classification problem and the full classification problem with a two model classifier (called two-shot). These two classifiers had accuracies on test data of 90.76% and 84% respectively. MarshCover is publicly available online.

Multiclass classification of environmental chemical stimuli from unbalanced plant electrophysiological data.

Plant electrophysiological response contains useful signature of its environment and health which can be utilized using suitable statistical analysis for developing an inverse model to classify the stimulus applied to the plant. In this paper, we have presented a statistical analysis pipeline to tackle a multiclass environmental stimuli classification problem with unbalanced plant electrophysiological data. The objective here is to classify three different environmental chemical stimuli, using fifteen statistical features, extracted from the plant electrical signals and compare the performance of eight different classification algorithms. A comparison using reduced dimensional projection of the high dimensional features via principal component analysis (PCA) has also been presented. Since the experimental data is highly unbalanced due to varying length of the experiments, we employ a random under-sampling approach for the two majority classes to create an ensemble of confusion matrices to compare the classification performances. Along with this, three other multi-classification performance metrics commonly used for unbalanced data viz. balanced accuracy, F1-score and Matthews correlation coefficient have also been analyzed. From the stacked confusion matrices and the derived performance metrics, we choose the best feature-classifier setting in terms of the classification performances carried out in the original high dimensional vs. the reduced feature space, for this highly unbalanced multiclass problem of plant signal classification due to different chemical stress. Difference in the classification performances in the high vs. reduced dimensions are also quantified using the multivariate analysis of variance (MANOVA) hypothesis testing. Our findings have potential real-world applications in precision agriculture for exploring multiclass classification problems with highly unbalanced datasets, employing a combination of existing machine learning algorithms. This work also advances existing studies on environmental pollution level monitoring using plant electrophysiological data.

Machine learning performance comparison for main propulsive shafting systems alignment

The ship shaft alignment is crucial to achieve a high-performance propulsion system. This alignment is carried out, still in drydock, by adjusting the bearings' offsets. However, there are numerous available combinations sets of vertical offsets which results in good alignment final condition. Defining an optimum set of vertical displacements is costing shipyards a valuable time. Therefore, this work proposes the use of machine learning algorithms to predict if a given set of offsets, following the mandatory rules in force, should be implemented or not. In this manner, this paper presents a case study where a finite element model of an Anchor Handling Tug Supply (AHTS) vessel’ shaft line was assembled in order to obtain the bearings reactions, shaft line deflections, shear forces, bending moments and, finally, the influence coefficients matrix (ICM). Those data were used to feed machine learning (ML) models based on four different algorithms, namely, K-Nearest-Neighbors, Extremely Randomized Trees, Random Forest and Gradient Boosting in order to solve multiclass classification problems concerning the offset's set to be applied in the alignment of the shaft line. Results showed that the Gradient Boosting and Random Forest algorithms presented the best results, achieving up to 98.1% of precision in one of the 36 scenarios tested, and an average precision above 96.0%. Such results corroborated with the authors' proposal to implement ML techniques as a way to speed up the currently very time-consuming shaft alignment process.

Multi-Class Transfer Learning and Domain Selection for Cross-Subject EEG Classification

Transfer learning (TL) has been proven to be one of the most significant techniques for cross-subject classification in electroencephalogram (EEG)-based brain-computer interfaces (BCI). Hence, it is widely used to address the challenges of cross-session and cross-subject variability with more accurate intention prediction. In this case, TL utilizes knowledge (signal features) in the source domain(s) to improve the classification in the target domain. However, current existing transfer learning approaches on EEG-based BCI are mostly limited to two-class cross-subject classification problems, while multi-class problems are only implemented with a focus on within-subject classification due to the complexity of multi-class cross-subject classification problems. In this paper, we first extended the transfer learning approaches to a multi-class cross-subject scenario, then investigated the reason for transfer learning performance being poor in multi-class cross-subject classification. Secondly, we address the challenge of significant sessional and subject-to-subject variations originating from both known and unknown factors. It is discovered that such variations have a massive influence on the classification because of the negative transfer (NT) across domains. Based on this discovery, we propose a multi-class transfer learning approach based on multi-source manifold feature transfer learning (MMFT) framework and an enhanced version to minimize the effects of NT. The proposed multi-class transfer learning approach extends the existing MMFT to multi-class cases. Then enhanced multi-class MMFT firstly searches for domains with high transferability and selects only the best combination among source domains (SD), then utilize the best-selected combination of domains for transfer learning. Experimental results illustrate that the proposed multi-class MMFT can be employed in the cross-subject classification of both three-class and four-class problems. Experimental results also demonstrated that the enhanced multi-class MMFT could effectively minimize the effect of negative transfer and significantly increase the prediction rates across individual target domains (TD). The highest classification accuracy (CA) of 98% is obtained by the enhanced multi-class MMFT.

Learning Optimal Topology for Ad-Hoc Robot Networks

In this letter, we synthesize a data-driven method to predict the optimal topology of an ad-hoc robot network. This problem is technically a multi-task classification problem. However, we divide it into a class of multi-class classification problems that can be more efficiently solved. For this purpose, we first compose an algorithm to create ground-truth optimal topologies associated with various configurations of a robot network. This algorithm incorporates a complex collection of optimality criteria that our learning model successfully manages to learn. This model is an stacked ensemble whose output is the topology prediction for a particular robot. Each stacked ensemble instance constitutes three low-level estimators whose outputs will be aggregated by a high-level boosting blender. Applying our model to a network of 10 robots displays over 80% accuracy in the prediction of optimal topologies corresponding to various configurations of the cited network.

Multi-class classification based on quantum state discrimination

We present a general framework for the problem of multi-class classification using classification functions that can be interpreted as fuzzy sets. We specialize these functions in the domain of Quantum-inspired classifiers, which are based on quantum state discrimination techniques. In particular, we use unsharp observables (Positive Operator-Valued Measures) that are determined by the training set of a given dataset to construct these classification functions. We show that such classifiers can be tested on near-term quantum computers once these classification functions are “distilled” (on a classical platform) from the quantum encoding of a training dataset. We compare these experimental results with their theoretical counterparts and we pose some questions for future research.

Critical Assessment of the Biomarker Discovery and Classification Methods for Multiclass Metabolomics.

Multiclass metabolomics has been widely applied in clinical practice to understand pathophysiological processes involved in disease progression and diagnostic biomarkers of various disorders. In contrast to the binary problem, the multiclass classification problem is more difficult in terms of obtaining reliable and stable results due to the increase in the complexity of determining exact class decision boundaries. In particular, methods of biomarker discovery and classification have a significant effect on the multiclass model because different methods with significantly varied theories produce conflicting results even for the same dataset. However, a systematic assessment for selecting the most appropriate methods of biomarker discovery and classification for multiclass metabolomics is still lacking. Therefore, a comprehensive assessment is essential to measure the suitability of methods in multiclass classification models from multiple perspectives. In this study, five biomarker discovery methods and nine classification methods were assessed based on four benchmark datasets of multiclass metabolomics. The performance assessment of the biomarker discovery and classification methods was performed using three evaluation criteria: assessment a (cluster analysis of sample grouping), assessment b (biomarker consistency in multiple subgroups), and assessment c (accuracy in the classification model). As a result, 13 combining strategies with superior performance were selected under multiple criteria based on these benchmark datasets. In conclusion, superior strategies that performed consistently well are suggested for the discovery of biomarkers and the construction of a classification model for multiclass metabolomics.

A joint training network for learning more distinguishable relation features in relation classification

Relation classification is an important task in natural language processing, which aims to predict the semantic relation between a given entity pair in a sentence. There are datasets, like TACRED, that contain a large number of “no_relation” type samples. Most existing methods treat “no_relation” and normal relation types equally, and directly apply the softmax function over all relation types. In this paper, we propose a novel joint training network to learn more distinguishable relation features for relation classification. Specially, we convert the original multi-class classification problem into two joint optimized modules, binary classification of whether a relation is “no_relation” and multi-class classification of normal relation types. To further differentiate between similar normal relation types, we introduce a self-supervised contrastive learning method to learn more distinguishable features for them. We jointly optimize the above modules. Experimental results agree well with our design intention and demonstrate that our joint training network not only achieves superior performance against existing competitive models, but also is robust to “no_relation” problem.

Deep learning for understanding multilabel imbalanced Chest X-ray datasets

Over the last few years, convolutional neural networks (CNNs) have dominated the field of computer vision thanks to their ability to extract features and their outstanding performance in classification problems, for example in the automatic analysis of X-rays. Unfortunately, these neural networks are considered black-box algorithms, i.e. it is impossible to understand how the algorithm has achieved the final result. To apply these algorithms in different fields and test how the methodology works, we need to use eXplainable AI techniques. Most of the work in the medical field focuses on binary or multiclass classification problems. However, in many real-life situations, such as chest X-rays, radiological signs of different diseases can appear at the same time. This gives rise to what is known as ”multilabel classification problems”. A disadvantage of these tasks is class imbalance, i.e. different labels do not have the same number of samples. The main contribution of this paper is a Deep Learning methodology for imbalanced, multilabel chest X-ray datasets. It establishes a baseline for the currently underutilised PadChest dataset and a new eXplainable AI technique based on heatmaps. This technique also includes probabilities and inter-model matching. The results of our system are promising, especially considering the number of labels used. Furthermore, the heatmaps match the expected areas, i.e. they mark the areas that an expert would use to make a decision.

A low-resolution real-time face recognition using extreme learning machine and its variants

ABSTRACT Extreme Learning Machine (ELM) is an excellent candidate for its exemplary performance as a generalized Single Layer Feed-forward Network (SLFN). In this regard, ELM has attracted widespread attention for tackling multiclass classification and regression problems with relative ease. Due to the random allocation of weights to the input and hidden layer biases, the running time spans of ELM are observed to be in seconds and agree with real-time events whilst giving a competitive performance. In this work, the suitability of ELM and its Online Sequential variant for recognition of unknown face samples is investigated under uncontrolled environments, including varying pose, variance, and illumination. The presented face recognition approach is evaluated sing four datasets from YALE, CMU, BIOID, and LFW. To track the face features of a sample face in its spatial domain with varying windows corresponding to different datasets, the Viola-Jones object detection framework is used. The face features are then extracted and evaluated using a Histogram of Oriented Gradients, which forms the dataset to be fed to ELM and OS-ELM classifiers. The running time for training the proposed face recognition approach is found to span within milliseconds to seconds with the computed time complexity of , justifying the suitability of ELM and OS-ELM classifiers for real-time face recognition applications with a high recognition rate outcome.

TSMC-Net: Deep-Learning Multigas Classification Using THz Absorption Spectra.

The identification of gas mixture speciation from a complex multicomponent absorption spectrum is a problem in gas sensing that can be addressed using machine-learning approaches. Here, we report on a deep convolutional neural network for multigas classification using terahertz (THz) absorption spectra, THz spectra mixture classifier network or TSMC-Net. TSMC-Net has been developed to identify eight volatile organic compounds in mixtures based on their fingerprint rotational absorption spectra in the 220-330 GHz frequency range. A data set consisting of simulated absorption spectra for randomly generated mixtures, with absorption greater than thresholds representing detectable limits and annotated with multiple labels, was prepared for model development. The supervised multilabel classification problem, i.e., the identification of individual gases in a mixture, is converted to a supervised multiclass classification problem via label powerset conversion. The trained model is validated and tested against simulated spectra for gas mixtures, with and without white Gaussian noise. The trained model exhibits high precision, recall, and accuracy for each pure compound. Class activation maps illustrate the complex decision-making process of the model and highlight relevant frequency regions that are needed to identify unique mixtures. Finally, the model was demonstrated against measured THz absorption spectra for pure species and mixtures, acquired using a microelectronics-based THz absorption spectrometer. The data set generation strategy and deep convolutional neural network approach are generalized and can be extrapolated to other spectroscopy types, frequency ranges, and sensors.