Multi-class Classification Problem Research Articles

Machine Learning-Based Framework to Predict Finger Movement for Prosthetic Hand

Loss of limb is critical for human life. The prosthetic hand can be an alternate for biological hand up to some extent although it can never mimic the original. This letter presents a machine learning (ML) framework to detect finger movement for the prosthetic hand. The data generated corresponding to different movements of different fingers make the classification problem as multiclass. To address this problem, four ML-based classifiers—K-nearest neighbors, decision tree, random forest, and extreme gradient boosting—have been employed. These classifiers possess inherent property of handling the multiclass classification problem. The employed classifiers have been evaluated for finger movement prediction with respect to precision, recall, F1-score, and accuracy. The experiments show that the XGBoost classifier outperformed the other multiclass classifiers in terms of accuracy.

Feature Extraction Method Based on Filter Banks and Riemannian Tangent Space in Motor-Imagery BCI.

Optimal feature extraction for multi-category motor imagery brain-computer interfaces (MI-BCIs) is a research hotspot. The common spatial pattern (CSP) algorithm is one of the most widely used methods in MI-BCIs. However, its performance is adversely affected by variance in the operational frequency band and noise interference. Furthermore, the performance of CSP is not satisfactory when addressing multi-category classification problems. In this work, we propose a fusion method combining Filter Banks and Riemannian Tangent Space (FBRTS) in multiple time windows. FBRTS uses multiple filter banks to overcome the problem of variance in the operational frequency band. It also applies the Riemannian method to the covariance matrix extracted by the spatial filter to obtain more robust features in order to overcome the problem of noise interference. In addition, we use a One-Versus-Rest support vector machine (OVR-SVM) model to classify multi-category features. We evaluate our FBRTS method using BCI competition IV dataset 2a and 2b. The experimental results show that the average classification accuracy of our FBRTS method is 77.7% and 86.9% in datasets 2a and 2b respectively. By analyzing the influence of the different numbers of filter banks and time windows on the performance of our FBRTS method, we can identify the optimal number of filter banks and time windows. Additionally, our FBRTS method can obtain more distinctive features than the filter banks common spatial pattern (FBCSP) method in two-dimensional embedding space. These results show that our proposed method can improve the performance of MI-BCIs.

Explainable Vision Transformers and Radiomics for COVID-19 Detection in Chest X-rays.

The rapid spread of COVID-19 across the globe since its emergence has pushed many countries’ healthcare systems to the verge of collapse. To restrict the spread of the disease and lessen the ongoing cost on the healthcare system, it is critical to appropriately identify COVID-19-positive individuals and isolate them as soon as possible. The primary COVID-19 screening test, RT-PCR, although accurate and reliable, has a long turn-around time. More recently, various researchers have demonstrated the use of deep learning approaches on chest X-ray (CXR) for COVID-19 detection. However, existing Deep Convolutional Neural Network (CNN) methods fail to capture the global context due to their inherent image-specific inductive bias. In this article, we investigated the use of vision transformers (ViT) for detecting COVID-19 in Chest X-ray (CXR) images. Several ViT models were fine-tuned for the multiclass classification problem (COVID-19, Pneumonia and Normal cases). A dataset consisting of 7598 COVID-19 CXR images, 8552 CXR for healthy patients and 5674 for Pneumonia CXR were used. The obtained results achieved high performance with an Area Under Curve (AUC) of 0.99 for multi-class classification (COVID-19 vs. Other Pneumonia vs. normal). The sensitivity of the COVID-19 class achieved 0.99. We demonstrated that the obtained results outperformed comparable state-of-the-art models for detecting COVID-19 on CXR images using CNN architectures. The attention map for the proposed model showed that our model is able to efficiently identify the signs of COVID-19.

Concat_CNN: A Model to Detect COVID-19 from Chest X-ray Images with Deep Learning.

COVID-19 is creating havoc on the lives of human beings all around the world. It continues to affect the normal lives of people. As number of cases are high, a cost effective and fast system is required to detect COVID-19 at appropriate time to provide the necessary healthcare. Chest X-rays have emerged as an easiest way to detect COVID-19 in no time as RT-PCR takes time to detect the infection. In this paper we propose a concatenation-based CNN model that will detect COVID-19 from chest X-rays. We have developed a multiclass classification problem which can detect and classify a chest X-ray image as either COVID + ve, or viral pneumonia, or normal. We have used chest X-rays collected from different open sources. To maintain class balancing, we took 500 images of COVID, 500 normal images, and 500 pneumonia images. We divided our dataset in training, validation, and test set in 70:10:20 ratio respectively. We used four CNNs as feature extractors from the images and concatenated their feature maps to get better efficiency of the network. After training our model for 5 folds, we have obtained around 96.31% accuracy, 95.8% precision, 92.99% recall, and 98.02% AUC. We have compared our work with state-of-the-art pretrained transfer learning algorithms and other state-of-the-art CNN models referred in different research papers. The proposed model (Concat_CNN) exhibits better accuracy than the state-of-the-art models. We hope our proposed model will help to classify chest X-rays effectively and help medical professionals with their treatment.

A decision tree-based method for ordinal classification problems

In ordinal classification problems, the class value exhibits a natural order. Usually, these problems are solved as multiclass classification problems while discarding the ordering form of the class. Recently, several research studies have proposed novel methods for ordinal classification problems while aiming to predict a predefined objective value. These methods are based on modification of the splitting criteria of decision tree-based algorithms. These methods consider the ordinal nature of the data and the magnitude of the potential classification error from a predefined predicted objective. This research aims to consolidate these methods and generalize them for any decision tree-based method while constructing decision trees according to any predefined objective value calculated from the training dataset. Furthermore, to evaluate the proposed method, a comprehensive experimental study is performed based on known ordinal datasets. The proposed method is found to significantly outperform its counterpart nonordinal models. This observation confirms that ordering information benefits ordinal methods and improves their performance. Additionally, the ordinal decision tree-based methods achieved competitive performance compared to state-of-the-art ordinal techniques. The results are validated with various performance measures that are commonly used for ordinal and non-ordinal classification problems.

A supervised discriminant data representation: application to pattern classification

The performance of machine learning and pattern recognition algorithms generally depends on data representation. That is why, much of the current effort in performing machine learning algorithms goes into the design of preprocessing frameworks and data transformations able to support effective machine learning. The method proposed in this work consists of a hybrid linear feature extraction scheme to be used in supervised multi-class classification problems. Inspired by two recent linear discriminant methods: robust sparse linear discriminant analysis (RSLDA) and inter-class sparsity-based discriminative least square regression (ICS_DLSR), we propose a unifying criterion that is able to retain the advantages of these two powerful methods. The resulting transformation relies on sparsity-promoting techniques both to select the features that most accurately represent the data and to preserve the row-sparsity consistency property of samples from the same class. The linear transformation and the orthogonal matrix are estimated using an iterative alternating minimization scheme based on steepest descent gradient method and different initialization schemes. The proposed framework is generic in the sense that it allows the combination and tuning of other linear discriminant embedding methods. According to the experiments conducted on several datasets including faces, objects, and digits, the proposed method was able to outperform competing methods in most cases.

Explaining Artificial Intelligence with Care

In the recent past, several popular failures of black box AI systems and regulatory requirements have increased the research interest in explainable and interpretable machine learning. Among the different available approaches of model explanation, partial dependence plots (PDP) represent one of the most famous methods for model-agnostic assessment of a feature’s effect on the model response. Although PDPs are commonly used and easy to apply they only provide a simplified view on the model and thus risk to be misleading. Relying on a model interpretation given by a PDP can be of dramatic consequences in an application area such as forensics where decisions may directly affect people’s life. For this reason in this paper the degree of model explainability is investigated on a popular real-world data set from the field of forensics: the glass identification database. By means of this example the paper aims to illustrate two important aspects of machine learning model development from the practical point of view in the context of forensics: (1) the importance of a proper process for model selection, hyperparameter tuning and validation as well as (2) the careful used of explainable artificial intelligence. For this purpose, the concept of explainability is extended to multiclass classification problems as e.g. given by the glass data.

COLAFOS: a hybrid machine learning model to forecast potential coseismic landslides severity

ABSTRACT Timely and rational prediction of coseismic landslides is crucial for the design and development of key infrastructure capable to protect human lives in seismically active regions. This research introduces the novel Hybrid Coseismic Landslide Forecasting model (COLAFOS) that takes into consideration three parameters namely: The Average Slope of the Active Areas, the Slope Aspect and the types of Geological forms. The developed model was tested on two datasets from the island of Lefkada Greece, for years 2003 and 2015. COLAFOS is a hybrid model, employing the Fuzzy c-Means clustering, the Ensemble Adaptive Boosting (ENS_AdaBoost) and the Ensemble Subspace k-Nearest Neighbour (ENSUB k-NN) algorithms. The introduced model managed to correctly classify the coseismic landslides according to their severity, with a success rate of 70.07% and 72.88% for 2003 and 2015, respectively. The algorithm has shown very good performance for the classes of major severity, reaching an accuracy up to 92%. Accuracy, Sensitivity, Specificity, Precision and F-1 Score, were used to evaluate the performance of the model. Given the fact that this is a multi-class classification problem, ‘One Versus All’ Strategy was used in the evaluation process. Although the datasets were relatively unbalanced, the evaluation indices sealed the efficiency of the model.

Bitcoin price change and trend prediction through twitter sentiment and data volume

Twitter sentiment has been shown to be useful in predicting whether Bitcoin’s price will increase or decrease. Yet the state-of-the-art is limited to predicting the price direction and not the magnitude of increase/decrease. In this paper, we seek to build on the state-of-the-art to not only predict the direction yet to also predict the magnitude of increase/decrease. We utilise not only sentiment extracted from tweets, but also the volume of tweets. We present results from experiments exploring the relation between sentiment and future price at different temporal granularities, with the goal of discovering the optimal time interval at which the sentiment expressed becomes a reliable indicator of price change. Two different neural network models are explored and evaluated, one based on recurrent nets and one based on convolutional networks. An additional model is presented to predict the magnitude of change, which is framed as a multi-class classification problem. It is shown that this model yields more reliable predictions when used alongside a price trend prediction model. The main research contribution from this paper is that we demonstrate that not only can price direction prediction be made but the magnitude in price change can be predicted with relative accuracy ( 63%).

Developing plastic recycling classifier by deep learning and directed acyclic graph residual network

Recycling is one of the most important approaches to safeguard the environment since it aims to reduce waste in landfills while conserving natural resources. Using deep Learning networks, this group of wastes may be automatically classified on the belts of a waste sorting plant. However, a basic set of connected layers may not be adequate to give satisfactory accuracy for such multi output classifier tasks. To optimize the gradient flow and enable deeper training for network design with multi label classifier, this study suggests a residual-based deep learning convolutional neural network. For network training, ten classes have been explored. The Directed Acyclic Graph (DAG) is a structure with hidden layers that have inputs, outputs, and other layers. The DAG network's residual-based architecture features shortcut connections that bypass some levels of the network, allowing gradients of network parameters to travel freely among the network output layers for deeper training. The methodology includes: 1) preparing the data and creating an augmented image data store; 2) defining the main serially-connected branches of the network architecture; 3) defining the residual interconnections that bypass the main branch layers; 4) defining layers, and finally; 5) creating a residual-based deeper layer graph. The concept is to split down the multiclass classification problem into minor binary states, where every classifier performs as an expert by concentrating on discriminating between only two labels, improving total accuracy. The results achieve (2.861 %) training error and (9.76 %) a validation error. The training results of this classifier are evaluated by finding the training error, validation error, and showing the confusion matrix of validation data

Dimension Reduction and Classification with Weighted Supervised PCA

Supervised principal component analysis (PCA) is a useful dimension reduction method which is able to induce more discriminative features compared to the conventional PCA in regression or classification problems. This is because supervised PCA tries to extract principal directions by considering not only data variance in input space but also the information of output variable. In this paper, we introduce a simple weighted supervised PCA and apply it to a multi-category classification problem. The weights in the proposed method represent some importance or discriminative power of each input variable for classifying the labels of the output variable. Therefore, we modify directly the correlation matrix using the weights. PCA from this adjusted correlation matrix can produce principal directions that are more useful for classifying the output labels. Some data analyses verify that the proposed method is simple to apply and show good prediction accuracy.

Deep Learning for Ocular Disease Recognition: An Inner-Class Balance.

It can be challenging for doctors to identify eye disorders early enough using fundus pictures. Diagnosing ocular illnesses by hand is time-consuming, error-prone, and complicated. Therefore, an automated ocular disease detection system with computer-aided tools is necessary to detect various eye disorders using fundus pictures. Such a system is now possible as a consequence of deep learning algorithms that have improved image classification capabilities. A deep-learning-based approach to targeted ocular detection is presented in this study. For this study, we used state-of-the-art image classification algorithms, such as VGG-19, to classify the ODIR dataset, which contains 5000 images of eight different classes of the fundus. These classes represent different ocular diseases. However, the dataset within these classes is highly unbalanced. To resolve this issue, the work suggested converting this multiclass classification problem into a binary classification problem and taking the same number of images for both classifications. Then, the binary classifications were trained with VGG-19. The accuracy of the VGG-19 model was 98.13% for the normal (N) versus pathological myopia (M) class; the model reached an accuracy of 94.03% for normal (N) versus cataract (C), and the model provided an accuracy of 90.94% for normal (N) versus glaucoma (G). All of the other models also improve the accuracy when the data is balanced.

A self-driven approach for multi-class discrimination in Alzheimer's disease based on wearable EEG

Early detection is critical to control Alzheimer's disease (AD) progression and postpone cognitive decline. Traditional medical procedures such as magnetic resonance imaging are costly, involve long waiting lists, and require complex analysis. Alternatively, for the past years, researchers have successfully evaluated AD detection approaches based on machine learning and electroencephalography (EEG). Nonetheless, these approaches frequently rely upon manual processing or involve non-portable EEG hardware. These aspects are suboptimal regarding automated diagnosis, since they require additional personnel and hinder portability. In this work, we report the preliminary evaluation of a self-driven AD multi-class discrimination approach based on a commercial EEG acquisition system using sixteen channels. For this purpose, we recorded the EEG of three groups of participants: mild AD, mild cognitive impairment (MCI) non-AD, and controls, and we implemented a self-driven analysis pipeline to discriminate the three groups. First, we applied automated artifact rejection algorithms to the EEG recordings. Then, we extracted power, entropy, and complexity features from the preprocessed epochs. Finally, we evaluated a multi-class classification problem using a multi-layer perceptron through leave-one-subject-out cross-validation. The preliminary results that we obtained are comparable to the best in literature (0.88 F1-score), what suggests that AD can potentially be detected through a self-driven approach based on commercial EEG and machine learning. We believe this work and further research could contribute to opening the door for the detection of AD in a single consultation session, therefore reducing the costs associated to AD screening and potentially advancing medical treatment.

Online Mixed-Integer Optimization in Milliseconds

We propose a method to approximate the solution of online mixed-integer optimization (MIO) problems at very high speed using machine learning. By exploiting the repetitive nature of online optimization, we can greatly speed up the solution time. Our approach encodes the optimal solution into a small amount of information denoted as strategy using the voice of optimization framework. In this way, the core part of the optimization routine becomes a multiclass classification problem that can be solved very quickly. In this work, we extend that framework to real-time and high-speed applications focusing on parametric mixed-integer quadratic optimization. We propose an extremely fast online optimization method consisting of a feedforward neural network evaluation and a linear system solution where the matrix has already been factorized. Therefore, this online approach does not require any solver or iterative algorithm. We show the speed of the proposed method both in terms of total computations required and measured execution time. We estimate the number of floating point operations required to completely recover the optimal solution as a function of the problem dimensions. Compared with state-of-the-art MIO routines, the online running time of our method is very predictable and can be lower than a single matrix factorization time. We benchmark our method against the state-of-the-art solver Gurobi obtaining up to two to three orders of magnitude speedups on examples from fuel cell energy management, sparse portfolio optimization, and motion planning with obstacle avoidance. Summary of Contribution: We propose a technique to approximate the solution of online optimization problems at high speed using machine learning. By exploiting the repetitive nature of online optimization, we learn the mapping between the key problem parameters and an encoding of the optimal solution to greatly speed up the solution time. This allows us to significantly improve the computation time and resources needed to solve online mixed-integer optimization problems. We obtain a simple method with a very low computing time variance, which is crucial in online settings.

Optical fringe analysis using support vector machine

An optical fringe analysis is very important in many scientific fields, such as optical metrology, rheology, and Fourier optics. It is proposed to analyze optical fringe patterns, obtained by a diffraction grating and Michelson’s interferometer, by treating them as training datasets for a multiclass classification problem, which is solved by using a support vector machine (SVM). The relation between a decision boundary function obtained by SVM and various optical parameters in the lab coordinate system is derived. The information that is extracted by the SVM algorithm is (1) the orientation angle of straight-line fringes; (2) the location of a smeared diffraction spot; (3) upper bounds on the spot size of each diffracted order; (4) the ratio between laser beam propagation distances; (5) the ratio between the radii of circular fringes of a Michelson’s interferogram; and (6) the orientation of elliptic fringes.

The severity prediction of the binary and multi-class cardiovascular disease − A machine learning-based fusion approach

In today’s world, a massive amount of data is available in almost every sector. This data has become an asset as we can use this enormous amount of data to find information. Mainly health care industry contains many data consisting of patient and disease-related information. By using the machine learning technique, we can look for hidden data patterns to predict various diseases. Recently CVDs, or cardiovascular disease, have become a leading cause of death around the world. The number of death due to CVDs is frightening. That is why many researchers are trying their best to design a predictive model that can save many lives using the data mining model. In this research, some fusion models have been constructed to diagnose CVDs along with its severity. Machine learning(ML) algorithms like artificial neural network, SVM, logistic regression, decision tree, random forest, and AdaBoost have been applied to the heart disease dataset to predict disease. Randomoversampler only for multi-class classification to make the imbalanced dataset balanced. To improve the performance of classification, a weighted score fusion approach was taken. At first, the models were trained. After training, two algorithms’ decision was combined using a weighted sum rule. A total of three fusion models have been developed from the six ML algorithms. The results were promising in the performance parameter. The proposed approach has been experimented with different test training ratios for binary and multiclass classification problems, and for both of them, the fusion models performed well. The highest accuracy for multiclass classification was found as 75%, and it was 95% for binary.

Relevance, redundancy, and complementarity trade-off (RRCT): A principled, generic, robust feature-selection tool

SummaryWe present a new heuristic feature-selection (FS) algorithm that integrates in a principled algorithmic framework the three key FS components: relevance, redundancy, and complementarity. Thus, we call it relevance, redundancy, and complementarity trade-off (RRCT). The association strength between each feature and the response and between feature pairs is quantified via an information theoretic transformation of rank correlation coefficients, and the feature complementarity is quantified using partial correlation coefficients. We empirically benchmark the performance of RRCT against 19 FS algorithms across four synthetic and eight real-world datasets in indicative challenging settings evaluating the following: (1) matching the true feature set and (2) out-of-sample performance in binary and multi-class classification problems when presenting selected features into a random forest. RRCT is very competitive in both tasks, and we tentatively make suggestions on the generalizability and application of the best-performing FS algorithms across settings where they may operate effectively.

Construction and Optimization of Mental Health Education Consultation Management System Based on Decision Tree Association Rule Mining

This paper studies association rule mining and decision tree algorithm, focusing on the extended research of association rule mining, including the number of generated rules, mining association rules of long itemsets with low support, attribute selection criteria and multivalue attributes in decision tree algorithm. This paper conducts in-depth research and analysis on the design and optimization of the mental health education consultation management system using the association rule decision tree algorithm. This paper analyzes the meaning of parameters under the support-confidence-interest model, and uses regression method to design equations between the number of rules and parameters. We use the multiple correlation coefficient to test the fitting effect of the equation, and use the significance test to verify whether the coefficient of the parameter is significantly zero. On the one hand, the widely used psychological crisis prevention measures generally include the screening of the SCL psychological scale in the early stage of first-year enrolment, the holding of general psychological knowledge lectures and courses, and the opening of psychological counselling rooms with a low penetration rate, but these practices are to a certain extent. In other words, it cannot enable the student administrator to grasp the psychological status of the students in a timely, effective, and dynamic manner, to timely intervene in the possible crisis. Not only the number of attribute values of the current node is considered but also the size of the variable precision clear area of the lower node is considered, that is, the two-layer nodes of the tree are considered at the same time. The new attribute selection method not only overcomes the shortcomings of the original algorithm, but also has the advantages of variable precision rough sets. This paper uses a new criterion for attribute selection, weighted roughness and complexity, which comprehensively considers the classification accuracy and the number of branches. In order to reduce the influence of noisy data and missing values, the algorithm uses a class prediction method based on matching degree. Through comparative experiments, the effectiveness of the method proposed in this paper is verified. We propose a new calculation formula for the similarity of the child nodes of the label set to evaluate the effect of attribute classification, and comprehensively consider the situation that the elements in the two multilabel sets appear or not appear at the same time, so that the calculation of the similarity of the label set is more comprehensive. The experimental results show that the model proposed in this paper can excavate the dialectical combination of multiple factors. By comparing with the existing algorithms, the classification effect of the proposed algorithm is verified. The classification algorithm proposed in this paper is more suitable for dealing with multi-value attributes and multiclass data classification problems. The psychological evaluation and counselling system designed in this paper has achieved the expected goal. The results of this paper can improve the problems existing in the work of psychological counselling services.

AI and Clinical Decision Making: The Limitations and Risks of Computational Reductionism in Bowel Cancer Screening

Advances in artificial intelligence in healthcare are frequently promoted as ‘solutions’ to improve the accuracy, safety, and quality of clinical decisions, treatments, and care. Despite some diagnostic success, however, AI systems rely on forms of reductive reasoning and computational determinism that embed problematic assumptions about clinical decision-making and clinical practice. Clinician autonomy, experience, and judgement are reduced to inputs and outputs framed as binary or multi-class classification problems benchmarked against a clinician’s capacity to identify or predict disease states. This paper examines this reductive reasoning in AI systems for colorectal cancer (CRC) to highlight their limitations and risks: (1) in AI systems themselves due to inherent biases in (a) retrospective training datasets and (b) embedded assumptions in underlying AI architectures and algorithms; (2) in the problematic and limited evaluations being conducted on AI systems prior to system integration in clinical practice; and (3) in marginalising socio-technical factors in the context-dependent interactions between clinicians, their patients, and the broader health system. The paper argues that to optimise benefits from AI systems and to avoid negative unintended consequences for clinical decision-making and patient care, there is a need for more nuanced and balanced approaches to AI system deployment and evaluation in CRC.

Object-Based Scene Classification Modeled by Hidden Markov Models Architecture

Multiclass classification problems such as document classification, medical diagnosis or scene classification are very challenging to address due to similarities between mutual classes. The use of reliable tools is necessary to get good classification results. This paper addresses the scene classification problem using objects as attributes. The process of classification is modeled by a famous mathematical tool: The Hidden Markov Models. We introduce suitable relations that scale the parameters of the Hidden Markov Model into variables of scene classification. The construction of Hidden Markov Chains is supported with weight measures and sorting functions. Lastly, inference algorithms extract most suitable scene categories from the Discrete Markov Chain. A parallelism approach constructs several Discrete Markov Chains in order to improve the accuracy of the classification process. We provide numerous tests on different datasets and compare classification accuracies with some state of the art methods. The proposed approach distinguishes itself by outperforming the other.