Multi-class Learning Research Articles

Joint segmentation of sternocleidomastoid and skeletal muscles in computed tomography images using a multiclass learning approach

Deep-learning-based methods can improve robustness against individual variations in computed tomography (CT) images of the sternocleidomastoid muscle, which is a challenge when using conventional methods based on probabilistic atlases are used for automatic segmentation. Thus, this study proposes a novel multiclass learning approach for the joint segmentation of the sternocleidomastoid and skeletal muscles in CT images, and it employs a two-dimensional U-Net architecture. The proposed method concurrently learns and segmented segments the sternocleidomastoid muscle and the entire skeletal musculature. Consequently, three-dimensional segmentation results are generated for both muscle groups. Experiments conducted on a dataset of 30 body CT images demonstrated segmentation accuracies of 82.94% and 92.73% for the sternocleidomastoid muscle and entire skeletal muscle compartment, respectively. These results outperformed those of conventional methods, such as the single-region learning of a target muscle and multiclass learning of specific muscle pairs. Moreover, the multiclass learning paradigm facilitated a robust segmentation performance regardless of the input image range. This highlights the method’s potential for cases that present muscle atrophy or reduced muscle strength. The proposed method exhibits promising capabilities for the high-accuracy joint segmentation of the sternocleidomastoid and skeletal muscles and is effective in recognizing skeletal muscles, thus, it holds promise for integration into computer-aided diagnostic systems for comprehensive musculoskeletal analysis. These findings are expected to enhance medical image analysis techniques and their applications in clinical decision support systems.

Hierarchical Label Distribution Learning for Disease Prediction.

The prediction of disease can facilitate early intervention, comprehensive diagnosis and treatment, thereby benefiting healthcare and reducing medical costs. While single class and multi-class learning methods have been applied for disease prediction, they are inadequate in distinguishing between primary and secondary diagnoses, which is crucial for treatments. In this paper, label distribution is suggested to describe the diagnosis, which assigns the description degree to quantify the diagnosis. Additionally, a novel hierarchical label distribution learning (HLDL) model is proposed to make fine-grained predictions based on the hierarchical classification of diseases, taking into account the relationship among diseases. The experimental results on real-world datasets demonstrate that the HLDL model outperforms the baselines with statistical significance.

Collaborative and dynamic kernel discriminant analysis for large-scale problems: applications in multi-class learning and novelty detection

Collaborative and dynamic kernel discriminant analysis for large-scale problems: applications in multi-class learning and novelty detection

Robust Supervised and Semisupervised Least Squares Regression Using ℓ2,p-Norm Minimization.

Least squares regression (LSR) is widely applied in statistics theory due to its theoretical solution, which can be used in supervised, semisupervised, and multiclass learning. However, LSR begins to fail and its discriminative ability cannot be guaranteed when the original data have been corrupted and noised. In reality, the noises are unavoidable and could greatly affect the error construction in LSR. To cope with this problem, a robust supervised LSR (RSLSR) is proposed to eliminate the effect of noises and outliers. The loss function adopts l2,p -norm ( ) instead of square loss. In addition, the probability weight is added to each sample to determine whether the sample is a normal point or not. Its physical meaning is very clear, in which if the point is normal, the probability value is 1; otherwise, the weight is 0. To effectively solve the concave problem, an iterative algorithm is introduced, in which additional weights are added to penalize normal samples with large errors. We also extend RSLSR to robust semisupervised LSR (RSSLSR) to fully utilize the limited labeled samples. A large number of classification performances on corrupted data illustrate the robustness of the proposed methods.

An Efficient Multi-Class Privacy-Preserving-Based Encryption Framework for Large Distributed Databases

This paper introduces a novel hybrid filter-based ensemble multi-class classification model for distributed privacy-preserving applications. The conventional privacy-preserving multi-class learning models have limited capacity to enhance the true positive rate, mainly due to computational time and memory constraints, as well as the static nature of metrics for parameter optimization and multi-class perturbation processes. In this research, we develop the proposed model on large medical and market databases with the aim of enhancing multi-party data confidentiality through a security framework during the privacy-preserving process. Moreover, we also introduce a secure multi-party data perturbation process to improve computational efficiency and privacy-preserving performance. Experimental results were evaluated on different real-time privacy-preserving datasets, such as medical and market datasets, using different statistical metrics. The evaluation results demonstrate that the proposed multi-party-based multi-class privacy-preserving model performs statistically better than conventional approaches.

Building hierarchical class structures for extreme multi-class learning

Building hierarchical class structures for extreme multi-class learning

One-Class Arabic Signature Verification: A Progressive Fusion of Optimal Features

Signature verification is regarded as the most beneficial behavioral characteristic-based biometric feature in security and fraud protection. It is also a popular biometric authentication technology in forensic and commercial transactions due to its various advantages, including noninvasiveness, user-friendliness, and social and legal acceptability. According to the literature, extensive research has been conducted on signature verification systems in a variety of languages, including English, Hindi, Bangla, and Chinese. However, the Arabic Offline Signature Verification (OSV) system is still a challenging issue that has not been investigated as much by researchers due to the Arabic script being distinguished by changing letter shapes, diacritics, ligatures, and overlapping, making verification more difficult. Recently, signature verification systems have shown promising results for recognizing signatures that are genuine or forgeries; however, performance on skilled forgery detection is still unsatisfactory. Most existing methods require many learning samples to improve verification accuracy, which is a major drawback because the number of available signature samples is often limited in the practical application of signature verification systems. This study addresses these issues by presenting an OSV system based on multifeature fusion and discriminant feature selection using a genetic algorithm (GA). In contrast to existing methods, which use multiclass learning approaches, this study uses a one-class learning strategy to address imbalanced signature data in the practical application of a signature verification system. The proposed approach is tested on three signature databases (SID-Arabic handwriting signatures, CEDAR (Center of Excellence for Document Analysis and Recognition), and UTSIG (University of Tehran Persian Signature), and experimental results show that the proposed system outperforms existing systems in terms of reducing the False Acceptance Rate (FAR), False Rejection Rate (FRR), and Equal Error Rate (ERR). The proposed system achieved 5% improvement.

Feature selection based on ACO and knowledge graph for Arabic text classification

ABSTRACT Nowadays, multi-class learning processes have attracted a vast amount of attention due to their complexities and progressive development in various fields. This study focuses on the prediction of semantic categories of Arabic hadith texts by combining the Knowledge-Graph (KG) with the Ant Colony Optimisation (ACO) method. Knowledge-Graph is constructed based on the correlation between features and categories. The links’ weights depend on how frequently the features and the categories occur together, either directly or indirectly. Knowledge-Graph also directly gauges the weights of the links among features. Finally, the ants traverse the graph to select features the most prominent. Features are important whenever they are mentioned frequently in Hadith texts, and they belong to one or fewer categories. It is worth considering that the more relationships between one feature and multiple categories, the less likely it is to be selected. In our analytical experiments, six famous books of the Prophetic Sunnah were used, including more than thirty thousand hadiths with a number of categories slightly under 120. On the experimental side, we found promising results when the ACO-KG model has combined with machine learning classifiers, namely an increase of 3% in the classification task.

Detecting relevant app reviews for software evolution and maintenance through multimodal one-class learning

Mobile app reviews are a rich source of information for software evolution and maintenance. Several studies have shown the effectiveness of exploring relevant reviews in the software development lifecycle , such as release planning and requirements engineering tasks. Popular apps receive even millions of reviews, thereby making manual extraction of relevant information an impractical task. The literature presents several machine learning approaches to detect relevant reviews. However, these approaches use multi-class learning, implying more user effort for data labeling since users must label a significant set of relevant and irrelevant reviews. This article investigates methods for detecting relevant app reviews considering scenarios with small sets of labeled data. We evaluated unimodal and multimodal representations, different labeling levels, as well as different app review domains and languages. We present a one-class multimodal learning method for detecting relevant reviews. Our approaches have two main contributions. First, we use one-class learning that requires only the labeling of relevant app reviews, thereby minimizing the labeling effort. Second, to handle the smaller amount of labeled reviews without harming classification performance, we also present methods to improve feature extraction and reviews representation. We propose the Multimodal Autoencoder and the Multimodal Variational Autoencoder. The methods learn representations which explore both textual data and visual information based on the density of the reviews. Density information can be interpreted as a summary of the main topics or clusters extracted from the reviews. Our methods achieved competitive results even using only 25% of labeled reviews compared to models that used the entire training set. Also, our multimodal approaches obtain the highest F 1 -Score and AUC-ROC in twenty-three out of twenty-four scenarios. Our one-class multimodal methods proved to be a competitive alternative for detecting relevant reviews and promising for practical scenarios involving data-driven software evolution and maintenance.

Towards convergence rate analysis of random forests for classification

Random forests have been one of the successful ensemble algorithms in machine learning, and the basic idea is to construct a large number of random trees individually and make predictions based on an average of their predictions. The great successes have attracted much attention on theoretical understandings of random forests, mostly focusing on regression problems. This work takes one step towards the convergence rates of random forests for classification. We present the first finite-sample rate O(n−1/(8d+2)) on the convergence of purely random forests for binary classification, which can be improved to be of O(n−1/(3.87d+2)) by considering the midpoint splitting mechanism. We introduce another variant of random forests, which follows Breiman's original random forests but with different mechanisms on splitting dimensions and positions. We present the convergence rate O(n−1/(d+2)(ln⁡n)1/(d+2)) for the variant of random forests, which reaches the minimax rate, except for a factor (ln⁡n)1/(d+2), of the optimal plug-in classifier under the L-Lipschitz assumption. We achieve the tighter convergence rate O(ln⁡n/n) under some assumptions over structural data. This work also takes one step towards the convergence rate of random forests for multi-class learning, and presents the same convergence rates of random forests for multi-class learning as that of binary classification, yet with different constants. We finally provide empirical studies to support the theoretical analysis.

Non-Destructive Defect Detection Using MASK R-CNN on Industrial Radiography of Machines

Abstract: Many industrial processes, particularly those requiring casting or welding, rely heavily on quality control. Manual quality control processes, on the other hand, are frequently time-consuming and error-prone. To address the increased demand for high-quality products, sophisticated visual inspection technologies are becoming increasingly important in manufacturing lines. Convolutional Neural Networks have recently demonstrated exceptional performance in image classification and localization tasks. Based on the Mask Region-based CNN architecture, this research proposes a solution for detecting casting errors in X-ray pictures. The suggested defect detection system conducts flaw identification and segmentation on input pictures at the same time, making it appropriate for a variety of defect detection jobs. It is demonstrated that training the network to conduct defect detection and defect instance segmentation at the same time leads in greater defect detection accuracy than training on defect detection alone. Transfer learning is used to minimizetraining data requirements while increasing the trained model's prediction accuracy. More precisely, the model is trained using two huge publically available picture datasets before being fine-tuned using a relatively modest metal casting X-ray dataset.The trained model's accuracy outperforms state-of-the-art performance on the GRIMA database of Xray images (GDXray) Castings dataset and is quick enough to be deployed in production. On the GDXray Welds dataset, the system likewise works well.A variety of in-depth research are being undertaken to investigate how transfer learning, multi-task learning, and multi-class learning affect the trained system's performance.

Multi-label, multi-domain learning identifies compounding effects of HIV and cognitive impairment.

Older individuals infected by Human Immunodeficiency Virus (HIV) are at risk for developing HIV-Associated Neurocognitive Disorder (HAND), i.e., from reduced cognitive functioning similar to HIV-negative individuals with Mild Cognitive Impairment (MCI) or to Alzheimer's Disease (AD) if more severely affected. Incompletely understood is how brain structure can serve to differentiate cognitive impairment (CI) in the HIV-positive (i.e., HAND) from the HIV-negative cohort (i.e., MCI and AD). To that end, we designed a multi-label classifier that labels the structural magnetic resonance images (MRI) of individuals by their HIV and CI status via two binary variables. Proper training of such an approach traditionally requires well-curated datasets containing large number of samples for each of the corresponding four cohorts (healthy controls, CI HIV-negative adults a.k.a. CI-only, HIV-positive patients without CI a.k.a. HIV-only, and HAND). Because of the rarity of such datasets, we proposed to improve training of the multi-label classifier via a multi-domain learning scheme that also incorporates domain-specific classifiers on auxiliary single-label datasets specific to either binary label. Specifically, we complement the training dataset of MRIs of the four cohorts (Control: 156, CI-only: 335, HIV-only: 37, HAND: 145) acquired by the Memory and Aging Center at the University of California - San Francisco with a CI-specific dataset only containing MRIs of HIV-negative subjects (Controls: 229, CI-only: 397) from the Alzheimer's Disease Neuroimaging Initiative and an HIV-specific dataset (Controls: 75, HIV-only: 75) provided by SRI International. Based on cross-validation on the UCSF dataset, the multi-domain and multi-label learning strategy leads to superior classification accuracy compared with one-domain or multi-class learning approaches, specifically for the undersampled HIV-only cohort. The 'prediction logits' of CI computed by the multi-label formulation also successfully stratify motor performance among the HIV-positive subjects (including HAND). Finally, brain patterns driving the subject-level predictions across all four cohorts characterize the independent and compounding effects of HIV and CI in the HAND cohort.

A Multiclass Nonparallel Parametric-Margin Support Vector Machine

The twin parametric-margin support vector machine (TPMSVM) is an excellent kernel-based nonparallel classifier. However, TPMSVM was originally designed for binary classification, which is unsuitable for real-world multiclass applications. Therefore, this paper extends TPMSVM for multiclass classification and proposes a novel K multiclass nonparallel parametric-margin support vector machine (MNP-KSVC). Specifically, our MNP-KSVC enjoys the following characteristics. (1) Under the “one-versus-one-versus-rest” multiclass framework, MNP-KSVC encodes the complicated multiclass learning task into a series of subproblems with the ternary output {−1,0,+1}. In contrast to the “one-versus-one” or “one-versus-rest” strategy, each subproblem not only focuses on separating the two selected class instances but also considers the side information of the remaining class instances. (2) MNP-KSVC aims to find a pair of nonparallel parametric-margin hyperplanes for each subproblem. As a result, these hyperplanes are closer to their corresponding class and at least one distance away from the other class. At the same time, they attempt to bound the remaining class instances into an insensitive region. (3) MNP-KSVC utilizes a hybrid classification and regression loss joined with the regularization to formulate its optimization model. Then, the optimal solutions are derived from the corresponding dual problems. Finally, we conduct numerical experiments to compare the proposed method with four state-of-the-art multiclass models: Multi-SVM, MBSVM, MTPMSVM, and Twin-KSVC. Experimental results demonstrate the feasibility and effectiveness of MNP-KSVC in terms of multiclass accuracy and learning time.

Adaptive Decision Support System for On-Line Multi-Class Learning and Object Detection

A new online multi-class learning algorithm is proposed with three main characteristics. First, in order to make the feature pool fitter for the pattern pool, the adaptive feature pool is proposed to dynamically combine the three general features, Haar-like, Histogram of Oriented Gradient (HOG), and Local Binary Patterns (LBP). Second, the external model is integrated into the proposed model without re-training to enhance the efficacy of the model. Third, a new multi-class learning and updating mechanism are proposed that help to find unsuitable decisions and adjust them automatically. The performance of the proposed model is validated with multi-class detection and online learning system. The proposed model achieves a better score than other non-deep learning algorithms used in public pedestrian and multi-class databases. The multi-class databases contain data for pedestrians, faces, vehicles, motorcycles, bicycles, and aircraft.

Intra-Pulse Modulation Recognition of Dual-Component Radar Signals Based on Deep Convolutional Neural Network

In an ever-increasingly complicated electromagnetic environment with explosive radar signals density, accurate and fast recognition of dual-component radar signals has become an urgent problem in the current radar reconnaissance system. This letter proposes a novel multi-class learning framework based on deep convolutional neural network (DCNN) for recognizing eight types of randomly overlapping dual-component radar signals. The framework mainly includes dual-component radar signals preprocessing, DCNN model that aimed to extract more effective dual-component radar signals features, and multi-class classification. The results shown that the average classification accuracy of dual-component radar signals can be up to 96.17% when the signal-to-noise ratio (SNR) is −8 dB, which demonstrates the superior performance over others. The proposed model possesses the larger improvement for 4FSK, BPSK, EQFM, and FRANK, especially at the lower SNR. This work provides a sound guidance for further improving multi-component radar signals recognition.

Performance and Preferences: Interactive Refinement of Machine Learning Procedures

Problem-solving procedures have been typically aimed at achieving well-defined goals or satisfying straightforward preferences. However, learners and solvers may often generate rich multiattribute results with procedures guided by sets of controls that define different dimensions of quality. We explore methods that enable people to explore and express preferences about the operation of classification models in supervised multiclass learning. We leverage a leave-one-out confusion matrix that provides users with views and real-time controls of a model space. The approach allows people to consider in an interactive manner the global implications of local changes in decision boundaries. We focus on kernel classifiers and show the effectiveness of the methodology on a variety of tasks.

DGA-based botnet detection toward imbalanced multiclass learning

Botnets based on the Domain Generation Algorithm (DGA) mechanism pose great challenges to the main current detection methods because of their strong concealment and robustness. However, the complexity of the DGA family and the imbalance of samples continue to impede research on DGA detection. In the existing work, the sample size of each DGA family is regarded as the most important determinant of the resampling proportion; thus, differences in the characteristics of various samples are ignored, and the optimal resampling effect is not achieved. In this paper, a Long Short-Term Memory-based Property and Quantity Dependent Optimization (LSTM.PQDO) method is proposed. This method takes advantage of LSTM to automatically mine the comprehensive features of DGA domain names. It iterates the resampling proportion with the optimal solution based on a comprehensive consideration of the original number and characteristics of the samples to heuristically search for a better solution around the initial solution in the right direction; thus, dynamic optimization of the resampling proportion is realized. The experimental results show that the LSTM.PQDO method can achieve better performance compared with existing models to overcome the difficulties of unbalanced datasets; moreover, it can function as a reference for sample resampling tasks in similar scenarios.

Mahalanobis-ANOVA criterion for optimum feature subset selection in multi-class planetary gear fault diagnosis

The empirical analysis of a typical gear fault diagnosis of five different classes has been studied in this article. The analysis was used to develop novel feature selection criteria that provide an optimum feature subset over feature ranking genetic algorithms for improving the planetary gear fault classification accuracy. We have considered traditional approach in the fault diagnosis, where the raw vibration signal was divided into fixed-length epochs, and statistical time-domain features have been extracted from the segmented signal to represent the data in a compact discriminative form. Scale-invariant Mahalanobis distance–based feature selection using ANOVA statistic test was used as a feature selection criterion to find out the optimum feature subset. The Support Vector Machine Multi-Class machine learning algorithm was used as a classification technique to diagnose the gear faults. It has been observed that the highest gear fault classification accuracy of 99.89% (load case) was achieved by using the proposed Mahalanobis-ANOVA Criterion for optimum feature subset selection followed by Support Vector Machine Multi-Class algorithm. It is also noted that the developed feature selection criterion is a data-driven model which will contemplate all the nonlinearity in a signal. The fault diagnosis consistency of the proposed Support Vector Machine Multi-Class learning algorithm was ensured through 100 Monte Carlo runs, and the diagnostic ability of the classifier has been represented using confusion matrix and receiver operating characteristics.

Anomaly recognition from surveillance videos using 3D convolution neural network

Anomalous activity recognition deals with identifying the patterns and events that vary from the normal stream. In a surveillance paradigm, these events range from abuse to fighting and road accidents to snatching, etc. Due to the sparse occurrence of anomalous events, anomalous activity recognition from surveillance videos is a challenging research task. The approaches reported can be generally categorized as handcrafted and deep learning-based. Most of the reported studies address binary classification i.e. anomaly detection from surveillance videos. But these reported approaches did not address other anomalous events e.g. abuse, fight, road accidents, shooting, stealing, vandalism, and robbery, etc. from surveillance videos. Therefore, this paper aims to provide an effective framework for the recognition of different real-world anomalies from videos. This study provides a simple, yet effective approach for learning spatiotemporal features using deep 3-dimensional convolutional networks (3D ConvNets) trained on the University of Central Florida (UCF) Crime video dataset. Firstly, the frame-level labels of the UCF Crime dataset are provided, and then to extract anomalous spatiotemporal features more efficiently a fine-tuned 3D ConvNets is proposed. Findings of the proposed study are twofold 1) There exist specific, detectable, and quantifiable features in UCF Crime video feed that associate with each other 2) Multiclass learning can improve generalizing competencies of the 3D ConvNets by effectively learning frame-level information of dataset and can be leveraged in terms of better results by applying spatial augmentation. The proposed study extracted 3D features by providing frame level information and spatial augmentation to a fine-tuned pre-trained model, namely 3DConvNets. Besides, the learned features are compact enough and the proposed approach outperforms significantly from state of art approaches in terms of accuracy on anomalous activity recognition having 82% AUC.

Classifier Selection and Ensemble Model for Multi-class Imbalance Learning in Education Grants Prediction

ABSTRACT Ensemble learning combines base classifiers to improve the performance of the models and obtains a higher classification accuracy than a single classifier. We propose a multi-classification method to predict the level of grant for each college student based on feature integration and ensemble learning. It extracted from expense, score, in/out dormitory, book loan conditions of 10885 students’ daily behavior data and constructed a 21-dimensional feature. The ensemble learning method integrated gradient boosting decision tree, random forest, AdaBoost, and Support Vector Machine classifiers for college grant classification. The proposed method is evaluated with 10885 students set and experiments show that the proposed method has an average accuracy of 0.954 5 and can be used as an effective means of assisting decision-making for college student grants.