Multi-class Data Classification Research Articles

A Classification Algorithm Inspired by the Chromatographic Separation Mechanism Dedicated to the Classification of Variable-length and Multi-class Vectors

Nowadays, one of the critical problems related to data mining is the processing of large data sets. This article presents an algorithm that may apply to the issues associated with classifying large-volume data sets. The motivation behind defining this type of algorithm was that the methods used to process this data type are subject to several significant limitations. The first considerable limitation of using classical classification methods is ensuring a constant data size. The second type of constraint is related to the data dimension. The last limitation in using classic classification algorithms is associated with the situation in which a given input vector may contain data belonging to many classes simultaneously, in which case we are talking about so-called multi-class vectors. The presented algorithm is inspired by the method of chromatographic separation of chemical substances. This method is widely and successfully used in analytical chemistry. As we know, in the case of chromatographic separation, we are dealing with a similar class of problems that occur when processing large data sets, firstly: the molecules of a chemical substance have a different number of molecules - i.e., they have different lengths, which corresponds to the situation that occurs when processing large data sets. In this work, a classification algorithm inspired by the mechanism of resolution chromatography is presented. The article presents the results of calculations for sample data sets. It discusses issues related to the properties of the defined algorithm, which concern the algorithm training process and the classification of single-class and multi-class data.

Merge Loss Calculation Method for Highly Imbalanced Data Multiclass Classification.

In real classification scenarios, the number distribution of modeling samples is usually out of proportion. Most of the existing classification methods still face challenges in comprehensive model performance for imbalanced data. In this article, a novel theoretical framework is proposed that establishes a proportion coefficient independent of the number distribution of modeling samples and a general merge loss calculation method independent of class distribution. The loss calculation method of the imbalanced problem focuses on both the global and batch sample levels. Specifically, the loss function calculation introduces the true-positive rate (TPR) and the false-positive rate (FPR) to ensure the independence and balance of loss calculation for each class. Based on this, global and local loss weight coefficients are generated from the entire dataset and batch dataset for the multiclass classification problem, and a merge weight loss function is calculated after unifying the weight coefficient scale. Furthermore, the designed loss function is applied to different neural network models and datasets. The method shows better performance on imbalanced datasets than state-of-the-art methods.

Imbalanced Multiclass Data Classification Using Combined Data Sampling and Deep Learning Method

Multiclass Classification for finding pattern refers to classifying each data to part of the classes or labels that are generally more than two. The foremost challenge in classifying is with imbalanced data that have large portion data known to be majority class, and small portion known as minority class that leads to poor understanding of samples and less accurate results. The existing works discussed Random Upsampling, Random Downsampling, SMOTE methods individually with FeedForward Neural Network and found Random Oversampling gave better results .However, it generates more duplicate data and has less accuracy. Hence , this research work put forward Combined Random Over-Under Sampling approach that was preprocessed prior with Replacing Missing value with mean, Feature selection, Noise Filtering. Meanwhile this work extends the existing FeedForward Neural Network to Deep Learning . The proposed work is implemented in Rapidminer tool, assessed with appropriate evaluation measures for training and testing data individually.

3DMASC: Accessible, explainable 3D point clouds classification. Application to bi-spectral topo-bathymetric lidar data

Three-dimensional data have become increasingly present in earth observation over the last decades. However, many 3D surveys are still underexploited due to the lack of accessible and explainable automatic classification methods, for example, new topo-bathymetric lidar data. In this work, we introduce explainable machine learning for 3D data classification using Multiple Attributes, Scales, and Clouds under 3DMASC, a new workflow. This workflow introduces multi-cloud classification through dual-cloud features, encrypting local spectral and geometrical ratios and differences. 3DMASC uses classical multi-scale descriptors adapted to all types of 3D point clouds and new ones based on their spatial variations. In this paper, we present the performances of 3DMASC for multi-class classification of topo-bathymetric lidar data in coastal and fluvial environments. We show how multivariate and embedded feature selection allows the building of optimized predictor sets of reduced complexity, and we identify features particularly relevant for coastal and riverine scene descriptions. Our results show the importance of dual-cloud features, lidar return-based attributes averaged over specific scales, and of statistics of dimensionality-based and spectral features. Additionally, they indicate that small to medium spherical neighbourhood diameters (<7 m) are sufficient to build effective classifiers, namely when combined with distance-to-ground or distance-to-water-surface features. Without using optional RGB information, and with a maximum of 37 descriptors, we obtain classification accuracies between 91 % for complex multi-class tasks and 98 % for lower-level processing using models trained on less than 2000 samples per class. Comparisons with classical point cloud classification methods show that 3DMASC features have a significantly improved descriptive power. Our contributions are made available through a plugin in the CloudCompare software, allowing non-specialist users to create classifiers for any type of 3D data characterized by 1 or 2 point clouds (airborne or terrestrial lidar, structure from motion), and two labelled topo-bathymetric lidar datasets, available on https://opentopography.org/.

Hybrid filtering methods for feature selection in high-dimensional cancer data

&lt;span lang="EN-US"&gt;Statisticians in both academia and industry have encountered problems with high-dimensional data. The rapid feature increase has caused the feature count to outstrip the instance count. There are several established methods when selecting features &lt;span&gt;from massive amounts of breast cancer data. Even so, overfitting continues to be a problem. The challenge of choosing important features with minimum loss in a different sample size is another area with room for development. As a result, the feature selection technique is crucial for dealing with high-dimensional&lt;/span&gt; data classification issues. This paper proposed a new architecture for high-dimensional breast cancer data using filtering techniques and a logistic regression model. Essential features are filtered out using a combination of hybrid chi–square and hybrid information gain (hybrid IG) with logistic regression as classifier. The results showed that hybrid IG performed the best for high-dimensional breast and prostate cancer data. The top 50 and 22 features outperformed the other configurations, with the highest classification accuracies of 86.96% and 82.61%, respectively, after integrating the hybrid &lt;span&gt;information gain and logistic function (hybrid IG+LR) with a sample size of 75. In the future, multiclass classification of multidimensional medical data to be evaluated using data from&lt;/span&gt; a different domain&lt;span&gt;.&lt;/span&gt;&lt;/span&gt;

Customer Segmentation Analysis Using Random Forest &amp; Naïve Bayes Method In The Case of Multi-Class Classification at PT. XYZ

Cases of the COVID-19 pandemic are gradually decreasing every day in Indonesia, but the impact of the COVID-19 pandemic has greatly affected various sectors, especially the economy and business. Sales transactions have not yet reached the company's target due to weak public purchasing power. The accuracy of customer segmentation analysis and attractive promo voucher offers are needed to increase the opportunity for people's purchasing power for a product. This study aimed to predict the level of customer purchasing power using the random forest and naïve Bayes methods in the case of multi-class data classification at PT. XYZ. The classification is carried out to determine the type of promo voucher suitable to be offered to customers according to the level of customer purchasing power. The data used is historical daily transaction data from January 1, 2022, to December 31, 2022, which is the transition period for the COVID-19 pandemic. Evaluation using the random forest method produces an accuracy of 99.99%, while the naïve Bayes method produces an accuracy of 92.99%. The random forest and naïve Bayes methods can work very well on large data volumes. However, from the comparison results, it can be concluded that the performance of the random forest method is better than the naïve Bayes method in the multi-class classification case in predicting the level of customer purchasing power at PT. XYZ.

Multi-class Classification of Imbalanced Intelligent Data using Deep Neural Network

In recent years, studies in the field of deep learning have made significant progress. These studies have focusedmore on datasets with balanced classification, and less research has been done on imbalanced datasets, whichare of great importance in the real world and present significant challenges for classification. This articlestudies the problem of classifying imbalanced data, introduces dynamic sampling for deep neural networks,investigates the imbalanced multiclass problem, and proposes a dynamic sampling method for deep learning.In our proposed method, all samples are fed to the current deep neural network for each training iteration,and the accuracy, precision, and mean error of the deep neural network are estimated. The proposed methoddynamically selects informative data for training the deep neural network. Comprehensive experiments wereconducted to evaluate and understand its strengths and weaknesses. The results of 13 imbalanced multiclassdatasets show that the proposed method outperforms other methods, such as initial sampling techniques,active learning, cost-sensitive learning, and reinforcement learning.

Machine Learning-Assisted Improved Anomaly Detection for Structural Health Monitoring.

The importance of civil engineering infrastructure in modern societies has increased lately due to the growth of the global economy. It forges global supply chains facilitating enormous economic activity. The bridges usually form critical links in complex supply chain networks. Structural health monitoring (SHM) of these infrastructures is essential to reduce life-cycle costs, and determine their remaining life using advanced sensing techniques and data fusion methods. However, the data obtained from the SHM systems describing the health condition of the infrastructure systems may contain anomalies (i.e., distortion, drift, bias, outlier, noise etc.). An automated framework is required to accurately classify these anomalies and evaluate the current condition of these systems in a timely and cost-effective manner. In this paper, a recursive and interpretable decision tree framework is proposed to perform multiclass classification of acceleration data collected from a real-life bridge. The decision nodes of the decision tree are random forest classifiers that are invoked recursively after synthetically augmenting the training data before successive iterations until suitable classification performance is obtained. This machine-learning-based classification model evolved from a simplistic decision tree where statistical features are used to perform classification. The feature vectors defined for training the random forest classifiers are calculated using similar statistical features that are easy to interpret, enhancing the interpretability of the classifier models. The proposed framework could classify non-anomalous (i.e., normal) time-series of the test dataset with 98% accuracy.

Canonical correlation analysis as a feature extraction method to classify active sonar targets with shallow neural networks.

Sonar target recognition remains an active area of research due to the complex entanglement of features from various acoustic scatterers, background clutter, and distortion by waveguide propagation effects. An equally challenging issue is due to different acoustic echoes returned from the target (including different target elements) itself. This work investigates the sonar target classification problem from a statistical perspective and aims to extract salient target feature vectors. Specifically, a multivariate statistical method is employed, canonical correlation analysis (CCA), as a feature extraction technique prior to multi-class classification of active sonar field data. The intuition behind using CCA is that persistent features slowly morph over time due to the changing aspect angles and platform positions and can be represented by maximally correlated projections of consecutive pings. CCA is applied using a sliding window, and the projections are used as feature vectors to train a neural network classifier. The smallest increase in classification accuracy when comparing the projection feature vectors to unprocessed feature vectors was 10%. The largest increase was 34%. The results are further examined through the use of confusion matrices and layer-wise relevance propagation, which distributes the trained networks output score to the input layer.

Dynamic noise filtering for multi-class classification of beehive audio data

Honeybees are the most specialized insect pollinators and are critical not only for honey production but, also, for keeping the environmental balance by pollinating the flowers of a wide variety of crops.Recording and analyzing bee sounds became a fundamental part of recent initiatives in the development of so-called smart hives. The majority of researches on beehive sound analytics are focusing on swarming detection, a relatively simple binary classification task (due to the obvious difference in the sound of a swarming and a non-swarming bee colony) where machine learning models achieve good performance even when trained on small data.However, in the case of more complex tasks of beehive sound analytics, even modern machine learning approaches perform poorly. First, training such models would need a large dataset but, according to our knowledge, there is no publicly available large-scale beehive audio data. Second, due to the specifics of beehive sounds, efficient noise filtering methods would be required, however, we could not find a noise filtering method that would increase the performance of machine learning models substantially.In this paper, we propose a dynamic noise filtering method applicable on spectrograms (image representations of audio data) which is superior to the most popular image noise filtering baselines. Further, we introduce a multi-class classification task of bee sounds and a large-scale dataset consisting of 10.000 beehive audio recordings. Finally, we provide the results of a large-scale experiment involving various combinations of audio feature extraction and noise filtering methods together with various deep learning models. We believe that the contributions of this paper will facilitate further research in the area of (beehive) sound analytics.

Multi-Class Classification of Agricultural Data Based on Random Forest and Feature Selection

Agricultural production and operation produce a large amount of data, which hides valuable knowledge. Data mining technology can effectively explore the connection between various factors from the massive agricultural data. Classification prediction is one of the most valuable agricultural data mining techniques. This paper presents a new algorithm consisting of machine learning algorithms, feature ranking method and instance filter, which aims to enhance the capability of the random forest algorithm and better solve the problem of agricultural multi-class classification. The performance of the new algorithm was tested by using four standard agricultural multi-class datasets, and the experimental results showed that the newly proposed method performed well on all datasets. Among them, substantial rise in classification accuracy is observed for Eucalyptus dataset. Applying random forest algorithm on Eucalyptus dataset results in classification accuracy as 53.4% and after applying the new algorithm (rough set) the classification accuracy significantly increases to 83.7%.

ROULETTE: A neural attention multi-output model for explainable Network Intrusion Detection

Network Intrusion Detection (NID) systems are one of the most powerful forms of defense for protecting public and private networks. Most of the prominent methods applied to NID problems consist of Deep Learning methods that have achieved outstanding accuracy performance. However, even though they are effective, these systems are still too complex to interpret and explain. In recent years this lack of interpretability and explainability has begun to be a major drawback of deep neural models, even in NID applications. With the aim of filling this gap, we propose ROULETTE: a method based on a new neural model with attention for an accurate, explainable multi-class classification of network traffic data. In particular, attention is coupled with a multi-output Deep Learning strategy that helps to discriminate better between network intrusion categories. We report the results of extensive experimentation on two benchmark datasets, namely NSL-KDD and UNSW-NB15, which show the beneficial effects of the proposed attention mechanism and multi-output learning strategy on both the accuracy and explainability of the decisions made by the method.

Construction and Optimization of Mental Health Education Consultation Management System Based on Decision Tree Association Rule Mining

This paper studies association rule mining and decision tree algorithm, focusing on the extended research of association rule mining, including the number of generated rules, mining association rules of long itemsets with low support, attribute selection criteria and multivalue attributes in decision tree algorithm. This paper conducts in-depth research and analysis on the design and optimization of the mental health education consultation management system using the association rule decision tree algorithm. This paper analyzes the meaning of parameters under the support-confidence-interest model, and uses regression method to design equations between the number of rules and parameters. We use the multiple correlation coefficient to test the fitting effect of the equation, and use the significance test to verify whether the coefficient of the parameter is significantly zero. On the one hand, the widely used psychological crisis prevention measures generally include the screening of the SCL psychological scale in the early stage of first-year enrolment, the holding of general psychological knowledge lectures and courses, and the opening of psychological counselling rooms with a low penetration rate, but these practices are to a certain extent. In other words, it cannot enable the student administrator to grasp the psychological status of the students in a timely, effective, and dynamic manner, to timely intervene in the possible crisis. Not only the number of attribute values of the current node is considered but also the size of the variable precision clear area of the lower node is considered, that is, the two-layer nodes of the tree are considered at the same time. The new attribute selection method not only overcomes the shortcomings of the original algorithm, but also has the advantages of variable precision rough sets. This paper uses a new criterion for attribute selection, weighted roughness and complexity, which comprehensively considers the classification accuracy and the number of branches. In order to reduce the influence of noisy data and missing values, the algorithm uses a class prediction method based on matching degree. Through comparative experiments, the effectiveness of the method proposed in this paper is verified. We propose a new calculation formula for the similarity of the child nodes of the label set to evaluate the effect of attribute classification, and comprehensively consider the situation that the elements in the two multilabel sets appear or not appear at the same time, so that the calculation of the similarity of the label set is more comprehensive. The experimental results show that the model proposed in this paper can excavate the dialectical combination of multiple factors. By comparing with the existing algorithms, the classification effect of the proposed algorithm is verified. The classification algorithm proposed in this paper is more suitable for dealing with multi-value attributes and multiclass data classification problems. The psychological evaluation and counselling system designed in this paper has achieved the expected goal. The results of this paper can improve the problems existing in the work of psychological counselling services.

Empowering multi-class medical data classification by Group-of-Single-Class-predictors and transfer optimization: Cases of structured dataset by machine learning and radiological images by deep learning

In the medical domain, data are often collected over time, evolving from simple to refined categories. The data and the underlying structures of the medical data as to how they have grown to today’s complexity can be decomposed into crude forms when data collection starts. For instance, the cancer dataset is labeled either benign or malignant at its simplest or perhaps the earliest form. As medical knowledge advances and/or more data become available, the dataset progresses from binary class to multi-class, having more labels of sub-categories of the disease added. In machine learning, inducing a multi-class model requires more computational power. Model optimization is enforced over the multi-class models for the highest possible accuracy, which of course, is necessary for life-and-death decision making. This model optimization task consumes an extremely long model training time. In this paper, a novel strategy called Group-of-Single-Class prediction (GOSC) coupled with majority voting and model transfer is proposed for achieving maximum accuracy by using only a fraction of the model training time. The main advantage is the ability to achieve an optimized multi-class classification model that has the highest possible accuracy near to the absolute maximum, while the training time could be saved by up to 70%. Experiments on machine learning over liver dataset classification and deep learning over COVID19 lung CT images were tested. Preliminary results suggest the feasibility of this new approach.

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures.

Medical data includes clinical trials and clinical data such as patient-generated health data, laboratory results, medical imaging, and different signals coming from continuous health monitoring. Some commonly used data analysis techniques are text mining, big data analytics, and data mining. These techniques can be used for classification, clustering, and machine learning tasks. Machine learning could be described as an automatic learning process derived from concepts and knowledge without deliberate system coding. However, finding a suitable machine learning architecture for a specific task is still an open problem. In this work, we propose a machine learning model for the multi-class classification of medical data. This model is comprised of two components—a restricted Boltzmann machine and a classifier system. It uses a discriminant pruning method to select the most salient neurons in the hidden layer of the neural network, which implicitly leads to a selection of features for the input patterns that feed the classifier system. This study aims to investigate whether information-entropy measures may provide evidence for guiding discriminative pruning in a neural network for medical data processing, particularly cancer research, by using three cancer databases: Breast Cancer, Cervical Cancer, and Primary Tumour. Our proposal aimed to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired by the information-entropy theory; the results obtained after pruning the neural network were favourable. Specifically, for the Breast Cancer dataset, the reported results indicate a 10.68% error rate, while our error rates range from 10% to 15%; for the Cervical Cancer dataset, the reported best error rate is 31%, while our proposal error rates are in the range of 4% to 6%; lastly, for the Primary Tumour dataset, the reported error rate is 20.35%, and our best error rate is 31%.

MiNB: Minority Sensitive Naïve Bayesian Algorithm for Multi-Class Classification of Unbalanced Data

The unbalanced nature of data makes it tough to achieve the desire performance goal for classification algorithms. The sub-optimal prediction system isn't a viable solution due to the high misclassification cost of minority events. Thus accurate imbalanced data classification could be a path changer for prediction in domains like medical diagnosis, judiciary, and disaster management systems. To date, most of the existing studies of imbalanced data are for the binary class dataset and supported by data sampling techniques that suffer from loss of information and over-fitting. In this paper, we present the modified naïve Bayesian algorithm for unbalanced data classification that eliminates the requirement of data level sampling. We compared our proposed model with the data sampling technique and cost-sensitive techniques. We use minority sensitive TP Rate, class-specific misclassification rate, and overall performance parameters such as accuracy, f-measure and G-mean. The result shows that our proposed algorithm shows a more optimal result for unbalanced data classification. Results shows reduction in misclassification rate and improve predictive performance for the minority class.

On the classification error rates in terms of semivariograms for Gaussian universal kriging models

Bayes multiclass classification of spatial Gaussian data following the universal kriging model is considered. The closed-form expressions for the maximum likelihood (ML) estimator of regression parameters and the actual error rate (AER) in terms of semivariograms are derived.

A graphical approach for multiclass classification and for correcting the labeling errors in mislabeled training data

Multiclass data classification, where the goal is to segment data into classes, is an important task in machine learning. However, the task is challenging due to reasons including the scarcity of labeled training data; in fact, most machine learning algorithms require a large amount of labeled examples to perform well. Moreover, the accuracy of a classifier can be dependent on the accuracy of the training labels which can be corrupted. In this paper, we present an efficient and unconditionally stable semi-supervised graph-based method for multiclass data classification which requires considerably less labeled training data to accurately classify a data set compared to current techniques, due to properties such as the embedding of data into a similarity graph. In particular, it performs very well and more accurately than current approaches in the common scenario of few labeled training elements. Morever, we show that the algorithm performs with good accuracy even with a large number of mislabeled examples and is also able to incorporate class size information. The proposed method uses a modified auction dynamics technique. Extensive experiments on benchmark datasets are performed and the results are compared to other methods.

Multi Class Data Classification to Improve Accuracy in Sentiment Analysis using Machine Learning

Sentiment analysis means classifying a text into different emotional classes. These days most of the sentiment analysis techniques divide the text into either binary or ternary classification in this paper we are classifying the movie reviews into 5 classes. Multi class sentiment analysis is a technique which can be used to know the exact sentiment of a review not just polarity of a given textual statement from positive to negative. So that one can know the precise sentiment of a review . Multi class sentiment analysis has always been a challenging task as natural languages are difficult to represent mathematically. The number of features are also generally large which requires huge computational power so to reduce the number of features we will use parts-of-speech tagging using textblob to extract the important features. Sentiment analysis is done using machine learning, where it requires training data and testing data to train a model. Various kinds of models are trained and tested at last one model is selected based on its accuracy and confusion matrix. It is important to analyze the reviews in textual form because large amount of reviews is present all over the web. Analyzing textual reviews can help the firms that are trying to find out the response of their products in the market. In this paper sentiment analysis is demonstrated by analyzing the movie reviews, reviews are taken from IMDB website.

Weighted support vector machine using fuzzy rough set theory

The existence of both uncertainty and imprecision has detrimental impact on efficiency of decision-making applications and some machine learning methods, in particular support vector machine in which noisy samples diminish the performance of SVM training. Therefore, it is important to introduce a special method in order to improve this problem. Fuzzy aspects can handle mentioned problem which has been considered in some classification methods. This paper presents a novel weighted support vector machine to improve the noisy sensitivity problem of standard support vector machine for multiclass data classification. The basic idea is considered to add a weighted coefficient to the penalty term Lagrangian formula for optimization problem, which is called entropy degree, using lower and upper approximation for membership function in fuzzy rough set theory. As a result, noisy samples have low degree and important samples have high degree. To evaluate the power of the proposed method WSVM-FRS (Weighted SVM-Fuzzy Rough Set), several experiments have been conducted based on tenfold cross-validation over real-world data sets from UCI repository and MNIST data set. Experimental results show that the proposed method is superior than the other state-of-the-art competing methods regarding accuracy, precision and recall metrics.