Multi-class Data Research Articles

Scalable feature selection using ReliefF aided by locality‐sensitive hashing

Feature selection algorithms, such as ReliefF, are very important for processing high-dimensionality data sets. However, widespread use of popular and effective such algorithms is limited by their computational cost. We describe an adaptation of the ReliefF algorithm that simplifies the costliest of its step by approximating the nearest neighbor graph using locality-sensitive hashing (LSH). The resulting ReliefF-LSH algorithm can process data sets that are too large for the original ReliefF, a capability further enhanced by distributed implementation in Apache Spark. Furthermore, ReliefF-LSH obtains better results and is more generally applicable than currently available alternatives to the original ReliefF, as it can handle regression and multiclass data sets. The fact that it does not require any additional hyperparameters with respect to ReliefF also avoids costly tuning. A set of experiments demonstrates the validity of this new approach and confirms its good scalability.

Improved multi-class discrimination by Common-Subset-of-Independent-Variables Partial-Least-Squares Discriminant Analysis

In this study, a new PLS-DA modelling approach for multi-class discriminant analysis, called Common-Subset-of-Independent-Variables Partial-Least-Squares Discriminant Analysis is proposed and evaluated. Because in this method Partial-Least-Squares models for one component are used, it is denoted as CSIV-PLS1-DA. In this method for each class vector, individual PLS1 models with individual model complexities are developed, based on one common set of independent variables, obtained after variable selection by the Final Complexity Adapted Models method, using the absolute values of the PLS regression coefficients, denoted as FCAM-REG. CSIV-PLS1-DA combines a common variable set for all class vectors, which is a characteristic of PLS2-DA, with the individual model complexity for each class vector, which is a characteristic of PLS1-DA. These characteristics make CSIV-PLS1-DA more flexible than PLS2-DA. CSIV-PLS1-DA is found to be an alternative for PLS1-DA or PLS2-DA when the correlations between the responses are low, which is often the case in discriminant analysis.The performance of the CSIV-PLS1-DA method is investigated using one simulated and eight real multi-class data sets from different sources. The classification abilities, measured by the percentage classification accuracy rates (%Acc), resulting from CSIV-PLS1-DA, are statistically compared with those of PLS1-DA and PLS2-DA, using one-tailed paired t-tests at the 95% confidence level.The results show that the %Acc values resulting from the CSIV-PLS1-DA method are significantly higher than those of the corresponding PLS1-DA and PLS2-DA methods, meaning that the classification ability of the CSIV-PLS1-DA method is significantly better.

Designing a grey wolf optimization based hyper-parameter optimized convolutional neural network classifier for skin cancer detection

In recent history, Convolutional Neural Networks have attained breakthroughs in addressing many intractable problems in the domain of image processing. But its performance builds upon its chosen hyper parameters and it is a tedious job to manually fine tune these hyper parameters. Hence, in this research, an Automated Hyper-parameter Optimized Convolution Neural Net is proposed which is further applied to uncover the class of skin cancer. The method has utilized a Grey Wolf Optimization algorithm for optimizing the hyper parameters of CNN, by adopting a proper encoding scheme. The effectiveness of the model is verified by comparing it with the performance of Particle Swarm Optimization and Genetic algorithm based hyper-parameter optimized CNN applied on the International Skin Imaging Collaboration skin lesion multi class data set. Simulation results infer that the proposed model is able to produce a testing accuracy up to 98.33% which is around 4% and 1% more compared to PSO and GA based models respectively. Similarly with the proposed model, the testing loss realized is around 0.17% which is 39.2% and 15% less compared to PSO and GA based models respectively. The experimental results clearly demonstrate that the proposed model performs competitively compared to other reported models.

Application of improved recursive regression algorithm on I&O model under COVID-19

Regression algorithm is an important data prediction method of machine learning. It can establish calculation matrix through the existing data and obtain the hidden characteristics in the data and other valuable information. Due to the excellent data processing ability, regression is widely used in gray data prediction, algorithm model building, data analysis and other fields. In this paper, the multi-layer recursive regression is adopted and improved for the regression algorithm with double influence factors, which increases the processing ability of multi class data. Data of import and export trade between China and the United States in the past five years is collected to create the future prediction model when the epidemic does not occur. Meanwhile, the data of four months after the outbreak to calculate the condition of the future prediction model of the epidemic situation in COVID-19. Using two opposite conditions as different data of regression algorithm calculation, different prediction algorithm formulas are obtained. After multiple calculations, the average value increases the sensitivity of the model. The calculation results show that the improved recursive regression algorithm has higher stability and sensitivity.

Important Features of CICIDS-2017 Dataset For Anomaly Detection in High Dimension and Imbalanced Class Dataset

The growth in internet traffic volume presents a new issue in anomaly detection, one of which is the high data dimension. The feature selection technique has been proven to be able to solve the problem of high data dimension by producing relevant features. On the other hand, high-class imbalance is a problem in feature selection. In this study, two feature selection approaches are proposed that are able to produce the most ideal features in the high-class imbalanced dataset. CICIDS-2017 is a reliable dataset that has a problem in high-class imbalance, therefore it is used in this study. Furthermore, this study performs experiments in Information Gain feature selection technique on the imbalance class datasaet. For validation, the Random Forest classification algorithm is used, because of its ability to handle multi-class data. The experimental results show that the proposed approaches have a very surprising performance, and surpass the state-of-the-art methods.

AsEmo: Automatic Approach for EEG-Based Multiple Emotional State Identification.

An electroencephalogram (EEG) is the most extensively used physiological signal in emotion recognition using biometric data. However, these EEG data are difficult to analyze, because of their anomalous characteristic where statistical elements vary according to time as well as spatial-temporal correlations. Therefore, new methods that can clearly distinguish emotional states in EEG data are required. In this paper, we propose a new emotion recognition method, named AsEmo. The proposed method extracts effective features boosting classification performance on various emotional states from multi-class EEG data. AsEmo Automatically determines the number of spatial filters needed to extract significant features using the explained variance ratio (EVR) and employs a Subject-independent method for real-time processing of Emotion EEG data. The advantages of this method are as follows: (a) it automatically determines the spatial filter coefficients distinguishing emotional states and extracts the best features; (b) it is very robust for real-time analysis of new data using a subject-independent technique that considers subject sets, and not a specific subject; (c) it can be easily applied to both binary-class and multi-class data. Experimental results on real-world EEG emotion recognition tasks demonstrate that AsEmo outperforms other state-of-the-art methods with a 2-8% improvement in terms of classification accuracy.

Meta Learning for Few-Shot One-Class Classification

We propose a method that can perform one-class classification given only a small number of examples from the target class and none from the others. We formulate the learning of meaningful features for one-class classification as a meta-learning problem in which the meta-training stage repeatedly simulates one-class classification, using the classification loss of the chosen algorithm to learn a feature representation. To learn these representations, we require only multiclass data from similar tasks. We show how the Support Vector Data Description method can be used with our method, and also propose a simpler variant based on Prototypical Networks that obtains comparable performance, indicating that learning feature representations directly from data may be more important than which one-class algorithm we choose. We validate our approach by adapting few-shot classification datasets to the few-shot one-class classification scenario, obtaining similar results to the state-of-the-art of traditional one-class classification, and that improves upon that of one-class classification baselines employed in the few-shot setting.

Remote Sensing Image Recognition Based on Multi-attention Residual Fusion Networks

Since each sample in a hyperspectral remote sensing image is made up of high-dimensional features and contains a wealth of remote sensing features, feature selection and mining become more difficult. To address this issue, a multi-attention residual integrated network (MARB-Net) algorithm is proposed, which reduces redundant features while increasing feature fusion and, as a result, improves hyperspectral image recognition. First, assign multiple weights to each feature using multiple attention mechanism models; then, deep mine and integrate the features using the residual network; and finally, perform contextual semantic integration on the deep fusion features using the Bi-LSTM network. The recognition task should be completed by the Softmax classifier. The experimental results on three multi-class public data sets show that the MARB-Net algorithm proposed in this paper is effective.

Weighted support vector machine using fuzzy rough set theory

The existence of both uncertainty and imprecision has detrimental impact on efficiency of decision-making applications and some machine learning methods, in particular support vector machine in which noisy samples diminish the performance of SVM training. Therefore, it is important to introduce a special method in order to improve this problem. Fuzzy aspects can handle mentioned problem which has been considered in some classification methods. This paper presents a novel weighted support vector machine to improve the noisy sensitivity problem of standard support vector machine for multiclass data classification. The basic idea is considered to add a weighted coefficient to the penalty term Lagrangian formula for optimization problem, which is called entropy degree, using lower and upper approximation for membership function in fuzzy rough set theory. As a result, noisy samples have low degree and important samples have high degree. To evaluate the power of the proposed method WSVM-FRS (Weighted SVM-Fuzzy Rough Set), several experiments have been conducted based on tenfold cross-validation over real-world data sets from UCI repository and MNIST data set. Experimental results show that the proposed method is superior than the other state-of-the-art competing methods regarding accuracy, precision and recall metrics.

Deep Neural Networks Classification via Binary Error-Detecting Output Codes

One-hot encoding is the prevalent method used in neural networks to represent multi-class categorical data. Its success stems from its ease of use and interpretability as a probability distribution when accompanied by a softmax activation function. However, one-hot encoding leads to very high dimensional vector representations when the categorical data’s cardinality is high. The Hamming distance in one-hot encoding is equal to two from the coding theory perspective, which does not allow detection or error-correcting capabilities. Binary coding provides more possibilities for encoding categorical data into the output codes, which mitigates the limitations of the one-hot encoding mentioned above. We propose a novel method based on Zadeh fuzzy logic to train binary output codes holistically. We study linear block codes for their possibility of separating class information from the checksum part of the codeword, showing their ability not only to detect recognition errors by calculating non-zero syndrome, but also to evaluate the truth-value of the decision. Experimental results show that the proposed approach achieves similar results as one-hot encoding with a softmax function in terms of accuracy, reliability, and out-of-distribution performance. It suggests a good foundation for future applications, mainly classification tasks with a high number of classes.

Collaborative filtering with q-divergence-based fuzzy clustering for spherical data

Although recommendation systems are the most powerful tool to help people choose items, a higher recommendation accuracy is required to satisfy the needs of the people. Motivated by this requirement, this study proposes a novel collaborative filtering (CF) algorithm, which is the underlying technology of a recommendation system. It filters items for a target user based on the reactions of similar users. Cluster analysis helps detect similar users by grouping a set of users such that users in the same group are more similar to each other than to those in other groups. However, in most representative CF algorithms such as GroupLens algorithm, users are considered as spherical data, and as categorical multivariate data in the clustering phase of a previous study. This study overcomes this logic gap by proposing a novel CF method using fuzzy clustering for spherical data based on q-divergence as both the clustering phase and the GroupLens algorithm consistently deal with users as spherical data. Experiments were conducted on six real datasets—BookCrossing, Epinions, Jester, LibimSeTi, MovieLens, and SUSHI, to compare the performance of the proposed method with GroupLens and the method using fuzzy clustering for categorical multivariate data based on q-divergence, which are conventional methods, where the performance is measured by the area under the receiver operating curve. The results of the experiments indicate that the proposed algorithm outperforms the others in terms of recommendation accuracy.

Constructing classifiers for imbalanced data using diversity optimisation

Imbalanced data is challenging in classification. This paper proposes a new approach to address imbalanced data by adopting diversity optimisation to generate synthetic instances for over-sampling the minority class. Diversity optimisation assures that the generated instances are close to the minority group but not identical. It also ensures the optimal spread of the generated instances in the space. We develop two formulations named as Diversity-based Average Distance Over-sampling (DADO) and Diversity-based Instance-Wise Over-sampling (DIWO). We evaluate the proposed formulations’ performance by designing experiments using both synthetic and real data with unbalanced classes. We examine the performance through area under curve (AUC), F1-score and g-mean measures in comparison with comparable synthetic over-sampling methods. We compare the methods using the obtained measures of the best performing classifier and statistical testing of all combinations over three imbalance levels using seven classifiers. The results show that both proposed formulations perform competitive to improve the performance of classifiers, and DIWO outperforms other comparable methods. Both perform robust by reducing the classifiers’ variance. We discuss the strengths and limitations of these formulations using the real data examples, runtime complexity and sensitivity analysis. We also demonstrate the possibility of utilising DADO and DIWO for multi-class imbalanced data.

Latent Gaussian process for anomaly detection in categorical data

We propose a semi-supervised approach towards anomaly detection in multivariate categorical data. Our goal is to learn a model that can distinguish the anomalous data, given a small set of training data from the normal class. To this end, our approach learns the probability distribution of normal instances with the assumption that the categorical data are generated from a continuous latent space. Gaussian process is adopted to construct the generative model. As a non-parametric Bayesian model, Gaussian process can adapt its model complexity according to the data size. Hence, our approach can be effective when the training dataset is small. Comprehensive experiments over different benchmarks clearly demonstrate the effectiveness of our approach.

Chemical profiling of dispersed fentanyl analogues sampled from indoor surfaces using multivariate data classification to determine synthesis methods

Fentanyl and its analogues are potent synthetic opioids used for pain relief or as anaesthetics in healthcare. However, they are also illicitly synthesised for use as recreational drugs. Dispersed fentanyl agents may be sampled as a means of collecting evidence of illegal fentanyl use. Previously, we showed that synthesis methods of fentanyl analogues could be classified based on impurity profiles, irrespective of acyl derivatization, using a multivariate classification model. In this study, we investigated whether traces of fentanyl analogues on surfaces could be sampled and the associated impurity profiles extracted. Fentanyl analogues were dispersed onto three different floor surfaces and one cloth surface commonly found in indoor environments. Samples were collected from hard surfaces after 24 h using cotton swabs. Analogues were extracted from the cotton swabs or directly from cloth surfaces, analysed by GC–MS and UHPLC–HRMS, and the resulting chemical profiles were classified using a previously developed classification model. The findings showed that the impurity profiles of swab-samples collected from different surfaces exposed to fentanyl could be used to determine the method used to synthesise specific fentanyl analogues.

ATR-FTIR spectroscopy based differentiation of typhoid and dengue fever in infected human sera

In this study, we have evaluated the potential of attenuated total reflection Fourier transform infrared (ATR-FTIR) spectroscopy to differentially diagnose bimolecular markers in Salmonella typhi and dengue infected freeze-dried sera samples along with healthy freeze-dried sera samples. Spectral signatures were recorded from 18 S. typhi infected, 23 dengue infected and 28 healthy freeze-dried sera samples. In order to achieve enhanced classification accuracy between the three sample classes, multivariate data classification algorithms principal component analysis (PCA) and linear discriminant analysis (LDA) were used. Outcomes of the PCA scatter plot showed 95% inter-class variation. A classification accuracy of 100% was achieved using LDA; therefore, we propose that ATR-FTIR coupled with chemo metric data classification algorithms holds substantial potential for bimolecular differentiation between S. typhi and dengue infected freeze-dried sera samples.

Smart Approach for Glioma Segmentation in Magnetic Resonance Imaging using Modified Convolutional Network Architecture (U-NET)

Segmentation of a brain tumor from magnetic resonance multimodal images is a challenging task in the field of medical imaging. The vast diversity in potential target regions, appearance and multifarious intensity threshold levels of various tumor types are few of the major factors that affect segmentation results. An accurate diagnosis and its treatment demand strict delineation of the tumor affected tissues. Herein, we focus on a smart, automated, and robust segmentation approach for brain tumor using a modified 3D U-Net architecture. The pre-operative multimodal 3D-MRI scans of High-Grade Glioma (HGG) and Low-Grade Glioma (LGG) are used as data. Our proposed approach solves the problem of memory and system resource constraints by robustly applying dense network training on image patches of 3D volumes. It improves the border region artifact detection by applying convolutions at an appropriate phase in the proposed neural network. Multi-class imbalance data are handled by using Categorical Cross Entropy (CCE) loss developed by combining the Weighted Cross Entropy (WCE) with Weighted Multi-class Dice Loss (WMDL) functions, which enables the network to perform smart segmentation of the smaller tumorous regions. The proposed approach is tested and evaluated for the challenge datasets of multimodal MRI volumes of tumor patients. Experiments are performed to compute the average dice scores on BraTS-2019 and BraTS-2020 datasets for the whole tumor region.

Cost-sensitive probability for weighted voting in an ensemble model for multi-class classification problems

Ensemble learning is an algorithm that utilizes various types of classification models. This algorithm can enhance the prediction efficiency of component models. However, the efficiency of combining models typically depends on the diversity and accuracy of the predicted results of ensemble models. However, the problem of multi-class data is still encountered. In the proposed approach, cost-sensitive learning was implemented to evaluate the prediction accuracy for each class, which was used to construct a cost-sensitivity matrix of the true positive (TP) rate. This TP rate can be used as a weight value and combined with a probability value to drive ensemble learning for a specified class. We proposed an ensemble model, which was a type of heterogenous model, namely, a combination of various individual classification models (support vector machine, Bayes, K-nearest neighbour, naïve Bayes, decision tree, and multi-layer perceptron) in experiments on 3-, 4-, 5- and 6-classifier models. The efficiencies of the propose models were compared to those of the individual classifier model and homogenous models (Adaboost, bagging, stacking, voting, random forest, and random subspaces) with various multi-class data sets. The experimental results demonstrate that the cost-sensitive probability for the weighted voting ensemble model that was derived from 3 models provided the most accurate results for the dataset in multi-class prediction. The objective of this study was to increase the efficiency of predicting classification results in multi-class classification tasks and to improve the classification results.

A-DBNF: adaptive deep belief network framework for regression and classification tasks

Many machine learning methods and models have been proposed for multivariate data regression and classification in recent years. Most of them are supervised learning methods, which require a large number of labeled data. Moreover, current methods need exclusive human labor and supervision to fine-tune the model hyperparameters. In this paper, we propose an adaptive deep belief network framework (A-DBNF) that can adapt to different datasets with minimum human labor. The proposed framework employs a deep belief network (DBN) to extract representative features of the datasets in the unsupervised learning phase and then fine-tune the network parameters by using few labeled data in the supervised learning phase. We integrate the DBN model with a genetic algorithm (GA) to select and optimize the model hyperparameters and further improve the network performance. We validate the performance of the proposed framework on several benchmark datasets, comparing the regression and classification accuracy with state-of-the-art methods. A-DBNF showed a noticeable performance improvement on three regression tasks using only 40–50% of labeled data. Our model outperformed most of the related methods in classification tasks by using 23–48% of labeled data.

Information granule-based classifier: A development of granular imputation of missing data

Granular Computing (GrC) is a human-centric way to discover the fundamental structure of data sets. The resulting information granules can be efficiently exploited to organize knowledge and reveal data descriptions, which can play a pivotal role in the classification problems. Furthermore, information granules are abstract collections of data entities and exhibit flexibility and tolerance when it comes to the representation of incomplete data. However, most of the existing methods focused on the data imputation and classification separately. They also require better interpretability. The crux of this study is to develop a novel information granule-based classification method for incomplete data and a way of representing missing entities and regarding them as information granules in a unified framework. The first aspect focuses on revealing the structural backbone of multiple labeled subspaces of data by fuzzy clustering of missing values. It emerges a classifier with interpretable “IF-THEN” rules by the refinement of fuzzy prototypes in a supervised mode to capture the critical relationship of the multi-class incomplete data. The second aspect concerns the construction of some information granules to impute and represent missing values according to the refined prototypes and classification findings. The experimental studies involved synthetic and publicly available datasets in quantifying the advantages of the classification and representation abilities of the proposed methods on incomplete data.

ENSEMBLE META CLASSIFIER WITH SAMPLING AND FEATURE SELECTION FOR DATA WITH IMBALANCE MULTICLASS PROBLEM

Ensemble learning by combining several single classifiers or another ensemble classifier is one of the procedures to solve the imbalance problem in multiclass data. However, this approach still faces the question of how the ensemble methods obtain their higher performance. In this paper, an investigation was carried out on the design of the meta classifier ensemble with sampling and feature selection for multiclass imbalanced data. The specific objectives were: 1) to improve the ensemble classifier through data-level approach (sampling and feature selection); 2) to perform experiments on sampling, feature selection, and ensemble classifier model; and 3 ) to evaluate t he performance of the ensemble classifier. To fulfil the objectives, a preliminary data collection of Malaysian plants’ leaf images was prepared and experimented, and the results were compared. The ensemble design was also tested with three other high imbalance ratio benchmark data. It was found that the design using sampling, feature selection, and ensemble classifier method via AdaboostM1 with random forest (also an ensemble classifier) provided improved performance throughout the investigation. The result of this study is important to the on-going problem of multiclass imbalance where specific structure and its performance can be improved in terms of processing time and accuracy.