Classification Of High-dimensional Data Research Articles

Evaluation of High-Dimensional Data Classification for Skin Malignancy Detection Using DL-Based Techniques

Skin cancer can be detected through visual screening and skin analysis based on the biopsy and pathological state of the human body. The survival rate of cancer patients is low, and millions of people are diagnosed annually. By determining the different comparative analyses, the skin malignancy classification is evaluated. Using the Isomap with the vision transformer, we analyze the high-dimensional images with dimensionality reduction. Skin cancer can present with severe cases and life-threatening symptoms. Overall performance evaluation and classification tend to improve the accuracy of the high-dimensional skin lesion dataset when completed. In deep learning methodologies, the distinct phases of skin malignancy classification are determined by its accuracy, specificity, F1 recall, and sensitivity while implementing the classification methodology. A nonlinear dimensionality reduction technique called Isomap preserves the data’s underlying nonlinear relationships intact. This is essential for the categorization of skin malignancies, as the features that separate malignant from benign skin lesions may not be linearly separable. Isomap decreases the data’s complexity while maintaining its essential characteristics, making it simpler to analyze and explain the findings. High-dimensional datasets for skin lesions have been evaluated and classified more effectively when evaluated and classified using Isomap with the vision transformer.

A joint learning framework for optimal feature extraction and multi-class SVM

In high-dimensional data classification, effectively extracting discriminative features while eliminating redundancy is crucial for enhancing the performances of classifiers, such as Support Vector Machine (SVM). However, previous studies have decoupled the process of feature extraction from the development of SVM, leading to suboptimal classification accuracy. To address this problem, we propose a novel joint learning framework that combines optimal feature extraction and multi-class SVM, incorporating a generalized regression form to learn a discriminative latent subspace. The projected data in this subspace are more likely to have a larger margin between different classes and align with the properties of the SVM classification mechanism, enhancing the overall classification performance. Three iterative algorithms were presented to obtain optimal solutions with guaranteed convergence, and theoretical analyses were also conducted to reveal their fundamental nature. The optimal linear projection subspace is equivalent to that obtained from Linear Discriminant Analysis (LDA) in some special cases. We conducted extensive experiments using diverse datasets to evaluate the performances of the proposed algorithms. Our algorithms achieved an accuracy improvement of up to 7.55% compared to other conventional methods.

Classification of high-dimensional imbalanced biomedical data based on spectral clustering SMOTE and marine predators algorithm

The research of biomedical data is crucial for disease diagnosis, health management, and medicine development. However, biomedical data are usually characterized by high dimensionality and class imbalance, which increase computational cost and affect the classification performance of minority class, making accurate classification difficult. In this paper, we propose a biomedical data classification method based on feature selection and data resampling. First, use the minimal-redundancy maximal-relevance (mRMR) method to select biomedical data features, reduce the feature dimension, reduce the computational cost, and improve the generalization ability; then, a new SMOTE oversampling method (Spectral-SMOTE) is proposed, which solves the noise sensitivity problem of SMOTE by an improved spectral clustering method; finally, the marine predators algorithm is improved using piecewise linear chaotic maps and random opposition-based learning strategy to improve the algorithm’s optimization seeking ability and convergence speed, and the key parameters of the spectral-SMOTE are optimized using the improved marine predators algorithm, which effectively improves the performance of the over-sampling approach. In this paper, five real biomedical datasets are selected to test and evaluate the proposed method using four classifiers, and three evaluation metrics are used to compare with seven data resampling methods. The experimental results show that the method effectively improves the classification performance of biomedical data. Statistical test results also show that the proposed PRMPA-Spectral-SMOTE method outperforms other data resampling methods.

Sparse Trace Ratio LDA for Supervised Feature Selection.

Classification is a fundamental task in the field of data mining. Unfortunately, high-dimensional data often degrade the performance of classification. To solve this problem, dimensionality reduction is usually adopted as an essential preprocessing technique, which can be divided into feature extraction and feature selection. Due to the ability to obtain category discrimination, linear discriminant analysis (LDA) is recognized as a classic feature extraction method for classification. Compared with feature extraction, feature selection has plenty of advantages in many applications. If we can integrate the discrimination of LDA and the advantages of feature selection, it is bound to play an important role in the classification of high-dimensional data. Motivated by the idea, we propose a supervised feature selection method for classification. It combines trace ratio LDA with l2,p -norm regularization and imposes the orthogonal constraint on the projection matrix. The learned row-sparse projection matrix can be used to select discriminative features. Then, we present an optimization algorithm to solve the proposed method. Finally, the extensive experiments on both synthetic and real-world datasets indicate the effectiveness of the proposed method.

Feature selection based on dynamic crow search algorithm for high-dimensional data classification

High-dimensional biomedical data plays an important role in disease diagnosis and classification. To analyze the high-dimensional biomedical data, machine learning algorithms are widely used. However, a great amount of redundant, irrelevant or weak correlation features are prevalent in these datasets, which significantly deteriorate the capability of machine learning techniques. It is necessary to select the most informative features and discard the useless ones for enhancing the classification accuracy and reducing the dimensionality. To address this issue, an improved Crow Search Algorithm (CSA) which is known as the Dynamic CSA (DCSA) is proposed to find the optimal feature subset in high-dimensional classification tasks. In DCSA, three modifications are investigated. Firstly, we propose a dynamic bi-level awareness probability to guide the transition between global search and local search. Consequently, the balance between exploration and exploitation is improved. Secondly, levy flight with a unfixed step length control parameter is employed as the global search and the random search mechanism of the original CSA is abandoned. Thirdly, a dynamic flight length strategy is adopted to enhance the local search and speed up convergence. All in all, DCSA is developed as a wrapper feature selection model, where KNN classifier is applied as the feature subset evaluator. The performance of DCSA is measured on seven high-dimensional biomedical datasets. Experimental results show that DCSA outperforms other state-of-the-art methods. Specifically, DCSA achieved the highest average accuracy in six data. In terms of numbers of selected features, DCSA gained the smallest subset size in a seven data. And for SRBCT data, the prediction accuracy of DCSA is at 100%.

Research on Online Condition monitoring for Complex System based on Modified Broad Learning Systems

Complex systems contain numerous interacting components, thus deep learning methods with powerful performance and complex structure are often used to achieve condition monitoring. However, the deep learning methods are always too time-consuming and hardware-demanding to be loaded into complex systems for online training and updates. To achieve accurate and timely monitoring of complex system state, based on broad learning system (BLS), an online condition monitoring method is proposed in this paper. General BLSs are based on a randomly generated hidden-layer, usually perform poorly in high-dimensional data classification tasks. In this work, based on correlation and causality, two modified BLSs are proposed and mixed to establish the online monitoring system. Specifically, logistic regression (LR) and structural causal model (SCM) are considered to form rough predictions of the system state, thus to replace the randomly generated ones with no practical significance. The effectiveness of the proposed online monitoring method is verified by both simulation data and real data.

Deep convolutional neural networks with genetic algorithm-based synthetic minority over-sampling technique for improved imbalanced data classification

Imbalanced data classification presents a challenge in machine learning, inducing biased model learning. Moreover, data dimensionality poses another challenge as it highly impacts classifier performance. This paper proposes a new deep-learning method that combines feature selection with oversampling to address these challenges. The proposed approach, GA-SMOTE-DCNN, integrates a genetic algorithm (GA) for feature selection, SMOTE for oversampling, and a deep 1D-convolutional neural network (DCNN) for classification. This study reveals that pre-splitting the data into training and testing sets before applying SMOTE results in higher accuracy, showing an improvement in accuracy ranging between 1.94% and 3.98% compared to post-SMOTE splitting for each dataset. This method achieved accuracy rates of 86.81% for the Balance Scale dataset, 86.15% for the Oil Spill dataset, 89.21% for the Yeast dataset, 91.32% for the Mammography dataset, 88.23% for the Australian credit dataset, and 89.53% for the German Credit dataset when compared with benchmark methods, underscoring its significance in tackling high-dimensional and imbalanced data classification problems. This method demonstrates scalability in effectively addressing challenges associated with high-dimensional and imbalanced data classification across various domains.

Search space division method for wrapper feature selection on high-dimensional data classification

Feature selection (FS) is an essential pre-processing technique for high-dimensional data. Wrapper-based FS techniques are known for their superior performance over filter FS. However, when the dimensionality of data is very high the wrapper techniques become computationally very expensive. To solve this problem of scalability, this paper proposes the concept of search space division (SSD) which leads to smaller search spaces and hence reduced computational cost. The proposed SSD approach is generic in nature and can be integrated with any wrapper-based FS technique. The SSD approach divides search space into multiple parts and the wrapper-based FS is independently applied to each part. To facilitate the interaction of features over the complete search space, all feature subsets obtained from each part are combined and the wrapper-based FS is again applied to get the final feature subset. Moreover, a new wrapper FS technique named Binary Rao (BRAO) Algorithm has been proposed. BRAO is based on the metaphor-less, parameter-less meta-heuristic optimization algorithm Rao-1. The proposed combination of SSD and BRAO algorithm named SSDR has produced better classification accuracies with a smaller number of features in a much shorter time as compared to other well-regarded wrapper FS techniques as well as classical FS techniques over thirteen benchmark high-dimensional datasets.

Numerically stable locality-preserving partial least squares discriminant analysis for efficient dimensionality reduction and classification of high-dimensional data

Dimensionality reduction plays a pivotal role in preparing high-dimensional data for classification and discrimination tasks by eliminating redundant features and enhancing the efficiency of classifiers. The effectiveness of a dimensionality reduction algorithm hinges on its numerical stability. When data projections are numerically stable, they lead to enhanced class separability in the lower-dimensional embedding, consequently yielding higher classification accuracy. This paper investigates the numerical attributes of dimensionality reduction and discriminant subspace learning, with a specific focus on Locality-Preserving Partial Least Squares Discriminant Analysis (LPPLS-DA). High-dimensional data frequently introduce singularity in the scatter matrices, posing a significant challenge. To tackle this issue, the paper explores two robust implementations of LPPLS-DA. These approaches not only optimize data projections but also capture more discriminative features, resulting in a marked improvement in classification accuracy. Empirical evidence supports these findings through numerical experiments conducted on synthetic and spectral datasets. The results demonstrate the superior performance of the proposed methods when compared to several state-of-the-art dimensionality reduction techniques in terms of both classification accuracy and dimension reduction.

Discriminative feature selection using signed Laplacian restricted Boltzmann machine for speed and generalization improvement of high dimensional data classification

Deep classifiers require lots of computations due to the use of multi-layer structure and processing of bulk input samples. Selecting effective features that have the same accuracy as full features, has recently been considered. Using feature selection, less calculation is needed to update the weights and consequently the speed of the training/testing process is increased. In this research, a new deep feature selection method called Discriminative Deep Feature Selection using Signed Laplacian Restricted Boltzmann Machine (DDFS-SLRBM) is proposed. In the approach, full training samples are fed into a SLRBM model. In the updates, the weight matrix is recomputed using the neighborhood matrix of similar and dissimilar classes which results in the discriminative property of the approach. Then, selected features are identified based on the minimum reconstruction error criterion. The efficiency of the proposed algorithm is demonstrated by performing different experiments on MNIST, GISETTE and Protein datasets. The experiments show that the proposed method is able to improve the classification accuracy and the generalizability of the approach while reducing the processing time. Also, the proposed approach shows scalability on the type of the problem, number of original features, number of samples and the number of final selected features.

Sparse Additive Machine With the Correntropy-Induced Loss.

Sparse additive machines (SAMs) have shown competitive performance on variable selection and classification in high-dimensional data due to their representation flexibility and interpretability. However, the existing methods often employ the unbounded or nonsmooth functions as the surrogates of 0-1 classification loss, which may encounter the degraded performance for data with outliers. To alleviate this problem, we propose a robust classification method, named SAM with the correntropy-induced loss (CSAM), by integrating the correntropy-induced loss (C-loss), the data-dependent hypothesis space, and the weighted lq,1 -norm regularizer ( q ≥ 1 ) into additive machines. In theory, the generalization error bound is estimated via a novel error decomposition and the concentration estimation techniques, which shows that the convergence rate O(n-1/4) can be achieved under proper parameter conditions. In addition, the theoretical guarantee on variable selection consistency is analyzed. Experimental evaluations on both synthetic and real-world datasets consistently validate the effectiveness and robustness of the proposed approach.

Minimum redundancy maximum relevance and VNS based gene selection for cancer classification in high-dimensional data

Minimum redundancy maximum relevance and VNS based gene selection for cancer classification in high-dimensional data

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Opinion mining and machine learning analysis : What emotions twitter data tell us about telemedicine?

The Covid-19 pandemic has made telemedicine one of the most relevant topics in recent years, so data mining about telemedicine obtained from Twitter offers a unique opportunity. The motivation of this study is to identify the emotional groupings of Twitter data users’ opinions for telemedicine using opinion mining techniques such as Sentiment Analysis combined with high-dimensional data classification and prediction methods. Data was collected from Twitter in August and September of 2021 related to telemedicine and official World Health Organization and World Bank documents for the years 2018 and 2021, respectively. Sentiment Analysis showed that 56.2% (n = 5351 tweets) of the sample had predominantly positive emotions toward telemedicine. Telemedicine, telehealth, and ivermectin are the most frequent words in the word cloud. Twitter data users’ opinions consist of nine mood classes (SIL index = 0.80) and the differences between these classes are statistically significant in terms of positive (p < 0.05), negative (p < 0.05), and neutral (p < 0.05) emotions. Neural Network (AUC = 0.842, F1 = 0.492) and Random Forest (AUC = 0.841, F1 = 0.494) are the best predictors of Twitter users’ mood classes compared with other machine learning techniques. The pythagorean tree generated by Random Forest showed that retweet is the best predictor of Twitter users’ opinions and emotional classes towards telemedicine. Future studies will create big social media datasets for a deep understanding of the emotional classes of individuals towards telemedicine technologies.

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification.

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Improved Contraction-Expansion Subspace Ensemble for High-Dimensional Imbalanced Data Classification

Improved Contraction-Expansion Subspace Ensemble for High-Dimensional Imbalanced Data Classification

Support Matrix Machine via Joint l2,1 and Nuclear Norm Minimization Under Matrix Completion Framework for Classification of Corrupted Data.

Traditional support vector machines (SVMs) are fragile in the presence of outliers; even a single corrupt data point can arbitrarily alter the quality of the approximation. If even a small fraction of columns is corrupted, then classification performance will inevitably deteriorate. This article considers the problem of high-dimensional data classification, where a number of the columns are arbitrarily corrupted. An efficient Support Matrix Machine that simultaneously performs matrix Recovery (SSMRe) is proposed, i.e. feature selection and classification through joint minimization of l2,1 (the nuclear norm of L ). The data are assumed to consist of a low-rank clean matrix plus a sparse noisy matrix. SSMRe works under incoherence and ambiguity conditions and is able to recover an intrinsic matrix of higher rank in the presence of data densely corrupted. The objective function is a spectral extension of the conventional elastic net; it combines the property of matrix recovery along with low rank and joint sparsity to deal with complex high-dimensional noisy data. Furthermore, SSMRe leverages structural information, as well as the intrinsic structure of data, avoiding the inevitable upper bound. Experimental results on different real-time applications, supported by the theoretical analysis and statistical testing, show significant gain for BCI, face recognition, and person identification datasets, especially in the presence of outliers, while preserving a reasonable number of support vectors.

QDA classification of high-dimensional data with rare and weak signals

QDA classification of high-dimensional data with rare and weak signals

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Hybrid deep learning approach to improve classification of low-volume high-dimensional data

BackgroundThe performance of machine learning classification methods relies heavily on the choice of features. In many domains, feature generation can be labor-intensive and require domain knowledge, and feature selection methods do not scale well in high-dimensional datasets. Deep learning has shown success in feature generation but requires large datasets to achieve high classification accuracy. Biology domains typically exhibit these challenges with numerous handcrafted features (high-dimensional) and small amounts of training data (low volume).MethodA hybrid learning approach is proposed that first trains a deep network on the training data, extracts features from the deep network, and then uses these features to re-express the data for input to a non-deep learning method, which is trained to perform the final classification.ResultsThe approach is systematically evaluated to determine the best layer of the deep learning network from which to extract features and the threshold on training data volume that prefers this approach. Results from several domains show that this hybrid approach outperforms standalone deep and non-deep learning methods, especially on low-volume, high-dimensional datasets. The diverse collection of datasets further supports the robustness of the approach across different domains.ConclusionsThe hybrid approach combines the strengths of deep and non-deep learning paradigms to achieve high performance on high-dimensional, low volume learning tasks that are typical in biology domains.