High-dimensional Classification Research Articles

Data-adaptive binary classifiers in high dimensions using random partitioning

Classification in high dimensions has been highlighted for the past two decades since Fisher's linear discriminant analysis (LDA) is not optimal in a smaller sample size n comparing the number of covariates p, i.e. p>n, which is mostly due to the singularity of the sample covariance matrix. Rather than modifying how to estimate the sample covariance and sample mean vector in constructing a classifier, we build two types of high-dimensional classifiers using data splitting, i.e. single data splitting (SDS) and multiple data splitting (MDS). Moreover, we introduce a weighted version of MDS classifier that improves classification performance as illustrated in numerical studies. Each of the split data sets has a smaller size of covariates compared to the sample size so that LDA is applicable, and classification results can be combined with respect to minimizing the misclassification rate. We present theoretical justification backing up our proposed methods by comparing misclassification rates with LDA in high dimension. We also conduct a wide range of simulations and analyse four microarray data sets, which demonstrates that our proposed methods outperform some existing methods or at least yield comparable performances.

An Interpretable Deep-Stacked TSK Fuzzy Classifier for High-Dimensional Problems

This study proposes a novel deep-stacked fully interpretable and more generalization Takagi-Sugeno-Kang fuzzy system (D-FIMG-TSK) to obtain short fuzzy rules with high interpretability in each layer and simultaneously earn more generalization capability for high-dimensional data classification tasks. With the help of both discriminative and residual information, D-FIMG-TSK combines several fully interpretable TSK classifier FIMG-TSK in a stacked manner to conveniently move apart the manifolds existing in the original data space to achieve better linear separability. Besides, a feature selection method is designed to make feature selection from original feature sets on all layers to guarantee the classification performances. On the training process of D-FIMG-TSK, the importance of all original features and the antecedent and consequent parts of all fuzzy rules in each FIMG-TSK can be determined at the same time. The effectiveness of D-FIMG-TSK is manifested by the experimental results on eight binary high-dimensional classification datasets.

Harnessing machine learning for enhanced quantitative financial strategies: Applications and case studies

Machine learning algorithms have long been an essential tool for quantitative finance, as they play an important role in the asset pricing, portfolio optimisation and financial risk management. This work discusses how to apply Random Forest, Support Vector Machine (SVM), and Neural Networks to further develop and enhance the quantitative investment research. With automatic feature selection, reconstruction of high-dimensional data and resist against over-fitting, Random Forest has been an efficient ensemble approach with remarkable prediction performance. For this reason, Random Forest is leveraged for the stock price prediction. SVM, particularly known for its robustness in high-dimensional classification task, has a non-linear optimisation ability that can be employed to capture the inherent non-linear dependencies in options prices. Neural Networks, as a learning approach capable of discovering complex relationships, is employed for the portfolio optimisation, with a promise for further improving the risk-adjusted portfolio return. The work is conducted through practical case studies to simulate how Random Forest, SVM and Neural Networks enable us to further improve the financial performance, and to illustrate its advantage, disadvantage and practical application in quantitative finance.

Random projection ensemble conformal prediction for high-dimensional classification

In classification problems, many models with superior performance fail to provide confidence estimates or intervals for each prediction. This lack of reliability poses risks in real-world applications, making these models difficult to trust. Conformal prediction, as distribution-free and model-free approaches with finite-sample coverage guarantee, have recently been widely used to construct prediction sets for classification models. However, traditional conformal prediction methods only produce set-valued results without specifying a definitive predicted class. Particularly in complex settings, these methods fail to assist models in effectively addressing challenges such as high dimensionality, resulting in ambiguous prediction sets with low statistical efficiency, i.e. the prediction sets contain many false classes. In this study, a novel Ensemble Conformal Prediction algorithm based on Random Projection and a designed voting strategy, RPECP, is developed to tackle these challenges. Initially, a procedure for selecting the approximately oracle random projections and classifiers is executed to best leverage the internal information and structure of the data. Subsequently, based on the approximately oracle random projections and underlying classifiers, conformal prediction is performed on new test samples in a lower-dimensional space, resulting in multiple independent prediction sets. Finally, an accurate predicted class and a precise prediction set with high coverage and statistical efficiency are produced through a designed voting strategy. Compared to several base classifiers, RPECP obtain higher classification accuracy; against other conformal prediction algorithms, it achieves less ambiguous prediction sets with fewer false classes while guaranteeing high coverage. For illustration, this paper demonstrates RPECP's superiority over other methods in four cases: two high-dimensional settings and two real-world datasets.

Changes in tumor microbiome and underlying value according to response to neoadjuvan chemotherapy for in patients with gastric cancer

Objective: To investigate the relationship between efficacy of neoadjuvant chemotherapy (NACT) for gastric cancer and gastric microecology. Methods: This was a retrospective observational study using fresh frozen operative specimens. The specimens had been stored in the tumor sample bank of the Department of Gastrointestinal Surgery of the Affiliated Hospital of Qingdao University from January 2017 to January 2023 after having been collected from 31 patients with pathologically diagnosed gastric cancer who had no metastases and had received only neoadjuvant chemotherapy preoperatively. The study patients had all successfully undergone radical gastric cancer surgery. Patients with metastases or other primary tumor foci and/or had received other therapies within 1 month prior to surgery, including immunotherapy, targeted therapies and probiotics, were excluded. The tumors were graded and grouped in accordance with the 8th edition of the American Joint Committee on Cancer staging system and the Tumor Regression Grading System (TRG) of the College of American Pathologists. Those with TRG Grades 0-1, ypT0-1 and ypN0 were classified as responsive (Group R, 12 cases), whereas those with TRG Grades 2-3 and ypT2-4 or ypN+ were classified as non-responsive (Group NR, 19 cases). The fresh frozen samples were processed and analyzed using 16S rRNA sequencing. Alpha and beta diversity analyses were performed using the Q2-diversity plug-in for QIIME2 and STAMP was used to determine the default parameters and differentially enriched bacterial taxa in the two groups. High-dimensional class comparisons were performed by effect size linear discriminant analysis, and potential functional distributions of microbiomes were predicted using PICRUST2 (v2.3.0-b) software. Results: Groups R and NR did not differ significantly in sex, age, body mass index, smoking history, tumor location, cTNM stage before NACT, and neoadjuvant chemotherapy (all P>0.05), whereas tumor size and ypTNM stage after NACT differed significantly between the two groups (both P=0.001). Alpha and beta diversity analysis of the gastric microbiota did not reveal a statistically significant difference in alpha diversity between the two groups (P>0.05), whereas there was a statistically significant difference in beta diversity between the two groups (P=0.004). Four family-level bacterial taxa, namely Coriobacteriaceae, Ruminococcaceae, Veillonellaceae, and Lachnospiraceae, were enriched in the R group, whereas four bacterial taxa dominated by phylum Proteobacteria were enriched in the NR group. Metabolic pathways of various amino acids, including citric acid cycle and alanine, were found to be potentially predictive. Conclusions: There are significant differences in the abundance and composition of gastric microecology in gastric cancer patients with different responses to NACT.

Social coevolution and Sine chaotic opposition learning Chimp Optimization Algorithm for feature selection

Feature selection is a hot problem in machine learning. Swarm intelligence algorithms play an essential role in feature selection due to their excellent optimisation ability. The Chimp Optimisation Algorithm (CHoA) is a new type of swarm intelligence algorithm. It has quickly won widespread attention in the academic community due to its fast convergence speed and easy implementation. However, CHoA has specific challenges in balancing local and global search, limiting its optimisation accuracy and leading to premature convergence, thus affecting the algorithm’s performance on feature selection tasks. This study proposes Social coevolution and Sine chaotic opposition learning Chimp Optimization Algorithm (SOSCHoA). SOSCHoA enhances inter-population interaction through social coevolution, improving local search. Additionally, it introduces sine chaotic opposition learning to increase population diversity and prevent local optima. Extensive experiments on 12 high-dimensional classification datasets demonstrate that SOSCHoA outperforms existing algorithms in classification accuracy, convergence, and stability. Although SOSCHoA shows advantages in handling high-dimensional datasets, there is room for future research and optimization, particularly concerning feature dimensionality reduction.

Towards multi-objective high-dimensional feature selection via evolutionary multitasking

Feature selection (FS) plays a crucial role in high-dimensional classification problems by identifying relevant features that contribute to model performance. Evolutionary multitasking (EMT) has recently shown success in FS problems. However, existing EMT-based FS methods have limitations in terms of diversity in task construction, evolutionary search, and knowledge transfer, leading to inadequate acquisition, exploration, and utilization of knowledge. To this end, this paper develops a novel EMT framework for multi-objective high-dimensional FS problems, namely MO-FSEMT. In particular, multiple auxiliary tasks are constructed by distinct formulation methods to provide diverse search spaces and information representations and then simultaneously addressed with the original task by leveraging multiple evolutionary solvers with different biases and search preferences. A task-specific-based knowledge transfer mechanism is designed to leverage the advantageous information from each task, facilitating the discovery and effective transmission of high-quality solutions during the search process. Comprehensive experimental results on 27 real high-dimensional datasets demonstrate the superiority of MO-FSEMT over state-of-the-art FS methods in terms of effectiveness and efficiency. Ablation studies further confirm the contributions of key components of the proposed MO-FSEMT.

Urinary volatile organic compounds (VOCs) based prostate cancer diagnosis via high-dimensional classification

Early detection of prostate cancer is critical for successful treatment and survival. However, current diagnostic methods such as prostate-specific antigen (PSA) testing and digital rectal examination (DRE) have limitations in accuracy, specificity, and sensitivity. Recent research suggests that urinary volatile organic compounds (VOCs) could serve as potential biomarkers for prostate cancer diagnosis. In this study, urine samples from 337 PCa-positive and 233 PCa-negative patients were collected to develop a diagnosis model. The study involves a high dimensional (HD) classification problem due to the vast number of measured VOCs. Our findings reveal that regularized logistic regression outperforms numerous other classifiers when analyzing the collected data. In particular, we have selected a regularized logistic model with the SCAD (smoothly clipped absolute deviation) penalty as the final model, which attains an AUC (area under the ROC curve) of 0.748, in contrast to a PSA-based AUC of 0.540. These results underscore the potential of VOC-based diagnosis as a clinically feasible approach for PCa screening.

Embedded feature selection approach based on TSK fuzzy system with sparse rule base for high-dimensional classification problems

In high-dimensional problems of fuzzy rule-based embedded feature selection, the challenges include loss of interpretability, curse of dimensionality, and arithmetic underflow, among others. The primary reason for these problems is the exponential increase in the number of fuzzy rules with an increase in input dimension. In this study, an embedded feature selection approach, the Takagi–Sugeno–Kang (TSK) fuzzy system with sparse rule base (TSK-SRB), is proposed for high-dimensional data. Based on clustering, a broader initial rule base is designed with a suitable number of rules. In the rule layer, refined softmin (Ref-softmin) is introduced to calculate the firing strength, which can approximate the minimum T-norm while avoiding arithmetic underflow. Two Group Lasso regularization terms are used to realize feature and rule selection. In addition, an automatic threshold segmentation is introduced to determine the appropriate number of selected features/rules. Extensive experiments on 17 classification datasets showed that TSK-SRB is effective and competitive in high-dimensional feature selection.

A Multi-Task Decomposition-Based Evolutionary Algorithm for Tackling High-Dimensional Bi-Objective Feature Selection

Evolutionary algorithms have been widely applied for solving multi-objective optimization problems, while the feature selection in classification can also be treated as a discrete bi-objective optimization problem if attempting to minimize both the classification error and the ratio of selected features. However, traditional multi-objective evolutionary algorithms (MOEAs) may have drawbacks for tackling large-scale feature selection, due to the curse of dimensionality in the decision space. Therefore, in this paper, we concentrated on designing an multi-task decomposition-based evolutionary algorithm (abbreviated as MTDEA), especially for handling high-dimensional bi-objective feature selection in classification. To be more specific, multiple subpopulations related to different evolutionary tasks are separately initialized and then adaptively merged into a single integrated population during the evolution. Moreover, the ideal points for these multi-task subpopulations are dynamically adjusted every generation, in order to achieve different search preferences and evolutionary directions. In the experiments, the proposed MTDEA was compared with seven state-of-the-art MOEAs on 20 high-dimensional classification datasets in terms of three performance indicators, along with using comprehensive Wilcoxon and Friedman tests. It was found that the MTDEA performed the best on most datasets, with a significantly better search ability and promising efficiency.

Double data piling: a high-dimensional solution for asymptotically perfect multi-category classification

For high-dimensional classification, interpolation of training data manifests as the data piling phenomenon, in which linear projections of data vectors from each class collapse to a single value. Recent research has revealed an additional phenomenon known as the ‘second data piling’ for independent test data in binary classification, providing a theoretical understanding of asymptotically perfect classification. This paper extends these findings to multi-category classification and provides a comprehensive characterization of the double data piling phenomenon. We define the maximal data piling subspace, which maximizes the sum of pairwise distances between piles of training data in multi-category classification. Furthermore, we show that a second data piling subspace that induces data piling for independent data exists and can be consistently estimated by projecting the negatively-ridged discriminant subspace onto an estimated ‘signal’ subspace. By leveraging this second data piling phenomenon, we propose a bias-correction strategy for class assignments, which asymptotically achieves perfect classification. The present research sheds light on benign overfitting and enhances the understanding of perfect multi-category classification of high-dimensional discrimination with a help of high-dimensional asymptotics.

Feature selection based on dynamic crow search algorithm for high-dimensional data classification

High-dimensional biomedical data plays an important role in disease diagnosis and classification. To analyze the high-dimensional biomedical data, machine learning algorithms are widely used. However, a great amount of redundant, irrelevant or weak correlation features are prevalent in these datasets, which significantly deteriorate the capability of machine learning techniques. It is necessary to select the most informative features and discard the useless ones for enhancing the classification accuracy and reducing the dimensionality. To address this issue, an improved Crow Search Algorithm (CSA) which is known as the Dynamic CSA (DCSA) is proposed to find the optimal feature subset in high-dimensional classification tasks. In DCSA, three modifications are investigated. Firstly, we propose a dynamic bi-level awareness probability to guide the transition between global search and local search. Consequently, the balance between exploration and exploitation is improved. Secondly, levy flight with a unfixed step length control parameter is employed as the global search and the random search mechanism of the original CSA is abandoned. Thirdly, a dynamic flight length strategy is adopted to enhance the local search and speed up convergence. All in all, DCSA is developed as a wrapper feature selection model, where KNN classifier is applied as the feature subset evaluator. The performance of DCSA is measured on seven high-dimensional biomedical datasets. Experimental results show that DCSA outperforms other state-of-the-art methods. Specifically, DCSA achieved the highest average accuracy in six data. In terms of numbers of selected features, DCSA gained the smallest subset size in a seven data. And for SRBCT data, the prediction accuracy of DCSA is at 100%.

Long access heroin self-administration significantly alters gut microbiome composition and structure.

It is well known that chronic opioid use disorder is associated with alterations in gastrointestinal (GI) function that include constipation, reduced motility, and increased bacterial translocation due to compromised gut barrier function. These signs of disrupted GI function can be associated with alterations in the gut microbiome. However, it is not known if long-access opioid self-administration has effects on the gut microbiome. We used 16S rRNA gene sequencing to investigate the gut microbiome in three independent cohorts (N=40 for each) of NIH heterogeneous stock rats before onset of long-access heroin self-administration (i.e., naïve status), at the end of a 15-day period of self-administration, and after post-extinction reinstatement. Measures of microbial α- and β-diversity were evaluated for all phases. High-dimensional class comparisons were carried out with MaAsLin2. PICRUSt2 was used for predicting functional pathways impacted by heroin based on marker gene sequences. Community α-diversity was not altered by heroin at any of the three phases by comparison to saline-yoked controls. Analyses of β-diversity showed that the heroin and saline-yoked groups clustered significantly apart from each other using the Bray-Curtis (community structure) index. Heroin caused significant alterations at the ASV level at the self-administration and extinction phases. At the phylum level, the relative abundance of Firmicutes was increased at the self-administration phase. Deferribacteres was decreased in heroin whereas Patescibacteria was increased in heroin at the extinction phase. Potential biomarkers for heroin emerged from the MaAsLin2 analysis. Bacterial metabolomic pathways relating to degradation of carboxylic acids, nucleotides, nucleosides, carbohydrates, and glycogen were increased by heroin while pathways relating to biosynthesis of vitamins, propionic acid, fatty acids, and lipids were decreased. These findings support the view that long access heroin self-administration significantly alters the structure of the gut microbiome by comparison to saline-yoked controls. Inferred metabolic pathway alterations suggest the development of a microbial imbalance favoring gut inflammation and energy expenditure. Potential microbial biomarkers and related functional pathways likely invoked by heroin self-administration could be targets for therapeutic intervention.

Enhanced NSGA-II-based feature selection method for high-dimensional classification

Feature selection in high-dimensional data faces significant challenges owing to large and discrete decision spaces. In this study, we propose a feature selection method based on the nondominated sorting genetic algorithm-II (NSGA-II) to enhance the performance of feature selection in high-dimensional data. This study makes four contributions: 1) The sparse initialization strategy is used to sparsen the search space and accelerate the convergence speed of the algorithm; 2) the guided selection operator is employed to strike a balance between exploration and exploitation abilities; 3) an intra-population evolution-based mutation operator dynamically shrinks the search space; and 4) a greedy repair strategy is adopted to generate improved feature subsets. The proposed method was validated on 15 publicly available high-dimensional datasets and compared with eight competitive multi-objective feature selection methods. The results demonstrate that the proposed method can achieve superior classification accuracy in a shorter time, with a smaller subset of features containing less redundancy.

A Locally Adaptive Shrinkage Approach to False Selection Rate Control in High-Dimensional Classification

A Locally Adaptive Shrinkage Approach to False Selection Rate Control in High-Dimensional Classification

A Space Transformation-Based Multiform Approach for Multiobjective Feature Selection in High-Dimensional Classification

A Space Transformation-Based Multiform Approach for Multiobjective Feature Selection in High-Dimensional Classification

An Efficient Greedy Search Algorithm for High-dimensional Linear Discriminant Analysis.

High-dimensional classification is an important statistical problem that has applications in many areas. One widely used classifier is the Linear Discriminant Analysis (LDA). In recent years, many regularized LDA classifiers have been proposed to solve the problem of high-dimensional classification. However, these methods rely on inverting a large matrix or solving large-scale optimization problems to render classification rules-methods that are computationally prohibitive when the dimension is ultra-high. With the emergence of big data, it is increasingly important to develop more efficient algorithms to solve the high-dimensional LDA problem. In this paper, we propose an efficient greedy search algorithm that depends solely on closed-form formulae to learn a high-dimensional LDA rule. We establish theoretical guarantee of its statistical properties in terms of variable selection and error rate consistency; in addition, we provide an explicit interpretation of the extra information brought by an additional feature in a LDA problem under some mild distributional assumptions. We demonstrate that this new algorithm drastically improves computational speed compared with other high-dimensional LDA methods, while maintaining comparable or even better classification performance.

Binning high-dimensional classifier output for HEP analyses through a clustering algorithm

Binning high-dimensional classifier output for HEP analyses through a clustering algorithm

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification.

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Skin Microbiome Composition is Associated with Radiation Dermatitis in Breast Cancer Patients Undergoing Radiation after Reconstructive Surgery: A Prospective, Longitudinal Study

Approximately 95% of breast cancer (BC) patients receiving radiotherapy (RT) develop varying degrees of radiation dermatitis (RD), which can greatly affect the patient's quality of life and aesthetics. Severe acute RD can lead to interruption or delay of RT. Currently, there is no consensus on the prevention and management of RD. The skin microbiota (SM), which are mainly composed of bacteria and fungi, are essential for skin homeostasis and microbial dysbiosis is correlated with the onset and progression of many common skin diseases. However, to date, research on the role of the SM in RD remains scarce. This prospective, longitudinal study aims to analyze the association of SM with RD. We collected 200 SM samples both before and after RT from the region in the irradiated chest wall of 100 BC patients receiving RT after reconstructive surgery and samples from the corresponding region in the contralateral breast for bacterial 16S and fungal ITS (internal transcribed spacer) rRNA sequencing. Acute RD was graded according to the Toxicity Criteria of the Radiation Therapy Oncology Group (RTOG). Patients were divided into no or mild dermatitis (N/MD, RTOG grade 0 or 1) and severe dermatitis (SD, RTOG grade 2 and above). The compositional differences across groups were compared using STAMP and high-dimensional class comparisons by linear discriminant analysis of effect size (LEfSe). Differences in metabolic function between groups were predicted by the Phylogenetic Investigation of Communities by Reconstruction of Unobserved States (PICRUSt) method. Significant differences were observed in the diversity and composition of the SM between N/MD and SD both before and after RT. Analysis of bacterial 16S sequencing (n = 89, 63 N/MD, 26 SD) showed significantly higher relative abundance of particular taxa such as Ralstonia, Truepera, Methyloversatilis genera and lower abundance of particular taxa including Staphylococcus, Corynebacterium genera in N/MD patients. Besides, analysis of fungal ITS sequencing (n = 71, 53 N/MD, 18 SD) showed significantly higher relative abundance of particular taxa such as Hypocreaceae family and lower abundance of particular taxa such as Sporidiobolus genus in N/MD patients. Pathways including fatty acid synthesis were predicted to be enriched in N/MD patients. The SM and pathway markers were identified in this study to be associated with the severity of acute RD in BC patients undergoing RT after reconstructive surgery. More patient data is needed to verify the current findings and the results of metagenomic, metatranscriptomic, and metabolomic analyses will further mine key biomarkers at the compositional and functional level.