High-dimensional Classification Research Articles

Changes in tumor microbiome and underlying value according to response to neoadjuvan chemotherapy for in patients with gastric cancer

Objective: To investigate the relationship between efficacy of neoadjuvant chemotherapy (NACT) for gastric cancer and gastric microecology. Methods: This was a retrospective observational study using fresh frozen operative specimens. The specimens had been stored in the tumor sample bank of the Department of Gastrointestinal Surgery of the Affiliated Hospital of Qingdao University from January 2017 to January 2023 after having been collected from 31 patients with pathologically diagnosed gastric cancer who had no metastases and had received only neoadjuvant chemotherapy preoperatively. The study patients had all successfully undergone radical gastric cancer surgery. Patients with metastases or other primary tumor foci and/or had received other therapies within 1 month prior to surgery, including immunotherapy, targeted therapies and probiotics, were excluded. The tumors were graded and grouped in accordance with the 8th edition of the American Joint Committee on Cancer staging system and the Tumor Regression Grading System (TRG) of the College of American Pathologists. Those with TRG Grades 0-1, ypT0-1 and ypN0 were classified as responsive (Group R, 12 cases), whereas those with TRG Grades 2-3 and ypT2-4 or ypN+ were classified as non-responsive (Group NR, 19 cases). The fresh frozen samples were processed and analyzed using 16S rRNA sequencing. Alpha and beta diversity analyses were performed using the Q2-diversity plug-in for QIIME2 and STAMP was used to determine the default parameters and differentially enriched bacterial taxa in the two groups. High-dimensional class comparisons were performed by effect size linear discriminant analysis, and potential functional distributions of microbiomes were predicted using PICRUST2 (v2.3.0-b) software. Results: Groups R and NR did not differ significantly in sex, age, body mass index, smoking history, tumor location, cTNM stage before NACT, and neoadjuvant chemotherapy (all P>0.05), whereas tumor size and ypTNM stage after NACT differed significantly between the two groups (both P=0.001). Alpha and beta diversity analysis of the gastric microbiota did not reveal a statistically significant difference in alpha diversity between the two groups (P>0.05), whereas there was a statistically significant difference in beta diversity between the two groups (P=0.004). Four family-level bacterial taxa, namely Coriobacteriaceae, Ruminococcaceae, Veillonellaceae, and Lachnospiraceae, were enriched in the R group, whereas four bacterial taxa dominated by phylum Proteobacteria were enriched in the NR group. Metabolic pathways of various amino acids, including citric acid cycle and alanine, were found to be potentially predictive. Conclusions: There are significant differences in the abundance and composition of gastric microecology in gastric cancer patients with different responses to NACT.

Social coevolution and Sine chaotic opposition learning Chimp Optimization Algorithm for feature selection

Feature selection is a hot problem in machine learning. Swarm intelligence algorithms play an essential role in feature selection due to their excellent optimisation ability. The Chimp Optimisation Algorithm (CHoA) is a new type of swarm intelligence algorithm. It has quickly won widespread attention in the academic community due to its fast convergence speed and easy implementation. However, CHoA has specific challenges in balancing local and global search, limiting its optimisation accuracy and leading to premature convergence, thus affecting the algorithm’s performance on feature selection tasks. This study proposes Social coevolution and Sine chaotic opposition learning Chimp Optimization Algorithm (SOSCHoA). SOSCHoA enhances inter-population interaction through social coevolution, improving local search. Additionally, it introduces sine chaotic opposition learning to increase population diversity and prevent local optima. Extensive experiments on 12 high-dimensional classification datasets demonstrate that SOSCHoA outperforms existing algorithms in classification accuracy, convergence, and stability. Although SOSCHoA shows advantages in handling high-dimensional datasets, there is room for future research and optimization, particularly concerning feature dimensionality reduction.

Urinary volatile organic compounds (VOCs) based prostate cancer diagnosis via high-dimensional classification

Early detection of prostate cancer is critical for successful treatment and survival. However, current diagnostic methods such as prostate-specific antigen (PSA) testing and digital rectal examination (DRE) have limitations in accuracy, specificity, and sensitivity. Recent research suggests that urinary volatile organic compounds (VOCs) could serve as potential biomarkers for prostate cancer diagnosis. In this study, urine samples from 337 PCa-positive and 233 PCa-negative patients were collected to develop a diagnosis model. The study involves a high dimensional (HD) classification problem due to the vast number of measured VOCs. Our findings reveal that regularized logistic regression outperforms numerous other classifiers when analyzing the collected data. In particular, we have selected a regularized logistic model with the SCAD (smoothly clipped absolute deviation) penalty as the final model, which attains an AUC (area under the ROC curve) of 0.748, in contrast to a PSA-based AUC of 0.540. These results underscore the potential of VOC-based diagnosis as a clinically feasible approach for PCa screening.

Embedded feature selection approach based on TSK fuzzy system with sparse rule base for high-dimensional classification problems

In high-dimensional problems of fuzzy rule-based embedded feature selection, the challenges include loss of interpretability, curse of dimensionality, and arithmetic underflow, among others. The primary reason for these problems is the exponential increase in the number of fuzzy rules with an increase in input dimension. In this study, an embedded feature selection approach, the Takagi–Sugeno–Kang (TSK) fuzzy system with sparse rule base (TSK-SRB), is proposed for high-dimensional data. Based on clustering, a broader initial rule base is designed with a suitable number of rules. In the rule layer, refined softmin (Ref-softmin) is introduced to calculate the firing strength, which can approximate the minimum T-norm while avoiding arithmetic underflow. Two Group Lasso regularization terms are used to realize feature and rule selection. In addition, an automatic threshold segmentation is introduced to determine the appropriate number of selected features/rules. Extensive experiments on 17 classification datasets showed that TSK-SRB is effective and competitive in high-dimensional feature selection.

A Multi-Task Decomposition-Based Evolutionary Algorithm for Tackling High-Dimensional Bi-Objective Feature Selection

Evolutionary algorithms have been widely applied for solving multi-objective optimization problems, while the feature selection in classification can also be treated as a discrete bi-objective optimization problem if attempting to minimize both the classification error and the ratio of selected features. However, traditional multi-objective evolutionary algorithms (MOEAs) may have drawbacks for tackling large-scale feature selection, due to the curse of dimensionality in the decision space. Therefore, in this paper, we concentrated on designing an multi-task decomposition-based evolutionary algorithm (abbreviated as MTDEA), especially for handling high-dimensional bi-objective feature selection in classification. To be more specific, multiple subpopulations related to different evolutionary tasks are separately initialized and then adaptively merged into a single integrated population during the evolution. Moreover, the ideal points for these multi-task subpopulations are dynamically adjusted every generation, in order to achieve different search preferences and evolutionary directions. In the experiments, the proposed MTDEA was compared with seven state-of-the-art MOEAs on 20 high-dimensional classification datasets in terms of three performance indicators, along with using comprehensive Wilcoxon and Friedman tests. It was found that the MTDEA performed the best on most datasets, with a significantly better search ability and promising efficiency.

Double data piling: a high-dimensional solution for asymptotically perfect multi-category classification

AbstractFor high-dimensional classification, interpolation of training data manifests as the data piling phenomenon, in which linear projections of data vectors from each class collapse to a single value. Recent research has revealed an additional phenomenon known as the ‘second data piling’ for independent test data in binary classification, providing a theoretical understanding of asymptotically perfect classification. This paper extends these findings to multi-category classification and provides a comprehensive characterization of the double data piling phenomenon. We define the maximal data piling subspace, which maximizes the sum of pairwise distances between piles of training data in multi-category classification. Furthermore, we show that a second data piling subspace that induces data piling for independent data exists and can be consistently estimated by projecting the negatively-ridged discriminant subspace onto an estimated ‘signal’ subspace. By leveraging this second data piling phenomenon, we propose a bias-correction strategy for class assignments, which asymptotically achieves perfect classification. The present research sheds light on benign overfitting and enhances the understanding of perfect multi-category classification of high-dimensional discrimination with a help of high-dimensional asymptotics.

Feature selection based on dynamic crow search algorithm for high-dimensional data classification

High-dimensional biomedical data plays an important role in disease diagnosis and classification. To analyze the high-dimensional biomedical data, machine learning algorithms are widely used. However, a great amount of redundant, irrelevant or weak correlation features are prevalent in these datasets, which significantly deteriorate the capability of machine learning techniques. It is necessary to select the most informative features and discard the useless ones for enhancing the classification accuracy and reducing the dimensionality. To address this issue, an improved Crow Search Algorithm (CSA) which is known as the Dynamic CSA (DCSA) is proposed to find the optimal feature subset in high-dimensional classification tasks. In DCSA, three modifications are investigated. Firstly, we propose a dynamic bi-level awareness probability to guide the transition between global search and local search. Consequently, the balance between exploration and exploitation is improved. Secondly, levy flight with a unfixed step length control parameter is employed as the global search and the random search mechanism of the original CSA is abandoned. Thirdly, a dynamic flight length strategy is adopted to enhance the local search and speed up convergence. All in all, DCSA is developed as a wrapper feature selection model, where KNN classifier is applied as the feature subset evaluator. The performance of DCSA is measured on seven high-dimensional biomedical datasets. Experimental results show that DCSA outperforms other state-of-the-art methods. Specifically, DCSA achieved the highest average accuracy in six data. In terms of numbers of selected features, DCSA gained the smallest subset size in a seven data. And for SRBCT data, the prediction accuracy of DCSA is at 100%.

Enhanced NSGA-II-based feature selection method for high-dimensional classification

Feature selection in high-dimensional data faces significant challenges owing to large and discrete decision spaces. In this study, we propose a feature selection method based on the nondominated sorting genetic algorithm-II (NSGA-II) to enhance the performance of feature selection in high-dimensional data. This study makes four contributions: 1) The sparse initialization strategy is used to sparsen the search space and accelerate the convergence speed of the algorithm; 2) the guided selection operator is employed to strike a balance between exploration and exploitation abilities; 3) an intra-population evolution-based mutation operator dynamically shrinks the search space; and 4) a greedy repair strategy is adopted to generate improved feature subsets. The proposed method was validated on 15 publicly available high-dimensional datasets and compared with eight competitive multi-objective feature selection methods. The results demonstrate that the proposed method can achieve superior classification accuracy in a shorter time, with a smaller subset of features containing less redundancy.

A Locally Adaptive Shrinkage Approach to False Selection Rate Control in High-Dimensional Classification

A Locally Adaptive Shrinkage Approach to False Selection Rate Control in High-Dimensional Classification

An Efficient Greedy Search Algorithm for High-dimensional Linear Discriminant Analysis.

High-dimensional classification is an important statistical problem that has applications in many areas. One widely used classifier is the Linear Discriminant Analysis (LDA). In recent years, many regularized LDA classifiers have been proposed to solve the problem of high-dimensional classification. However, these methods rely on inverting a large matrix or solving large-scale optimization problems to render classification rules-methods that are computationally prohibitive when the dimension is ultra-high. With the emergence of big data, it is increasingly important to develop more efficient algorithms to solve the high-dimensional LDA problem. In this paper, we propose an efficient greedy search algorithm that depends solely on closed-form formulae to learn a high-dimensional LDA rule. We establish theoretical guarantee of its statistical properties in terms of variable selection and error rate consistency; in addition, we provide an explicit interpretation of the extra information brought by an additional feature in a LDA problem under some mild distributional assumptions. We demonstrate that this new algorithm drastically improves computational speed compared with other high-dimensional LDA methods, while maintaining comparable or even better classification performance.

Binning high-dimensional classifier output for HEP analyses through a clustering algorithm

Binning high-dimensional classifier output for HEP analyses through a clustering algorithm

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification.

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Long access heroin self-administration significantly alters gut microbiome composition and structure.

It is well known that chronic opioid use disorder is associated with alterations in gastrointestinal (GI) function that include constipation, reduced motility, and increased bacterial translocation due to compromised gut barrier function. These signs of disrupted GI function can be associated with alterations in the gut microbiome. However, it is not known if long-access opioid self-administration has effects on the gut microbiome. We used 16S rRNA gene sequencing to investigate the gut microbiome in three independent cohorts (N=40 for each) of NIH heterogeneous stock rats before onset of long-access heroin self-administration (i.e., naïve status), at the end of a 15-day period of self-administration, and after post-extinction reinstatement. Measures of microbial α- and β-diversity were evaluated for all phases. High-dimensional class comparisons were carried out with MaAsLin2. PICRUSt2 was used for predicting functional pathways impacted by heroin based on marker gene sequences. Community α-diversity was not altered by heroin at any of the three phases by comparison to saline-yoked controls. Analyses of β-diversity showed that the heroin and saline-yoked groups clustered significantly apart from each other using the Bray-Curtis (community structure) index. Heroin caused significant alterations at the ASV level at the self-administration and extinction phases. At the phylum level, the relative abundance of Firmicutes was increased at the self-administration phase. Deferribacteres was decreased in heroin whereas Patescibacteria was increased in heroin at the extinction phase. Potential biomarkers for heroin emerged from the MaAsLin2 analysis. Bacterial metabolomic pathways relating to degradation of carboxylic acids, nucleotides, nucleosides, carbohydrates, and glycogen were increased by heroin while pathways relating to biosynthesis of vitamins, propionic acid, fatty acids, and lipids were decreased. These findings support the view that long access heroin self-administration significantly alters the structure of the gut microbiome by comparison to saline-yoked controls. Inferred metabolic pathway alterations suggest the development of a microbial imbalance favoring gut inflammation and energy expenditure. Potential microbial biomarkers and related functional pathways likely invoked by heroin self-administration could be targets for therapeutic intervention.

Skin Microbiome Composition is Associated with Radiation Dermatitis in Breast Cancer Patients Undergoing Radiation after Reconstructive Surgery: A Prospective, Longitudinal Study

Approximately 95% of breast cancer (BC) patients receiving radiotherapy (RT) develop varying degrees of radiation dermatitis (RD), which can greatly affect the patient's quality of life and aesthetics. Severe acute RD can lead to interruption or delay of RT. Currently, there is no consensus on the prevention and management of RD. The skin microbiota (SM), which are mainly composed of bacteria and fungi, are essential for skin homeostasis and microbial dysbiosis is correlated with the onset and progression of many common skin diseases. However, to date, research on the role of the SM in RD remains scarce. This prospective, longitudinal study aims to analyze the association of SM with RD. We collected 200 SM samples both before and after RT from the region in the irradiated chest wall of 100 BC patients receiving RT after reconstructive surgery and samples from the corresponding region in the contralateral breast for bacterial 16S and fungal ITS (internal transcribed spacer) rRNA sequencing. Acute RD was graded according to the Toxicity Criteria of the Radiation Therapy Oncology Group (RTOG). Patients were divided into no or mild dermatitis (N/MD, RTOG grade 0 or 1) and severe dermatitis (SD, RTOG grade 2 and above). The compositional differences across groups were compared using STAMP and high-dimensional class comparisons by linear discriminant analysis of effect size (LEfSe). Differences in metabolic function between groups were predicted by the Phylogenetic Investigation of Communities by Reconstruction of Unobserved States (PICRUSt) method. Significant differences were observed in the diversity and composition of the SM between N/MD and SD both before and after RT. Analysis of bacterial 16S sequencing (n = 89, 63 N/MD, 26 SD) showed significantly higher relative abundance of particular taxa such as Ralstonia, Truepera, Methyloversatilis genera and lower abundance of particular taxa including Staphylococcus, Corynebacterium genera in N/MD patients. Besides, analysis of fungal ITS sequencing (n = 71, 53 N/MD, 18 SD) showed significantly higher relative abundance of particular taxa such as Hypocreaceae family and lower abundance of particular taxa such as Sporidiobolus genus in N/MD patients. Pathways including fatty acid synthesis were predicted to be enriched in N/MD patients. The SM and pathway markers were identified in this study to be associated with the severity of acute RD in BC patients undergoing RT after reconstructive surgery. More patient data is needed to verify the current findings and the results of metagenomic, metatranscriptomic, and metabolomic analyses will further mine key biomarkers at the compositional and functional level.

A bidirectional dynamic grouping multi-objective evolutionary algorithm for feature selection on high-dimensional classification

As a key preprocessing step in classification, feature selection involves two conflicting objectives: maximizing the classification accuracy and minimizing the number of selected features. Therefore, multi-objective optimization is widely used in feature selection due to its excellent trade-off between the convergence of two objectives. However, most existing multi-objective feature selection methods still face the issues of the “curse of dimensionality” and high computational costs, especially when the search space is large. To solve the above issues, this paper proposes a bidirectional dynamic grouping multi-objective evolutionary approach for high-dimensional feature selection, referred to as BDGMOEA. This approach transforms a high-dimensional feature selection problem into a feature selection task with a smaller search space by the idea of feature grouping, in which one bit of an individual represents a group of features. Specifically, a grouping search strategy is developed to divide the features into different quadrants according to the importance of the features obtained by different evaluation techniques. Then, the features in each quadrant are grouped by sector. This strategy can effectively narrow the search space and quickly locate promising feature regions. In addition, a bidirectional dynamic adjustment mechanism is presented by considering the evolutionary state of the population, and it can be used to explore each feature in more detail and comprehensively to prevent good features from being ignored in unselected groups. The experimental results demonstrate that the proposed BDGMOEA method performs the best in most cases, indicating that BDGMOEA not only achieves better classification performance but also reduces the training time.

An Evolutionary Multitasking Algorithm With Multiple Filtering for High-Dimensional Feature Selection

Recently, evolutionary multitasking (EMT) has been successfully used in the field of high-dimensional classification. However, the generation of multiple tasks in the existing EMT-based feature selection (FS) methods is relatively simple, using only the Relief-F method to collect related features with similar importance into one task, which cannot provide more diversified tasks for knowledge transfer. Thus, this paper devises a new EMT algorithm for FS in high-dimensional classification, which first adopts different filtering methods to produce multiple tasks and then modifies a competitive swarm optimizer to efficiently solve these related tasks via knowledge transfer. First, a diversified multiple task generation method is designed based on multiple filtering methods, which generates several relevant low-dimensional FS tasks by eliminating irrelevant features. In this way, useful knowledge for solving simple and relevant tasks can be transferred to simplify and speed up the solution of the original high-dimensional FS task. Then, a competitive swarm optimizer is modified to simultaneously solve these relevant FS tasks by transferring useful knowledge among them. Numerous empirical results demonstrate that the proposed EMT-based FS method can obtain a better feature subset than several state-of-the-art FS methods on eighteen high-dimensional datasets.

Correntropy-Based Logistic Regression with Automatic Relevance Determination for Robust Sparse Brain Activity Decoding.

Recent studies have used sparse classifications to predict categorical variables from high-dimensional brain activity signals to expose human's mental states and intentions, selecting the relevant features automatically in the model training process. However, existing sparse classification models will likely be prone to the performance degradation which is caused by the noise inherent in the brain recordings. To address this issue, we aim to propose a new robust and sparse classification algorithm in this study. To this end, we introduce the correntropy learning framework into the automatic relevance determination based sparse classification model, proposing a new correntropy-based robust sparse logistic regression algorithm. To demonstrate the superior brain activity decoding performance of the proposed algorithm, we evaluate it on a synthetic dataset, an electroencephalogram (EEG) dataset, and a functional magnetic resonance imaging (fMRI) dataset. The extensive experimental results confirm that not only the proposed method can achieve higher classification accuracy in a noisy and high-dimensional classification task, but also it would select those more informative features for the decoding tasks. Integrating the correntropy learning approach with the automatic relevance determination technique will significantly improve the robustness with respect to the noise, leading to more adequate robust sparse brain decoding algorithm. It provides a more powerful approach in the real-world brain activity decoding and the brain-computer interfaces.

An Adaptive Neuro-Fuzzy System With Integrated Feature Selection and Rule Extraction for High-Dimensional Classification Problems

A major limitation of fuzzy or neuro-fuzzy systems is their failure to deal with high-dimensional datasets. This happens primarily due to the use of T-norm, particularly, product or minimum (or a softer version of it). Thus, there are hardly any work dealing with datasets having features more than hundred or so. Here, we propose a neuro-fuzzy framework that can handle datasets with even more than 7000 features! In this context, we propose an adaptive softmin (Ada-softmin) which effectively overcomes the drawbacks of “numeric underflow” and “fake minimum” that arise for existing fuzzy systems while dealing with high-dimensional problems. We call it an Adaptive Takagi-Sugeno-Kang (AdaTSK) fuzzy system. We then equip the AdaTSK system to perform feature selection and rule extraction in an integrated manner. In this context, a novel gate function is introduced and embedded only in the consequent parts, which can determine the useful features and rules, in two successive phases of learning. Unlike conventional fuzzy rule bases, we design an enhanced fuzzy rule base (En-FRB), which maintains adequate rules but does not grow the number of rules exponentially with features that typically happens for fuzzy neural networks. The integrated Feature Selection and Rule Extraction AdaTSK (FSRE-AdaTSK) system consists of three sequential phases: (i) feature selection, (ii) rule extraction, and (iii) fine tuning. The effectiveness of the FSRE-AdaTSK is demonstrated on 19 datasets of which five are in more than 2000 dimension including two with more than 7000 features. This may be the first attempt to develop fuzzy rule based classifiers that can directly deal with more than 7000 features without requiring separate selection of features or any other dimensionality reduction method.

Efficient Bayesian High-Dimensional Classification via Random Projection with Application to Gene Expression Data

Inspired by the impressive successes of compress sensing-based machine learning algorithms, data augmentation-based efficient Gibbs samplers for Bayesian high-dimensional classification models are developed by compressing the design matrix to a much lower dimension. Ardent care is exercised in the choice of the projection mechanism, and an adaptive voting rule is employed to reduce sensitivity to the random projection matrix. Focusing on the high-dimensional Probit regression model, we note that the naive implementation of the data augmentation-based Gibbs sampler is not robust to the presence of co-linearity in the design matrix – a setup ubiquitous in $n\lt p$ problems. We demonstrate that a simple fix based on joint updates of parameters in the latent space circumnavigates this issue. With a computationally efficient MCMC scheme in place, we introduce an ensemble classifier by creating R ($\sim 25$–50) projected copies of the design matrix, and subsequently running R classification models with the R projected design matrix in parallel. We combine the output from the R replications via an adaptive voting scheme. Our scheme is inherently parallelizable and capable of taking advantage of modern computing environments often equipped with multiple cores. The empirical success of our methodology is illustrated in elaborate simulations and gene expression data applications. We also extend our methodology to a high-dimensional logistic regression model and carry out numerical studies to showcase its efficacy.

Multiobjective optimization algorithm with dynamic operator selection for feature selection in high-dimensional classification

Feature selection (FS) is an important technique in data preprocessing that aims to reduce the number of features for training while maintaining a high accuracy for classification. In recent studies, FS has been extended to optimize multiple objectives simultaneously in classification. To better solve this problem, this paper proposes a new multiobjective optimization algorithm with dynamic operator selection for feature selection in high-dimensional classification, called FS-DOS. First, two complementary search operators with different characteristics are designed, where the first operator is a quick search (QS) operator aiming to accelerate the convergence speed, and the other operator is a modified binary differential evolution (BDE) operator that can prevent the algorithm from falling into a local optimum. In addition, a dynamic selection strategy based on the idea of resource allocation is also designed to dynamically select the most suitable operator for each solution according to its corresponding performance improvement on aggregated objective values. The simulation results on fifteen different real-world high-dimensional FS datasets show that FS-DOS can obtain a feature subset with higher quality than several state-of-the-art FS algorithms. Importantly, in terms of error rate, FS-DOS wins 55 out of 75 comparisons. In terms of dimensionality reduction, the number of features selected by FS-DOS is between one hundredth and one thousandth of the original dataset.