High-dimensional Datasets Research Articles

Consistency approximation: Incremental feature selection based on fuzzy rough set theory

Fuzzy Rough Set Theory (FRST)-based feature selection has been widely used as a preprocessing step to handle dynamic and large datasets. However, large-scale or high-dimensional datasets remain intractable for FRST-based feature selection approaches due to high space complexity and unsatisfactory classification performance. To overcome these challenges, we propose a Consistency Approximation (CA)-based framework for incremental feature selection. By exploring CA, we introduce a novel significance measure and a tri-accelerator. The CA-based significance measure provides a mechanism for each sample in the universe to keep members with different class labels within its fuzzy neighbourhood as far as possible, while keeping members with the same label as close as possible. Furthermore, our tri-accelerator reduces the search space and decreases the computational space with a theoretical lower bound. The experimental results demonstrate the superiority of our proposed algorithm compared to state-of-the-art methods on efficiency and classification accuracy, especially for large-scale and high-dimensional datasets.

Cross-spectrum method for acoustic source identification and visualization of airfoil noise

The identification and visualization of airfoil noise sources are critical for comprehending the interplay between flow and acoustics, and for understanding the generation and propagation of sound. In this paper, we introduce an innovative method, specifically designed to identify and visualize flow-induced noise sources associated with airfoil noise. This approach employs cross power spectral density analysis to distinctly identify flow-induced noise sources. The data for our study comes from large-eddy simulations of a NACA 0012 airfoil, characterized by a Reynolds number of 4×105, a Mach number of 0.058, and an angle of attack of 6.25∘. Different cross-spectrum formulations are examined and evaluated. The analysis includes a comparison of the cross-spectrum method's strengths and limitations relative to established data-driven approaches like dynamic mode decomposition (DMD) and spectral proper orthogonal decomposition (SPOD). We find that the cross-spectrum method provides both spectral magnitudes and phase topologies, allowing it to effectively compare sound intensities among various sources at specific frequencies, while retaining high-resolution spatiotemporal coherent flow and acoustic dynamics. One of the notable advantages of the cross-spectrum method over DMD or SPOD modes is its lesser reliance on extensive data manipulation in the form of large-size data matrix, making it a more efficient and user-friendly approach for practitioners, particularly when dealing with complex systems or high-dimensional datasets. This characteristic enhances its practicality and accessibility in the field of flow and acoustic visualization. Finally, the newly developed method is applied to three distinct flow transition scenarios to evaluate its proficiency in distinguishing acoustic generation and propagation mechanisms, depending on the specific transition case.

Visually exploring canonical correlation patterns of high-dimensional industrial control datasets based on multi-sensor fusion

Visually exploring canonical correlation patterns of high-dimensional industrial control datasets based on multi-sensor fusion

A two-stage clonal selection algorithm for local feature selection on high-dimensional data

Various evolutionary computation algorithms have shown their excellent performance for high-dimensional feature selection (FS). However, most of current FS methods choose a global feature subset and ignore the correlations between feature subsets and sample subspaces, which limits the performance of methods in the sample space with different probability distributions. To solve the issue, we propose a two-stage clonal selection algorithm for filter-based local feature selection (TSCSA-LFS) that integrates symmetric uncertainty and a discrete clonal selection algorithm with three contributions. First, unlike conventional feature selection methods, TSCSA-LFS introduces local sample behaviors and assigns subsets of features for different sample regions. Second, an improved discrete clonal selection algorithm is developed for searching relevant features, which contains mutual information-based individual initialization, a differential evolution-based mutation strategy pool and a local search technique. Third, a two-part antibody representation is employed for automatical adjustment of the weight-related parameter. Our method shows the promising experimental results compared with well-known global FS and clonal selection-based local FS methods on fourteen high-dimensional datasets. For instance, our method can obviously outperform local FS, filter-based and hybrid methods on at least nine datasets

Robust quantum federated learning with noise

Federated learning is a decentralized machine learning approach where multiple participants collaboratively train machine learning models. With the development of quantum computing, there has been significant potential in the integration of quantum computing and federated learning. However, existing research has demonstrated that, similar to classical federated learning models, quantum federated learning models also face various security threats and privacy leakage issues. This paper proposes a quantum federated learning model based on quantum noise. Adding quantum noise to the model not only addresses privacy leakage, but also enhances the model robustness, effectively resists adversarial attacks. Specifically, extensive numerical simulations are conducted using various datasets to evaluate the effectiveness of the proposed method. The results reveal a more pronounced variation in robust training in high-dimensional datasets compared to low-dimensional datasets. Furthermore, the impact of noise intensity on model robustness is explored. Experimental demonstrate that a small amount of quantum noise does not have a significant impact on accuracy, and as the noise increases, the robustness of the model also improves. Finally, three different types of quantum noise were used for robustness testing in the paper to analyze the impact of quantum noise on the robustness of quantum machine learning models. The abundant experimental results have verified that the noise can improve the security of distributed quantum machine learning.

High-dimensional mediation analysis for continuous outcome with confounders using overlap weighting method in observational epigenetic study

BackgroundMediation analysis is a powerful tool to identify factors mediating the causal pathway of exposure to health outcomes. Mediation analysis has been extended to study a large number of potential mediators in high-dimensional data settings. The presence of confounding in observational studies is inevitable. Hence, it’s an essential part of high-dimensional mediation analysis (HDMA) to adjust for the potential confounders. Although the propensity score (PS) related method such as propensity score regression adjustment (PSR) and inverse probability weighting (IPW) has been proposed to tackle this problem, the characteristics with extreme propensity score distribution of the PS-based method would result in the biased estimation.MethodsIn this article, we integrated the overlapping weighting (OW) technique into HDMA workflow and proposed a concise and powerful high-dimensional mediation analysis procedure consisting of OW confounding adjustment, sure independence screening (SIS), de-biased Lasso penalization, and joint-significance testing underlying the mixture null distribution. We compared the proposed method with the existing method consisting of PS-based confounding adjustment, SIS, minimax concave penalty (MCP) variable selection, and classical joint-significance testing.ResultsSimulation studies demonstrate the proposed procedure has the best performance in mediator selection and estimation. The proposed procedure yielded the highest true positive rate, acceptable false discovery proportion level, and lower mean square error. In the empirical study based on the GSE117859 dataset in the Gene Expression Omnibus database using the proposed method, we found that smoking history may lead to the estimated natural killer (NK) cell level reduction through the mediation effect of some methylation markers, mainly including methylation sites cg13917614 in CNP gene and cg16893868 in LILRA2 gene.ConclusionsThe proposed method has higher power, sufficient false discovery rate control, and precise mediation effect estimation. Meanwhile, it is feasible to be implemented with the presence of confounders. Hence, our method is worth considering in HDMA studies.

Spatial prediction of plant invasion using a hybrid of machine learning and geostatistical method.

Modeling ecological patterns and processes often involve large-scale and complex high-dimensional spatial data. Due to the nonlinearity and multicollinearity of ecological data, traditional geostatistical methods have faced great challenges in model accuracy. As machine learning has increased our ability to construct models on big data, the main focus of the study is to propose the use of statistical models that hybridize machine learning and spatial interpolation methods to cope with increasingly large-scale and complex ecological data. Here, two machine learning algorithms, boosted regression tree (BRT) and least absolute shrinkage and selection operator (LASSO), were combined with ordinary kriging (OK) to model plant invasions across the eastern United States. The accuracies of the hybrid models and conventional models were evaluated by 10-fold cross-validation. Based on an invasive plants dataset of 15 ecoregions across the eastern United States, the results showed that the hybrid algorithms were significantly better at predicting plant invasion when compared to commonly used algorithms in terms of RMSE and paired-samples t-test (with the p-value < .0001). Besides, the additional aspect of the combined algorithms is to have the ability to select influential variables associated with the establishment of invasive cover, which cannot be achieved by conventional geostatistics. Higher accuracy in the prediction of large-scale biological invasions improves our understanding of the ecological conditions that lead to the establishment and spread of plants into novel habitats across spatial scales. The results demonstrate the effectiveness and robustness of the hybrid BRTOK and LASOK that can be used to analyze large-scale and high-dimensional spatial datasets, and it has offered an optional source of models for spatial interpolation of ecology properties. It will also provide a better basis for management decisions in early-detection modeling of invasive species.

The Effects of Shocks on the Real Economy in Romania. A Bayesian FAVAR Approach

Abstract The last five years have marked the global economy with a series of shocks, such as the COVID-19 pandemic, the war in Ukraine, supply chain bottlenecks, the surge in inflation, and the energy crisis, whose magnitude and duration were challenging to predict. Moreover, the impact of those shocks significantly affected the accuracy of macroeconomic forecasts. Thus, assessing the effects of the shocks became extremely important and conditioned by the econometric models that can capture a large amount of information. This paper proposes a Bayesian Factor-Augmented Vector Autoregressive Model (FAVAR) to assess the effects of the most prominent shocks that recently hit the real Romanian economy. We use a high-dimensional data set of quarterly indicators from 2005Q1 to 2023Q3, covering the real sector, price indexes, sentiment indicators and financial variables. Our main results suggest that tightening monetary policy shock decreases both the real economic activity and prices, while a supply shock produces a delayed negative and persistent effect on real economic activity and on the majority of the indicators introduced in the model. The results of the uncertainty shocks are characterized by wider confidence intervals and the effects are estimated to have a lower magnitude.

Evaluation of sequential feature selection in improving the K-nearest neighbor classifier for diabetes prediction

The K-nearest neighbor (KNN) classifier employs distance metrics to measure the distance between the test instance and the samples used in training. With smaller samples, the KNN classifier achieves higher accuracy with low computational time. However, computing the distance between the test instance and all training samples to determine the class of the test instance requires higher computational time for a high-dimensional dataset. This research employs sequential feature selection (SFS) to select the optimal feature for diabetes prediction while reducing the computational time complexity of the KNN classifier. The KNN classifier showed effectiveness with an accuracy rate of 84.41% with nine features. The performance of the KNN improves by 2.6% when trained on the optimal features selected with the SFS. The result revealed glucose level, blood pressure (BP), skin thickness (ST), diabetes pedigree function (DPF), age, and body mass index (BMI) as the most representative features in diabetes prediction. The KNN classifier gives higher accuracy with these features. However, insulin and the number of times a woman is pregnant do not show a significant effect on the KNN classifier.

Efficient Feature Clustering for High-Dimensional Datasets: A Non-Parametric Approach

Clustering methods have become widely popular in the field of data analysis, as they enable the grouping of similar data points. However, the challenge of clustering high-dimensional data remains a significant obstacle due to the ”curse of dimensionality.” Traditional methods like Principal Component Analysis (PCA) and hierarchy-based feature agglomeration have been used to overcome this challenge by reducing the dimensionality of the dataset. However, there is currently no efficient method for clustering features of a high-dimensional dataset. To address this issue, we present a non-parametric clustering method in this paper, specifically designed for identifying features in a high-dimensional dataset. Our proposed feature clustering algorithm utilizes a partitioning algorithm that creates clusters of features. Through experimental results using publicly available datasets, we demonstrate the effectiveness of our method. Our approach offers a valuable tool for researchers and practitioners to better understand and analyze high-dimensional data.

Exploring Dimensionality Reduction Techniques for Improved Breast Cancer Diagnosis

Breast cancer diagnosis is a critical area in medical research, where the challenge lies not only in accurate identification but also in managing the inherent complexity of high-dimensional datasets. This paper navigates this challenge by exploring dimensionality reduction techniques to enhance diagnostic accuracy. The primary objective of this research was to employ dimensionality reduction methods to refine breast cancer diagnosis, with a focus on improving accuracy and interpretability. The study investigates the impact of preprocessing techniques on a high-dimensional dataset, aiming to uncover meaningful patterns for effective diagnostic models. Starting with a dataset including 569 observations and 30 attributes, careful examination reveals imbalances in the dataset (63% benign, 37% malignant). To deal with multicollinearity, we use the coefficients of Pearson correlation to find and eliminate highly correlated features. Subsequent data transformation, utilizing min-max normalization, ensures uniform weighting. Principal Component Analysis (PCA) is then leveraged for comprehensive dimensionality reduction. Visualizations through scree plots and bi-plots underscore the efficacy of early principal components in distinguishing benign from malignant cases. Our results demonstrate a notable 24% reduction in data dimensionality, affirming the process's efficiency. This abstract beckon the exploration of detailed findings, emphasizing dimensionality reduction's pivotal role in refining breast cancer diagnosis for more accurate, efficient, and interpretable models.

Advancing Nanomaterial Toxicology Screening Through Efficient and Cost-Effective Quantitative Proteomics.

Proteomic investigations yield high-dimensional datasets, yet their application to large-scale toxicological assessments is hindered by reproducibility challenges due to fluctuating measurement conditions. To address these limitations, this study introduces an advanced tandem mass tag (TMT) labeling protocol. Although labeling approaches shorten data acquisition time by multiplexing samples compared to traditional label-free quantification (LFQ) methods in general, the associated costs may surge significantly with large sample sets, for example, in toxicological screenings. However, the introduced advanced protocol offers an efficient, cost-effective alternative, reducing TMT reagent usage (by a factor of ten) and requiring minimal biological material (1µg), while demonstrating increased reproducibility compared to LFQ. To demonstrate its effectiveness, the advanced protocol is employed to assess the toxicity of nine benchmark nanomaterials (NMs) on A549 lung epithelial cells. While LFQ measurements identify 3300 proteins, they proved inadequate to reveal NM toxicity. Conversely, despite detecting 2600 proteins, the TMT protocol demonstrates superior sensitivity by uncovering alterations induced by NM treatment. In contrast to previous studies, the introduced advanced protocol allows simultaneous and straightforward assessment of multiple test substances, enabling prioritization, ranking, and grouping for hazard evaluation. Additionally, it fosters the development of New Approach Methodologies (NAMs), contributing to innovative methodologies in toxicological research.

GHOST: Graph-based higher-order similarity transformation for classification

Exploring and identifying a good feature representation to describe high-dimensional datasets is a challenge of prime importance. However, plenty of feature selection techniques and distance metrics exist, which entails an intricacy for identifying the one best suited to the task. This paper provides an algorithm to design high-order distance metrics over a sparse selection of features dedicated to classification. Our approach is based on Conditional Random Field (CRF) energy minimization and Dual Decomposition, which allow efficiency and great flexibility in the considered features. The optimization technique ensures the tractability of high-dimensionality problems using hundreds of features and samples. Our approach is evaluated on synthetic data as well as on Covid-19 patient stratification. Comparisons with state-of-the-art baselines and our proposed method on different classification results prove the learned metric’s relevance.

A Knowledge-Guided Competitive Co-Evolutionary Algorithm for Feature Selection

In real-world applications, feature selection is crucial for enhancing the performance of data science and machine learning models. Typically, feature selection is a complex combinatorial optimization problem and a multi-objective optimization problem. Its primary goals are to reduce the dimensionality of the dataset and enhance the performance of machine learning algorithms. The selection of features in high-dimensional datasets is challenging due to the intricate relationships between features, which pose significant challenges to the performance and computational efficiency of algorithms. This paper introduces a Knowledge-Guided Competitive Co-Evolutionary Algorithm (KCCEA) for feature selection, especially for high-dimensional features. In the proposed algorithm, we make improvements to the foundational dominance-based multi-objective evolutionary algorithm in two aspects. First, the use of feature correlation as knowledge to guide evolution enhances the search speed and quality of traditional multi-objective evolutionary algorithm solutions. Second, a dynamically allocated competitive–cooperative evolutionary mechanism is proposed, integrating the improved knowledge-guided evolution with traditional evolutionary algorithms, further enhancing the search efficiency and diversity of solutions. Through rigorous empirical testing on various datasets, the KCCEA demonstrates superior performance compared to basic multi-objective evolutionary algorithms, providing effective solutions to multi-objective feature selection problems while enhancing the interpretability and effectiveness of prediction models.

Evaluation of High-Dimensional Data Classification for Skin Malignancy Detection Using DL-Based Techniques

Skin cancer can be detected through visual screening and skin analysis based on the biopsy and pathological state of the human body. The survival rate of cancer patients is low, and millions of people are diagnosed annually. By determining the different comparative analyses, the skin malignancy classification is evaluated. Using the Isomap with the vision transformer, we analyze the high-dimensional images with dimensionality reduction. Skin cancer can present with severe cases and life-threatening symptoms. Overall performance evaluation and classification tend to improve the accuracy of the high-dimensional skin lesion dataset when completed. In deep learning methodologies, the distinct phases of skin malignancy classification are determined by its accuracy, specificity, F1 recall, and sensitivity while implementing the classification methodology. A nonlinear dimensionality reduction technique called Isomap preserves the data’s underlying nonlinear relationships intact. This is essential for the categorization of skin malignancies, as the features that separate malignant from benign skin lesions may not be linearly separable. Isomap decreases the data’s complexity while maintaining its essential characteristics, making it simpler to analyze and explain the findings. High-dimensional datasets for skin lesions have been evaluated and classified more effectively when evaluated and classified using Isomap with the vision transformer.

Improving golden jackel optimization algorithm: An application of chemical data classification

One of the main issues affecting the effectiveness of the quantitative structure-activity relationship (QSAR) classification techniques in chemometrics is high dimensionality. Applying feature selection is a critical procedure that determines the most relevant and important aspects of a dataset. It improves the effectiveness and accuracy of prediction models by effectively lowering the number of features. This decrease increases classification accuracy, reduces computing strain, and improves overall performance. Recently, the golden jackal optimization (GJO) algorithm was introduced, which has been successfully used to solve various continuous optimization issues. Therefore, this study proposes an improvement in the GJO algorithm employing chaotic maps, abbreviated as CGJO, to enhance the exploration and exploitation capability of the GJO algorithm in picking the essential descriptors in QSAR classification models with high classification accuracy and less computation time. Experimental findings based on four different high-dimensional chemical datasets show that the proposed CGJO algorithm can maximize classification accuracy while simultaneously decreasing the number of chosen descriptors and lowering the time required for computing. Thus, the proposed algorithm can be useful for chemical data classification in other QSAR modeling.

Sydney’s residential relocation landscape: Machine learning and feature selection methods unpack the whys and whens

This study investigates household residential relocation timing, an aspect vital for transport and urban planning. Analyzing a high-dimensional dataset from 1,024 relocations in Sydney, Australia, the research contrasts ten machine learning survival techniques with three classical survival models. Results indicate that when classical models are paired with tree-based automated feature selectors, they align closely with machine learning outcomes. Notably, the GBM, XGBoost, and Random Forest models emerge as standout performers. The study provides a comprehensive comparison between automatic and manual feature selection, shedding light on variables influencing households’ duration of stay. While stacked ensemble modeling, which leverages predictions from various models, is used to enhance accuracy, the improvements are marginal, underscoring inherent modeling challenges, particularly the recurring issue of misclassifying specific pairs of households in the concordance index measure. A thorough feature analysis highlights homeownership as the foremost predictor, underscoring the importance of recent life events and accessibility features in relocation decisions. The research emphasizes the importance of considering the accessibility of both current and future homes in relocation models, with 20% feature significance in model outcomes. Building on these foundational insights, the study paves the way for a deeper understanding of individual decision-making processes in sustainable urban planning.

Improved Kepler Optimization Algorithm for enhanced feature selection in liver disease classification

Liver diseases represent a significant healthcare challenge, impacting millions globally and posing complexities in diagnosis. To address this global health concern, this paper introduces a groundbreaking enhancement to the Kepler Optimization Algorithm, termed I-KOA, designed specifically for feature selection in high-dimensional datasets. By harnessing the synergies of Opposition-Based Learning and a Local Escaping Operator grounded in the k-nearest Neighbor (kNN) classifier, I-KOA asserts itself as a potent tool for local exploitation, balanced exploration, and evasion of local optima. To our knowledge, this is the first work to exploit KOA as a feature selection method. Pioneering the utilization of KOA as a feature selection method, the paper rigorously tests I-KOA in two extensive experiments, tackling the complex CEC’22 benchmark suite functions and the intricate landscape of five liver disease datasets. Results underscore I-KOA’s unparalleled performance, validated through the Friedman test, where it surpasses seven rival optimization algorithms. Achieving an outstanding overall classification accuracy of 93.46%, Feature selection size of 0.1042, sensitivity of 97.46%, precision of 94.37%, and F1-score of 90.35% across the liver disease datasets, I-KOA’s randomized algorithm ensures robust feature selection, striking a compelling balance between subset size and classification efficacy. Acknowledging computational demands and generalization nuances, I-KOA is a formidable tool ready to revolutionize medical diagnosis and decision support systems. The open source codes of the proposed I-KOA are available at https://www.mathworks.com/matlabcentral/fileexchange/161376-improved-kepler-optimization-algorithm.

An Improved Feature Removal Approach for Classification of High Dimensional Feature Dataset

Extracting and evaluating pertinent information in a high-dimensional feature set is extremely difficult when dealing with a high-dimensional feature space. Classification methods require additional training time to generate a classification model. Every feature is not equally relevant in a high-dimensional feature collection. As a result, feature selection is a productive method for identifying vital features and eliminating unnecessary ones. Feature selection serves as a method of pre-processing before classification. It decreases the dimension of the dataset to shorten the training period required to develop a classifier. This research study aims to propose a novel feature subset selection method that establishes the relative importance of each feature using several criteria. The proposed approach ranks available features from high to low using a variety of feature-ranking approaches. Different feature ranking algorithms perform differently on the same dataset. It is challenging to obtain robust performance with just one feature ranking algorithm. To overcome this problem, we have used the Schulze rank aggregation method. The Schulze method combines multiple feature ranking techniques to assign a rank to each feature inside the dataset. This study presents an optimization strategy for heuristic search based on the backward feature removal method. It eliminates features according to the rank determined by the Schulze rank aggregation technique. In this paper, we evaluated the performance of the proposed method against the current state-of-the-art feature ranking techniques for high-dimensional feature set classification.

A Hybrid Rider Optimization with Deep Learning Driven Intrusion Detection Farmwork in Wireless Sensor Network

Introduction: an array of hazards currently exists in cyberspace, prompting extensive research to tackle these concerns. Intrusion Detection Systems (IDS) are a mechanism used to provide security in Wireless Sensor Networks (WSN). The IDS continue to encounter significant challenges in accurately identifying unknown attacks. Conventional Intrusion Detection Systems (IDS) commonly rely on Deep Learning (DL) algorithms, which utilise binary classifiers to classify attacks. The data dimension attribute is affected inside large-scale high-dimensional data sets. Methods: this research introduces a hybrid GFSO (HGFSO) model combined with Deep Learning Driven Intrusion Detection (HGFSO-DLIDS) to tackle this problem. The HGFSO approach is developed by merging the parameter selection methods of the Felis Margarita Swarm Optimisation (FMSO), the Grampus optimisation algorithm (GOA), and the Deep Convolutional Neural Network (DCNN) with BiLSTM (Bidirectional Long Short-Term Memory) algorithm. Results: the model training utilised real-time traffic statistics, including the KDDCup 99 and WSN-DS datasets. After being trained and validated using the datasets, the model’s performance is assessed by multi-class classification, achieving accuracy rates of 99,89 % and 99,64 % respectively. Conclusion: as a result, this occurrence leads to a decrease in the overall effectiveness of detecting assaults. Deep learning may enhance the creation of an intrusion detection system by eliminating complex features in the raw data, resulting in a more precise classification method.