Dimensionality Reduction Approach Research Articles

Integrating Unsupervised and Supervised ML Models for Analysis of Synthetic Data From VAE, GAN, and Clustering of Variables

Clustering of variables is a specialized approach for dimensionality reduction. This strategy is evaluated for data reduction with a Kaggle diabetes dataset. Since the original dataset is small, Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE) are used to generate 100,000 records and tested for resemblance to the real data using standard statistical methods. VAE-data is more representative of the real data than GAN-data when analyzed using machine learning (ML) models. Applying Clustering of Variables on VAE-data yields new synthetic variables (SV). SV-data is then augmented with target variable data. Random Forest model is used on VAE and SV data. SV-data results matched VAE-data, proving the new data's quality. SV-data also provides insights into correlations and data dispersion patterns. This analysis implements a combination of Unsupervised learning (clustering of variables) and Supervised learning (classification) which is reflected in the results.

Exploring local interpretability in dimensionality reduction: Analysis and use cases

Dimensionality reduction is a crucial area in artificial intelligence that enables the visualization and analysis of high-dimensional data. The main use of dimensionality reduction is to lower the dimensional complexity of data, improving the performance of machine learning models. Non-linear dimensionality reduction approaches, which provide higher quality representations than linear ones, lack interpretability, prohibiting their application in tasks requiring interpretability. This paper presents LXDR (Local eXplanation of Dimensionality Reduction), a local, model-agnostic technique that can be applied to any DR technique. LXDR trains linear models around a neighborhood of a specific instance and provides local interpretations using a variety of neighborhood generation techniques. Variations of the proposed technique are also introduced. The effectiveness of LXDR’s interpretations is evaluated by quantitative and qualitative experiments, as well as demonstrations of its practical implementation in diverse use cases. The experiments emphasize the importance of interpretability in dimensionality reduction and how LXDR reinforces it.

A computationally efficient dimensionality reduction and attack classification approach for network intrusion detection

A computationally efficient dimensionality reduction and attack classification approach for network intrusion detection

Machine Learning and Feature Selection for soil spectroscopy. An evaluation of Random Forest wrappers to predict soil organic matter, clay, and carbonates

Soil spectroscopy estimates soil properties using the absorption features in soil spectra. However, modelling soil properties with soil spectroscopy is challenging due to the high dimensionality of spectral data. Feature Selection wrapper methods are promising approaches to reduce the dimensionality but are barely used in soil spectroscopy. The aim of this study is to evaluate the performance of two feature selection wrapper methods, Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS) built using the Random Forest (RF) algorithm, for dimensionality reduction of spectral data and predictive modelling of modelling soil organic matter (SOM), clay and carbonates. The reflectance of 100 soil samples, acquired from Sierra de las Nieves (Spain), was measured under laboratory conditions using ASD FieldSpec Pro JR. Four different datasets were obtained after applying two spectral preprocessing methods to raw spectra: raw spectra, Continuum Removal (CR), Multiplicative Scatter Correction (MSC), and a so-called “Global” dataset composed of raw, CR and MSC features. The performance of RF models built with feature selection methods was compared to that of Partial Least Squares Regression (PLSR) and RF (alone).RF models built with SFS and SFFS outperformed PLSR and RF alone models: The best RF models with feature selection had a respective ratio of performance to interquartile distance of 1.93, 0.38 and 2.56. PLSR models had an accuracy of 1.41, 0.29 and 1.81 for SOM, carbonates, and clay, respectively. RF alone had a respective performance of 1.29, 0.29 and 1.81. The application of feature selection wrapper methods reduced the number of features to less than 1 % of the starting features. Features were selected across all spectra for SOM and clay, and around 900 nm, 1900 nm, and 2350 nm for carbonates. However, feature selection highlighted features around 1100 nm in SOM modelling, as well as other features around 2200 nm, which is considered a main absorption feature of clay. The application of feature selection with Random Forest was very important in improving modelling accuracy, reducing the redundant features and avoiding the curse of dimensionality or Hughes effect. Thus, this research showed an alternative to dimensionality reduction approaches that have been applied to date to model soil properties with spectroscopy and paves the way for further scientific investigation based on feature selection methods and machine learning.

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. The proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10−7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10−5. The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

Are national or regional surveys useful for nowcasting regional jobseekers? The case of the French region of Pays‐de‐la‐Loire

AbstactIn this paper we develop nowcasting models for the Pays‐de‐la‐Loire's jobseekers, a dynamic French regional economy. We ask whether these regional nowcasts are more accurate by only using the regional data or by combining the national and regional data. For this purpose, we use penalized regressions, random forest, and dynamic factor models as well as dimension reduction approaches. The best nowcasting performance is provided by the DFM estimated on the regional and regional‐national databases as well as the Elastic‐Net model with a prior screening step for which the national data are the most frequently selected data. For the latter, it appears that the Change in foreign orders in the industry sector, the OECD Composite leading indicator, and the BdF Business sentiment indicator are among the major predictors.

Predicting the Equity Premium with Combination Forecasts: A Reappraisal

Abstract This paper reappraises the usefulness of combining individual forecasts for predicting the U.S. equity premium. For comparison, we also consider penalized regression and dimension reduction approaches. We fail to find evidence of predictive ability in recent decades, regardless of the forecasting method used. Further analysis shows that an increase in the correlation of individual forecast errors is an important factor in the declining performance of combination forecasts.()

Distance Analysis and Dimensionality Reduction using PCA on Brain Tumour MRI Scans

INTRODUCTION: Compression of MRI images while maintaining essential information, makes it easier to distinguish between different types of brain tumors. It also assesses the effect of PCA on picture representation modification and distance analysis between tumor classes.OBJECTIVES: The objective of this work is to enhance the interpretability and classification accuracy of highdimensional MRI scans of patients with brain tumors by utilising Principle Component Analysis (PCA) to reduce their complexity.METHODS:This study uses PCA to compress high-dimensional MRI scans of patients with brain tumors, focusing on improving classification using dimensionality reduction approaches and making the scans easier to understand.RESULTS: PCA efficiently reduced MRI data, enabling better discrimination between different types of brain tumors and significant changes in distance matrices, which emphasize structural changes in the data.CONCLUSION: PCA is crucial for improving the interpretability of MRI data.

Machine learning approach for intrusion detection system using dimensionality reduction

As cyberspace has emerged, security in all the domains like networks, cloud, and databases has become a greater concern in real-time distributed systems. Existing systems for detecting intrusions (IDS) are having challenges coping with constantly changing threats. The proposed model, DR-DBMS (dimensionality reduction in database management systems), creates a unique strategy that combines supervised machine learning algorithms, dimensionality reduction approaches and advanced rule-based classifiers to improve intrusion detection accuracy in terms of different types of attacks. According to simulation results, the DR-DBMS system detected the intrusion attack in 0.07 seconds and with a smaller number of features using the dimensionality reduction and feature selection techniques efficiently.

MISTRA: Misogyny Detection through Text–Image Fusion and Representation Analysis

Detecting misogynous memes poses a significant challenge due to the presence of multiple modalities (image + text). The inherent complexity arises from the lack of direct correspondence between the textual and visual elements, where an image and overlaid text often convey disparate meanings. Additionally, memes conveying messages of hatred or taunting, particularly targeted towards women, present additional comprehension difficulties. This article introduces the MISTRA framework, which leverages variational autoencoders for dimensionality reduction of the large-sized image features before fusing multimodal features. The framework also harnesses the capabilities of large language models through transfer learning to develop fusion embeddings by extracting and concatenating features from different modalities (image, text, and image-generated caption text) for the misogynous classification task. The components of the framework include state-of-the-art models such as the Vision Transformer model (ViT), textual model (DistilBERT), CLIP (Contrastive Language–Image Pre-training), and BLIP (Bootstrapping Language–Image Pre-training for Unified Vision-Language Understanding and Generation) models. Our experiments are conducted on the SemEval-2022 Task 5 MAMI dataset. To establish a baseline model, we perform separate experiments using the Naive Bayes machine learning classifier on meme texts and ViT on meme images. We evaluate the performance on six different bootstrap samples and report evaluation metrics such as precision, recall, and Macro-F1 score for each bootstrap sample. Additionally, we compute the confidence interval on our evaluation scores and conduct paired t-tests to understand whether our best-performing model has significant differences from the other experiments or not. The experimental results demonstrate that the dimensionality reduction approach on multimodal features with a multilayer perceptron classifier achieved the highest performance with a Macro–F1 score of 71.5 percent, outperforming the baseline approaches in individual modalities.

A novel approach for dimensionality reduction of high-dimensional stochastic dynamical systems using symbolic regression

This paper introduces an innovative data-driven approach for dimensionality reduction in stochastic dynamical systems. The method combines stochastic averaging theory with symbolic regression. First, this approach generates simulation data from the high-dimensional stochastic dynamical system that needs to be reduced. Then, a loss function is designed based on stochastic averaging theory, and symbolic regression is used to obtain a parameterized one-dimensional equation from the simulation data. Finally, the effectiveness of the approach proposed in this paper is demonstrated through validation on Hamiltonian systems with 5, 10, and 20 degrees of freedom (DOF) subjected to noisy excitation. The primary advantage of this method is that it ultimately provides a closed form rather than a black-box process. Furthermore, it overcomes the numerical integration challenges that stochastic averaging methods often face when dealing with high-dimensional systems, therefore it performs effectively with high-dimensional stochastic dynamical systems.

Gender classification based on gait analysis using ultrawide band radar augmented with artificial intelligence

The identification of individuals based on their walking patterns, also known as gait recognition, has garnered considerable interest as a biometric trait. The use of gait patterns for gender classification has emerged as a significant research domain with diverse applications across multiple fields. The present investigation centers on the classification of gender based on gait utilizing data from Ultra-wide band radar. A total of 181 participants were included in the study, and data was gathered using Ultra-wide band radar technology. This study investigates various preprocessing techniques, feature extraction methods, and dimensionality reduction approaches to efficiently process Ultra-wide band radar data. The data quality is improved through the utilization of a two-pulse canceller and discrete wavelet transform. The hybrid feature dataset is generated through the creation of gray-level co-occurrence matrices and subsequent extraction of statistical features. Principal Component Analysis is utilized for dimensionality reduction, and prediction probabilities are incorporated as features for classification optimization. The present study employs k-fold cross-validation to train and assess machine learning classifiers, Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, Multi-Layer Perceptron, K-Nearest Neighbors, and Extra Tree Classifier. The Multilayer Perceptron exhibits superior performance, achieving an accuracy of 0.936. The Support Vector Machine and k-Nearest Neighbors classifiers closely trail behind, both achieving an accuracy of 0.934. This research is of the utmost importance due to its capacity to offer solutions to crucial problems in multiple domains. The findings indicate that the utilization of UWB radar data for gait-based gender classification holds promise in diverse domains, including biometrics, surveillance, and healthcare. The present study makes a valuable contribution to the progress of gender classification systems that rely on gait patterns.

Interpretable linear dimensionality reduction based on bias-variance analysis

AbstractOne of the central issues of several machine learning applications on real data is the choice of the input features. Ideally, the designer should select a small number of the relevant, nonredundant features to preserve the complete information contained in the original dataset, with little collinearity among features. This procedure helps mitigate problems like overfitting and the curse of dimensionality, which arise when dealing with high-dimensional problems. On the other hand, it is not desirable to simply discard some features, since they may still contain information that can be exploited to improve results. Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower dimensional space, possibly considering all the original features. However, the projected features resulting from the application of dimensionality reduction techniques are usually difficult to interpret. In this paper, we seek to design a principled dimensionality reduction approach that maintains the interpretability of the resulting features. Specifically, we propose a bias-variance analysis for linear models and we leverage these theoretical results to design an algorithm, Linear Correlated Features Aggregation (LinCFA), which aggregates groups of continuous features with their average if their correlation is “sufficiently large”. In this way, all features are considered, the dimensionality is reduced and the interpretability is preserved. Finally, we provide numerical validations of the proposed algorithm both on synthetic datasets to confirm the theoretical results and on real datasets to show some promising applications.

Abstract 38: Prediction of Coronary Artery Disease Subtypes With Polygenic Risk Scores

Background: Coronary Artery Disease (CAD) is a complex heterogeneous disease with multiple known etiological mechanisms. Consequently, CAD could be considered to have multiple subtypes, reflecting alternative driving etiologies that may warrant distinct therapeutic approaches. Motivated by this, we developed and optimized statistical models to predict CAD subtypes informed by genetic and clinical risk factors. Methods: We developed three models, each using logistic regression: (a) clinical risk factors only, (b) clinical risk factors plus genome-wide polygenic risk scores (PRS), and (c) clinical risk factors and pathway-based PRSs. We defined 5 different CAD subtypes on the basis of the literature: (1) stable vs. unstable, (2) occlusive vs non-occlusive, (3) high vs normal LDL, (4) high vs normal Lp(a), and (5) high vs normal ASCVD score. Clinical risk factors included: ASCVD score, ApoA, ApoB, LDL, HDL, triglycerides, and C-reactive protein. We calculated 4,402 pathway PRSs obtained from six genomic pathway databases (e.g. KEGG). We optimized the models using 80% of the samples as training, and then performed model evaluation (Bonferroni-corrected P-values) in an unseen 20% of the samples as validation. Results: Among 20,935 UK Biobank participants with CAD (mean age 63 years, 58% female), addition of pathway-based PRSs improved prediction of high LDL (P=1e-19) and high Lp(a) subtypes (P=1.8e-78). Using a lasso dimension reduction approach, the pathways most associated with the high Lp(a) subtype of CAD were: Fibronectin Binding (P=6xe-260), Apolipoprotein Binding (P=6.2e-252) and Serine-Type Endopeptidase Activity (P=1.7e-203). Conclusions: We demonstrate that prediction models distinguish CAD subtypes and that a novel pathway PRS approach can increase predictive power and provide insights into the genetic underpinnings of CAD subtypes. Pathway PRSs hold promise for the targeted therapeutic management for subgroups of CAD patients.

Mayfly algorithm-based semi-supervised band selection with enhanced bitonic filter for spectral-spatial hyperspectral image classification

ABSTRACT Hyperspectral imaging effectively distinguishes land cover classes with discriminative spectral-spatial characteristics. However, hyperspectral image (HSI) classification encounters three significant challenges: the curse of dimensionality, the scarcity of training samples, and the necessity to incorporate spatial information within the classification process. The large number of redundant and irrelevant spectral bands in HSI can negatively impact the classification performance. Furthermore, the annotation process for training samples is expensive and time-consuming. To address these challenges, this work presents a novel semi-supervised approach for hyperspectral image dimensionality reduction using the Mayfly Algorithm (MA). The proposed method aims to preserve the most informative spectral bands while leveraging both labelled and unlabelled samples in the process. The MA algorithm effectively balances exploration and exploitation strategies, enhancing population diversity and mitigating challenges such as premature convergence and local optima. Additionally, the inclusion of structural variation and thresholding (SVT) in the Bitonic filter enables the incorporation of spatial features during the classification phase. This improves the separability of pixels belonging to different objects and leads to significant improvements in classification performance. Finally, two machine learning classifiers, namely Random Forest (RF) and Support Vector Machine (SVM), are applied at the pixel level for hyperspectral image classification. Extensive experimental results on three real hyperspectral datasets demonstrate the superiority of the proposed approach compared to state-of-the-art algorithms like Harmony Search (HS), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and more with an average improvement of 3.2%.

A New Hybridized Dimensionality Reduction Approach Using Genetic Algorithm and Folded Linear Discriminant Analysis Applied to Hyperspectral Imaging for Effective Rice Seed Classification

A New Hybridized Dimensionality Reduction Approach Using Genetic Algorithm and Folded Linear Discriminant Analysis Applied to Hyperspectral Imaging for Effective Rice Seed Classification

대용량 자료의 이항 분류에서의 충분 차원 축소를 위한 전향적 접근 방법

Sufficient dimension reduction, aimed at finding a lower-dimensional subspace in explanatory variables that contains response variable information, typically relies on inverse-based methodologies. These methods are easy to implement but often require linear or constant variance conditions. To address these limitations, techniques for forwardly estimating the central subspace have been developed. In particular, methods utilizing the Reproducing Kernel Hilbert Space have gained attention, but their use in analyzing large datasets is limited due to the characteristics of the kernel space. In this paper, we study a novel forward approach for sufficient dimension reduction in binomial classification of large-scale data. We propose a method that employs a divide-and-conquer technique to split data into subsets, then independently perform dimension reduction on each subset before synthesizing them into a final model. It was shown that when the number of partitions of data was appropriately selected, the loss in prediction accuracy was not significant compared to the existing method, while being efficient in terms of storage space and calculation cost. In addition, simulations in various models showed superior prediction accuracy than other inverse-based techniques. The utility of the proposed method was confirmed through the real data analysis.

Extensive dynamic changes in the human transcriptome and its circadian organization during prolonged bed rest

Physiological and molecular processes including the transcriptome change across the 24-h day, driven by molecular circadian clocks and behavioral and systemic factors. It is not known how the temporal organization of the human transcriptome responds to a long-lasting challenge. This may, however, provide insights into adaptation, disease, and recovery. We investigated the human 24-h time series transcriptome in 20 individuals during a 90-day constant bed rest protocol. We show that the protocol affected 91% of the transcriptome with 76% of the transcriptome still affected after 10days of recovery. Dimensionality-reduction approaches revealed that many affected transcripts were associated with mRNA translation and immune function. The number, amplitude, and phase of rhythmic transcripts, including clock genes, varied significantly across the challenge. These findings of long-lasting changes in the temporal organization of the transcriptome have implications for understanding the mechanisms underlying health consequences of conditions such as microgravity and bed rest.

Multi-dimensional scaling techniques unveiled gain1q&loss13q co-occurrence in Multiple Myeloma patients with specific genomic, transcriptional and adverse clinical features

The complexity of Multiple Myeloma (MM) is driven by several genomic aberrations, interacting with disease-related and/or -unrelated factors and conditioning patients’ clinical outcome. Patient’s prognosis is hardly predictable, as commonly employed MM risk models do not precisely partition high- from low-risk patients, preventing the reliable recognition of early relapsing/refractory patients. By a dimensionality reduction approach, here we dissect the genomic landscape of a large cohort of newly diagnosed MM patients, modelling all the possible interactions between any MM chromosomal alterations. We highlight the presence of a distinguished cluster of patients in the low-dimensionality space, with unfavorable clinical behavior, whose biology was driven by the co-occurrence of chromosomes 1q CN gain and 13 CN loss. Presence or absence of these alterations define MM patients overexpressing either CCND2 or CCND1, fostering the implementation of biology-based patients’ classification models to describe the different MM clinical behaviors.

Recognition of differential culture conditions with dimensional reduction approach

Microorganisms constantly modify their gene expression and metabolic profiles in response to alterations in their surrounding environment. Monitoring these changes is crucial for regulating microbial production of substances. However, it remains challenging to identify differential culture conditions through the extraction of differentially expressed genes and clustering of gene expression profiles. In this study, we employed a dimensionality reduction technique for yeast gene expression data obtained under multiple culture conditions to visualize discrepancies among culture conditions. Our findings indicate that this approach is effective in identifying multiple culture conditions.