Dimensionality Reduction Method Research Articles

Supervised Clustering of Persian Handwritten Images Using Regularization and Dimension Reduction Methods

Clustering, as a fundamental exploratory data technique, not only is used to discover patterns and structures in complex datasets but also is utilized to group variables in high-dimensional data analysis. Dimension reduction through clustering helps identify important variables and reduce data dimensions without losing significant information. High-dimensional image datasets, such as Persian handwritten images, have numerous pixels, making statistical inference difficult. Such high-dimensionality property pose challenges for analysis and processing, requiring specialized techniques like clustering to extract information. Incorporating response variable information enhances clustering analysis, transforming it into a supervised method. This article evaluates a supervised clustering approach using Ridge and Lasso penalties, comparing them in analyzing a real dataset while identifying important variables. We demonstrate that despite choosing a small number of variables as important variables, Lasso penalty performs relatively well in predicting the labels of new observations for this multi-class dataset.

Dissecting muscle synergies in the task space

The muscle synergy is a guiding concept in motor control research that relies on the general notion of muscles ‘working together’ towards task performance. However, although the synergy concept has provided valuable insights into motor coordination, muscle interactions have not been fully characterised with respect to task performance. Here, we address this research gap by proposing a novel perspective to the muscle synergy that assigns specific functional roles to muscle couplings by characterising their task-relevance. Our novel perspective provides nuance to the muscle synergy concept, demonstrating how muscular interactions can ‘work together’ in different ways: (1) irrespective of the task at hand but also (2) redundantly or (3) complementarily towards common task-goals. To establish this perspective, we leverage information- and network-theory and dimensionality reduction methods to include discrete and continuous task parameters directly during muscle synergy extraction. Specifically, we introduce co-information as a measure of the task-relevance of muscle interactions and use it to categorise such interactions as task-irrelevant (present across tasks), redundant (shared task information), or synergistic (different task information). To demonstrate these types of interactions in real data, we firstly apply the framework in a simple way, revealing its added functional and physiological relevance with respect to current approaches. We then apply the framework to large-scale datasets and extract generalizable and scale-invariant representations consisting of subnetworks of synchronised muscle couplings and distinct temporal patterns. The representations effectively capture the functional interplay between task end-goals and biomechanical affordances and the concurrent processing of functionally similar and complementary task information. The proposed framework unifies the capabilities of current approaches in capturing distinct motor features while providing novel insights and research opportunities through a nuanced perspective to the muscle synergy.

Dissecting muscle synergies in the task space.

The muscle synergy is a guiding concept in motor control research that relies on the general notion of muscles 'working together' towards task performance. However, although the synergy concept has provided valuable insights into motor coordination, muscle interactions have not been fully characterised with respect to task performance. Here, we address this research gap by proposing a novel perspective to the muscle synergy that assigns specific functional roles to muscle couplings by characterising their task-relevance. Our novel perspective provides nuance to the muscle synergy concept, demonstrating how muscular interactions can 'work together' in different ways: (1) irrespective of the task at hand but also (2) redundantly or (3) complementarily towards common task-goals. To establish this perspective, we leverage information- and network-theory and dimensionality reduction methods to include discrete and continuous task parameters directly during muscle synergy extraction. Specifically, we introduce co-information as a measure of the task-relevance of muscle interactions and use it to categorise such interactions as task-irrelevant (present across tasks), redundant (shared task information), or synergistic (different task information). To demonstrate these types of interactions in real data, we firstly apply the framework in a simple way, revealing its added functional and physiological relevance with respect to current approaches. We then apply the framework to large-scale datasets and extract generalizable and scale-invariant representations consisting of subnetworks of synchronised muscle couplings and distinct temporal patterns. The representations effectively capture the functional interplay between task end-goals and biomechanical affordances and the concurrent processing of functionally similar and complementary task information. The proposed framework unifies the capabilities of current approaches in capturing distinct motor features while providing novel insights and research opportunities through a nuanced perspective to the muscle synergy.

A Reduced Sparse Dictionary Reconstruction Algorithm Based on Grid Selection

A sparse dictionary reconstruction algorithm based on grid selection is introduced to solve the grid mismatch when using the sparse recovery space time adaptive processing (SR-STAP) algorithm. First, the atom most closely related to clutter is selected from the traditional dictionary through the spectral value dimensionality reduction method. The local mesh is divided around the selected atoms to create mesh cells, and the mesh cells that are most likely to appear in the real clutter points are judged according to the local selection iteration criteria. In this way, the mesh spacing is refined, the local mesh selection is carried out step by step, and the optimal atoms in the local region are constantly adjusted and selected to narrow the search region until the iteration termination condition is met. Finally, the space-time plane is divided using a novel meshing technique that centers around the optimal atom. By removing atoms beyond the maximum range of spatial and Doppler frequencies, the simplified sparse dictionary can overcome the mesh mismatch problem. The simulation results demonstrate that the algorithm enhances the sparse recovery accuracy of clutter space-time spectrum, mitigates the mesh mismatch effect, and boosts STAP performance.

A New and Effective Classification Method for Complex Time Series Based on Information Measure

The growing importance of time series information measure raises questions about how to effectively cluster a large number of nonlinear complex time series data and accurately extract more hidden information from them. In this paper, a clustering measurement and classification method for complex time series, the symmetrical exponential Tsallis relative information (SETRI) measure, is proposed, which aims to address these problems. The intrinsic characteristics of different types of time series information could be validly identified by this method. The modified multidimensional scaling (MDS) method, based on the SETRI measure, has the ability to display the data in the form of graphs for intuitive exhibition and accomplish the process of dimension reduction. The introduction of weighted permutation patterns allows a higher-accuracy classification not only for the time series dissimilarity quantification, but also for avoiding dispensable errors. Besides, the feasibility of the modified MDS classification method is visually and quantitatively verified by the simulated and real-world data. Compared with other MDS methods, the proposed method has better performance, which is reflected in the validity and rationality of the clustering results, thus further verifying the feasibility of the proposed method. Therefore, the new results will be helpful to develop complex data clustering and dimensionality reduction methods.

Machine Learning Deciphered Molecular Mechanistics with Accurate Kinetic and Thermodynamic Prediction.

Time-lagged independent component analysis (tICA) and the Markov state model (MSM) have been extensively employed for extracting conformational dynamics and kinetic community networks from unbiased trajectory ensembles. However, these techniques may not be the optimal choice for elucidating transition mechanisms within low-dimensional representations, especially for intricate biosystems. Unraveling the association mechanism in such complex systems always necessitates permutations of several essential independent components or collective variables, a process that is inherently obscure and may require empirical knowledge for selection. To address these challenges, we have implemented an integrated unsupervised dimension reduction model: uniform manifold approximation and projection (UMAP) with hierarchy density-based spatial clustering of applications with noise (HDBSCAN). This approach effectively generates low-dimensional configurational embeddings. The hierarchical application of this architecture, in conjunction with MSM, reveals global kinetic connectivity while identifying local conformational states. Consequently, our methodology establishes a multiscale mechanistic elucidation framework. Leveraging the benefits of the uniform sample distribution and a denoising approach, our model demonstrates robustness in preserving global and local data structures compared to traditional dimension reduction methods in the field of MD analysis area. The interpretability of hyperparameter selection and compatibility with downstream tasks are cross-validated across various simulation data sets, utilizing both computational evaluation metrics and experimental kinetic observables. Furthermore, the predicted Mcl1-BH3 association kinetics (0.76 s-1) is in close agreement with surface plasmon resonance experiments (0.12 s-1), affirming the plausibility of the identified pathway composed of representative conformations. We anticipate that the devised workflow will serve as a foundational framework for studying recognition patterns in complex biological systems. Its contributions extend to the exploration of protein functional dynamics and rational drug design, offering a potent avenue for advancing research in these domains.

Epilepsy EEG signals classification based on sparse principal component logistic regression model

Epilepsy is a chronic brain disease caused by excessive discharge of brain neurons. Long-term recurrent seizures bring a lot of trouble to patients and their families. Prediction of different stages of epilepsy is of great significance. We extract pearson correlation coefficients (PCC) between channels in different frequency bands as features of EEG signals for epilepsy stages prediction. However, the features are of large feature dimension and serious multi-collinearity. To eliminate these adverse influence, the combination of traditional dimension reduction method principal component analysis (PCA) and logistic regression method with regularization term is proposed to avoid over-fitting and achieve the feature sparsity. The experiments are conducted on the widely used CHB-MIT dataset using different regularization terms L 1 and L 2 , respectively. The proposed method identifies various stages of epilepsy quickly and efficiently, and it presents the best average accuracy of 94.86%, average precision of 96.71%, average recall of 93.48%, average kappa value of 0.90 and average Matthews correlation coefficient (MCC) value of 0.90 for all patients.

Chronic malaria exposure is associated with inhibitory markers on T cells that correlate with atypical memory and marginal zone-like B cells.

Chronic immune activation from persistent malaria infections can induce immunophenotypic changes associated with T-cell exhaustion. However, associations between T and B cells during chronic exposure remain undefined. We analyzed peripheral blood mononuclear cells from malaria-exposed pregnant women from Papua New Guinea and Spanish malaria-naïve individuals using flow cytometry to profile T-cell exhaustion markers phenotypically. T-cell lineage (CD3, CD4, and CD8), inhibitory (PD1, TIM3, LAG3, CTLA4, and 2B4), and senescence (CD28-) markers were assessed. Dimensionality reduction methods revealed increased PD1, TIM3, and LAG3 expression in malaria-exposed individuals. Manual gating confirmed significantly higher frequencies of PD1+CD4+ and CD4+, CD8+, and double-negative (DN) T cells expressing TIM3 in malaria-exposed individuals. Increased frequencies of T cells co-expressing multiple markers were also found in malaria-exposed individuals. T-cell data were analyzed with B-cell populations from a previous study where we reported an alteration of B-cell subsets, including increased frequencies of atypical memory B cells (aMBC) and reduction in marginal zone (MZ-like) B cells during malaria exposure. Frequencies of aMBC subsets and MZ-like B cells expressing CD95+ had significant positive correlations with CD28+PD1+TIM3+CD4+ and DN T cells and CD28+TIM3+2B4+CD8+ T cells. Frequencies of aMBC, known to associate with malaria anemia, were inversely correlated with hemoglobin levels in malaria-exposed women. Similarly, inverse correlations with hemoglobin levels were found for TIM3+CD8+ and CD28+PD1+TIM3+CD4+ T cells. Our findings provide further insights into the effects of chronic malaria exposure on circulating B- and T-cell populations, which could impact immunity and responses to vaccination.

Predicting S. aureus antimicrobial resistance with interpretable genomic space maps.

Increasing antimicrobial resistance (AMR) represents a global healthcare threat. To decrease the spread of AMR and associated mortality, methods for rapid selection of optimal antibiotic treatment are urgently needed. Machine learning (ML) models based on genomic data to predict resistant phenotypes can serve as a fast screening tool prior to phenotypic testing. Nonetheless, many existing ML methods lack interpretability. Therefore, we present a methodology for visualization of sequence space and AMR prediction based on the non-linear dimensionality reduction method - generative topographic mapping (GTM). This approach, applied to AMR data of >5000 S. aureus isolates retrieved from the PATRIC database, yielded GTM models with reasonable accuracy for all drugs (balanced accuracy values ≥0.75). The Generative Topographic Maps (GTMs) represent data in the form of illustrative maps of the genomic space and allow for antibiotic-wise comparison of resistant phenotypes. The maps were also found to be useful for the analysis of genetic determinants responsible for drug resistance. Overall, the GTM-based methodology is a useful tool for both the illustrative exploration of the genomic sequence space and AMR prediction.

Joint modeling of an outcome variable and integrated omics datasets using GLM-PO2PLS

In many studies of human diseases, multiple omics datasets are measured. Typically, these omics datasets are studied one by one with the disease, thus the relationship between omics is overlooked. Modeling the joint part of multiple omics and its association to the outcome disease will provide insights into the complex molecular base of the disease. Several dimension reduction methods which jointly model multiple omics and two-stage approaches that model the omics and outcome in separate steps are available. Holistic one-stage models for both omics and outcome are lacking. In this article, we propose a novel one-stage method that jointly models an outcome variable with omics. We establish the model identifiability and develop EM algorithms to obtain maximum likelihood estimators of the parameters for normally and Bernoulli distributed outcomes. Test statistics are proposed to infer the association between the outcome and omics, and their asymptotic distributions are derived. Extensive simulation studies are conducted to evaluate the proposed model. The method is illustrated by modeling Down syndrome as outcome and methylation and glycomics as omics datasets. Here we show that our model provides more insight by jointly considering methylation and glycomics.

A Review of Machine Learning Algorithms for Biomedical Applications.

As the amount and complexity of biomedical data continue to increase, machine learning methods are becoming a popular tool in creating prediction models for the underlying biomedical processes. Although all machine learning methods aim to fit models to data, the methodologies used can vary greatly and may seem daunting at first. A comprehensive review of various machine learning algorithms per biomedical applications is presented. The key concepts of machine learning are supervised and unsupervised learning, feature selection, and evaluation metrics. Technical insights on the major machine learning methods such as decision trees, random forests, support vector machines, and k-nearest neighbors are analyzed. Next, the dimensionality reduction methods like principal component analysis and t-distributed stochastic neighbor embedding methods, and their applications in biomedical data analysis were reviewed. Moreover, in biomedical applications predominantly feedforward neural networks, convolutional neural networks, and recurrent neural networks are utilized. In addition, the identification of emerging directions in machine learning methodology will serve as a useful reference for individuals involved in biomedical research, clinical practice, and related professions who are interested in understanding and applying machine learning algorithms in their research or practice.

Seeing is Learning in High Dimensions: The Synergy Between Dimensionality Reduction and Machine Learning

High-dimensional data are a key study object for both machine learning (ML) and information visualization. On the visualization side, dimensionality reduction (DR) methods, also called projections, are the most suited techniques for visual exploration of large and high-dimensional datasets. On the ML side, high-dimensional data are generated and processed by classifiers and regressors, and these techniques increasingly require visualization for explanation and exploration. In this paper, we explore how both fields can help each other in achieving their respective aims. In more detail, we present both examples that show how DR can be used to understand and engineer better ML models (seeing helps learning) and also applications of DL for improving the computation of direct and inverse projections (learning helps seeing). We also identify existing limitations of DR methods used to assist ML and of ML techniques applied to improve DR. Based on the above, we propose several high-impact directions for future work that exploit the analyzed ML-DR synergy.

Enhanced Sampling with Machine Learning.

Molecular dynamics (MD) enables the study of physical systems with excellent spatiotemporal resolution but suffers from severe timescale limitations. To address this, enhanced sampling methods have been developed to improve the exploration of configurational space. However, implementing these methods is challenging and requires domain expertise. In recent years, integration of machine learning (ML) techniques into different domains has shown promise, prompting their adoption in enhanced sampling as well. Although ML is often employed in various fields primarily due to its data-driven nature, its integration with enhanced sampling is more natural with many common underlying synergies. This review explores the merging of ML and enhanced MD by presenting different shared viewpoints. It offers a comprehensive overview of this rapidly evolving field, which can be difficult to stay updated on. We highlight successful strategies such as dimensionality reduction, reinforcement learning, and flow-based methods. Finally, we discuss open problems at the exciting ML-enhanced MD interface.

An adaptive data-driven subspace polynomial dimensional decomposition for high-dimensional uncertainty quantification based on maximum entropy method and sparse Bayesian learning

Polynomial dimensional decomposition (PDD) is a surrogate method originated from the ANOVA (analysis of variance) decomposition, and has shown powerful performance in uncertainty quantification (UQ) accuracy and convergence recently. However, complex high-dimensional problems result in a large number of polynomial basis functions, leading to heavy computational burden, and the probability distributions of the input random variables are indispensable for PDD modeling and UQ, which may be unavailable in practical engineering. This study establishes an adaptive data-driven subspace PDD (ADDSPDD) for high-dimensional UQ, which employs two types of data for modeling the PDD basis function and the low-dimensional subspace directly, namely, the data of input random variables and the input-response samples. Firstly, we propose a data-driven zero-entropy criterion-based maximum entropy method for reconstructing the probability density functions (PDF) of input variables. Then, with the aid of the established PDFs, a data-driven subspace PDD (DDSPDD) is proposed based on the whitening transformation. To recover the subspace of the function of interest accurately and efficiently, we put forward an approximate active subspace method (AAS) based on the Taylor expansion under some mild premises. Finally, we integrate an adaptive learning algorithm into the DDSPDD framework based on the sparse Bayesian learning theory, obtaining our ADDSPDD; thus, the real subspace and the significant PDD basis functions can be identified with limited computational budget. We validate the proposed method by using four examples, and systematically compare four existing dimension-reduction methods with the AAS. Results show that the proposed framework is effective and the AAS is a good choice when the corresponding assumptions are satisfied.

Feasibility analysis of arterial CT radiomics model to predict the risk of local and metastatic recurrence after radical cystectomy for bladder cancer

PurposeTo construct a radiomics-clinical nomogram model for predicting the risk of local and metastatic recurrence within 3 years after radical cystectomy (RC) of bladder cancer (BCa) based on the radiomics features and important clinical risk factors for arterial computed tomography (CT) images and to evaluate its efficacy.MethodsPreoperative CT datasets of 134 BCa patients (24 recurrent) who underwent RC were collected and divided into training (n = 93) and validation sets (n = 41). Radiomics features were extracted from a 1.5 mm CT layer thickness image in the arterial phase. A radiomics score (Rad-Score) model was constructed using the feature dimension reduction method and a logistic regression model. Combined with important clinical factors, including gender, age, tumor size, tumor number and grade, pathologic T stage, lymph node stage and histology type of the archived lesion, and CT image signs, a radiomics-clinical nomogram was developed, and its performance was evaluated in the training and validation sets. Decision curve analyses (DCA) the potential clinical usefulness.ResultsThe radiomics model is finally linear combined by 8 features screened by LASSO regression, and after coefficient weighting, achieved good predictive results. The radiomics nomogram developed by combining two independent predictors, Rad-Score and pathologic T stage, was developed in the training set [AUC, 0.840; 95% confidence interval (CI) 0.743–0.937] and validation set (AUC, 0.883; 95% CI 0.777–0.989). The calibration curve showed good agreement between the predicted probability of the radiomics-clinical model and the actual recurrence rate within 3 years after RC for BCa. DCA show the clinical application value of the radiomics-clinical model.ConclusionThe radiomics-clinical nomogram model constructed based on the radiomics features of arterial CT images and important clinical risk factors is potentially feasible for predicting the risk of recurrence within 3 years after RC for BCa.

Research on Nuclear Power Plant Operational Task Complexity Assessment Method Based on Group Decision Making and Deep Learning of EEG Signals

This study aims to explore the correlation between the operational task complexity of nuclear power plant (NPP) operators and electroencephalogram (EEG) features. Initially, we segmented EEG signals according to operational steps and extracted a total of 120 time domain, frequency domain, and time-frequency domain features. Subsequently, we applied an adaptive principal component analysis (PCA) dimensionality reduction method to process the features. On the other hand, three experts were invited to evaluate the complexity of the operational tasks, and their evaluation data were synthesized using a group decision-making approach. A correlation analysis was performed between these data and the PCA-reduced feature data, identifying the features with the highest correlation coefficient for each participant. Then we built a long short-term memory model with the data of the first group of participants to predict the task complexity value and tested it with the data of the second group of participants. Testing the model with data from the second group yielded favorable results, with a training set mean squared error (MSE) of 0.025 and a testing set MSE of 0.078. The results of this study indicate a significant correlation between specific EEG features and task complexity in the operational tasks of NPP operators. The model established through a combination of group decision making and machine learning methods effectively predicted the task complexity levels for operators in different operational tasks. This research provides a new perspective on NPP operators’ cognitive load and operational tasks, holding practical significance for operator training and workload management.

Alzheimer's disease detection using data fusion with a deep supervised encoder.

Alzheimer's disease (AD) is affecting a growing number of individuals. As a result, there is a pressing need for accurate and early diagnosis methods. This study aims to achieve this goal by developing an optimal data analysis strategy to enhance computational diagnosis. Although various modalities of AD diagnostic data are collected, past research on computational methods of AD diagnosis has mainly focused on using single-modal inputs. We hypothesize that integrating, or "fusing," various data modalities as inputs to prediction models could enhance diagnostic accuracy by offering a more comprehensive view of an individual's health profile. However, a potential challenge arises as this fusion of multiple modalities may result in significantly higher dimensional data. We hypothesize that employing suitable dimensionality reduction methods across heterogeneous modalities would not only help diagnosis models extract latent information but also enhance accuracy. Therefore, it is imperative to identify optimal strategies for both data fusion and dimensionality reduction. In this paper, we have conducted a comprehensive comparison of over 80 statistical machine learning methods, considering various classifiers, dimensionality reduction techniques, and data fusion strategies to assess our hypotheses. Specifically, we have explored three primary strategies: (1) Simple data fusion, which involves straightforward concatenation (fusion) of datasets before inputting them into a classifier; (2) Early data fusion, in which datasets are concatenated first, and then a dimensionality reduction technique is applied before feeding the resulting data into a classifier; and (3) Intermediate data fusion, in which dimensionality reduction methods are applied individually to each dataset before concatenating them to construct a classifier. For dimensionality reduction, we have explored several commonly-used techniques such as principal component analysis (PCA), autoencoder (AE), and LASSO. Additionally, we have implemented a new dimensionality-reduction method called the supervised encoder (SE), which involves slight modifications to standard deep neural networks. Our results show that SE substantially improves prediction accuracy compared to PCA, AE, and LASSO, especially in combination with intermediate fusion for multiclass diagnosis prediction.

Application of Dimension Reduction Methods to High-Dimensional Single-Cell 3D Genomic Contact Data

The volume and complexity of data in various fields, particularly in biology, are increasing exponentially, posing a challenge to existing analytical methods, which often struggle with high-dimensional data such as single-cell Hi-C data. To address this issue, we employ unsupervised methods, specifically Principal Component Analysis (PCA) and t-Distributed Stochastic Neighbor Embedding (t-SNE), to reduce data dimensions for visualization. Furthermore, we assess the information retention of the decomposed components using a Linear Discriminant Analysis (LDA) classifier model. Our findings indicate that these dimensionality reduction techniques effectively capture and present information not readily apparent in the original high-dimensional data, facilitating the visualization and interpretation of complex biological data. The LDA classifier's performance suggests that PCA and t-SNE maintain critical information necessary for accurate classification. In conclusion, our study demonstrates that PCA and t-SNE are powerful tools for visualizing and analyzing high-dimensional biological data, enabling researchers to gain new insights and understandings that are challenging to achieve with traditional approaches.

How to choose a proper representation of compositional data for mineral exploration?

Regional mineral exploration is based on geochemical data of which the nature is compositional and frequently involves a large number of components. Consequently, it mostly needs multivariate dimension reduction methods such as principal component analysis (PCA) and its various robust versions. The application of such methods, defined for real random variables, require the data to be represented in coordinates supported in the real space. However, a common problem in exploration geochemistry is to select the appropriate representation. Using centered (clr) and isometric (ilr) logratio coordinates to discriminate anomalous zones for orogeny gold exploration throughout Sweden revealed that there is, as expected, no difference between the two representation methods. The main difference affects the interpretation of the coordinates used. This is observed for regional scale exploration, while it is also needed to study different ways of representing geochemical data in local scale.

Autonomous Ride-Sharing Service Using Graph Embedding and Dial-a-Ride Problem: Application to the Last-Mile Transit in Lyon City

Autonomous vehicles are anticipated to revolutionize ride-sharing services and subsequently enhance the public transportation systems through a first–last-mile transit service. Within this context, a fleet of autonomous vehicles can be modeled as a Dial-a-Ride Problem with certain features. In this study, we propose a holistic solving approach to this problem, which combines the mixed-integer linear programming formulation with a novel graph dimension reduction method based on the graph embedding framework. This latter method is effective since accounting for heterogeneous travel demands of the covered territory tends to increase the size of the routing graph drastically, thus rendering the exact solving of small instances computationally infeasible. An application is provided for the real transport demand of the industrial district of “Vallée de la Chimie” in Lyon city, France. Instances involving more than 50 transport requests and 10 vehicles could be easily solved. Results suggest that this method generates routes of reduced nodes with lower vehicle kilometers traveled compared to the constrained K-means-based reduction. Reductions in terms of GHG emissions are estimated to be around 75% less than the private vehicle mode in our applied service. A sensitivity analysis is also provided.