Low-dimensional Representation Research Articles

Low-dimensional signal representations for massive black hole binary signals analysis from LISA data

Low-dimensional signal representations for massive black hole binary signals analysis from LISA data

Semi-Supervised Learning for Multi-View Data Classification and Visualization

Data visualization has several advantages, such as representing vast amounts of data and visually demonstrating patterns within it. Manifold learning methods help us estimate lower-dimensional representations of data, thereby enabling more effective visualizations. In data analysis, relying on a single view can often lead to misleading conclusions due to its limited perspective. Hence, leveraging multiple views simultaneously and interactively can mitigate this risk and enhance performance by exploiting diverse information sources. Additionally, incorporating different views concurrently during the graph construction process using interactive visualization approach has improved overall performance. In this paper, we introduce a novel algorithm for joint consistent graph construction and label estimation. Our method simultaneously constructs a unified graph and predicts the labels of unlabeled samples. Furthermore, the proposed approach estimates a projection matrix that enables the prediction of labels for unseen samples. Moreover, it incorporates the information in the label space to further enhance the accuracy. In addition, it merges the information in different views along with the labels to construct a consensus graph. Experimental results conducted on various image databases demonstrate the superiority of our fusion approach compared to using a single view or other fusion algorithms. This highlights the effectiveness of leveraging multiple views and simultaneously constructing a unified graph for improved performance in data classification and visualization tasks in semi-supervised contexts.

Coincident Learning for Unsupervised Anomaly Detection of Scientific Instruments

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g., industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called CoAD, which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, F̂ᵦ, out of analogy to the supervised classification Fβ statistic. CoAD uses F̂ᵦ to train an anomaly detection algorithm on unlabeled data, based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

Pre-frontal cortex guides dimension-reducing transformations in the occipito-ventral pathway for categorization behaviors.

To interpret our surroundings, the brain uses a visual categorization process. Current theories and models suggest that this process comprises a hierarchy of different computations that transforms complex, high-dimensional inputs into lower-dimensional representations (i.e., manifolds) in support of multiple categorization behaviors. Here, we tested this hypothesis by analyzing these transformations reflected in dynamic MEG source activity while individual participants actively categorized the same stimuli according to different tasks: face expression, face gender, pedestrian gender, and vehicle type. Results reveal three transformation stages guided by the pre-frontal cortex. At stage 1 (high-dimensional, 50-120ms), occipital sources represent both task-relevant and task-irrelevant stimulus features; task-relevant features advance into higher ventral/dorsal regions, whereas task-irrelevant features halt at the occipital-temporal junction. At stage 2 (121-150ms), stimulus feature representations reduce to lower-dimensional manifolds, which then transform into the task-relevant features underlying categorization behavior over stage 3 (161-350ms). Our findings shed light on how the brain's network mechanisms transform high-dimensional inputs into specific feature manifolds that support multiple categorization behaviors.

Generating extremely low-dimensional representation of subsurface earth models using vector quantization and deep Autoencoder

Geological model compression is crucial for making large and complex models more manageable. By reducing the size of these models, compression techniques enable efficient storage, enhance computational efficiency, making it feasible to perform complex simulations and analyses in a shorter time. This is particularly important in applications such as reservoir management, groundwater hydrology, and geological carbon storage, where large geomodels with millions of grid cells are common. This study presents a comprehensive overview of previous work on geomodel compression and introduces several autoencoder-based deep-learning architectures for low-dimensional representation of modified Brugge-field geomodels. The compression and reconstruction efficiencies of autoencoders (AE), variational autoencoders (VAE), vector-quantized variational autoencoders (VQ-VAE), and vector-quantized variational autoencoders 2 (VQ-VAE2) were tested and compared to the traditional singular value decomposition (SVD) method. Results show that the deep-learning-based approaches significantly outperform SVD, achieving higher compression ratios while maintaining or even exceeding the reconstruction quality. Notably, VQ-VAE2 achieves the highest compression ratio of 667:1 with a structural similarity index metric (SSIM) of 0.92, far surpassing the 10:1 compression ratio of SVD with a SSIM of 0.9. The result of this work shows that, unlike traditional approaches, which often rely on linear transformations and can struggle to capture complex, non-linear relationships within geological data, VQ-VAE's use of vector quantization helps in preserving high-resolution details and enhances the model's ability to generalize across varying geological complexities.

Replacing non-biomedical concepts improves embedding of biomedical concepts.

Concept embeddings are low-dimensional vector representations of concepts such as MeSH:D009203 (Myocardial Infarction), whose similarity in the embedded vector space reflects their semantic similarity. Here, we test the hypothesis that non-biomedical concept synonym replacement can improve the quality of biomedical concepts embeddings. We developed an approach that leverages WordNet to replace sets of synonyms with the most common representative of the synonym set. We tested our approach on 1055 concept sets and found that, on average, the mean intracluster distance was reduced by 8% in the vector-space. Assuming that homophily of related concepts in the vector space is desirable, our approach tends to improve the quality of embeddings. This pilot study shows that non-biomedical synonym replacement tends to improve the quality of embeddings of biomedical concepts using the Word2Vec algorithm. We have implemented our approach in a freely available Python package available at https://github.com/TheJacksonLaboratory/wn2vec .

P-149 A visual interpretability method to unbox ‘black-box’ deep learning image-based classification of embryo properties

Abstract Study question Can we decipher the underlying visual properties that drive image-based AI embryo classification models to assist clinical decisions and biological discovery? Summary answer Our framework interpreted which annotated and non-explicitly-annotated phenotypes impact model predictions and rank their importance. These discoveries were aligned with known blastocyst quality criteria. What is known already Deep learning models have shown great promise for complex pattern recognition when applied to embryo images. The success of these models relies on their ability to perform non-linear optimization of feature extraction during model construction. However, this involves their entanglement of multiple classification-driving image properties, thereby producing ‘black-box’ systems that lack user confidence, trust and interpretability. Therefore, there is an urgent need for an interpretability method that can uncover the semantic image properties that contribute to ‘black box’ embryo image-based AI classification model predictions to assist in blastocyst selection. Study design, size, duration 11,211 time-lapse videos were retrospectively collected from three IVF centers. A deep convolutional neural network is first trained to discriminate high-versus-low quality blastocysts. We then developed DISCOVER, a general-purpose interpretability method designed to discover underlying visual properties driving the classifier. DISCOVER encodes an image to an interpretable lower dimensional representation which is correlated to the classifier and encapsulates a different distinct phenotype in each one of the dimensions. Participants/materials, setting, methods The encoding of embryo images to low dimensional representations enables interpretability globally and locally. Globally, the embryo images are synthetically altered by amplifying subtle properties that affect the classification decision. With our method this can be done one property at a time, therefore separating confounding properties. By evaluating the altered images, embryologists can decipher their meaning. Locally, each one of the discovered properties can be ranked by its importance for a specific embryo instance. Main results and the role of chance Using DISCOVER, we interpreted the classification model driving features. We quantitatively linked the top two classification features as blastocyst size (as proxy to degree of expansion and development) and trophectoderm quality, by embryologists evaluation and annotations. We then asked whether DISCOVER can identify non-explicitly annotated latent features that encode morphologic properties not defined by ASEBIR/Gardner criteria. Expert embryologist interpreted the third top classification feature to be the blastocoel. DISCOVER interpreted high quality embryos as having denser and more granular blastocoelic regions, suggesting that this change in the blastocoel appearance is one of the encoded classification-driving morphologic properties. This visualization indicates that there are additional parameters of the blastocoel beyond its volume expansion associated with its quality. We showed how embryo properties can be weighted differently by the classifier on a per embryo basis, giving clinical insight to which properties influence the classification of a specific instance. These results indicate that DISCOVER enables expert-in-the-loop interpretation of the classification model both globally, discovering the overall main properties driving the classifier, and locally, showing a per instance explanation. Limitations, reasons for caution DISCOVER failed to interpret the inner cell mass (ICM) as a classification-driving feature in its latent representation, though it was explicitly used to label the data for training the classification model. It is possible that other properties collectively contained the discriminative information encoded in the ICM. Wider implications of the findings This deep analysis demonstrates the feasibility of providing interpretability for biomedical image-based classification models for clinical use in the IVF clinic. Trial registration number not applicable

Variational autoencoder-based dimension reduction of Ichimoku features for improved financial market analysis

Financial markets are complex and dynamic, and accurately predicting market trends is crucial for traders and financial analysts. Ichimoku-based features have gained significant attention in financial market analysis due to their ability to capture essential market signals and patterns. This significant compression retains essential patterns related to trends, support/resistance levels, and trading signals. The reduced dimensionality improves computational efficiency and could allow for more accurate predictive modeling by traders. However, real-world testing is needed because compressing data risks losing useful nuances. In this study, we utilize an autoencoder for the dimensionality reduction of Ichimoku-based features in financial market analysis. The autoencoder, a neural network architecture, compresses high-dimensional data into a lower-dimensional representation by learning important features and patterns. The experiments conducted on a Euro/Dollar market dataset spanning 1990, comprising 16 columns with Ichimoku features, reveal the remarkable reduction of dataset size from 2,269,500 to 756,375, equivalent to a decrease of 66.67 %. These results highlight the efficiency of the proposed approach in reducing the dimensionality of financial market data, suggesting its potential as a valuable tool for traders and financial analysts to predict market trends and make informed decisions in the financial markets.

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

Reduced-order modeling of unsteady fluid flow using neural network ensembles

The use of deep learning has become increasingly popular in reduced-order models (ROMs) to obtain low-dimensional representations of full-order models. Convolutional autoencoders (CAEs) are often used to this end as they are adept at handling data that are spatially distributed, including solutions to partial differential equations. When applied to unsteady physics problems, ROMs also require a model for time-series prediction of the low-dimensional latent variables. Long short-term memory (LSTM) networks, a type of recurrent neural network useful for modeling sequential data, are frequently employed in data-driven ROMs for autoregressive time-series prediction. When making predictions at unseen design points over long time horizons, error propagation is a frequently encountered issue, where errors made early on can compound over time and lead to large inaccuracies. In this work, we propose using bagging, a commonly used ensemble learning technique, to develop a fully data-driven ROM framework referred to as the CAE-eLSTM ROM that uses CAEs for spatial reconstruction of the full-order model and LSTM ensembles for time-series prediction. When applied to two unsteady fluid dynamics problems, our results show that the presented framework effectively reduces error propagation and leads to more accurate time-series prediction of latent variables at unseen points.

Multi-sample non-negative spatial factorization.

It is important to model biological variation when analyzing spatial transcriptomics data from multiple samples. One approach to multi-sample analysis is to spatially align samples, but this is a challenging problem. Here, we provide an alignment-free framework for generalizing a one-sample spatial factorization model to multi-sample data. Using this framework, we develop a method, called multi-sample non-negative spatial factorization (mNSF) that extends the one-sample non-negative spatial factorization (NSF) framework to a multi-sample dataset. Our model allows for a sample-specific model for the spatial correlation structure and extracts a low-dimensional representation of the data. We illustrate the performance of mNSF by simulation studies and real data. mNSF identifies true factors in simulated data, identifies shared anatomical regions across samples in real data and reveals region-specific biological functions. mNSFs performance is similar to alignment based methods when alignment is possible, but extends analysis to situations where spatial alignment is impossible. We expect multi-sample factorization methods to be a powerful class of methods for analyzing spatially resolved transcriptomics data.

Real-time Bayesian inversion in resin transfer moulding using neural surrogates

In Resin Transfer Moulding (RTM), local variations in reinforcement properties (porosity and permeability) and the formation of gaps along the reinforcement edges result in non-uniform resin flow patterns, which may cause defects in the produced composite component. The ensemble Kalman inversion (EKI) algorithm has previously been used to invert in-process data to estimate local reinforcement properties. However, implementation of this algorithm in some applications is limited by the requirement to run thousands of computationally expensive resin flow simulations. In this study, a machine learning approach is used to train a surrogate model which can emulate resin flow simulations near-instantaneously. A partition of the flow domain into a low-dimensional representation enables an artificial neural network (ANN) surrogate to make accurate predictions, with a simple architecture. When the ANN is integrated within the EKI algorithm, estimates for local reinforcement permeability and porosity can be achieved in real time, as was verified by virtual and lab experiments. Since EKI utilises the Bayesian framework, estimates are given within confidence intervals and statements can be made on-line regarding the probability of defects within sections of the reinforcement. The proposed framework has shown good predictive capabilities for the set of laboratory experiments and estimates for reinforcement properties were always computed within 1 s

Reduced‐Order Modeling for Linearized Representations of Microphysical Process Rates

AbstractRepresenting cloud microphysical processes in large scale atmospheric models is challenging because many processes depend on the details of the droplet size distribution (DSD, the spectrum of droplets with different sizes in a cloud). While full or partial statistical moments of droplet size distributions are the typical variables used in bulk models, prognostic moments are limited in their ability to represent microphysical processes across the range of conditions experienced in the atmosphere. Microphysical parameterizations employing prognostic moments are known to suffer from structural uncertainty in their representations of inherently higher dimensional cloud processes, which limit model fidelity and lead to forecasting errors. Here we investigate how data‐driven reduced‐order modeling can be used to learn predictors for microphysical process rates in bulk microphysics schemes in an unsupervised manner from higher dimensional bin distributions. Using simulations characteristic of marine stratiform clouds, we simultaneously learn lower dimensional representations of droplet size distributions and predict the evolution of the microphysical state of the system. Droplet collision‐coalescence, the main process for generating warm rain, is estimated to have an intrinsic dimension of three. This intrinsic dimension provides a lower limit on the number of degrees of freedom needed to accurately represent collision‐coalescence in models. We demonstrate how deep learning based reduced‐order modeling can be used to discover intrinsic coordinates describing the microphysical state of the system, where process rates such as collision‐coalescence are globally linearized. These implicitly learned representations of the DSD retain more information about the DSD than typical moment‐based representations.

ST-CellSeg: Cell segmentation for imaging-based spatial transcriptomics using multi-scale manifold learning.

Spatial transcriptomics has gained popularity over the past decade due to its ability to evaluate transcriptome data while preserving spatial information. Cell segmentation is a crucial step in spatial transcriptomic analysis, as it enables the avoidance of unpredictable tissue disentanglement steps. Although high-quality cell segmentation algorithms can aid in the extraction of valuable data, traditional methods are frequently non-spatial, do not account for spatial information efficiently, and perform poorly when confronted with the problem of spatial transcriptome cell segmentation with varying shapes. In this study, we propose ST-CellSeg, an image-based machine learning method for spatial transcriptomics that uses manifold for cell segmentation and is novel in its consideration of multi-scale information. We first construct a fully connected graph which acts as a spatial transcriptomic manifold. Using multi-scale data, we then determine the low-dimensional spatial probability distribution representation for cell segmentation. Using the adjusted Rand index (ARI), normalized mutual information (NMI), and Silhouette coefficient (SC) as model performance measures, the proposed algorithm significantly outperforms baseline models in selected datasets and is efficient in computational complexity.

Consensus Subspace Graph Regularization based on prior information for multiplex network clustering

Multiplex network clustering can identify the common cluster structure shared by all layers, which is of great significance for downstream research such as personalized recommendations of social relationships and the mining of social media communication behaviors. Traditional multiplex network clustering methods rely only on the topological structure and are suitable for networks with clear structures. However, real-world networks often suffer from sparse connectivity and noisy edges. In addition, the current methods based on prior information only use the obtained consensus prior information as a preprocessing method for multiplex networks, and do not fully utilize and integrate the prior information in the clustering process, resulting in a low accuracy. To solve the problem of insufficient utilization of prior information, we propose a Consensus Subspace Graph Regularization (CSGR) approach for multiplex network clustering, which integrates topological information with the consensus prior information of a network. We first construct the consensus prior matrix of a network by a non-overlapping greedy search method, which represents a subset of the network that consists of nodes with high edge density. Then we construct a graph regularization term, which encodes the consensus prior information, to optimize the generation of a joint low-dimensional representation within the consensus subspace for the entire multiplex network. Finally, we employ the consensus prior matrix to denoise each network layer, thereby yielding a precise low-dimensional representation of each layer for consensus subspace fusion. In addition, comprehensive experiments have demonstrated that CSGR improves the clustering accuracy of real-world networks by an average of 1.56%.

Temporal information in the anterior cingulate cortex relates to accumulated experiences

Anterior cingulate cortex (ACC) activity is important for operations that require the ability to integrate multiple experiences over time, such as rule learning, cognitive flexibility, working memory, and long-term memory recall. To shed light on this, we analyzed neuronal activity while rats repeated the same behaviors during hour-long sessions to investigate how activity changed over time. We recorded neuronal ensembles as rats performed a decision-free operant task with varying reward likelihoods at three different response ports (n= 5). Neuronal state space analysis revealed that each repetition of a behavior was distinct, with more recent behaviors more similar than those further apart in time. ACC activity was dominated by a slow, gradual change in low-dimensional representations of neural state space aligning with the pace of behavior. Temporal progression, or drift, was apparent on the top principal component for every session and was driven by the accumulation of experiences and not an internal clock. Notably, these signals were consistent across subjects, allowing us to accurately predict trial numbers based on a model trained on data from a different animal. We observed that non-continuous ramping firing rates over extended durations (tens of minutes) drove the low-dimensional ensemble representations. 40% of ACC neurons' firing ramped over a range of trial lengths and combinations of shorter duration ramping neurons created ensembles that tracked longer durations. These findings provide valuable insights into how the ACC, at an ensemble level, conveys temporal information by reflecting the accumulation of experiences over extended periods.

Chemistry and Local Environment Adaptive Representation graphs as material descriptors

Accelerating materials design and property prediction via supervised learning involves manually generating feature vectors or performing complicated atom coordinates transformation, which restricts the model to a limited set of crystal structures or makes it challenging to provide chemical insights. In this work, we devised a unique low-dimensional featurization technique known as "Chemistry and Local Environment Adaptive Representation" (CLEAR) graphs to automatically learn the properties of materials through the chemistry of the atom connections. CLEAR is an adaptive featurization method that integrates the chemistry of atoms with their atomic environment using Voronoi nearest neighbours (NN). In addition, integrating the CLEAR descriptors with explainable machine learning models unravels its potential to obtain numerous scientific insights. We applied this approach to study the stability of compositional and configurational diverse high entropy alloys (HEAs). The proposed framework provides a universal low-dimensional representation with chemistry-informed local environment descriptors, which outperforms the prediction of phases and formation energies in HEAs.

A deep-learning model for the density profiles of subhaloes in IllustrisTNG

ABSTRACT We present a machine-learning-based model for the total density profiles of subhaloes with masses $M \gtrsim 7\times 10^8\, h^{-1}{\rm M}_\odot$ in the IllustrisTNG100 simulation. The model is based on an interpretable variational encoder (IVE) which returns the independent factors of variation in the density profiles within a low-dimensional representation, as well as the predictions for the density profiles themselves. The IVE returns accurate and unbiased predictions on all radial ranges, including the outer region profile where the subhaloes experience tidal stripping; here its fit accuracy exceeds that of the commonly used Einasto profile. The IVE discovers three independent degrees of freedom in the profiles, which can be interpreted in terms of the formation history of the subhaloes. In addition to the two parameters controlling the normalization and inner shape of the profile, the IVE discovers a third parameter that accounts for the impact of tidal stripping on to the subhalo outer profile; this parameter is sensitive to the mass loss experienced by the subhalo after its infall on to its parent halo. Baryonic physics in the IllustrisTNG galaxy formation model does not impact the number of degrees of freedom identified in the profile compared to the pure dark matter expectations, nor their physical interpretation. Our newly proposed profile fit can be used in strong lensing analyses or other observational studies which aim to constrain cosmology from small-scale structures.

Identifying low-dimensional trajectories of mechanically-ventilated patient systems: Empirical phenotypes of joint patient+care processes to enhance temporal analysis in ARDS research.

Refined management of mechanically ventilation is an obvious target for improving patient outcomes, but is impeded by the nature of data for study and hypothesis generation. The connections between clinical outcomes and temporal development of iatrogenic injuries current lung-protective ventilator settings remain poorly understood. Analysis of lung-ventilator system (LVS) evolution at relevant timescales is frustrated by data volume and multiple sources of heterogeneity. This work motivates, presents, and validates a computational pipeline for resolving LVS systems into the joint evolution of data-conditioned model parameters and ventilator information. Applied to individuals, the workflow yields a concise low-dimensional representation of LVS behavior expressed in phenotypic breath waveforms suitable for analysis. The effectiveness of this approach is demonstrated through application to multi-day observational series of 35 patients. Individual patient analyses reveal multiple types of patient-oriented dynamics and breath behavior to expose the complexity of LVS evolution; less than 10% of phenotype changes related to ventilator settings changes. Dynamics are shown to including both stable and unstable phenotype transitions as well as both discrete and continuous changes unrelated to ventilator settings. At a cohort scale, 721 phenotypes constructed from individual data are condensed into a set of 16 groups that empirically organize around certain settings (positive end-expository pressure and ventilator mode) and structurally similar pressure-volume loop characterizations. Individual and cohort scale phenotypes, which may be refined by hypothesis-specific constructions, provide a common framework for ongoing temporal analysis and investigation of LVS dynamics.

Liam tackles complex multimodal single-cell data integration challenges.

Multi-omics characterization of single cells holds outstanding potential for profiling the dynamics and relations of gene regulatory states of thousands of cells. How to integrate multimodal data is an open problem, especially when aiming to combine data from multiple sources or conditions containing both biological and technical variation. We introduce liam, a flexible model for the simultaneous horizontal and vertical integration of paired single-cell multimodal data and mosaic integration of paired with unimodal data. Liam learns a joint low-dimensional representation of the measured modalities, which proves beneficial when the information content or quality of the modalities differ. Its integration accounts for complex batch effects using a tunable combination of conditional and adversarial training, which can be optimized using replicate information while retaining selected biological variation. We demonstrate liam's superior performance on multiple paired multimodal data types, including Multiome and CITE-seq data, and in mosaic integration scenarios. Our detailed benchmarking experiments illustrate the complexities and challenges remaining for integration and the meaningful assessment of its success.