Unsupervised Discovery Research Articles

New prebiotic molecules in the interstellar medium from the reaction between vinyl alcohol and CN radicals: unsupervised reaction mechanism discovery, accurate electronic structure calculations and kinetic simulations.

Vinyl alcohol (VyA) and cyanide (CN) radicals are relatively abundant in the interstellar medium (ISM). VyA is the enolic tautomer of acetaldehyde and has two low-lying conformers, characterized by the syn or anti placement of hydroxyl hydrogen with respect to the double bond. In this paper, we present a gas-phase model of the barrierless reactions of both VyA's conformers with CN employing accurate quantum chemical computations in the framework of a master equation approach based on the transition state theory. Our results indicate that both VyA conformers feature a similar reactivity with CN, starting with a barrierless addition to the double bond and followed by different isomerization, dissociation, and/or hydrogen elimination steps. The rate constants computed for temperatures up to 600 K show that several reaction channels are open even under the harsh conditions of the ISM, with the favoured one providing the first feasible formation route of a prebiotic molecule not yet detected in the ISM, namely cyanoacetaldehyde. This finding suggests looking for cyanoacetaldehyde in regions where both VyA and CN have already been detected, like, e.g., Sagittarius B2N or G+0.693-0.027.

Meta-topologies define distinct anatomical classes of brain tumours linked to histology and survival.

The current World Health Organization classification integrates histological and molecular features of brain tumours. The aim of this study was to identify generalizable topological patterns with the potential to add an anatomical dimension to the classification of brain tumours. We applied non-negative matrix factorization as an unsupervised pattern discovery strategy to the fine-grained topographic tumour profiles of 936 patients with neuroepithelial tumours and brain metastases. From the anatomical features alone, this machine learning algorithm enabled the extraction of latent topological tumour patterns, termed meta-topologies. The optimal part-based representation was automatically determined in 10 000 split-half iterations. We further characterized each meta-topology's unique histopathologic profile and survival probability, thus linking important biological and clinical information to the underlying anatomical patterns. In neuroepithelial tumours, six meta-topologies were extracted, each detailing a transpallial pattern with distinct parenchymal and ventricular compositions. We identified one infratentorial, one allopallial, three neopallial (parieto-occipital, frontal, temporal) and one unisegmental meta-topology. Each meta-topology mapped to distinct histopathologic and molecular profiles. The unisegmental meta-topology showed the strongest anatomical-clinical link demonstrating a survival advantage in histologically identical tumours. Brain metastases separated to an infra- and supratentorial meta-topology with anatomical patterns highlighting their affinity to the cortico-subcortical boundary of arterial watershed areas.Using a novel data-driven approach, we identified generalizable topological patterns in both neuroepithelial tumours and brain metastases. Differences in the histopathologic profiles and prognosis of these anatomical tumour classes provide insights into the heterogeneity of tumour biology and might add to personalized clinical decision-making.

A deep learning method to map tissue architecture.

A new study in Nature Methods describes a computational method named UTAG (unsupervised discovery of tissue architecture with graphs) that aims to identify and quantify higher-level tissue domains from biological images without previous knowledge.

Unsupervised Behavior Discovery With Quality-Diversity Optimization

Quality-diversity (QD) algorithms refer to a class of evolutionary algorithms designed to find a collection of diverse and high-performing solutions to a given problem. In robotics, such algorithms can be used for generating a collection of controllers covering most of the possible behaviors of a robot. To do so, these algorithms associate a behavioral descriptor (BD) to each of these behaviors. Each BD is used for estimating the novelty of one behavior compared to the others. In most existing algorithms, the BD needs to be hand-coded, thus requiring prior knowledge about the task to solve. In this article, we introduce: autonomous robots realizing their abilities, an algorithm that uses a dimensionality reduction technique to automatically learn BDs based on raw sensory data. The performance of this algorithm is assessed on three robotic tasks in simulation. The experimental results show that it performs similar to traditional hand-coded approaches without the requirement to provide any hand-coded BD. In the collection of diverse and high-performing solutions, it also manages to find behaviors that are novel with respect to more features than its hand-coded baselines. Finally, we introduce a variant of the algorithm which is robust to the dimensionality of the BD space.

Unsupervised discovery of nonlinear plasma physics using differentiable kinetic simulations

Plasma supports collective modes and particle–wave interactions that lead to complex behaviour in, for example, inertial fusion energy applications. While plasma can sometimes be modelled as a charged fluid, a kinetic description is often crucial for studying nonlinear effects in the higher-dimensional momentum–position phase space that describes the full complexity of the plasma dynamics. We create a differentiable solver for the three-dimensional partial-differential equation describing the plasma kinetics and introduce a domain-specific objective function. Using this framework, we perform gradient-based optimization of neural networks that provide forcing function parameters to the differentiable solver given a set of initial conditions. We apply this to an inertial-fusion-relevant configuration and find that the optimization process exploits a novel physical effect.

Unsupervised discovery of tissue architecture in multiplexed imaging.

Multiplexed imaging and spatial transcriptomics enable highly resolved spatial characterization of cellular phenotypes, but still largely depend on laborious manual annotation to understand higher-order patterns of tissue organization. As a result, higher-order patterns of tissue organization are poorly understood and not systematically connected to disease pathology or clinical outcomes. To address this gap, we developed an approach called UTAG to identify and quantify microanatomical tissue structures in multiplexed images without human intervention. Our method combines information on cellular phenotypes with the physical proximity of cells to accurately identify organ-specific microanatomical domains in healthy and diseased tissue. We apply our method to various types of images across healthy and disease states to show that it can consistently detect higher-level architectures in human tissues, quantify structural differences between healthy and diseased tissue, and reveal tissue organization patterns at the organ scale.

NN-EUCLID: Deep-learning hyperelasticity without stress data

We propose a new approach for unsupervised learning of hyperelastic constitutive laws with physics-consistent deep neural networks. In contrast to supervised learning, which assumes the availability of stress–strain pairs, the approach only uses realistically measurable full-field displacement and global reaction force data, thus it lies within the scope of our recent framework for Efficient Unsupervised Constitutive Law Identification and Discovery (EUCLID) and we denote it as NN-EUCLID. The absence of stress labels is compensated for by leveraging a physics-motivated loss function based on the conservation of linear momentum to guide the learning process. The constitutive model is based on input-convex neural networks, which are capable of learning a function that is convex with respect to its inputs. By employing a specially designed neural network architecture, multiple physical and thermodynamic constraints for hyperelastic constitutive laws, such as material frame indifference, material stability, and stress-free reference configuration are automatically satisfied. We demonstrate the ability of the approach to accurately learn several hidden isotropic and anisotropic hyperelastic constitutive laws – including e.g., Mooney–Rivlin, Arruda–Boyce, Ogden, and Holzapfel models – without using stress data. For anisotropic hyperelasticity, the unknown anisotropic fiber directions are automatically discovered jointly with the constitutive model. The neural network-based constitutive models show good generalization capability beyond the strain states observed during training and are readily deployable in a general finite element framework for simulating complex mechanical boundary value problems with good accuracy.

Few-shot learning with unsupervised part discovery and part-aligned similarity

Few-shot learning aims to recognize novel concepts with only a few examples. To this end, previous studies resort to acquiring a strong inductive bias via meta-learning on a group of similar tasks, which however needs a large labeled base dataset to sample training tasks. In this paper, we show that such inductive bias can be learned from a flat collection of unlabeled images, and instantiated as transferable representations among seen and unseen classes. Specifically, we propose a novel unsupervised Part Discovery Network (PDN) to learn transferable representations from unlabeled images, which automatically selects the most discriminative part from an input image and then maximizes its similarities to the global view of the input and other neighbors with similar semantics. To better leverage the learned representations for few-shot learning, we further propose Part-Aligned Similarity (PAS), the key of which is to measure image similarities based on a set of discriminative and aligned parts. We conduct extensive studies on five popular few-shot learning datasets to evaluate our approach. The experimental results show that our approach outperforms previous unsupervised methods by a large margin and is even comparable with state-of-the-art supervised methods.

Bayesian-EUCLID: Discovering hyperelastic material laws with uncertainties

Within the scope of our recent approach for Efficient Unsupervised Constitutive Law Identification and Discovery (EUCLID), we propose an unsupervised Bayesian learning framework for discovery of parsimonious and interpretable constitutive laws with quantifiable uncertainties. As in deterministic EUCLID, we do not resort to stress data, but only to realistically measurable full-field displacement and global reaction force data; as opposed to calibration of an a priori assumed model, we start with a constitutive model ansatz based on a large catalog of candidate functional features; we leverage domain knowledge by including features based on existing, both physics-based and phenomenological, constitutive models. In the new Bayesian-EUCLID approach, we use a hierarchical Bayesian model with sparsity-promoting priors and Monte Carlo sampling to efficiently solve the parsimonious model selection task and discover physically consistent constitutive equations in the form of multivariate multi-modal probabilistic distributions. We demonstrate and validate the ability to accurately and efficiently recover isotropic and anisotropic hyperelastic models like the Neo-Hookean, Isihara, Gent–Thomas, Arruda–Boyce, Ogden, and Holzapfel models in both elastostatics and elastodynamics. The discovered constitutive models are reliable under both epistemic uncertainties — i.e. uncertainties on the true features of the constitutive catalog – and aleatoric uncertainties – which arise from the noise in the displacement field data, and are automatically estimated by the hierarchical Bayesian model.

Wasserstein Unsupervised Reinforcement Learning

Unsupervised reinforcement learning aims to train agents to learn a handful of policies or skills in environments without external reward. These pre-trained policies can accelerate learning when endowed with external reward, and can also be used as primitive options in hierarchical reinforcement learning. Conventional approaches of unsupervised skill discovery feed a latent variable to the agent and shed its empowerment on agent’s behavior by mutual information (MI) maximization. However, the policies learned by MI-based methods cannot sufficiently explore the state space, despite they can be successfully identified from each other. Therefore we propose a new framework Wasserstein unsupervised reinforcement learning (WURL) where we directly maximize the distance of state distributions induced by different policies. Additionally, we overcome difficulties in simultaneously training N(N>2) policies, and amortizing the overall reward to each step. Experiments show policies learned by our approach outperform MI-based methods on the metric of Wasserstein distance while keeping high discriminability. Furthermore, the agents trained by WURL can sufficiently explore the state space in mazes and MuJoCo tasks and the pre-trained policies can be applied to downstream tasks by hierarchical learning.

Unsupervised discovery of non-trivial similarities between online communities

Language is used differently across communities. The differences may be manifested in vocabulary, style, and semantics. These differences enable the exploration of nuanced similarities and differences between communities. In this work, we introduce C3 — a novel unsupervised approach for community comparison. C3 creates contextual pairwise representations by aligning communities and tuning word embeddings according to both the lexical context and the social context reflected by the community’s structure and the community engagement patterns. Specifically, C3 takes into account the semantic relations between pairs of words, reflected by the embeddings model of each community, and leverages the social context and users’ role in their community to calculate a similarity measure between community pairs. C3 is evaluated over a dataset of 1565 active Reddit communities, comparing results against three competitive models. We show through an array of experiments and validations that C3 recovers nuanced and not-trivial similarities between communities that are not captured by any of the competitive models. We complement the quantitative results with a qualitative analysis, discussing recovered non-trivial similarities between community pairs such as: opiates and adhd, babyBumps and depression, wallStreetBets and sandersForPresident, all of which are recovered by C3 but not by any of the other models. This qualitative analysis demonstrates the exploratory power of our model.

Unsupervised discovery of behaviorally relevant brain states in rats playing hide-and-seek

SummaryIn classical neuroscience experiments, neural activity is measured across many identical trials of animals performing simple tasks and is then analyzed, associating neural responses to pre-defined experimental parameters. This type of analysis is not suitable for patterns of behavior that unfold freely, such as play behavior. Here, we attempt an alternative approach for exploratory data analysis on a single-trial level, applicable in more complex and naturalistic behavioral settings in which no two trials are identical. We analyze neural population activity in the prefrontal cortex (PFC) of rats playing hide-and-seek and show that it is possible to discover what aspects of the task are reflected in the recorded activity with a limited number of simultaneously recorded cells (≤ 31). Using hidden Markov models, we cluster population activity in the PFC into a set of neural states, each associated with a pattern of neural activity. Despite high variability in behavior, relating the inferred states to the events of the hide-and-seek game reveals neural states that consistently appear at the same phases of the game. Furthermore, we show that by applying the segmentation inferred from neural data to the animals’ behavior, we can explore and discover novel correlations between neural activity and behavior. Finally, we replicate the results in a second dataset and show that population activity in the PFC displays distinct sets of states during playing hide-and-seek and observing others play the game. Overall, our results reveal robust, state-like representations in the rat PFC during unrestrained playful behavior and showcase the applicability of population analyses in naturalistic neuroscience.

Unsupervised query-adaptive implicit subtopic discovery for diverse image retrieval based on intrinsic cluster quality

Unsupervised query-adaptive implicit subtopic discovery for diverse image retrieval based on intrinsic cluster quality

Discovering plasticity models without stress data

We propose an approach for data-driven automated discovery of material laws, which we call EUCLID (Efficient Unsupervised Constitutive Law Identification and Discovery), and we apply it here to the discovery of plasticity models, including arbitrarily shaped yield surfaces and isotropic and/or kinematic hardening laws. The approach is unsupervised, i.e., it requires no stress data but only full-field displacement and global force data; it delivers interpretable models, i.e., models that are embodied by parsimonious mathematical expressions discovered through sparse regression of a potentially large catalog of candidate functions; it is one-shot, i.e., discovery only needs one experiment. The material model library is constructed by expanding the yield function with a Fourier series, whereas isotropic and kinematic hardening is introduced by assuming a yield function dependency on internal history variables that evolve with the plastic deformation. For selecting the most relevant Fourier modes and identifying the hardening behavior, EUCLID employs physics knowledge, i.e., the optimization problem that governs the discovery enforces the equilibrium constraints in the bulk and at the loaded boundary of the domain. Sparsity promoting regularization is deployed to generate a set of solutions out of which a solution with low cost and high parsimony is automatically selected. Through virtual experiments, we demonstrate the ability of EUCLID to accurately discover several plastic yield surfaces and hardening mechanisms of different complexity.

Unsupervised Discovery of Non-Trivial Similarities between Online Communities

Unsupervised Discovery of Non-Trivial Similarities between Online Communities

Unsupervised discovery of dynamic cell phenotypic states from transmitted light movies.

Identification of cell phenotypic states within heterogeneous populations, along with elucidation of their switching dynamics, is a central challenge in modern biology. Conventional single-cell analysis methods typically provide only indirect, static phenotypic readouts. Transmitted light images, on the other hand, provide direct morphological readouts and can be acquired over time to provide a rich data source for dynamic cell phenotypic state identification. Here, we describe an end-to-end deep learning platform, UPSIDE (Unsupervised Phenotypic State IDEntification), for discovering cell states and their dynamics from transmitted light movies. UPSIDE uses the variational auto-encoder architecture to learn latent cell representations, which are then clustered for state identification, decoded for feature interpretation, and linked across movie frames for transition rate inference. Using UPSIDE, we identified distinct blood cell types in a heterogeneous dataset. We then analyzed movies of patient-derived acute myeloid leukemia cells, from which we identified stem-cell associated morphological states as well as the transition rates to and from these states. UPSIDE opens up the use of transmitted light movies for systematic exploration of cell state heterogeneity and dynamics in biology and medicine.

Using dynamic time warping self-organizing maps to characterize diurnal patterns in environmental exposures

Advances in measurement technology are producing increasingly time-resolved environmental exposure data. We aim to gain new insights into exposures and their potential health impacts by moving beyond simple summary statistics (e.g., means, maxima) to characterize more detailed features of high-frequency time series data. This study proposes a novel variant of the Self-Organizing Map (SOM) algorithm called Dynamic Time Warping Self-Organizing Map (DTW-SOM) for unsupervised pattern discovery in time series. This algorithm uses DTW, a similarity measure that optimally aligns interior patterns of sequential data, both as the similarity measure and training guide of the neural network. We applied DTW-SOM to a panel study monitoring indoor and outdoor residential temperature and particulate matter air pollution (PM2.5) for 10 patients with asthma from 7 households near Salt Lake City, UT; the patients were followed for up to 373 days each. Compared to previous SOM algorithms using timestamp alignment on time series data, the DTW-SOM algorithm produced fewer quantization errors and more detailed diurnal patterns. DTW-SOM identified the expected typical diurnal patterns in outdoor temperature which varied by season, as well diurnal patterns in PM2.5 which may be related to daily asthma outcomes. In summary, DTW-SOM is an innovative feature engineering method that can be applied to highly time-resolved environmental exposures assessed by sensors to identify typical diurnal (or hourly or monthly) patterns and provide new insights into the health effects of environmental exposures.

SMM: Leveraging Metadata for Contextually Salient Multi-Variate Motif Discovery

A common challenge in multimedia data understanding is the unsupervised discovery of recurring patterns, or motifs, in time series data. The discovery of motifs in uni-variate time series is a well studied problem and, while being a relatively new area of research, there are also several proposals for multi-variate motif discovery. Unfortunately, motif search among multiple variates is an expensive process, as the potential number of sub-spaces in which a pattern can occur increases exponentially with the number of variates. Consequently, many multi-variate motif search algorithms make simplifying assumptions, such as searching for motifs across all variates individually, assuming that the motifs are of the same length, or that they occur on a fixed subset of variates. In this paper, we are interested in addressing a relatively broad form of multi-variate motif detection, which seeks frequently occurring patterns (of possibly differing lengths) in sub-spaces of a multi-variate time series. In particular, we aim to leverage contextual information to help select contextually salient patterns and identify the most frequent patterns among all. Based on these goals, we first introduce the contextually salient multi-variate motif (CS-motif) discovery problem and then propose a salient multi-variate motif (SMM) algorithm that, unlike existing methods, is able to seek a broad range of patterns in multi-variate time series.

Phoneme Segmentation-Based Unsupervised Pattern Discovery and Clustering of Speech Signals

Phoneme Segmentation-Based Unsupervised Pattern Discovery and Clustering of Speech Signals

Link-PLSA-LDA: A New Unsupervised Model for Topics and Influence of Blogs

In this work, we address the twin problems of unsupervised topic discovery and estimation of topic specific influence of blogs. We propose a new model that can be used to provide a user with highly influential blog postings on the topic of the user's interest. We adopt the framework of an unsupervised model called Latent Dirichlet Allocation, known for its effectiveness in topic discovery. An extension of this model, which we call Link-LDA, defines a generative model for hyperlinks and thereby models topic specific influence of documents, the problem of our interest. However, this model does not exploit the topical relationship between the documents on either side of a hyperlink, i.e., the notion that documents tend to link to other documents on the same topic. We propose a new model, called Link-PLSA-LDA, that combines PLSA and LDA into a single framework, and explicitly models the topical relationship between the linking and the linked document. The output of the new model on blog data reveals very interesting visualizations of topics and influential blogs on each topic. We also perform quantitative evaluation of the model using log-likelihood of unseen data and on the task of link prediction. Both experiments show that that the new model performs better, suggesting its superiority over Link-LDA in modeling topics and topic specific influence of blogs.