Unsupervised Construction Research Articles

Unsupervised construction of health indicator for rotating machinery via multi-criterion feature selection and attentive variational autoencoder

Unsupervised construction of health indicator for rotating machinery via multi-criterion feature selection and attentive variational autoencoder

Remaining Useful Life Prediction by Distribution Contact Ratio Health Indicator and Consolidated Memory GRU

Facing the gap in the unsupervised construction of health indicator (HI) with a uniform failure threshold, a new unsupervised HI construction approach is developed. First, the distribution of the raw vibration signal is estimated by the Gaussian mixture model (GMM), then a distribution contact ratio metric (DCRM) is designed to compute the distance between two arbitrary distributions. With DCRM, a distribution contact ratio metric health indicator (DCRHI) is innovatively constructed for well representing the degradation process and obtaining a uniform failure threshold. Next, aiming at the challenge of prediction under limited samples, a novel consolidated memory gated recurrent unit (CMGRU) is proposed by making full use of the historical state information, and it can effectively slow down the forgetting speed of important trend information. Combing the proposed DCRHI and CMGRU, a novel remaining useful life (RUL) prediction methodology is put forward for enhancing the predictive performance. Via two public bearing datasets, several contrast experiments are implemented, and the comparative results show that DCRHI can better describe the degradation process of bearing than other typical unsupervised HIs, and CMGRU has a stronger prediction ability than other classical time series processing networks. Thus, the proposed methodology has great application value in the RUL prediction.

Unsupervised construction of gene regulatory network based on single-cell multi-omics data of colorectal cancer.

Identifying gene regulatory networks (GRNs) at the resolution of single cells has long been a great challenge, and the advent of single-cell multi-omics data provides unprecedented opportunities to construct GRNs. Here, we propose a novel strategy to integrate omics datasets of single-cell ribonucleic acid sequencing and single-cell Assay for Transposase-Accessible Chromatin using sequencing, and using an unsupervised learning neural network to divide the samples with high copy number variation scores, which are used to infer the GRN in each gene block. Accuracy validation of proposed strategy shows that approximately 80% of transcription factors are directly associated with cancer, colorectal cancer, malignancy and disease by TRRUST; and most transcription factors are prone to produce multiple transcript variants and lead to tumorigenesis by RegNetwork database, respectively. The source code access are available at: https://github.com/Cuily-v/Colorectal_cancer.

Unsupervised construction of computational graphs for gene expression data with explicit structural inductive biases.

MotivationGene expression data are commonly used at the intersection of cancer research and machine learning for better understanding of the molecular status of tumour tissue. Deep learning predictive models have been employed for gene expression data due to their ability to scale and remove the need for manual feature engineering. However, gene expression data are often very high dimensional, noisy and presented with a low number of samples. This poses significant problems for learning algorithms: models often overfit, learn noise and struggle to capture biologically relevant information. In this article, we utilize external biological knowledge embedded within structures of gene interaction graphs such as protein–protein interaction (PPI) networks to guide the construction of predictive models.ResultsWe present Gene Interaction Network Constrained Construction (GINCCo), an unsupervised method for automated construction of computational graph models for gene expression data that are structurally constrained by prior knowledge of gene interaction networks. We employ this methodology in a case study on incorporating a PPI network in cancer phenotype prediction tasks. Our computational graphs are structurally constructed using topological clustering algorithms on the PPI networks which incorporate inductive biases stemming from network biology research on protein complex discovery. Each of the entities in the GINCCo computational graph represents biological entities such as genes, candidate protein complexes and phenotypes instead of arbitrary hidden nodes of a neural network. This provides a biologically relevant mechanism for model regularization yielding strong predictive performance while drastically reducing the number of model parameters and enabling guided post-hoc enrichment analyses of influential gene sets with respect to target phenotypes. Our experiments analysing a variety of cancer phenotypes show that GINCCo often outperforms support vector machine, Fully Connected Multi-layer Perceptrons (MLP) and Randomly Connected MLPs despite greatly reduced model complexity.Availability and implementation https://github.com/paulmorio/gincco contains the source code for our approach. We also release a library with algorithms for protein complex discovery within PPI networks at https://github.com/paulmorio/protclus. This repository contains implementations of the clustering algorithms used in this article.Supplementary information Supplementary data are available at Bioinformatics online.

Targeted Adversarial Learning Optimized Sampling.

Boosting transitions of rare events is critical to simulations of chemical and biophysical dynamic systems in order to close the time scale gaps between theoretical modeling and experiments. We present a novel approach, called targeted adversarial learning optimized sampling (TALOS), to modify the potential energy surface in order to drive the system to a user-defined target distribution where the free-energy barrier is lowered. Combining statistical mechanics and generative learning, TALOS formulates a competing game between a sampling engine and a virtual discriminator, enables unsupervised construction of bias potentials, and seeks for an optimal transport plan that transforms the system into a target. Through multiple experiments, we show that on-the-fly training of TALOS benefits from the state-of-art optimization techniques in deep learning and thus is efficient, robust, and interpretable. TALOS is also closely connected to the actor-critic reinforcement learning and hence leads to a new way of flexibly manipulating the many-body Hamiltonian systems.

Dendritic Spines Taxonomy: The Functional and Structural Classification • Time-Dependent Probabilistic Model of Neuronal Activation.

Categorizing spines into four subpopulations, stubby, mushroom, thin, or filopodia, is one of the common approaches in morphological analysis. Most cellular models describing synaptic plasticity, long-term potentiation (LTP), and long-term depression associate synaptic strength with either spine enlargement or spine shrinkage. Unfortunately, although we have a lot of available software with automatic spine segmentation and feature extraction methods, at present none of them allows for automatic and unbiased distinction between dendritic spine subpopulations, or for the detailed computational models of spine behavior. Therefore, we propose structural classification based on two different mathematical approaches: unsupervised construction of spine shape taxonomy based on arbitrary features (SpineTool) and supervised classification exploiting convolution kernels theory (2dSpAn). We compared two populations of spines in a form of static and dynamic data sets gathered at three time points. The dynamic data contain two sets of spines: the active set and the control set. The first population was stimulated with LTP, and the other population in its resting state was used as a control population. We propose one equation describing the distribution of variables that best fits all dendritic spine parameters.

Natural-Annotation-based Unsupervised Construction of Korean-Chinese Domain Dictionary

The large-scale bilingual parallel resource is significant to statistical learning and deep learning in natural language processing. This paper addresses the automatic construction issue of the Korean-Chinese domain dictionary, and presents a novel unsupervised construction method based on the natural annotation in the raw corpus. We firstly extract all Korean-Chinese word pairs from Korean texts according to natural annotations, secondly transform the traditional Chinese characters into the simplified ones, and finally distill out a bilingual domain dictionary after retrieving the simplified Chinese words in an extra Chinese domain dictionary. The experimental results show that our method can automatically build multiple Korean-Chinese domain dictionaries efficiently.

Unsupervised construction of human body models

Unsupervised learning of a generalizable model of the visual appearance of humans from video data is of major importance for computing systems interacting naturally with their users and others. We propose a step towards automatic behavior understanding by making the posture estimation cycle more autonomous. The system extracts coherent motion from moving upper bodies and autonomously decides about limbs and their possible spatial relationships. The models from many videos are integrated into a meta-model, which shows good generalization with respect to different individuals, backgrounds, and attire. This model allows robust interpretation of single video frames without temporal continuity and posture mimicking by an android robot.

Computational Approach to Dendritic Spine Taxonomy and Shape Transition Analysis.

The common approach in morphological analysis of dendritic spines of mammalian neuronal cells is to categorize spines into subpopulations based on whether they are stubby, mushroom, thin, or filopodia shaped. The corresponding cellular models of synaptic plasticity, long-term potentiation, and long-term depression associate the synaptic strength with either spine enlargement or spine shrinkage. Although a variety of automatic spine segmentation and feature extraction methods were developed recently, no approaches allowing for an automatic and unbiased distinction between dendritic spine subpopulations and detailed computational models of spine behavior exist. We propose an automatic and statistically based method for the unsupervised construction of spine shape taxonomy based on arbitrary features. The taxonomy is then utilized in the newly introduced computational model of behavior, which relies on transitions between shapes. Models of different populations are compared using supplied bootstrap-based statistical tests. We compared two populations of spines at two time points. The first population was stimulated with long-term potentiation, and the other in the resting state was used as a control. The comparison of shape transition characteristics allowed us to identify the differences between population behaviors. Although some extreme changes were observed in the stimulated population, statistically significant differences were found only when whole models were compared. The source code of our software is freely available for non-commercial use1. Contact: d.plewczynski@cent.uw.edu.pl.

Manifold Spanning Graphs

Graph construction is the essential first step for nearly all manifold learning algorithms. While many applications assume that a simple k-nearest or epsilon-close neighbors graph will accurately model the topology of the underlying manifold, these methods often require expert tuning and may not produce high quality graphs. In this paper, the hyperparameter sensitivity of existing graph construction methods is demonstrated. We then present a new algorithm for unsupervised graph construction, based on minimal assumptions about the input data and its manifold structure.

Regression based D-optimality experimental design for sparse kernel density estimation

This paper derives an efficient algorithm for constructing sparse kernel density (SKD) estimates. The algorithm first selects a very small subset of significant kernels using an orthogonal forward regression (OFR) procedure based on the D-optimality experimental design criterion. The weights of the resulting sparse kernel model are then calculated using a modified multiplicative nonnegative quadratic programming algorithm. Unlike most of the SKD estimators, the proposed D-optimality regression approach is an unsupervised construction algorithm and it does not require an empirical desired response for the kernel selection task. The strength of the D-optimality OFR is owing to the fact that the algorithm automatically selects a small subset of the most significant kernels related to the largest eigenvalues of the kernel design matrix, which counts for the most energy of the kernel training data, and this also guarantees the most accurate kernel weight estimate. The proposed method is also computationally attractive, in comparison with many existing SKD construction algorithms. Extensive numerical investigation demonstrates the ability of this regression-based approach to efficiently construct a very sparse kernel density estimate with excellent test accuracy, and our results show that the proposed method compares favourably with other existing sparse methods, in terms of test accuracy, model sparsity and complexity, for constructing kernel density estimates.

UNSUPERVISED LEARNING OF A HIERARCHY OF TOPOLOGICAL MAPS USING OMNIDIRECTIONAL IMAGES

This paper presents a novel appearance-based method for path-based map learning by a mobile robot equipped with an omnidirectional camera. In particular, we focus on an unsupervised construction of topological maps, which provide an abstraction of the environment in terms of visual aspects. An unsupervised clustering algorithm is used to represent the images in multiple subspaces, forming thus a sensory grounded representation of the environment's appearance. By introducing transitional fields between clusters we are able to obtain a partitioning of the image set into distinctive visual aspects. By abstracting the low-level sensory data we are able to efficiently reconstruct the overall topological layout of the covered path. After the high level topology is estimated, we repeat the procedure on the level of visual aspects to obtain local topological maps. We demonstrate how the resulting representation can be used for modeling indoor and outdoor environments, how it successfully detects previously visited locations and how it can be used for the estimation of the current visual aspect and the retrieval of the relative position within the current visual aspect.

Automated wavelength selection for spectroscopic fuel models by symmetrically contracting repeated unmoving window partial least squares

The need for automated quality surveillance of liquid hydrocarbon fuels has driven the development of rapid fuel property modeling from spectroscopic sensor data. The correlation of near-infrared (NIR) and Raman spectroscopic data with jet and diesel fuel properties can be improved by the deliberate selection of continuous wavelength sub-ranges. An automatic wavelength selection strategy would allow for the unsupervised construction of partial least squares (PLS) regression models of increased predictive utility when supervised model construction and maintenance is not feasible. Changeable size moving window partial least squares (CSMWPLS) is one of the most thorough operations suited for this task. Unfortunately, the necessarily large number of PLS model constructions required by an automated version of this procedure limits the evaluation of the predictive ability of the resulting models through full cross-validation results. Presented here is a novel restricted version of the CSMWPLS algorithm in which the initial spectral range selection is accomplished through multiple interval PLS ( iPLS) analyses, where analysis windows for the refinement step no longer move, and size changes are limited to a series of symmetric attenuations. It is shown that the proposed algorithm can provide significant PLS model improvements during the course of a fully automated analysis of jet and diesel fuel spectra in less time than an automated CSMWPLS algorithm.

Unsupervised construction of fuzzy measures through self-organizing feature maps and its application in color image segmentation

The paper presents a framework for the segmentation of multi-dimensional images, e.g., color, satellite, multi-sensory images, based on the employment of the fuzzy integral, which undertakes the classification of the input features. The framework makes use of a self-organizing feature map, whereby the coefficients of the fuzzy measure are determined. This process is unsupervised and therefore constitutes one of the main contributions of the paper. The performance of the framework is shown by successfully realizing the segmentation of color images in two different applications. First, the features of the framework and its parameterization are analyzed by segmenting different images used as benchmark in image processing. Finally, the framework is applied in the segmentation of different images taken under difficult illumination conditions. The images serve the development of an automated cashier system, where the weak segmentation constitutes the first step for the identification of different market items. The presented framework succeeds in the segmentation of all these color images.

Integrating high-throughput characterization into combinatorial heterogeneous catalysis: unsupervised construction of quantitative structure/property relationship models

This paper presents a novel approach in the framework of heterogeneous combinatorial catalysis, which integrates into the global discovery strategy the use of inexpensive high-throughput characterization of libraries of catalysts, as multivariate spectral descriptors for catalytic quantitative structure/property relationship (QSPR) modeling. Moreover, QSPR models can be used to assist the design of new libraries and for extraction of rules and relationships, yielding knowledge about catalysis. This approach can be of special interest when experimental evaluation of catalytic behavior is very expensive or time-consuming, as, for instance, for catalyst deactivation studies, for testing under very severe conditions, or when high amounts of catalyst are demanded. This methodology has been applied to modeling of the behavior of epoxidation catalysts, with the composition vector of the starting synthesis gel and XRD spectra as descriptors. Dimensional reduction was conducted by principal components analysis, clustering, and Kohonen networks, and predictive models were obtained with the use of logistic equations, artificial neural networks, and decision tree techniques. The use of spectral descriptors made it possible to markedly improve the prediction performance obtained with synthesis descriptors alone.

Semisupervised learning of hierarchical latent trait models for data visualization

Recently, we have developed the hierarchical generative topographic mapping (HGTM), an interactive method for visualization of large high-dimensional real-valued data sets. We propose a more general visualization system by extending HGTM in three ways, which allows the user to visualize a wider range of data sets and better support the model development process. 1) We integrate HGTM with noise models from the exponential family of distributions. The basic building block is the latent trait model (LTM). This enables us to visualize data of inherently discrete nature, e.g., collections of documents, in a hierarchical manner. 2) We give the user a choice of initializing the child plots of the current plot in either interactive, or automatic mode. In the interactive mode, the user selects "regions of interest", whereas in the automatic mode, an unsupervised minimum message length (MML)-inspired construction of a mixture of LTMs is employed. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. 3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualization plots, since they can highlight the boundaries between data clusters. We illustrate our approach on a toy example and evaluate it on three more complex real data sets.