Nonlinear Feature Representation Research Articles

Algebraic shortcuts for leave-one-out cross-validation in supervised network inference.

Supervised machine learning techniques have traditionally been very successful at reconstructing biological networks, such as protein-ligand interaction, protein-protein interaction and gene regulatory networks. Many supervised techniques for network prediction use linear models on a possibly nonlinear pairwise feature representation of edges. Recently, much emphasis has been placed on the correct evaluation of such supervised models. It is vital to distinguish between using a model to either predict new interactions in a given network or to predict interactions for a new vertex not present in the original network. This distinction matters because (i) the performance might dramatically differ between the prediction settings and (ii) tuning the model hyperparameters to obtain the best possible model depends on the setting of interest. Specific cross-validation schemes need to be used to assess the performance in such different prediction settings. In this work we discuss a state-of-the-art kernel-based network inference technique called two-step kernel ridge regression. We show that this regression model can be trained efficiently, with a time complexity scaling with the number of vertices rather than the number of edges. Furthermore, this framework leads to a series of cross-validation shortcuts that allow one to rapidly estimate the model performance for any relevant network prediction setting. This allows computational biologists to fully assess the capabilities of their models. The machine learning techniques with the algebraic shortcuts are implemented in the RLScore software package: https://github.com/aatapa/RLScore.

VASC: Dimension Reduction and Visualization of Single-cell RNA-seq Data by Deep Variational Autoencoder

Single-cell RNA sequencing (scRNA-seq) is a powerful technique to analyze the transcriptomic heterogeneities at the single cell level. It is an important step for studying cell sub-populations and lineages, with an effective low-dimensional representation and visualization of the original scRNA-Seq data. At the single cell level, the transcriptional fluctuations are much larger than the average of a cell population, and the low amount of RNA transcripts will increase the rate of technical dropout events. Therefore, scRNA-seq data are much noisier than traditional bulk RNA-seq data. In this study, we proposed the deep variational autoencoder for scRNA-seq data (VASC), a deep multi-layer generative model, for the unsupervised dimension reduction and visualization of scRNA-seq data. VASC can explicitly model the dropout events and find the nonlinear hierarchical feature representations of the original data. Tested on over 20 datasets, VASC shows superior performances in most cases and exhibits broader dataset compatibility compared to four state-of-the-art dimension reduction and visualization methods. In addition, VASC provides better representations for very rare cell populations in the 2D visualization. As a case study, VASC successfully re-establishes the cell dynamics in pre-implantation embryos and identifies several candidate marker genes associated with early embryo development. Moreover, VASC also performs well on a 10× Genomics dataset with more cells and higher dropout rate.

Deriving enhanced geographical representations via similarity-based spectral analysis: predicting colorectal cancer survival curves in Iowa

Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use such models to explore different geographical feature representations in the context of predicting colorectal cancer survival for patients in the state of Iowa, spanning the years 1989 to 2013. Specifically, we compare model performance using area between the curves (ABC) to assess (a) whether survival can be reasonably predicted for colorectal cancer patients in the state of Iowa, (b) whether geographical features improve predictive performance, (c) whether a simple binary representation, or a richer, spectral analysis-elicited representation perform better, and (d) whether spectral analysis-based representations can be improved upon by leveraging geographically-descriptive features. In exploring (d), we devise a similarity-based spectral analysis procedure, which allows for the combination of geographically relational and geographically descriptive features. Our findings suggest that survival can be reasonably estimated on average, with predictive performance deviating at the five-year survival mark among all models. We also find that geographical features improve predictive performance, and that better performance is obtained using richer, spectral analysis-elicited features. Furthermore, we find that similarity-based spectral analysis-elicited representations improve upon the original spectral analysis results by approximately 40%.

Deriving enhanced geographical representations via similarity-based spectral analysis: predicting colorectal cancer survival curves in Iowa

Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use such models to explore different geographical feature representations in the context of predicting colorectal cancer survival curves for patients in the state of Iowa, spanning the years 1989 to 2013. Specifically, we compare model performance using area between the curves (ABC) to assess (a) whether survival curves can be reasonably predicted for colorectal cancer patients in the state of Iowa, (b) whether geographical features improve predictive performance, (c) whether a simple binary representation, or a richer, spectral analysis-elicited representation perform better, and (d) whether spectral analysis-based representations can be improved upon by leveraging geographically-descriptive features. In exploring (d), we devise a similarity-based spectral analysis procedure, which allows for the combination of geographically relational and geographically descriptive features. Our findings suggest that survival curves can be reasonably estimated on average, with predictive performance deviating at the five-year survival mark among all models. We also find that geographical features improve predictive performance, and that better performance is obtained using richer, spectral analysis-elicited features. Furthermore, we find that similarity-based spectral analysis-elicited representations improve upon the original spectral analysis results by approximately 40%.

Non-linear classifiers applied to EEG analysis for epilepsy seizure detection

Abstract This work presents a novel approach for automatic epilepsy seizure detection based on EEG analysis that exploits the underlying non-linear nature of EEG data. In this paper, two main contributions are presented and validated: the use of non-linear classifiers through the so-called kernel trick and the proposal of a Bag-of-Words model for extracting a non-linear feature representation of the input data in an unsupervised manner. The performance of the resulting system is validated with public datasets, previously processed to remove artifacts or external disturbances, but also with private datasets recorded under realistic and non-ideal operating conditions. The use of public datasets caters for comparison purposes whereas the private one shows the performance of the system under realistic circumstances of noise, artifacts, and signals of different amplitudes. Moreover, the proposed solution has been compared to state-of-the-art works not only for pre-processed and public datasets but also with the private datasets. The mean F1-measure shows a 10% improvement over the second-best ranked method including cross-dataset experiments. The obtained results prove the robustness of the proposed solution to more realistic and variable conditions.

Nonlinear and robust statistical process monitoring based on variant autoencoders

Autoencoders (AEs) are an effective means for nonlinear feature extraction and dimension reduction. Variant autoencoders are an improvement over traditional AEs in terms of robustness. This paper proposes a novel nonlinear and robust process-monitoring approach based on variant autoencoders (variant AEs), which include denoising autoencoders (DAE) and contractive autoencoders (CAE). The CAE and DAE are powerful for extracting robust and nonlinear feature representations or manifold structures underlying data from industrial processes. Next, an online monitoring model is built through constructing new test statistic H2 based on the robust feature representations. The control limits are determined by kernel density estimation. The proposed method was applied to the Tennessee Eastman process (TE process) to evaluate its monitoring performance, and it demonstrated outstanding process-monitoring performance through the experimental results, especially for the barely detectable faults, such as 3, 5, 9, 10, 11, 15, 19, 20 and 21. Variant AEs monitoring provides a simple and very effective process-monitoring method for industrial processes.

Bidirectional Nonnegative Deep Model and Its Optimization in Learning

Nonnegative matrix factorization (NMF) has been successfully applied in signal processing as a simple two-layer nonnegative neural network. Projective NMF (PNMF) with fewer parameters was proposed, which projects a high-dimensional nonnegative data onto a lower-dimensional nonnegative subspace. Although PNMF overcomes the problem of out-of-sample of NMF, it does not consider the nonlinear characteristic of data and is only a kind of narrow signal decomposition method. In this paper, we combine the PNMF with deep learning and nonlinear fitting to propose a bidirectional nonnegative deep learning (BNDL) model and its optimization learning algorithm, which can obtain nonlinear multilayer deep nonnegative feature representation. Experiments show that the proposed model can not only solve the problem of out-of-sample of NMF but also learn hierarchical nonnegative feature representations with better clustering performance than classical NMF, PNMF, and Deep Semi-NMF algorithms.

Iterative deep learning for image set based face and object recognition

We present a novel technique for image set based face/object recognition, where each gallery and query example contains a face/object image set captured from different viewpoints, background, facial expressions, resolution and illumination levels. While several image set classification approaches have been proposed in recent years, most of them represent each image set as a single linear subspace, mixture of linear subspaces or Lie group of Riemannian manifold. These techniques make prior assumptions in regards to the specific category of the geometric surface on which images of the set are believed to lie. This could result in a loss of discriminative information for classification. This paper alleviates these limitations by proposing an Iterative Deep Learning Model (IDLM) that automatically and hierarchically learns discriminative representations from raw face and object images. In the proposed approach, low level translationally invariant features are learnt by the Pooled Convolutional Layer (PCL). The latter is followed by Artificial Neural Networks (ANNs) applied iteratively in a hierarchical fashion to learn a discriminative non-linear feature representation of the input image sets. The proposed technique was extensively evaluated for the task of image set based face and object recognition on YouTube Celebrities, Honda/UCSD, CMU Mobo and ETH-80 (object) dataset, respectively. Experimental results and comparisons with state-of-the-art methods show that our technique achieves the best performance on all these datasets.

Multi-Layer and Recursive Neural Networks for Metagenomic Classification.

Recent advances in machine learning, specifically in deep learning with neural networks, has made a profound impact on fields such as natural language processing, image classification, and language modeling; however, feasibility and potential benefits of the approaches to metagenomic data analysis has been largely under-explored. Deep learning exploits many layers of learning nonlinear feature representations, typically in an unsupervised fashion, and recent results have shown outstanding generalization performance on previously unseen data. Furthermore, some deep learning methods can also represent the structure in a data set. Consequently, deep learning and neural networks may prove to be an appropriate approach for metagenomic data. To determine whether such approaches are indeed appropriate for metagenomics, we experiment with two deep learning methods: i) a deep belief network, and ii) a recursive neural network, the latter of which provides a tree representing the structure of the data. We compare these approaches to the standard multi-layer perceptron, which has been well-established in the machine learning community as a powerful prediction algorithm, though its presence is largely missing in metagenomics literature. We find that traditional neural networks can be quite powerful classifiers on metagenomic data compared to baseline methods, such as random forests. On the other hand, while the deep learning approaches did not result in improvements to the classification accuracy, they do provide the ability to learn hierarchical representations of a data set that standard classification methods do not allow. Our goal in this effort is not to determine the best algorithm in terms accuracy-as that depends on the specific application-but rather to highlight the benefits and drawbacks of each of the approach we discuss and provide insight on how they can be improved for predictive metagenomic analysis.

Discriminative quadratic feature learning for handwritten Chinese character recognition

In this paper, we propose a feature learning method for handwritten Chinese character recognition (HCCR), called discriminative quadratic feature learning (DQFL). Based on original gradient direction feature representation, quadratic correlation between features is used to promote the feature dimensionality, then discriminative feature extraction (DFE) is used for dimensionality reduction. By combining dimensionality promotion and reduction, we can learn a much more discriminative and nonlinear feature representation, which can then boost the classification accuracy significantly. For dimensionality promotion, two types of correlation are exploited, namely, statistical correlation and spatial correlation. Statistical correlation is computed on multiple local feature vectors in different regions of the character image; while spatial correlation encodes the dependency between features of two positions. Feature correlation increases the dimensionality by over 40,000. DFE then reduces the dimensionality to less than 300 without losing discriminability. Classification is performed using nearest prototype classifier (NPC), modified quadratic discriminant function (MQDF) and discriminative learning quadratic discriminant function (DLQDF). In experiments on the CASIA-HWDB1.1 standard dataset, the proposed DQFL method improves the test accuracies of NPC, MQDF and DLQDF by 4.94%, 1.83%, and 1.82%, respectively. The test accuracy is further improved by training set expansion. On the ICDAR 2013 Chinese handwriting recognition competition dataset, the proposed DQFL+DLQDF classifier outperforms the best participating system based on deep convolutional neural network (CNN), while the test speed is much faster.

Incremental two-dimensional kernel principal component analysis

In this paper, we propose a new online non-linear feature extraction method, called the incremental two-dimensional kernel principal component analysis (I2DKPCA), not only to reduce the computational cost but also to provide good feature representation. Batch type feature extraction methods such as principal component analysis (PCA) and two-dimensional PCA (2DPCA) require more computational time and memory usage, as they collect the entire training data to extract the basis vectors. Also, these linear feature extraction methods could not effectively represent the non-linear distribution of input data. Therefore, by adopting a non-linear kernel approach with chunk concept, the KPCA and 2DKPCA can effectively address the non-linear feature representation problem by adaptively changing the feature spaces. However, this kernel approach requires more computational time for processing images with high dimensional input data. In order to solve these problems, we combined the 2DKPCA with incremental learning for (1) solving the non-linear problem and (2) reducing the memory usage with computational time. In order to evaluate the performance of I2DKPCA, several experiments have been performed using well-known face and object image databases.

Saliency detection using sparse and nonlinear feature representation.

An important aspect of visual saliency detection is how features that form an input image are represented. A popular theory supports sparse feature representation, an image being represented with a basis dictionary having sparse weighting coefficient. Another method uses a nonlinear combination of image features for representation. In our work, we combine the two methods and propose a scheme that takes advantage of both sparse and nonlinear feature representation. To this end, we use independent component analysis (ICA) and covariant matrices, respectively. To compute saliency, we use a biologically plausible center surround difference (CSD) mechanism. Our sparse features are adaptive in nature; the ICA basis function are learnt at every image representation, rather than being fixed. We show that Adaptive Sparse Features when used with a CSD mechanism yield better results compared to fixed sparse representations. We also show that covariant matrices consisting of nonlinear integration of color information alone are sufficient to efficiently estimate saliency from an image. The proposed dual representation scheme is then evaluated against human eye fixation prediction, response to psychological patterns, and salient object detection on well-known datasets. We conclude that having two forms of representation compliments one another and results in better saliency detection.