Gaussian Process Latent Variable Model Research Articles

A Versatile Parameterization for Measured Material Manifolds

AbstractA popular approach for computing photorealistic images of virtual objects requires applying reflectance profiles measured from real surfaces, introducing several challenges: the memory needed to faithfully capture realistic material reflectance is large, the choice of materials is limited to the set of measurements, and image synthesis using the measured data is costly. Typically, this data is either compressed by projecting it onto a subset of its linear principal components or by applying non‐linear methods. The former requires many components to faithfully represent the input reflectance, whereas the latter necessitates costly extrapolation algorithms. We learn an underlying, low‐dimensional non‐linear reflectance manifold amenable to rapid exploration and rendering of real‐world materials. We can express interpolated materials as linear combinations of the measured data, despite them lying on an inherently non‐linear manifold. This allows us to efficiently interpolate and extrapolate measured BRDFs, and to render directly from the manifold representation. We exploit properties of Gaussian process latent variable models and use our representation for high‐performance and offline rendering with interpolated real‐world materials.

A New Application Based on GPLVM, LMNN, and NCA for Early Detection of the Stomach Cancer

ABSTRACTIn this article, speeded-up robust features (SURF) for each image have been calculated. Discrete Fourier transform (DFT) method has been applied to these SURF. High dimensions of these SURF–DFT feature vectors are reduced to low dimensions with large-margin nearest neighbor (LMNN), Gaussian process latent variable models (GPLVM), and neighborhood component analysis (NCA). When size reduction process was done, effect on the GPLVM, LMNN, and NCA of the 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 feature numbers has been examined. These features are classified by naive Bayes (NB) classifier. Thus, SURF_DFT_GPLVM_NB, SURF_DFT_NCA_NB, and SURF_DFT_LMNN_NB methods for gastric histopathological images have been developed. Classification results obtained with these methods have been compared. According to the obtained results, the highest classification result was obtained as 90.24% by using 4 features by SURF_DFT_GPLVM_NB method for second group images.

Humanoids skill learning based on real-time human motion imitation using Kinect

In this paper, a novel framework which enables humanoid robots to learn new skills from demonstration is proposed. The proposed framework makes use of real-time human motion imitation module as a demonstration interface for providing the desired motion to the learning module in an efficient and user-friendly way. This interface overcomes many problems of the currently used interfaces like direct motion recording, kinesthetic teaching, and immersive teleoperation. This method gives the human demonstrator the ability to control almost all body parts of the humanoid robot in real time (including hand shape and orientation which are essential to perform object grasping). The humanoid robot is controlled remotely and without using any sophisticated haptic devices, where it depends only on an inexpensive Kinect sensor and two additional force sensors. To the best of our knowledge, this is the first time for Kinect sensor to be used in estimating hand shape and orientation for object grasping within the field of real-time human motion imitation. Then, the observed motions are projected onto a latent space using Gaussian process latent variable model to extract the relevant features. These relevant features are then used to train regression models through the variational heteroscedastic Gaussian process regression algorithm which is proved to be a very accurate and very fast regression algorithm. Our proposed framework is validated using different activities concerned with both human upper and lower body parts and object grasping also.

Human Action Recognition Using Improved Sparse Gaussian Process Latent Variable Model and Hidden Conditional Random Filed

Owing to the fragile to external environment, current human activity recognition is still challenging in the area of computer vision. In particular, the recognition of subtle actions is a difficult task to distinguish from each other. In this paper, we proposed the methods for human action recognition based on improved sparse Gaussian process latent variable model (GPLVM) and hidden conditional random field (HCRF). We first fused the skeletal information and the motion characteristics of human body to extract human action features from a set of action sequences. And then, we proposed an improved sparse GPLVM algorithm for feature dimensionality reduction to satisfy the visualization and computation complexity for human actions recognition. Through sparse approximation with information vector machine and the dynamic process of human actions, the improved sparse GPLVM allows the action features to be represented by the low dimensional trajectory in manifold space. Furthermore, HCRF was applied to recognize the characteristics of human actions derived from manifold spaces and finally to classify human actions. The proposed methods are tested and evaluated on public action database. The experimental results show that our methods can achieve better performance of feature dimensionality reduction and visualization and obtain the average recognition accuracy of 93.68%.

Discriminative geodesic Gaussian process latent variable model for structure preserving dimension reduction in clustering and classification problems

Dimension reduction is a common approach for analyzing complex high-dimensional data and allows efficient implementation of classification and decision algorithms. Gaussian process latent variable model (GPLVM) is a widely applicable dimension reduction method which represents latent space without considering the class labels. Preserving the structure and topology of data are key factors that influence the performance of dimensionality reduction models. A conventional measure which reflects the topological structure of data points is geodesic distance. In this study, we propose an enriched GPLVM mapping between low-dimensional space and high-dimensional data. One of the contributions of the proposed approach is to calculate geodesic distance under the influence of class labels and introducing an improved GPLVM kernel using the distance. Also, the objective function of the model is reformulated to consider the trade-off between class separation and structure preservation which improves discrimination power and compactness of data. The efficiency of the proposed approach is compared with other dimension reduction techniques such as the kernel principal component analysis (KPCA), locally linear embedding (LLE), Laplacian eigenmaps and also discriminative and supervised extensions of standard GPLVM. Based on the experiments, it is suggested that the proposed model has a higher capacity for accurate classification and clustering of data as compared with the mentioned approaches.

Shared Autoencoder Gaussian Process Latent Variable Model for Visual Classification.

Multiview learning reveals the latent correlation among different modalities and utilizes the complementary information to achieve a better performance in many applications. In this paper, we propose a novel multiview learning model based on the Gaussian process latent variable model (GPLVM) to learn a set of nonlinear and nonparametric mapping functions and obtain a shared latent variable in the manifold space. Different from the previous work on the GPLVM, the proposed shared autoencoder Gaussian process (SAGP) latent variable model assumes that there is an additional mapping from the observed data to the shared manifold space. Due to the introduction of the autoencoder framework, both nonlinear projections from and to the observation are considered simultaneously. Additionally, instead of fully connecting used in the conventional autoencoder, the SAGP achieves the mappings utilizing the GP, which remarkably reduces the number of estimated parameters and avoids the phenomenon of overfitting. To make the proposed method adaptive for classification, a discriminative regularization is embedded into the proposed method. In the optimization process, an efficient algorithm based on the alternating direction method and gradient decent techniques is designed to solve the encoder and decoder parts alternatively. Experimental results on three real-world data sets substantiate the effectiveness and superiority of the proposed approach as compared with the state of the art.

Supervised Gaussian Process Latent Variable Model for Hyperspectral Image Classification

Discriminative features are significant for hyper-spectral image (HSI) classification. In this letter, we apply the supervised dimensionality reduction (DR) model termed supervised latent linear Gaussian process latent variable model (SLLGPLVM) for feature extraction. As a semiparametric classification model, the new model has ability in simultaneous feature extraction and classification and demonstrates high classification accuracy with only a small training set. This is therefore suitable for HSI classification. Experimental results on six real HSI data sets show that the proposed SLLGPLVM outperforms several conventional supervised DR models and the support vector machine implemented in the original spectral space.

Data driven flexible backbone protein design.

Protein design remains an important problem in computational structural biology. Current computational protein design methods largely use physics-based methods, which make use of information from a single protein structure. This is despite the fact that multiple structures of many protein folds are now readily available in the PDB. While ensemble protein design methods can use multiple protein structures, they treat each structure independently. Here, we introduce a flexible backbone strategy, FlexiBaL-GP, which learns global protein backbone movements directly from multiple protein structures. FlexiBaL-GP uses the machine learning method of Gaussian Process Latent Variable Models to learn a lower dimensional representation of the protein coordinates that best represent backbone movements. These learned backbone movements are used to explore alternative protein backbones, while engineering a protein within a parallel tempered MCMC framework. Using the human ubiquitin–USP21 complex as a model we demonstrate that our design strategy outperforms current strategies for the interface design task of identifying tight binding ubiquitin variants for USP21.

An Automated Classification of Pathological Gait Using Unobtrusive Sensing Technology.

This paper integrates an unobtrusive and affordable sensing technology with machine learning methods to discriminate between healthy and pathological gait patterns as a result of stroke or acquired brain injury. A feature analysis is used to identify the role of each body part in separating pathological patterns from healthy patterns. Gait features, including the orientations of the hips and spine (trunk), shoulders and neck (upper limb), knees and ankles (lower limb), are calculated during walking based on Kinect skeletal tracking sequences. Sequences of these features during three types of walking conditions were examined: 1) walking at self-pace (WSP); 2) walking at distracted (WD); and 3) walking at fast pace (WFP). Two machine learning approaches, an instance-based discriminative classifier ( -nearest neighbor) and a dynamical generative classifier (using Gaussian Process Latent Variable Model), are examined to distinguish between healthy and pathological gaits. Nested cross validation is implemented to evaluate the performance of the two classifiers using three metrics: F1-score, macro-averaged error, and micro-averaged error. The discriminative model outperforms the generative model in terms of the F1-score (discriminative: WSP > 0.95, WD > 0.96, and WFP > 0.95 and generative: WSP > 0.87, WD > 0.85, and WFP > 0.68) and macro-averaged error (discriminative: WSP < 0.08, WD < 0.1, and WFP < 0.09 and generative: WSP < 0.11, WD < 0.12, and WFP < 0.14). The dynamical generative model on the other hand obtains better micro-averaged error (discriminative: WSP < 0.37, WD < 0.3, and WFP < 0.35 and generative: WSP < 0.15, WD < 0.2, and WFP < 0.2). The high-dimensional gait features are divided into five subsets: lower limb, upper limb, trunk, velocity, and acceleration. An instance-based feature analysis method (ReliefF) is used to assign weights to each subset of features according to its discriminatory power. The feature analysis establishes the most informative features (upper limb, lower limb, and trunk) for identifying pathological gait.

A Multi-view Facial Expression Recognition Method Based on Discriminative Shared Gaussian Process Latent Variable Model

The traditional multi-view facial expression recognition method is adopted view-specific classifier to recognition view-specific sample. This approach ignores the fact that different views of a facial expression are just different manifestations of the same facial expression. To address this, a multi-view facial expression recognition method based on Discriminative Shared Gaussian Process Latent Variable Model is proposed. Firstly this method extraction Incremental Update Parallel Cascade of Linear Regression feature, then its uses PCA to select the feature, finally adopts Discriminative Shared Gaussian Process Latent Variable Model to recognition Multi-view facial expression. The experiment carried on CMU-PIE database and LFPW database show the effectiveness of our method.

Video-Based Human Walking Estimation Using Joint Gait and Pose Manifolds

We study two fundamental issues about video-based human walking estimation, where the goal is to estimate 3D gait kinematics (i.e., joint positions) from 2D gait appearances (i.e., silhouettes). One is how to model the gait kinematics from different walking styles, and the other is how to represent the gait appearances captured under different views and from individuals of distinct walking styles and body shapes. Our research is conducted in three steps. First, we propose the idea of joint gait-pose manifold (JGPM), which represents gait kinematics by coupling two nonlinear variables, pose (a specific walking stage) and gait (a particular walking style) in a unified latent space. We extend the Gaussian process latent variable model (GPLVM) for JGPM learning, where two heuristic topological priors, a torus and a cylinder, are considered and several JGPMs of different degrees of freedom (DoFs) are introduced for comparative analysis. Second, we develop a validation technique and a series of benchmark tests to evaluate multiple JGPMs and recent GPLVMs in terms of their performance for gait motion modeling. It is shown that the toroidal prior is slightly better than the cylindrical one, and the JGPM of 4 DoFs that balances the toroidal prior with the intrinsic data structure achieves the best performance. Third, a JGPM-based visual gait generative model (JGPM-VGGM) is developed, where JGPM plays a central role to bridge the gap between the gait appearances and the gait kinematics. Our proposed JGPM-VGGM is learned from Carnegie Mellon University MoCap data and tested on the HumanEva-I and HumanEva-II data sets. Our experimental results demonstrate the effectiveness and competitiveness of our algorithms compared with existing algorithms.

Autonomous exploration of mobile robots through deep neural networks

The exploration problem of mobile robots aims to allow mobile robots to explore an unknown environment. We describe an indoor exploration algorithm for mobile robots using a hierarchical structure that fuses several convolutional neural network layers with decision-making process. The whole system is trained end to end by taking only visual information (RGB-D information) as input and generates a sequence of main moving direction as output so that the robot achieves autonomous exploration ability. The robot is a TurtleBot with a Kinect mounted on it. The model is trained and tested in a real world environment. And the training data set is provided for download. The outputs of the test data are compared with the human decision. We use Gaussian process latent variable model to visualize the feature map of last convolutional layer, which proves the effectiveness of this deep convolution neural network mode. We also present a novel and lightweight deep-learning library libcnn especially for deep-learning processing of robotics tasks.

Multimodal Similarity Gaussian Process Latent Variable Model.

Data from real applications involve multiple modalities representing content with the same semantics from complementary aspects. However, relations among heterogeneous modalities are simply treated as observation-to-fit by existing work, and the parameterized modality specific mapping functions lack flexibility in directly adapting to the content divergence and semantic complicacy in multimodal data. In this paper, we build our work based on the Gaussian process latent variable model (GPLVM) to learn the non-parametric mapping functions and transform heterogeneous modalities into a shared latent space. We propose multimodal Similarity Gaussian Process latent variable model (m-SimGP), which learns the mapping functions between the intra-modal similarities and latent representation. We further propose multimodal distance-preserved similarity GPLVM (m-DSimGP) to preserve the intra-modal global similarity structure, and multimodal regularized similarity GPLVM (m-RSimGP) by encouraging similar/dissimilar points to be similar/dissimilar in the latent space. We propose m-DRSimGP, which combines the distance preservation in m-DSimGP and semantic preservation in m-RSimGP to learn the latent representation. The overall objective functions of the four models are solved by simple and scalable gradient decent techniques. They can be applied to various tasks to discover the nonlinear correlations and to obtain the comparable low-dimensional representation for heterogeneous modalities. On five widely used real-world data sets, our approaches outperform existing models on cross-modal content retrieval and multimodal classification.

3D Reconstruction Supported by Gaussian Process Latent Variable Model Shape Priors

This article presents a method that removes outliers, reduces noise and fills holes in a point cloud using a learned shape prior. The shape prior is learned from a set of training objects using the Gaussian process latent variable model. All training objects are represented by a signed distance function. To improve the time performance, the signed distance functions are compressed using the discrete cosine transform. As input data the method uses the estimated object pose from an object detector and a segmented point cloud. It is shown that the estimated shape prior is capable of modelling fine details to a certain degree. The limitations and difficulties of this method are also investigated. It is shown that by applying this method the accuracy and completeness of the reconstructed models can be significantly increased.

Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes

An intriguing application of transfer learning emerges when tasks arise with similar, but not identical, dynamics. Hidden Parameter Markov Decision Processes (HiP-MDP) embed these tasks into a low-dimensional space; given the embedding parameters one can identify the MDP for a particular task. However, the original formulation of HiP-MDP had a critical flaw: the embedding uncertainty was modeled independently of the agent's state uncertainty, requiring an arduous training procedure. In this work, we apply a Gaussian Process latent variable model to jointly model the dynamics and the embedding, leading to a more elegant formulation, one that allows for better uncertainty quantification and thus more robust transfer.

A Comparative Study for Postural Synergy Synthesis Using Linear and Nonlinear Methods

Human controls dozens of muscles for different hand postures in a coordinated manner. Such coordination is referred to as a postural synergy. Postural synergy has enabled an anthropomorphic robotic hand with many actuators to be applied as a prosthetic hand and controlled by two to three channels of biological signals. Principle component analysis (PCA) of the hand postures has become a popular way to extract the postural synergies. However, relatively big errors are often produced while the hand postures are reconstructed using these PCA-synthesized synergies due to the linearity nature of this method. This paper presents a comparative study in which the postural synergies are synthesized using both linear and nonlinear methods. Specifically, the Gaussian process latent variable model (GPLVM), as a nonlinear dimension reduction method, is implemented to produce nonlinear postural synergies and the hand postures can then be reconstructed from the two-dimensional synergy plane. Computational and experimental verifications show that the posture reconstruction errors are greatly reduced using this nonlinear method. The results suggest that the use of nonlinear postural synergies should be considered while applying a dexterous robotic hand as prosthesis. Versatile hand postures could be formed via only two channels of bio-signals.

Maximum entropy discrimination factor analyzers

Devising generative models that allow for inferring low dimensional latent feature representations of high-dimensional observations is a significant problem in statistical machine learning. Factor analysis (FA) is a well-established linear latent variable scheme addressing this problem by modeling the covariances between the elements of multivariate observations under a set of linear assumptions. FA is closely related to principal components analysis (PCA), and might be considered as a generalization of both PCA and its probabilistic version, PPCA. Recently, the invention of Gaussian process latent variable models (GP-LVMs) has given rise to a whole new family of latent variable modeling schemes that generalize FA under a nonparametric Bayesian inference framework. In this work, we examine generalization of FA models under a different Bayesian inference perspective. Specifically, we propose a large-margin formulation of FA under the maximum entropy discrimination (MED) framework. The MED framework integrates the large-margin principle with Bayesian posterior inference in an elegant and computationally efficient fashion, allowing to leverage existing high-performance solvers for convex optimization problems. We devise efficient mean-field inference algorithms for our model, and exhibit its advantages by evaluating it in a number of diverse application scenarios, dealing with high-dimensional data classification and reconstruction.

Improved data visualisation through multiple dissimilarity modelling

Popular dimension reduction and visualisation algorithms rely on the assumption that input dissimilarities are typically Euclidean, for instance Metric Multidimensional Scaling, t-distributed Stochastic Neighbour Embedding and the Gaussian Process Latent Variable Model. It is well known that this assumption does not hold for most datasets and often high-dimensional data sits upon a manifold of unknown global geometry. We present a method for improving the manifold charting process, coupled with Elastic MDS, such that we no longer assume that the manifold is Euclidean, or of any particular structure. We draw on the benefits of different dissimilarity measures allowing for the relative responsibilities, under a linear combination, to drive the visualisation process.

Dual Diversified Dynamical Gaussian Process Latent Variable Model for Video Repairing

In this paper, we propose a dual diversified dynamical Gaussian process latent variable model ( [Formula: see text]GPLVM) to tackle the video repairing issue. For preservation purposes, videos have to be conserved on media. However, storing on media, such as films and hard disks, can suffer from unexpected data loss, for instance, physical damage. So repairing of missing or damaged pixels is essential for better video maintenance. Most methods seek to fill in missing holes by synthesizing similar textures from local patches (the neighboring pixels), consecutive frames, or the whole video. However, these can introduce incorrect contexts, especially when the missing hole or number of damaged frames is large. Furthermore, simple texture synthesis can introduce artifacts in undamaged and recovered areas. To address aforementioned problems, we introduce two diversity encouraging priors to both of inducing points and latent variables for considering the variety in existing videos. In [Formula: see text]GPLVM, the inducing points constitute a smaller subset of observed data, while latent variables are a low-dimensional representation of observed data. Since they have a strong correlation with the observed data, it is essential that both of them can capture distinct aspects of and fully represent the observed data. The dual diversity encouraging priors ensure that the trained inducing points and latent variables are more diverse and resistant for context-aware and artifacts-free-based video repairing. The defined objective function in our proposed model is initially not analytically tractable and must be solved by variational inference. Finally, experimental testing results illustrate the robustness and effectiveness of our method for damaged video repairing.

Diversified Dynamical Gaussian Process Latent Variable Model for Video Repair

Videos can be conserved on different media. However, storing on media such as films and hard disks can suffer from unexpected data loss, for instance from physical damage. Repair of missing or damaged pixels is essential for video maintenance and preservation. Most methods seek to fill in missing holes by synthesizing similar textures from local or global frames. However, this can introduce incorrect contexts, especially when the missing hole or number of damaged frames is large. Furthermore, simple texture synthesis can introduce artifacts in undamaged and recovered areas. To address aforementioned problems, we propose the diversified dynamical Gaussian process latent variable model (D2GPLVM) for considering the variety in existing videos and thus introducing a diversity encouraging prior to inducing points. The aim is to ensure that the trained inducing points, which are a smaller set of all observed undamaged frames, are more diverse and resistant for context-aware and artifacts-free based video repair. The defined objective function in our proposed model is initially not analytically tractable and must be solved by variational inference. Finally, experimental testing illustrates the robustness and effectiveness of our method for damaged video repair.