Gaussian Process Latent Variable Model Research Articles

Articulated tracking with manifold regularized particle filter

In this paper, we investigate articulated human motion tracking from video sequences using Bayesian approach. We derive a generic particle-based filtering procedure with a low-dimensional manifold. The manifold can be treated as a regularizer that enforces a distribution over poses during tracking process to be concentrated around the low-dimensional embedding. We refer to our method as manifold regularized particle filter. We present a particular implementation of our method based on back-constrained gaussian process latent variable model and gaussian diffusion. The proposed approach is evaluated using the real-life benchmark dataset HumanEva. We show empirically that the presented sampling scheme outperforms sampling-importance resampling and annealed particle filter procedures.

Variational inference for latent variables and uncertain inputs in Gaussian processes

The Gaussian process latent variable model (GP-LVM) provides a flexible approach for non-linear dimensionality reduction that has been widely applied. However, the current approach for training GP-LVMs is based on maximum likelihood, where the latent projection variables are maximised over rather than integrated out. In this paper we present a Bayesian method for training GP-LVMs by introducing a non-standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a GP-LVM by maximising an analytic lower bound on the exact marginal likelihood. We apply this method for learning a GP-LVM from i.i.d. observations and for learning non-linear dynamical systems where the observations are temporally correlated. We show that a benefit of the variational Bayesian procedure is its robustness to overfitting and its ability to automatically select the dimensionality of the non-linear latent space. The resulting framework is generic, flexible and easy to extend for other purposes, such as Gaussian process regression with uncertain or partially missing inputs. We demonstrate our method on synthetic data and standard machine learning benchmarks, as well as challenging real world datasets, including high resolution video data.

Gaze-estimation for consumer-grade cameras using a Gaussian process latent variable model

Commercial gaze-tracking devices provide accurate measurements of the visual gaze and are applied to a broad range of problems in marketing, human-computer interaction, and health care technology. In some applications commercial systems are either unavailable or unaffordable. Therefore, developing low cost solutions using off the shelf components is worthwhile. In the paper at hand, we apply a hierarchy of Gaussian processes, a class of probabilistic function regressors, to the problem of visual gaze-tracking for consumer grade cameras. Gaussian process latent variable models lead to a lower dimensional manifold which represents the gaze space. Finally, a Gaussian process mapping from screen coordinates to gaze manifold enables us to seek for the users visual gaze point given a previously unseen eye-patch. In our experiments, we achieve mean errors of approximately 2 cm for a consumer grade webcam that is positioned 30-40 cm in front of the user.

Dynamic Gesture Recognition based on Image Sequence

This paper proposed an algorithm for 3D hands tracking on the learned hierarchical latent variable space, which employs a Hierarchical Gaussian Process Latent Variable Model(HGPLVM) to learn the hierarchical latent space of hands motion and the nonlinear mapping from the hierarchical latent space to the pose space simultaneously. Nonlinear mappings from the hierarchical latent space to the space of hand images are constructed using radial basis function interpolation method. With these mappings, particles can be projected into hand images and measured in the image space directly. Particle filters with fewer particles are used to track the hand on the learned hierarchical low-dimensional space. Then the Hierarchical Conditional Random Field, which can capture extrinsic class dynamics and learn the relationship between motions of hand parts and different hand gestures simultaneously, is presented to model the continuous hand gestures. Experimental results show that our proposed method can track articulated hand robustly and approving recognition performance has also been achieved on the user-defined hand gesture dataset.

Dynamic texture modeling and synthesis using multi-kernel Gaussian process dynamic model

Dynamic texture (DT) widely exists in various social video media. Therefore, DT modeling and synthesis plays an important role in social media analyzing and processing. In this paper, we propose a Bayesian-based nonlinear dynamic texture modeling method for dynamic texture synthesis. To capture the non-stationary distribution of DT, we utilize the Gaussian process latent variable model for dimensional reduction. Furthermore, we design a multi-kernel dynamic system for the latent dynamic behavior modeling. In our model, we do not make strong assumption on the nonlinear function. Instead, our model automatically constructs a suitable nonlinear kernel for dynamic modeling and therefore is capable of fitting various types of dynamics. We evaluate the effectiveness our methods on the DynTex database and compared with representative DT synthesis method. Experimental results show that our method can achieve synthesis results with higher visual quality.

Covariate dimension reduction for survival data via the Gaussian process latent variable model.

The analysis of high-dimensional survival data is challenging, primarily owing to the problem of overfitting, which occurs when spurious relationships are inferred from data that subsequently fail to exist in test data. Here, we propose a novel method of extracting a low-dimensional representation of covariates in survival data by combining the popular Gaussian process latent variable model with a Weibull proportional hazards model. The combined model offers a flexible non-linear probabilistic method of detecting and extracting any intrinsic low-dimensional structure from high-dimensional data. By reducing the covariate dimension, we aim to diminish the risk of overfitting and increase the robustness and accuracy with which we infer relationships between covariates and survival outcomes. In addition, we can simultaneously combine information from multiple data sources by expressing multiple datasets in terms of the same low-dimensional space. We present results from several simulation studies that illustrate a reduction in overfitting and an increase in predictive performance, as well as successful detection of intrinsic dimensionality. We provide evidence that it is advantageous to combine dimensionality reduction with survival outcomes rather than performing unsupervised dimensionality reduction on its own. Finally, we use our model to analyse experimental gene expression data and detect and extract a low-dimensional representation that allows us to distinguish high-risk and low-risk groups with superior accuracy compared with doing regression on the original high-dimensional data.

Modeling dynamic high-DOF finger postures from surface EMG using nonlinear synergies in latent space representation.

Accurate proportional myoelectric control of the hand is important in replicating dexterous manipulation in robot prostheses and orthoses. However, this is still difficult to achieve due to the complex and high degree-of-freedom (DOF) nature present in the governing musculoskeletal system. To address this problem, we suggest using a low dimensional encoding based on nonlinear synergies to represent both the high-DOF finger joint kinematics and the coordination of muscle activities taken from surface electromyographic (EMG) signals. Generating smooth multi-finger movements using EMG inputs is then done by using a shared Gaussian Process latent variable model that learns a dynamical model between both the kinematic and EMG data represented in a shared latent space. The experimental results show that the method is able to synthesize continuous movements of a full five-finger hand model, with total dimensions as large as 69 (although highly redundant and correlated). Finally, by comparing the estimation performances when the number of EMG latent dimensions are varied, we show that these synergistic features can capture the variance, shared and specific to the observed kinematics.

Age estimation based on improved discriminative Gaussian process latent variable model

Affected by various factors (genes, living habits and so on), different people present distinct aging patterns. To discover the underlying trend of aging patterns, we propose an effective age estimation method based on DGPLVM (Discriminative Gaussian Process Latent Variable Model). DGPLVM is a kind of discriminative latent variable method for manifold learning. It discovers the low-dimensional manifold by employing a discriminative prior distribution over the latent space. DGPLVM with KFDA (Kernel Fisher Discriminant Analysis) prior has been studied and successfully applied to face verification. Different with face verification which is a two-class problem, age estimation is a linearly inseparable multi-class problem. In this paper, DGPLVM with KFDA is reformulated to get the low-dimensional representations for age estimation. After low-dimensional representations are obtained, Gaussian process regression model is adopted to find the age regressor mapping low-dimensional representations to ages. Experimental results on two widely used databases FG-NET and MORPH show that reformulated DGPLVM with KFDA is a good application in age estimation and achieves comparable results to state-of-the arts.

Multilayer Joint Gait-Pose Manifolds for Human Gait Motion Modeling.

We present new multilayer joint gait-pose manifolds (multilayer JGPMs) for complex human gait motion modeling, where three latent variables are defined jointly in a low-dimensional manifold to represent a variety of body configurations. Specifically, the pose variable (along the pose manifold) denotes a specific stage in a walking cycle; the gait variable (along the gait manifold) represents different walking styles; and the linear scale variable characterizes the maximum stride in a walking cycle. We discuss two kinds of topological priors for coupling the pose and gait manifolds, i.e., cylindrical and toroidal, to examine their effectiveness and suitability for motion modeling. We resort to a topologically-constrained Gaussian process (GP) latent variable model to learn the multilayer JGPMs where two new techniques are introduced to facilitate model learning under limited training data. First is training data diversification that creates a set of simulated motion data with different strides. Second is the topology-aware local learning to speed up model learning by taking advantage of the local topological structure. The experimental results on the Carnegie Mellon University motion capture data demonstrate the advantages of our proposed multilayer models over several existing GP-based motion models in terms of the overall performance of human gait motion modeling.

Discriminative Shared Gaussian Processes for Multiview and View-Invariant Facial Expression Recognition

Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.

TPSLVM: a dimensionality reduction algorithm based on thin plate splines.

Dimensionality reduction (DR) has been considered as one of the most significant tools for data analysis. One type of DR algorithms is based on latent variable models (LVM). LVM-based models can handle the preimage problem easily. In this paper we propose a new LVM-based DR model, named thin plate spline latent variable model (TPSLVM). Compared to the well-known Gaussian process latent variable model (GPLVM), our proposed TPSLVM is more powerful especially when the dimensionality of the latent space is low. Also, TPSLVM is robust to shift and rotation. This paper investigates two extensions of TPSLVM, i.e., the back-constrained TPSLVM (BC-TPSLVM) and TPSLVM with dynamics (TPSLVM-DM) as well as their combination BC-TPSLVM-DM. Experimental results show that TPSLVM and its extensions provide better data visualization and more efficient dimensionality reduction compared to PCA, GPLVM, ISOMAP, etc.

Data‐Driven Reconstruction of Human Locomotion Using a Single Smartphone

AbstractGenerating a visually appealing human motion sequence using low‐dimensional control signals is a major line of study in the motion research area in computer graphics. We propose a novel approach that allows us to reconstruct full body human locomotion using a single inertial sensing device, a smartphone. Smartphones are among the most widely used devices and incorporate inertial sensors such as an accelerometer and a gyroscope. To find a mapping between a full body pose and smartphone sensor data, we perform low dimensional embedding of full body motion capture data, based on a Gaussian Process Latent Variable Model. Our system ensures temporal coherence between the reconstructed poses by using a state decomposition model for automatic phase segmentation. Finally, application of the proposed nonlinear regression algorithm finds a proper mapping between the latent space and the sensor data. Our framework effectively reconstructs plausible 3D locomotion sequences. We compare the generated animation to ground truth data obtained using a commercial motion capture system.

Manifold Learning for Object Tracking With Multiple Nonlinear Models

This paper presents a novel manifold learning algorithm for high-dimensional data sets. The scope of the application focuses on the problem of motion tracking in video sequences. The framework presented is twofold. First, it is assumed that the samples are time ordered, providing valuable information that is not presented in the current methodologies. Second, the manifold topology comprises multiple charts, which contrasts to the most current methods that assume one single chart, being overly restrictive. The proposed algorithm, Gaussian process multiple local models (GP-MLM), can deal with arbitrary manifold topology by decomposing the manifold into multiple local models that are probabilistic combined using Gaussian process regression. In addition, the paper presents a multiple filter architecture where standard filtering techniques are integrated within the GP-MLM. The proposed approach exhibits comparable performance of state-of-the-art trackers, namely multiple model data association and deep belief networks, and compares favorably with Gaussian process latent variable models. Extensive experiments are presented using real video data, including a publicly available database of lip sequences and left ventricle ultrasound images, in which the GP-MLM achieves state of the art results.

Probabilistic PCA of censored data: accounting for uncertainties in the visualization of high-throughput single-cell qPCR data

Motivation: High-throughput single-cell quantitative real-time polymerase chain reaction (qPCR) is a promising technique allowing for new insights in complex cellular processes. However, the PCR reaction can be detected only up to a certain detection limit, whereas failed reactions could be due to low or absent expression, and the true expression level is unknown. Because this censoring can occur for high proportions of the data, it is one of the main challenges when dealing with single-cell qPCR data. Principal component analysis (PCA) is an important tool for visualizing the structure of high-dimensional data as well as for identifying subpopulations of cells. However, to date it is not clear how to perform a PCA of censored data. We present a probabilistic approach that accounts for the censoring and evaluate it for two typical datasets containing single-cell qPCR data.Results: We use the Gaussian process latent variable model framework to account for censoring by introducing an appropriate noise model and allowing a different kernel for each dimension. We evaluate this new approach for two typical qPCR datasets (of mouse embryonic stem cells and blood stem/progenitor cells, respectively) by performing linear and non-linear probabilistic PCA. Taking the censoring into account results in a 2D representation of the data, which better reflects its known structure: in both datasets, our new approach results in a better separation of known cell types and is able to reveal subpopulations in one dataset that could not be resolved using standard PCA.Availability and implementation: The implementation was based on the existing Gaussian process latent variable model toolbox (https://github.com/SheffieldML/GPmat); extensions for noise models and kernels accounting for censoring are available at http://icb.helmholtz-muenchen.de/censgplvm.Contact: fbuettner.phys@gmail.comSupplementary information: Supplementary data are available at Bioinformatics online.

Dimensionality Reduction and Generation of Human Motion

To reuse existing motion data and generate new motion, a method of human motion nonlinear dimensionality reduction and generation, based on fast adaptive scaled Gaussian process latent variable models, is proposed. Through statistical learning on motion data, the motion data are mapped from high-dimensional observation space to low-dimensional latent space to implement nonlinear dimensionality reduction, and probability distributing of posture space which measures the nature of posture is obtained. The posture which meets constraints and has maximal probability can be computed as the solution of inverse kinematics. This method can avoid cockamamie computation and posture distortion existing in traditional inverse kinematics.The experiments show that our method has higher convergence velocity and precision and extends editing range of motion by adapting motion editing direction.

Joint view-identity manifold for infrared target tracking and recognition

We propose a new joint view-identity manifold (JVIM) for multi-view and multi-target shape modeling that is well-suited for automated target tracking and recognition (ATR) in infrared imagery. As a shape generative model, JVIM features a novel manifold structure that imposes a conditional dependency between the two shape-related factors, view and identity, in a unified latent space, which is embedded with one view-independent identity manifold and infinite identity-dependent view manifolds. A modified local linear Gaussian process latent variable model (LL-GPLVM) is proposed for JVIM learning where a stochastic gradient descent method is used to improve the learning efficiency. We also develop a local inference technique to speed up JVIM-based shape interpolation. Due to its probabilistic and continuous nature, JVIM provides effective shape synthesis and supports robust ATR inference for both known and unknown target types under arbitrary views. Experiments on both synthetic data and the SENSIAC infrared ATR database demonstrate the advantages of the proposed method over several existing techniques both qualitatively and quantitatively.

A Research on Gaussian Process Based Feature Extraction

Data sets of high–dimensional spaces are problematic when it comes to classification, compression, and visualization. The main issue is to find a reduced dimensionality representation that corresponds to the intrinsic dimensionality of the original data. In this paper we try to investigate a practical Bayesian method for feature extracting problem, in particular we will apply Gaussian Process Latent Variable Model (GPLVM) to a real world data set. Feature extraction experiments were performed on a cancer treatments’ components data set using GPLVM, then we used PCA on the same data set for comparison of the results.

Gaussian process latent variable model based on immune clonal selection for SAR target feature extraction and recognition

As a nonlinear dimension reduction algorithm,Gaussian process latent variable model(GPLVM) has been widely applied in pattern recognition and computer vision for its capability in dealing with small size and high-dimensional samples.As GPLVM can discover low-dimensional manifolds in high-dimensional data given only a small number of samples,a new SAR target recognition method was proposed,in which a modified GPLVM was used for feature extraction and Gaussian process classification was employed as the classifier.In GPLVM,the likelihood was optimized by using the scaled conjugate gradient.In order to avoid the noise effect to gradient estimate and overcome the disadvantage that the performance is severely affected by the step length,the immune clone selection algorithm based GPLVM was developed for target feature extraction where the immune clonal selection algorithm characterized by rapid convergence to global optimum was utilized to improve the performance.The experimental results show that the method not only reduces the dimension but also gets higher accuracy.

基于机器学习模型的沙漠腹地地下水含盐量变化过程及模拟研究

为了研究塔克拉玛干沙漠腹地的地下水盐分变化规律,模拟地下水盐分变化过程,评价适合该区域的地下水变化规律的模型。通过对研究区蒸发量、降水量、气温、气压、地下水位、地下水电导率数据的统计分析,揭示了地下水含盐量及其影响因素的特征;使用GP模型、GPLVM模型和BP人工神经网络模型以及综合模型,模拟了气候变化和人类活动双重影响下的地下水含盐量变化过程,并评价了模型的模拟结果。研究结果表明:(1)研究区地下水流动系统主要受气候变化和人类活动的影响,地下水位在局部地区随开采过程呈现波动变化。地下水位变化过程与气压的变化规律相一致;而气温和蒸发量的季节变化规律相一致。地下水盐分含量呈上升趋势。(2)GP模型对于地下水含盐量的预测效果最好;GPLVM模型对于已知地下水含盐量条件下,与其他环境因素进行多元回归分析的拟合效果最好。而GP、GPLVM和BP人工神经网络模型的综合模型,对于包括模型训练和模型预测的全体数据集的拟合和预测效果最好。;The Taklimakan Desert, in the center of the Tarim Basin, Northwest China, is near the arid center of the Eurasian Continent. It is an ideal place for carrying out research on circulation systems of ocean-continent and continent-continent, as well as on the environmental effects of the Tibetan Plateau uplift. Oil and natural gas resources are rich in the desert hinterlands, where weather is extremely dry and the ecological environment is extremely frail. How to achieve sustainable development of oil gas energy exploitation and water resources will become a key issue of rapid and sustained energy development in the exploitation area. Since 2000, with the increase of oil and gas production, as well as construction of ecological engineering in oil field areas, regional water use, i.e., water consumption by oil exploitation, life-service water, and ecological engineering water has been increasing. Under this condition, the response of regional groundwater to water-use behavior, and sustainable use of groundwater under scaled development of future oil gas energy, is a focus of discussion in academic circles. All the water used for oil field production, living, and an ecological shelterbelt program in the Tarim oil field was supplied by 19 local water wells. The main groundwater was from surrounding snow cover and glacier meltwater that leak into the groundwater system at a mountain pass, which then inflows slowly as runoff with terrain change. There is a single chemical type of groundwater in the study area, which is mainly alkali and a strong acid ion. Groundwater is an important resource within desert areas, for both human use and maintenance of the ecological environment. However, little information is available on the long-term change and evolutionary trends of groundwater salt content in the region. The aim of this study is to increase knowledge of such content and its influences there. We simulated this content, studied its change, and evaluated the effects and applicability of machine learning models for the hinterland of Taklimakan Desert. For these purposes, three machine learning models and an appropriate integrated model were designed to simulate the variation of groundwater salt content. Based on statistical analysis of evaporation, precipitation, air temperature, air pressure, groundwater level and conductivity, this paper reports some statistical features of that content and its influences. The temporal change of content was simulated using Gaussian Process (GP), Gaussian Process Latent Variable Model (GPLVM), Back Propagation Artificial Neural Network (BP-ANN), and an integrated machine learning model, under the impacts of climate change and human activities. The modeling was evaluated using root mean square error (RMSE) and uncentered R-squared (uncentered <em>R</em>²).The results show that the groundwater flow system is affected by climate change and human activities, and groundwater level appears steady with small fluctuation in the study area. The change of groundwater level was concordant with air pressure, and the change of air temperature was consistent with evaporation. There was an overall uptrend of groundwater salt content. Accurate prediction can be achieved with GP, whereas GPLVM was the better multiple regression method for a known groundwater salt content among the three models. The integrated model of GP, GPLVM, and BP-ANN gave the best results for all observation datasets, including training and prediction data. The results not only help clarify regional hydrologic cycle processes under extremely dry conditions, but can also provide appropriate theoretical evidence for sustainable utilization of water resources and safe production in desert oilfields.

A novel approach for resolving differences in single-cell gene expression patterns from zygote to blastocyst

Motivation: Single-cell experiments of cells from the early mouse embryo yield gene expression data for different developmental stages from zygote to blastocyst. To better understand cell fate decisions during differentiation, it is desirable to analyse the high-dimensional gene expression data and assess differences in gene expression patterns between different developmental stages as well as within developmental stages. Conventional methods include univariate analyses of distributions of genes at different stages or multivariate linear methods such as principal component analysis (PCA). However, these approaches often fail to resolve important differences as each lineage has a unique gene expression pattern which changes gradually over time yielding different gene expressions both between different developmental stages as well as heterogeneous distributions at a specific stage. Furthermore, to date, no approach taking the temporal structure of the data into account has been presented.Results: We present a novel framework based on Gaussian process latent variable models (GPLVMs) to analyse single-cell qPCR expression data of 48 genes from mouse zygote to blastocyst as presented by (Guo et al., 2010). We extend GPLVMs by introducing gene relevance maps and gradient plots to provide interpretability as in the linear case. Furthermore, we take the temporal group structure of the data into account and introduce a new factor in the GPLVM likelihood which ensures that small distances are preserved for cells from the same developmental stage. Using our novel framework, it is possible to resolve differences in gene expressions for all developmental stages. Furthermore, a new subpopulation of cells within the 16-cell stage is identified which is significantly more trophectoderm-like than the rest of the population. The trophectoderm-like subpopulation was characterized by considerable differences in the expression of Id2, Gata4 and, to a smaller extent, Klf4 and Hand1. The relevance of Id2 as early markers for TE cells is consistent with previously published results.Availability: The mappings were implemented based on Prof. Neil Lawrence's FGPLVM toolbox1; extensions for relevance analysis and including the structure of the data can be obtained from one of the authors' homepage.2Contact: f.buettner@helmholtz-muenchen.de