Twin Gaussian Processes Research Articles

Efficient Model-Free Calibration of a 5-Degree of Freedom Hybrid Robot

Abstract The pose accuracy is a crucial issue that limits the application of hybrid robots. The model-free calibration instead of complex error modeling is investigated to improve the pose accuracy of a 5-degrees-of-freedom (DOF) hybrid robot efficiently. To overcome the difficult problem of model-free calibration in high-dimension joint space that the required measurement data for accurate prediction increase exponentially, a dimensionality reduction method is proposed to decompose high-dimension joint space into two low-dimension subspaces. Then the pose errors can be respectively measured in two subspaces based on the calibrated standard poses to train their corresponding pose error predicators. The standard poses ensure the measured pose errors in two subspaces do not affect each other. Thus, a merging operation obtained by kinematic analysis can finally merge the predicted pose errors of two subspaces into the complete pose error. The error predicators established by several regression methods including artificial neural network, extreme learning machine (ELM) and Twin Gaussian process regression are compared on multi aspects, and ELM stands out among them due to its outstanding prediction accuracy, good anti-noise ability, and low training data requirements. In addition, different representations of pose and pose error are adopted at different calibration stages to deal with the influence of parasitic motion of hybrid robot for the implementation of proposed calibration method. The compensation experiment is executed and the results show that position and orientation errors are reduced by 92.4% and 88.2% on average after calibration and the pose accuracy can meet application requirements.

Structured Gaussian Process Regression of Music Mood

Modeling the music mood has wide applications in music categorization, retrieval, and recommendation systems; however, it is challenging to computationally model the affective content of music due to its subjective nature. In this work, a structured regression framework is proposed to model the valence and arousal mood dimensions of music using a single regression model at a linear computational cost. To tackle the subjectivity phenomena, a confidence-interval based estimated consensus is computed by modeling the behavior of various annotators (e.g. biased, adversarial) and is shown to perform better than using the average annotation values. For a compact feature representation of music clips, variational Bayesian inference is used to learn the Gaussian mixture model representation of acoustic features and chord-related features are used to improve the valence estimation by probing the chord progressions between chroma frames. The dimensionality of features is further reduced using an adaptive version of kernel PCA. Using an efficient implementation of twin Gaussian process for structured regression, the proposed work achieves a significant improvement in R2 for arousal and valence dimensions relative to state-of-the-art techniques on two benchmark datasets for music mood estimation.

MTGPLP Approach for Traffic Signal Intelligent Control

In this paper, we state a combining programming approach to optimize traffic signal control problem. The objective of the model is to minimize the total queue length with weight factors at the end of each phase. Then, modified Twin Gaussian Process (MTGP) is employed to predict the arrival rates for the traffic signal control problem. For achieving automatic control of the traffic signal, an intelligent control method of the traffic signal is proposed in view of the combining method, that is to say, the combining method of MTGP and LP, called MTGPLP, is embraced in the intelligent control system. Furthermore, some numerical experiments are proposed to test the validity of the model and the MTGPLP approach. In particular, the results of numerical experiments show that the model is effective with different arrival rates, departure rates, and weight factors and the combining method is successful.

Covariate Shift Adaptation for Structured Regression With Frank–Wolfe Algorithms

This paper concerns structured regression problems wherein the issue of covariate shift is addressed, which aims at reducing the discrepancy in training and test data distributions, using computationally efficient and sparse optimization principles. In particular, the projection-free Frank–Wolfe optimization algorithms are used to learn the importance weights and re-weight the training data in the context of covariate shift. To determine the unbiased estimates of the weights, Kullback–Leibler importance estimation procedure is used but its computational cost can be high since it is based on projected gradient optimization. Instead of using the standard Frank–Wolfe algorithm, we adapt its variants and propose away-steps Frank–Wolfe and pairwise Frank–Wolfe covariate shift algorithms to correct the covariate shift. The results highlight the improved computational efficiency and sparsity achieved while learning the importance weights on synthetic as well as benchmark datasets. Furthermore, importance weighted Sharma-Mittal twin Gaussian process structured regression framework is proposed to incorporate the learned weights from covariate shift algorithms, and its equations are derived for importance weighted derivatives and uncertainties. The performance of proposed algorithms is evaluated on two applications of structured regression, namely, human pose estimation and music mood estimation, where the benefit of handling covariate shift is demonstrated with improved performance relative to the state-of-the-art techniques.

Vibration-based Structural Damage Detection Using Twin Gaussian Process (TGP)

A new pattern recognition based structural damage detection method using structural mode-shapes along with natural frequencies is presented in this article. A three span RC bridge is selected to verify the proposed method. To conquer the complexities of finite element simulation of the real structure, SAP2000 is employed for FE modeling and analyzing the structure and thus generating the input parameters for damage assessment process.Having the modal data for the intact and damaged structure, extraction of damage properties becomes a “pattern recognition” or “feature discrimination” problem, while Twin Gaussian Process (TGP) as an advantageous pattern recognition technique is employed to compare the measured mode-shapes and frequencies with their healthy state values and identify the damaged elements.The results of this study shows that the suggested method performs precisely in damage identification of structures having an updated as-built model. Furthermore, it can be concluded that TGP, which is used for damage detection process in this study, is an efficient tool to detect both location and severity of structural damages in single and multiple damage scenarios. The proposed method can be widely used for model updating and damage detection of real structures without any restrictions in modeling.

Martial Arts, Dancing and Sports dataset: A challenging stereo and multi-view dataset for 3D human pose estimation

Human pose estimation is one of the most popular research topics in the past two decades, especially with the introduction of human pose datasets for benchmark evaluation. These datasets usually capture simple daily life actions. Here, we introduce a new dataset, the Martial Arts, Dancing and Sports (MADS), which consists of challenging martial arts actions (Tai-chi and Karate), dancing actions (hip-hop and jazz), and sports actions (basketball, volleyball, football, rugby, tennis and badminton). Two martial art masters, two dancers and an athlete performed these actions while being recorded with either multiple cameras or a stereo depth camera. In the multi-view or single-view setting, we provide three color views for 2D image-based human pose estimation algorithms. For depth-based human pose estimation, we provide stereo-based depth images from a single view. All videos have corresponding synchronized and calibrated ground-truth poses, which were captured using a Motion Capture system. We provide initial baseline results on our dataset using a variety of tracking frameworks, including a generative tracker based on the annealing particle filter and robust likelihood function, a discriminative tracker using twin Gaussian processes [1], and hybrid trackers, such as Personalized Depth Tracker [2]. The results of our evaluation suggest that discriminative approaches perform better than generative approaches when there are enough representative training samples, and that the generative methods are more robust to diversity of poses, but can fail to track when the motion is too quick for the effective search range of the particle filter. The data and the accompanying code will be made available to the research community.

An active inference approach to on-line agent monitoring in safety–critical systems

The current trend towards integrating software agents in safety–critical systems such as drones, autonomous cars and medical devices, which must operate in uncertain environments, gives rise to the need of on-line detection of an unexpected behavior. In this work, on-line monitoring is carried out by comparing environmental state transitions with prior beliefs descriptive of optimal behavior. The agent policy is computed analytically using linearly solvable Markov decision processes. Active inference using prior beliefs allows a monitor proactively rehearsing on-line future agent actions over a rolling horizon so as to generate expectations to discover surprising behaviors. A Bayesian surprise metric is proposed based on twin Gaussian processes to measure the difference between prior and posterior beliefs about state transitions in the agent environment. Using a sliding window of sampled data, beliefs are updated a posteriori by comparing a sequence of state transitions with the ones predicted using the optimal policy. An artificial pancreas for diabetic patients is used as a representative example.

Generalized Twin Gaussian processes using Sharma–Mittal divergence

There has been a growing interest in mutual information measures due to their wide range of applications in machine learning and computer vision. In this paper, we present a generalized structured regression framework based on Sharma---Mittal (SM) divergence, a relative entropy measure, which is introduced to in the machine learning community in this work. SM divergence is a generalized mutual information measure for the widely used Renyi, Tsallis, Bhattacharyya, and Kullback---Leibler (KL) relative entropies. Specifically, we study SM divergence as a cost function in the context of the Twin Gaussian processes (TGP) (Bo and Sminchisescu 2010), which generalizes over the KL-divergence without computational penalty. We show interesting properties of Sharma---Mittal TGP (SMTGP) through a theoretical analysis, which covers missing insights in the traditional TGP formulation. However, we generalize this theory based on SM-divergence instead of KL-divergence which is a special case. Experimentally, we evaluated the proposed SMTGP framework on several datasets. The results show that SMTGP reaches better predictions than KL-based TGP, since it offers a bigger class of models through its parameters that we learn from the data.

Analysis of music/speech via integration of audio content and functional brain response

Effective analysis of music/speech data such as clustering, retrieval, and classification has received significant attention in recent years. Traditional methods mainly rely on the low-level acoustic features derived from digital audio stream, and the accuracy of these methods is limited by the well-known semantic gap. To alleviate this problem, we propose a novel framework for music/speech clustering, retrieval, and classification by integrating the low-level acoustic features derived from audio content with the functional magnetic resonance imaging (fMRI) measured features that represent the brain’s functional response when subjects are listening to the music/speech excerpts. First, the brain networks and regions of interest (ROIs) involved in the comprehension of audio stimuli, such as the auditory, emotion, attention, and working memory systems, are located by a new approach named dense individualized and common connectivity-based cortical landmarks (DICCCOLs). Then the functional connectivity matrix measuring the similarity between the fMRI signals of different ROIs is adopted to represent the brain’s comprehension of audio semantics. Afterwards, we propose an improved twin Gaussian process (ITGP) model based on self-training to predict the fMRI-measured features of testing data without fMRI scanning. Finally, multi-view learning algorithms are proposed to integrate acoustic features with fMRI-measured features for music/speech clustering, retrieval, and classification, respectively. The experimental results demonstrate the superiority of our proposed work in comparison with existing methods and suggest the advantage of integrating functional brain responses via fMRI data for music/speech analysis.

Behavior monitoring under uncertainty using Bayesian surprise and optimal action selection

The increasing trend towards delegating tasks to autonomous artificial agents in safety–critical socio-technical systems makes monitoring an action selection policy of paramount importance. Agent behavior monitoring may profit from a stochastic specification of an optimal policy under uncertainty. A probabilistic monitoring approach is proposed to assess if an agent behavior (or policy) respects its specification. The desired policy is modeled by a prior distribution for state transitions in an optimally-controlled stochastic process. Bayesian surprise is defined as the Kullback–Leibler divergence between the state transition distribution for the observed behavior and the distribution for optimal action selection. To provide a sensitive on-line estimation of Bayesian surprise with small samples twin Gaussian processes are used. Timely detection of a deviant behavior or anomaly in an artificial pancreas highlights the sensitivity of Bayesian surprise to a meaningful discrepancy regarding the stochastic optimal policy when there exist excessive glycemic variability, sensor errors, controller ill-tuning and infusion pump malfunctioning. To reject outliers and leave out redundant information, on-line sparsification of data streams is proposed.

Covariate Shift Adaptation for Discriminative 3D Pose Estimation

Discriminative, or (structured) prediction, methods have proved effective for variety of problems in computer vision; a notable example is 3D monocular pose estimation. All methods to date, however, relied on an assumption that training (source) and test (target) data come from the same underlying joint distribution. In many real cases, including standard data sets, this assumption is flawed. In the presence of training set bias, the learning results in a biased model whose performance degrades on the (target) test set. Under the assumption of covariate shift, we propose an unsupervised domain adaptation approach to address this problem. The approach takes the form of training instance reweighting, where the weights are assigned based on the ratio of training and test marginals evaluated at the samples. Learning with the resulting weighted training samples alleviates the bias in the learned models. We show the efficacy of our approach by proposing weighted variants of kernel regression (KR) and twin Gaussian processes (TGP). We show that our weighted variants outperform their unweighted counterparts and improve on the state-of-the-art performance in the public (HumanEva) data set.

Domain Adaptation for Structured Regression

Discriminative regression models have proved effective for many vision applications (here we focus on 3D full-body and head pose estimation from image and depth data). However, dataset bias is common and is able to significantly degrade the performance of a trained model on target test sets. As we show, covariate shift, a form of unsupervised domain adaptation (USDA), can be used to address certain biases in this setting, but is unable to deal with more severe structural biases in the data. We propose an effective and efficient semi-supervised domain adaptation (SSDA) approach for addressing such more severe biases in the data. Proposed SSDA is a generalization of USDA, that is able to effectively leverage labeled data in the target domain when available. Our method amounts to projecting input features into a higher dimensional space (by construction well suited for domain adaptation) and estimating weights for the training samples based on the ratio of test and train marginals in that space. The resulting augmented weighted samples can then be used to learn a model of choice, alleviating the problems of bias in the data; as an example, we introduce SSDA twin Gaussian process regression (SSDA-TGP) model. With this model we also address the issue of data sharing, where we are able to leverage samples from certain activities (e.g., walking, jogging) to improve predictive performance on very different activities (e.g., boxing). In addition, we analyze the relationship between domain similarity and effectiveness of proposed USDA versus SSDA methods. Moreover, we propose a computationally efficient alternative to TGP (Bo and Sminchisescu 2010), and it's variants, called the direct TGP. We show that our model outperforms a number of baselines, on two public datasets: HumanEva and ETH Face Pose Range Image Dataset. We can also achieve 8---15 times speedup in computation time, over the traditional formulation of TGP, using the proposed direct formulation, with little to no loss in performance.

Twin Gaussian Processes for Structured Prediction

We describe twin Gaussian processes (TGP), a generic structured prediction method that uses Gaussian process (GP) priors on both covariates and responses, both multivariate, and estimates outputs by minimizing the Kullback-Leibler divergence between two GP modeled as normal distributions over finite index sets of training and testing examples, emphasizing the goal that similar inputs should produce similar percepts and this should hold, on average, between their marginal distributions. TGP captures not only the interdependencies between covariates, as in a typical GP, but also those between responses, so correlations among both inputs and outputs are accounted for. TGP is exemplified, with promising results, for the reconstruction of 3d human poses from monocular and multicamera video sequences in the recently introduced HumanEva benchmark, where we achieve 5 cm error on average per 3d marker for models trained jointly, using data from multiple people and multiple activities. The method is fast and automatic: it requires no hand-crafting of the initial pose, camera calibration parameters, or the availability of a 3d body model associated with human subjects used for training or testing.