Bayesian Generative Model Research Articles

A unifying framework for joint trait analysis under a non-infinitesimal model

MotivationA large proportion of risk regions identified by genome-wide association studies (GWAS) are shared across multiple diseases and traits. Understanding whether this clustering is due to sharing of causal variants or chance colocalization can provide insights into shared etiology of complex traits and diseases.ResultsIn this work, we propose a flexible, unifying framework to quantify the overlap between a pair of traits called UNITY (Unifying Non-Infinitesimal Trait analYsis). We formulate a Bayesian generative model that relates the overlap between pairs of traits to GWAS summary statistic data under a non-infinitesimal genetic architecture underlying each trait. We propose a Metropolis–Hastings sampler to compute the posterior density of the genetic overlap parameters in this model. We validate our method through comprehensive simulations and analyze summary statistics from height and body mass index GWAS to show that it produces estimates consistent with the known genetic makeup of both traits.Availability and implementationThe UNITY software is made freely available to the research community at: https://github.com/bogdanlab/UNITY.Supplementary information Supplementary data are available at Bioinformatics online.

A cost-effective and reliable measurement strategy for 3D printed parts by integrating low- and high-resolution measurement systems

ABSTRACTMetrology data are crucial to quality control of three-dimensional (3D) printed parts. Low-cost measurement systems are often unreliable due to their low resolutions, whereas high-resolution measurement systems usually induce high measurement costs. To balance the measurement cost and accuracy, a new cost-effective and reliable measurement strategy is proposed in this article, which jointly uses two-resolution measurement systems. Specifically, only a small sample of base parts are measured by both the low- and high-resolution measurement systems in order to save costs. The measurement accuracy of most parts with only low-resolution metrology data is improved by effectively integrating high-resolution metrology data of the base parts. A Bayesian generative model parameterizes a part-independent bias and variance pattern of the low-resolution metrology data and facilitates a between-part data integration via an efficient Markov chain Monte Carlo sampling algorithm. This multi-part two-resolution metrology data integration highlights the novelty and contribution of this article compared with the existing one-part data integration methods in the literature. Finally, an intensive experimental study involving a laser scanner and a machine visual system has validated the effectiveness of our measurement strategy in acquisition of reliable metrology data of 3D printed parts.

Hierarchical Spatial Concept Formation Based on Multimodal Information for Human Support Robots.

In this paper, we propose a hierarchical spatial concept formation method based on the Bayesian generative model with multimodal information e.g., vision, position and word information. Since humans have the ability to select an appropriate level of abstraction according to the situation and describe their position linguistically, e.g., “I am in my home” and “I am in front of the table,” a hierarchical structure of spatial concepts is necessary in order for human support robots to communicate smoothly with users. The proposed method enables a robot to form hierarchical spatial concepts by categorizing multimodal information using hierarchical multimodal latent Dirichlet allocation (hMLDA). Object recognition results using convolutional neural network (CNN), hierarchical k-means clustering result of self-position estimated by Monte Carlo localization (MCL), and a set of location names are used, respectively, as features in vision, position, and word information. Experiments in forming hierarchical spatial concepts and evaluating how the proposed method can predict unobserved location names and position categories are performed using a robot in the real world. Results verify that, relative to comparable baseline methods, the proposed method enables a robot to predict location names and position categories closer to predictions made by humans. As an application example of the proposed method in a home environment, a demonstration in which a human support robot moves to an instructed place based on human speech instructions is achieved based on the formed hierarchical spatial concept.

Human‐level face verification with intra‐personal factor analysis and deep face representation

The last two decades have seen an escalating interest in methods for large-scale unconstrained face recognition. While the promise of computer vision systems to efficiently and accurately verify and identify faces in naturally occurring circumstances still remains elusive, recent advances in deep learning are taking us closer to human-level recognition. In this study, the authors propose a new paradigm which employs deep features in a feature extractor and intra-personal factor analysis as a recogniser. The proposed new strategy represents the face changes of a person using identity specific components and the intra-personal variation through reinterpretation of a Bayesian generative factor analysis model. The authors employ the expectation-maximisation algorithm to calculate model parameters which cannot be observed directly. Recognition outcomes achieved through benchmarking on large-scale wild databases, Labeled Faces in the Wild (LFW) and Youtube Face (YTF), clearly prove that the proposed approach provides remarkable face verification performance improvement over state-of-the-art approaches.

Generative multi-view and multi-feature learning for classification

Multi-view based classification has attracted much attention in recent years. In general, an object can be represented with various views or modalities, and the exploitation of correlation across different views would contribute to improving the classification performance. However, each view can also be described with multiple features and this types of data is called multi-view and multi-feature data. Different from many existing multi-view methods which only model multiple views but ignore intrinsic information among the various features in each view, a generative bayesian model is proposed in this paper to not only jointly take the features and views into account, but also learn a discriminant representation across distinctive categories. A latent variable corresponding to each feature in each view is assumed and the raw feature is a projection of the latent variable from a more discriminant space. Particularly, the extracted variables in each view belonging to the same class are encouraged to follow the same gaussian distribution and those belonging to different classes are conducted to follow different distributions, greatly exploiting the label information. To optimize the presented approach, the proposed method is transformed into a class-conditional model and an effective algorithm is designed to alternatively estimate the parameters and variables. The experimental results on the extensive synthetic and four real-world datasets illustrate the effectiveness and superiority of our method compared with the state-of-the-art.

Lower Bounds on the Bayes Risk of the Bayesian BTL Model With Applications to Comparison Graphs

We consider the problem of aggregating pairwise comparisons to obtain a consensus ranking order over a collection of objects. We use the popular Bradley-Terry-Luce (BTL) model which allows us to probabilistically describe pairwise comparisons between objects. In particular, we employ the Bayesian BTL model which allows for meaningful prior assumptions and to cope with situations where the number of objects is large and the number of comparisons between some objects is small or even zero. For the conventional Bayesian BTL model, we derive information-theoretic lower bounds on the Bayes risk of estimators for norm-based distortion functions. We compare the information-theoretic lower bound with the Bayesian Cram\'{e}r-Rao lower bound we derive for the case when the Bayes risk is the mean squared error. We illustrate the utility of the bounds through simulations by comparing them with the error performance of an expectation-maximization based inference algorithm proposed for the Bayesian BTL model. We draw parallels between pairwise comparisons in the BTL model and inter-player games represented as edges in a comparison graph and analyze the effect of various graph structures on the lower bounds. We also extend the information-theoretic and Bayesian Cram\'{e}r-Rao lower bounds to the more general Bayesian BTL model which takes into account home-field advantage.

Cross-Situational Learning with Bayesian Generative Models for Multimodal Category and Word Learning in Robots.

In this paper, we propose a Bayesian generative model that can form multiple categories based on each sensory-channel and can associate words with any of the four sensory-channels (action, position, object, and color). This paper focuses on cross-situational learning using the co-occurrence between words and information of sensory-channels in complex situations rather than conventional situations of cross-situational learning. We conducted a learning scenario using a simulator and a real humanoid iCub robot. In the scenario, a human tutor provided a sentence that describes an object of visual attention and an accompanying action to the robot. The scenario was set as follows: the number of words per sensory-channel was three or four, and the number of trials for learning was 20 and 40 for the simulator and 25 and 40 for the real robot. The experimental results showed that the proposed method was able to estimate the multiple categorizations and to learn the relationships between multiple sensory-channels and words accurately. In addition, we conducted an action generation task and an action description task based on word meanings learned in the cross-situational learning scenario. The experimental results showed that the robot could successfully use the word meanings learned by using the proposed method.

Bayesian switching factor analysis for estimating time-varying functional connectivity in fMRI

There is growing interest in understanding the dynamical properties of functional interactions between distributed brain regions. However, robust estimation of temporal dynamics from functional magnetic resonance imaging (fMRI) data remains challenging due to limitations in extant multivariate methods for modeling time-varying functional interactions between multiple brain areas. Here, we develop a Bayesian generative model for fMRI time-series within the framework of hidden Markov models (HMMs). The model is a dynamic variant of the static factor analysis model (Ghahramani and Beal, 2000). We refer to this model as Bayesian switching factor analysis (BSFA) as it integrates factor analysis into a generative HMM in a unified Bayesian framework. In BSFA, brain dynamic functional networks are represented by latent states which are learnt from the data. Crucially, BSFA is a generative model which estimates the temporal evolution of brain states and transition probabilities between states as a function of time. An attractive feature of BSFA is the automatic determination of the number of latent states via Bayesian model selection arising from penalization of excessively complex models. Key features of BSFA are validated using extensive simulations on carefully designed synthetic data. We further validate BSFA using fingerprint analysis of multisession resting-state fMRI data from the Human Connectome Project (HCP). Our results show that modeling temporal dependencies in the generative model of BSFA results in improved fingerprinting of individual participants. Finally, we apply BSFA to elucidate the dynamic functional organization of the salience, central-executive, and default mode networks—three core neurocognitive systems with central role in cognitive and affective information processing (Menon, 2011). Across two HCP sessions, we demonstrate a high level of dynamic interactions between these networks and determine that the salience network has the highest temporal flexibility among the three networks. Our proposed methods provide a novel and powerful generative model for investigating dynamic brain connectivity.

Fast Approximate Inference for Arbitrarily Large Semiparametric Regression Models via Message Passing

ABSTRACTWe show how the notion of message passing can be used to streamline the algebra and computer coding for fast approximate inference in large Bayesian semiparametric regression models. In particular, this approach is amenable to handling arbitrarily large models of particular types once a set of primitive operations is established. The approach is founded upon a message passing formulation of mean field variational Bayes that utilizes factor graph representations of statistical models. The underlying principles apply to general Bayesian hierarchical models although we focus on semiparametric regression. The notion of factor graph fragments is introduced and is shown to facilitate compartmentalization of the required algebra and coding. The resultant algorithms have ready-to-implement closed form expressions and allow a broad class of arbitrarily large semiparametric regression models to be handled. Ongoing software projects such as Infer.NET and Stan support variational-type inference for particular model classes. This article is not concerned with software packages per se and focuses on the underlying tenets of scalable variational inference algorithms. Supplementary materials for this article are available online.

A General Bayesian Model for Speaker Verification

This paper presents a general Bayesian model for speaker verification tasks. It is a generative probability model. Due to its simple analytical property, a computationally efficient expectation-maximization algorithm can be derived to obtain the model parameters. A closedform solution, which allows the scalable size of enrollment set, is given in a full Bayesian way for making speaker verification decisions. Factor analysis technique is employed to model the speaker-specific components, then the redundant information in this model will be dropped. Experimental results are evaluated by both equal error rate and minimum detection cost function. The proposed approach shows promising results on the National institute of standards and technology (NIST) Speaker recognition evaluation (SRE) 2010 extended and 2012 core tasks. Significant improvement is obtained when comparing with Gaussian probabilistic linear discriminant analysis, especially under phone-call conditions and mismatched train-test channel conditions. Contrast experimental results with other popular generative probability models are also presented in this paper.

BayesWipe

Recent efforts in data cleaning of structured data have focused exclusively on problems like data deduplication, record matching, and data standardization; none of the approaches addressing these problems focus on fixing incorrect attribute values in tuples. Correcting values in tuples is typically performed by a minimum cost repair of tuples that violate static constraints like Conditional Functional Dependencies (which have to be provided by domain experts or learned from a clean sample of the database). In this article, we provide a method for correcting individual attribute values in a structured database using a Bayesian generative model and a statistical error model learned from the noisy database directly. We thus avoid the necessity for a domain expert or clean master data. We also show how to efficiently perform consistent query answering using this model over a dirty database, in case write permissions to the database are unavailable. We evaluate our methods over both synthetic and real data.

Preferential sampling and Bayesian geostatistics: Statistical modeling and examples.

Preferential sampling refers to any situation in which the spatial process and the sampling locations are not stochastically independent. In this paper, we present two examples of geostatistical analysis in which the usual assumption of stochastic independence between the point process and the measurement process is violated. To account for preferential sampling, we specify a flexible and general Bayesian geostatistical model that includes a shared spatial random component. We apply the proposed model to two different case studies that allow us to highlight three different modeling and inferential aspects of geostatistical modeling under preferential sampling: (1) continuous or finite spatial sampling frame; (2) underlying causal model and relevant covariates; and (3) inferential goals related to mean prediction surface or prediction uncertainty.

Robust Estimation and Tracking of Pitch Period Using an Efficient Bayesian Filter

In this paper, we introduce an algorithm for estimating and tracking the pitch period of audio signals using Bayesian filters. For this purpose, we propose a general Bayesian model, which is robust to the nonstationary variations of the amplitude and frequency of the input signal. We also employ a state-space model, which uses the delayed versions of the input signal to model the periodicity of nonstationary audio signals. This simple model allows a significant reduction of the required number of particles for the estimation of the pitch period compared to the state-of-the-art particle filtering methods. Moreover, we propose to estimate the logarithm of the period instead of the period itself. We show that the resulting algorithm does not require prior knowledge about the initial state and is robust to the octave error phenomenon, which is a common problem in pitch period estimation methods. Most of the existing methods require that the processing window be longer than the largest existing period of the input signal. In contrast, the proposed method does not impose such a limit. Our method often results in a higher time-domain resolution with no perceptible compromise on the frequency-domain resolution, especially for high-pitched audio signals such as music. Simulation results reveal that the proposed algorithm outperforms the state-of-the-art pitch period detection algorithms at low signal to noise ratios assuming no prior knowledge about the initial conditions.

The existence of Bayesian fuzzy equilibrium problems for a new general Bayesian abstract fuzzy economy model with differential private information

The existence of Bayesian fuzzy equilibrium problems for a new general Bayesian abstract fuzzy economy model with differential private information

Quantifying Interdependent Privacy Risks with Location Data

Co-location information about users is increasingly available online. For instance, mobile users more and more frequently report their co-locations with other users in the messages and in the pictures they post on social networking websites by tagging the names of the friends they are with. The users' IP addresses also constitute a source of co-location information. Combined with (possibly obfuscated) location information, such co-locations can be used to improve the inference of the users' locations, thus further threatening their location privacy: As co-location information is taken into account, not only a user's reported locations and mobility patterns can be used to localize her, but also those of her friends (and the friends of their friends and so on). In this paper, we study this problem by quantifying the effect of co-location information on location privacy, considering an adversary such as a social network operator that has access to such information. We formalize the problem and derive an optimal inference algorithm that incorporates such co-location information, yet at the cost of high complexity. We propose some approximate inference algorithms, including a solution that relies on the belief propagation algorithm executed on a general Bayesian network model, and we extensively evaluate their performance. Our experimental results show that, even in the case where the adversary considers co-locations of the targeted user with a single friend, the median location privacy of the user is decreased by up to 62 percent in a typical setting. We also study the effect of the different parameters (e.g., the settings of the location-privacy protection mechanisms) in different scenarios.

Bayesian exponential random graph modeling of whole-brain structural networks across lifespan

Descriptive neural network analyses have provided important insights into the organization of structural and functional networks in the human brain. However, these analyses have limitations for inter-subject or between-group comparisons in which network sizes and edge densities may differ, such as in studies on neurodevelopment or brain diseases. Furthermore, descriptive neural network analyses lack an appropriate generic null model and a unifying framework. These issues may be solved with an alternative framework based on a Bayesian generative modeling approach, i.e. Bayesian exponential random graph modeling (ERGM), which explains an observed network by the joint contribution of local network structures or features (for which we chose neurobiologically meaningful constructs such as connectedness, local clustering or global efficiency). We aimed to identify how these local network structures (or features) are evolving across the life-span, and how sensitive these features are to random and targeted lesions. To that aim we applied Bayesian exponential random graph modeling on structural networks derived from whole-brain diffusion tensor imaging-based tractography of 382 healthy adult subjects (age range: 20.2–86.2years), with and without lesion simulations. Networks were successfully generated from four local network structures that resulted in excellent goodness-of-fit, i.e. measures of connectedness, local clustering, global efficiency and intrahemispheric connectivity. We found that local structures (i.e. connectedness, local clustering and global efficiency), which give rise to the global network topology, were stable even after lesion simulations across the lifespan, in contrast to overall descriptive network changes – e.g. lower network density and higher clustering – during aging, and despite clear effects of hub damage on network topologies. Our study demonstrates the potential of Bayesian generative modeling to characterize the underlying network structures that drive the brain's global network topology at different developmental stages and/or under pathological conditions.

Modeling Multi‐factor Sequential User Behavior Data over Social Networks

Modeling dynamic user behavior over online social networks not only helps us understand user behavior patterns on social networks, but also improves the performance of behavior analysis tasks. Time-varying user behavior is commonly influenced by multiple factors: user habit, social influence and external events. Existing works either consider only a part of these factors, or fail to model the dynamics behind user behavior. Thus, they cannot precisely model the user behavior. We present a generative Bayesian model HES to model dynamic user behavior data. We take the influential factors and user's selection process as separate latent variables, based on which we can recover the evolving patterns underneath user behavior data sequences. Empirical results on large-scale social networks show that the proposed approach outperforms existing user behavior prediction models by at least 8% w.r.t. prediction accuracy. Our work also unveils some interesting insights underneath social behavior data.

Scalable models for computing hierarchies in information networks

Information hierarchies are organizational structures that often used to organize and present large and complex information as well as provide a mechanism for effective human navigation. Fortunately, many statistical and computational models exist that automatically generate hierarchies; however, the existing approaches do not consider linkages in information networks that are increasingly common in real-world scenarios. Current approaches also tend to present topics as an abstract probably distribution over words, etc., rather than as tangible nodes from the original network. Furthermore, the statistical techniques present in many previous works are not yet capable of processing data at Web-scale. In this paper, we present the hierarchical document-topic model (HDTM), which uses a distributed vertex-programming process to calculate a nonparametric Bayesian generative model. Experiments on three medium- size data sets and the entire Wikipedia data set show that HDTM can infer accurate hierarchies even over large information networks.

Inflation Prediction and Inflation Volatility for Turkey

In early development of macroeconomic literature, policy makers focused on stability and constancy in monetary policy. However until now, policy makers are unable to make reliable prognostication in economic literature. The monetarist claims that under active monetary policy, inflation deviation should be unpredictable. Recently policy makers apply complicated econometric models such as Dynamic Stochastic General Equilibrium model (DSGE) and Bayesian VAR model in order to calculate an appropriate forecasting, yet, researchers prove these models very poor and hence these models do not give any further insights for policy making. The aim of this paper is too select an alternate technique that is robust, reliable, unbiased and efficient. For this reason, we estimate the inflation and its volatility for Turkey based on stochastic differential equation (SDE) under mathematical simulation. After computing stochastic parameters under SDE, our results establish that inflation volatility behaviour follow a systematic pattern from ex-ante to ex-post period for the Turkey. We can certainly reason that mathematical techniques are robust and give sufficient traces for inflation uncertainty rather than choosing DSGE and Bayesian VAR. Moreover, Euler Discretization and Unbiased Discretization process explain that the intensity for inflation volatility is recovered in the long run. Euler Discretization and unbiased Discretization confirm that inflation deviation is predictable for Turkey. DOI: 10.5901/mjss.2016.v7n1p361

Fuzzy games for a general Bayesian abstract fuzzy economy model of product measurable spaces

In this paper, we introduce a general Bayesian abstract fuzzy economy model of product measurable spaces, and we prove the existence of Bayesian fuzzy equilibrium for this model. Our results extend and improve the corresponding recent results announced by Patriche and many authors from the literature. It captures the idea that the uncertainties characterize the individual feature of the decisions of the agents involved in different economic activities. In this paper, the uncertainties can be described by using random fuzzy mappings. Further attention is needed for the study of applications of the established result in the game theory and the fuzzy economic field.Copyright © 2015 John Wiley & Sons, Ltd.