Bayesian Programming Research Articles

Gene-level alignment of single-cell trajectories

AbstractSingle-cell data analysis can infer dynamic changes in cell populations, for example across time, space or in response to perturbation, thus deriving pseudotime trajectories. Current approaches comparing trajectories often use dynamic programming but are limited by assumptions such as the existence of a definitive match. Here we describe Genes2Genes, a Bayesian information-theoretic dynamic programming framework for aligning single-cell trajectories. It is able to capture sequential matches and mismatches of individual genes between a reference and query trajectory, highlighting distinct clusters of alignment patterns. Across both real world and simulated datasets, it accurately inferred alignments and demonstrated its utility in disease cell-state trajectory analysis. In a proof-of-concept application, Genes2Genes revealed that T cells differentiated in vitro match an immature in vivo state while lacking expression of genes associated with TNF signaling. This demonstrates that precise trajectory alignment can pinpoint divergence from the in vivo system, thus guiding the optimization of in vitro culture conditions.

Static Posterior Inference of Bayesian Probabilistic Programming via Polynomial Solving

Static Posterior Inference of Bayesian Probabilistic Programming via Polynomial Solving

Controlled Discovery and Localization of Signals via Bayesian Linear Programming

Scientists often must simultaneously localize and discover signals. For instance, in genetic fine-mapping, high correlations between nearby genetic variants make it hard to identify the exact locations of causal variants. So the statistical task is to output as many disjoint regions containing a signal as possible, each as small as possible, while controlling false positives. Similar problems arise, for example, when locating stars in astronomical surveys and in changepoint detection. Common Bayesian approaches to these problems involve computing a posterior distribution over signal locations. However, existing procedures to translate these posteriors into credible regions for the signals fail to capture all the information in the posterior, leading to lower power and (sometimes) inflated false discoveries. We introduce Bayesian Linear Programming (BLiP), which can efficiently convert any posterior distribution over signals into credible regions for signals. BLiP overcomes an extremely high-dimensional and nonconvex problem to verifiably nearly maximize expected power while controlling false positives. Applying BLiP to existing state-of-the-art analyses of UK Biobank data (for genetic fine-mapping) and the Sloan Digital Sky Survey (for astronomical point source detection) increased power by 30%–120% in just a few minutes of additional computation. BLiP is implemented in pyblip (Python) and blipr (R). Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Managing supply chain risks with dual sourcing: Bayesian learning of censored supply capacity

Dual sourcing is a common strategy for mitigating supply chain risks, with supply capacity uncertainty being one of the most significant risks. However, firms often have limited knowledge about uncertain supply capacity, and can only observe it when the delivered quantity is less than the ordered amount. This leads to censored supply capacity observations. To investigate how supply capacity learning affects dual-sourcing decisions, we propose a model of a firm sourcing non-storable substitute products from an expensive reliable supplier and an inexpensive unreliable supplier to meet a deterministic time-varying demand over a finite horizon. The unreliable supplier has a random capacity with an unknown parameter. The firm updates its belief about this parameter using Bayesian learning and adapts its sourcing decisions to balance the tradeoff between maximizing current profits based on its current belief (exploitation) and increasing the precision of its belief for future periods (exploration). We use Bayesian dynamic programming to determine the firm’s myopic sourcing policy and partially characterize its optimal sourcing policy. We apply a state space reduction technique to Weibull distributed capacity, resulting in an inductively solvable optimization problem. The optimal sourcing policy can be obtained under a condition that ensures splitting the demand between the suppliers is optimal. For exponentially distributed capacity, the optimization problem has a recursive closed-form solution, and the optimal sourcing policy is such that the optimal order quantity from the reliable (unreliable) supplier is smaller (greater) than the myopic quantity. We complement our analytical results with numerical examples and discuss extensions to random demand and inventory carryover.

Affine Monads and Lazy Structures for Bayesian Programming

We show that streams and lazy data structures are a natural idiom for programming with infinite-dimensional Bayesian methods such as Poisson processes, Gaussian processes, jump processes, Dirichlet processes, and Beta processes. The crucial semantic idea, inspired by developments in synthetic probability theory, is to work with two separate monads: an affine monad of probability, which supports laziness, and a commutative, non-affine monad of measures, which does not. (Affine means that T (1)≅ 1.) We show that the separation is important from a decidability perspective, and that the recent model of quasi-Borel spaces supports these two monads. To perform Bayesian inference with these examples, we introduce new inference methods that are specially adapted to laziness; they are proven correct by reference to the Metropolis-Hastings-Green method. Our theoretical development is implemented as a Haskell library, LazyPPL.

Closed-Loop Dynamic Blending Optimization Based on Variational Bayesian and its Application in Industry

There exist uncertainties in raw material components and distribution parameters in blending process. Considering the dynamic uncertainty of distribution parameters and the dynamic change of raw material inventory, a general closed-loop dynamic blending optimization (CDBO) is proposed. The proposed method consists of both data-driven system and model-based system. In the data-driven system, the feedback optimization problem is reconstructed into a linear regression problem, and the variational Bayesian is proposed to obtain the mean and variance of the distribution. Then, according to the incoming raw materials, the mean value of each distribution parameter is adjusted by expert rules. Last, in the model-based system, blending ratio is obtained by chance-constrained programming. To verify the effectiveness of CDBO, a detailed derivation process of variable Bayesian, expert rule and chance-constrained programming are established for zinc smelting process. Numerical studies and an industrial application for zinc smelting process are presented to demonstrate the advantages of the proposed method. Compared with the manual blending, the volatility index of chemical tests data is greatly reduced and the compliance rates of zinc and lead is increased by 6.7% and 3.3%, respectively.

Bayesian modeling of risks of various origin

Background. Financial as well as many other types of risks are inherent to all types of human activities. The problem isto construct adequate mathematical description for the formal representation of risks selected and to use it for possibleloss estimation and forecasting. The loss estimation can be based upon processing available data and relevant expertestimates characterizing history and current state of the processes considered. An appropriate instrumentation for mod-elling and estimating risks of possible losses provides probabilistic approach including Bayesian techniques known todayas Bayesian programming methodology.Objective. The purpose of the paper is to perform overview of some Bayesian data processing methods providing apossibility for constructing models of financial risks selected. To use statistical data to develop a new model of Bayesiantype so that to describe formally operational risk that can occur in the information processing procedures.Methods. The methods used for data processing and model constructing refer to Bayesian programming methodology.Also Bayes theorem was directly applied to operational risk assessment in its formulation for discrete events and discreteparameters.Results. The proposed approach to modelling was applied to building a model of operational risk associated with in-correct information processing. To construct and apply the model to risk estimation the risk problem was analysed,appropriate variables were selected, and prior conditional probabilities were estimated. Functioning of the models con-structed was demonstrated with illustrative examples.Conclusions. Modelling and estimating financial and other type of risks is important practical problem that can besolved using the methodology of Bayesian programming providing the possibility for identification and taking intoconsideration uncertainties of data and expert estimates. The risk model constructed with the methodology proposedillustrates the possibilities of applying the Bayesian methods to solving the risk estimation problems

Bayesian dynamic programming approach for tracking time-varying model properties in SHM

Structural health monitoring (SHM) quite often involves continually tracking the temporal variation of some properties of interest, where statistical information about variations under normal situation can be obtained and potential anomalies may be detected for further attention. Structural-related properties such as natural frequency and stiffness often need to be identified with a physics-based model that relates them to measured data. Conventional approaches empirically divide data into non-overlapping segments with equal lengths and identify the model parameters within each segment based on a time-invariant model. Potentially time-varying model properties are then investigated based on the variation of identified results from one data segment to another. Challenges do exist, e.g., how to choose segment length to balance modeling error and identification accuracy, how to set criterion for anomaly detection, etc., which should be addressed with proper extraction of probabilistic information from data. In this work, the SHM problem is formulated as a Bayesian model selection problem, where the data can be ‘partitioned’ in an arbitrary manner, whose optimal choice, and hence points of significant change, are determined together with the model inference process by maximizing the probabilistic evidence supported by data. An efficient algorithm based on dynamic programming is proposed to determine the optimal partitioning and associated piecewise-constant properties, which is otherwise computationally prohibitive. The methodology is applied to tracking modal properties, e.g., natural frequency and damping ratio, of structures using output-only ambient vibration data. It is investigated with synthetic data and then applied to field data of a tall building during a typhoon event.

Simulating Financial Risks on the Basis of Statistical Assessment Methods

Financial risks are summed up by type and form, macro- and micro-level. The importance of monitoring of financial risks in insurance, banking, monetary activities and various business processes is substantiated. It is emphasized that the sound assessment of risks constitutes an important tool in risk management. The essence, advantages and disadvantages of selected methods for financial risk assessment is shown: Value-at-Risk, Monte Carlo method, methods based on IRB approach, Shortfall, LDA, methods using Bayesian programming.
 The importance of statistical methods for the assessment of financial risks like non-parametric techniques of Kaplan – Meier and Cox proportional hazards model is substantiated. It is emphasized that the cumulative hazard function by Cox model reflects the cumulative level of bank losses, hence, its application in risk assessment is capable to protect and warn the bank about a potential threat. Kaplan – Meier method allows to assess the probability of risk occurrence and risk level in various client groups, which is a necessary component of risk monitoring. But its drawback is its incapability to account for several risks at the same time. In view of this, a sounder method for risk assessment is Cox proportional hazard regression. The input data for constructing this regression can include both categorical and continuous variables, thus enabling for accounting of a multiplicity of risk-related factors. 
 It is concluded that Kaplan – Meier method should be used with caution, because the survival function may overvalue the probability of occurrence of “critical” event, depending on the internal nature of data and their individual variances. Hence, applications of semiparametric techniques of Cox proportional hazards model should be an alternative approach to the survival analysis.

Bayesian machine learning analysis of single-molecule fluorescence colocalization images.

Multi-wavelength single-molecule fluorescence colocalization (CoSMoS) methods allow elucidation of complex biochemical reaction mechanisms. However, analysis of CoSMoS data is intrinsically challenging because of low image signal-to-noise ratios, non-specific surface binding of the fluorescent molecules, and analysis methods that require subjective inputs to achieve accurate results. Here, we use Bayesian probabilistic programming to implement Tapqir, an unsupervised machine learning method that incorporates a holistic, physics-based causal model of CoSMoS data. This method accounts for uncertainties in image analysis due to photon and camera noise, optical non-uniformities, non-specific binding, and spot detection. Rather than merely producing a binary 'spot/no spot' classification of unspecified reliability, Tapqir objectively assigns spot classification probabilities that allow accurate downstream analysis of molecular dynamics, thermodynamics, and kinetics. We both quantitatively validate Tapqir performance against simulated CoSMoS image data with known properties and also demonstrate that it implements fully objective, automated analysis of experiment-derived data sets with a wide range of signal, noise, and non-specific binding characteristics.

Combine Statistical Thinking With Open Scientific Practice: A Protocol of a Bayesian Research Project

Current developments in the statistics community suggest that modern statistics education should be structured holistically, that is, by allowing students to work with real data and to answer concrete statistical questions, but also by educating them about alternative frameworks, such as Bayesian inference. In this article, we describe how we incorporated such a holistic structure in a Bayesian research project on ordered binomial probabilities. The project was conducted with a group of three undergraduate psychology students who had basic knowledge of Bayesian statistics and programming, but lacked formal mathematical training. The research project aimed to (1) convey the basic mathematical concepts of Bayesian inference; (2) have students experience the entire empirical cycle including collection, analysis, and interpretation of data and (3) teach students open science practices.

Forecasting urban water consumption using bayesian networks and gene expression programming

Forecasting urban water consumption using bayesian networks and gene expression programming

A Two-Stage Fuzzy Optimization Model for Urban Land Use: A Case Study of Chongzhou City

Under the background of New-type Urbanization, with the continuous advancement of urbanization and the all-round development of cities, all kinds of demands are also rising. In the case of demand, it is difficult to quickly adjust from the land supply side and to guide the optimization of the structure and layout of land use is one of the methods to achieve this based on the current situation and shortage of urban land use structure and spatial arrangement. Because of the complexity, uncertainty and dynamics of the land use system, it is necessary to use an uncertain model to accurately describe and propose the approximate optimal solution, so this study analyzes the influencing mechanism of land use and optimize the land use structure under uncertainties by using a Bayesian network and fuzzy mathematical programming. Based on the results of the two stages of analysis, the cellular automata simulation is completed. The framework is applied to Chongzhou city in western China. The results indicated that the optimal land space for cultivated land is in the middle and the south based on the joint influence probability of arable land and urban construction land. The conversion probability of the area near the east is low, and the joint impact probability of construction land in all areas is generally similar except for the western protection area. After the optimization of the fuzzy planning, the optimal construction land scale is 69.42 km2. Under the condition that the cultivated land’s red line is guaranteed, there is still 98.87 km2 of space for the increase in cultivated land. It is found through simulation that the increase in construction land would occur in the central and western parts of Chongzhou, which may be caused by the urban siphon effect. According to Monte Carlo verification, when the conversion probability exceeds 50%, the cultivated land could be turned into urban construction land, with an accuracy of 91.99%. Therefore, this proposed framework is helpful to understand the process of land use and provides a reference for making scientific and reasonable territorial spatial planning and guiding land use practice under uncertainties.

A Bayesian econometrics and risk programming approach for analysing the impact of decoupled payments in the European Union*

We estimate a risk‐based programming, individual farm model and apply it to study the wealth effects of crop‐related, decoupled direct payments under the European Union (EU) Common Agricultural Policy. The model expands on previous work on estimating risk‐based programming models by applying a robust Bayesian econometric framework. The results indicate that the wealth effect varies greatly between individual farms, but that its impact on aggregate crop production is small. For larger farms, in particular, removing the decoupled payments, while keeping total land constant, increases the diversity of the cropping plan.

The geometry of Bayesian programming

AbstractWe give two geometry of interaction models for a typed λ-calculus with recursion endowed with operators for sampling from a continuous uniform distribution and soft conditioning, namely a paradigmatic calculus for higher-order Bayesian programming. The models are based on the category of measurable spaces and partial measurable functions, and the category of measurable spaces and s-finite kernels, respectively. The former is proved adequate with respect to both a distribution-based and a sampling-based operational semantics, while the latter is proved adequate with respect to a sampling-based operational semantics.

Increasing Interpretability of Bayesian Probabilistic Programming Models Through Interactive Representations

Bayesian probabilistic modeling is supported by powerful computational tools like probabilistic programming and efficient Markov Chain Monte Carlo (MCMC) sampling. However, the results of Bayesian inference are challenging for users to interpret in tasks like decision-making under uncertainty or model refinement. Decision-makers need simultaneous insight into both the model's structure and its predictions, including uncertainty in inferred parameters. This enables better assessment of the risk overall possible outcomes compatible with observations and thus more informed decisions. To support this, we see a need for visualization tools that make probabilistic programsinterpretableto reveal the interdependencies in probabilistic models and their inherent uncertainty. We propose the automatic transformation of Bayesian probabilistic models, expressed in a probabilistic programming language, into an interactive graphical representation of the model's structure at varying levels of granularity, with seamless integration of uncertainty visualization. This interactive graphical representation supports the exploration of the prior and posterior distribution of MCMC samples. The interpretability of Bayesian probabilistic programming models is enhanced through the interactive graphical representations, which provide human users with moreinformative, transparent, andexplainableprobabilistic models. We present a concrete implementation that translates probabilistic programs to interactive graphical representations and show illustrative examples for a variety of Bayesian probabilistic models.

Bayesian differential programming for robust systems identification under uncertainty.

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator-prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj.

Methodology of classification and recognition of the radar emission sources based on Bayesian programming

This study considers the Bayesian programming methodology for recognition and classification of radio emission sources. A mathematical model of Bayesian programming proposes forming a family of probability distributions based on known parameters contained in a training sample (database). The correlations between the object classes have been estimated according discussed methodology. The received assessment has been used to separate procedures of recognition and classification of radar emission sources. The simulation of methodology has carried out for four parameters of radar signals (frequency range, pulse width, pulse repetition interval and radar rotation frequency) by used database with 346 classes and 16 types of radar. Based on the existing database of radar emission sources, it is possible to predict the probability of class recognition for the general population of objects, if its distribution is known. This study demonstrates the consistency of the Bayesian programming methodology for object identification in ELINT systems.

Variance Regularization in Sequential Bayesian Optimization

Sequential Bayesian optimization constitutes an important and broad class of problems where model parameters are not known a priori but need to be learned over time using Bayesian updating. It is known that the solution to these problems can in principle be obtained by solving the Bayesian dynamic programming (BDP) equation. Although the BDP equation can be solved in certain special cases (for example, when posteriors have low-dimensional representations), solving this equation in general is computationally intractable and remains an open problem. A second unresolved issue with the BDP equation lies in its (rather generic) interpretation. Beyond the standard narrative of balancing immediate versus future costs—an interpretation common to all dynamic programs with or without learning—the BDP equation does not provide much insight into the underlying mechanism by which sequential Bayesian optimization trades off between learning (exploration) and optimization (exploitation), the distinguishing feature of this problem class. The goal of this paper is to develop good approximations (with error bounds) to the BDP equation that help address the issues of computation and interpretation. To this end, we show how the BDP equation can be represented as a tractable single-stage optimization problem that trades off between a myopic term and a “variance regularization” term that measures the total solution variability over the remaining planning horizon. Intuitively, the myopic term can be regarded as a pure exploitation objective that ignores the impact of future learning, whereas the variance regularization term captures a pure exploration objective that only puts value on solutions that resolve statistical uncertainty. We develop quantitative error bounds for this representation and prove that the error tends to zero like o(n-1) almost surely in the number of stages n, which as a corollary, establishes strong consistency of the approximate solution.

Information theory for ranking and selection

AbstractWe study the classical ranking and selection problem, where the ultimate goal is to find the unknown best alternative in terms of the probability of correct selection or expected opportunity cost. However, this paper adopts an alternative sampling approach to achieve this goal, where sampling decisions are made with the objective of maximizing information about the unknown best alternative, or equivalently, minimizing its Shannon entropy. This adaptive learning is formulated via a Bayesian stochastic dynamic programming problem, by which several properties of the learning problem are presented, including the monotonicity of the optimal value function in an information‐seeking setting. Since the state space of the stochastic dynamic program is unbounded in the Gaussian setting, a one‐step look‐ahead approach is used to develop a policy. The proposed policy seeks to maximize the one‐step information gain about the unknown best alternative, and therefore, it is called information gradient (IG). It is also proved that the IG policy is consistent, that is, as the sampling budget grows to infinity, the IG policy finds the true best alternative almost surely. Later, a computationally efficient estimate of the proposed policy, called approximated information gradient (AIG), is introduced and in the numerical experiments its performance is tested against recent benchmarks alongside several sensitivity analyses. Results show that AIG performs competitively against other algorithms from the literature.