Deterministic Inference Research Articles

Free-field characterization of locally reacting sound absorbers using Bayesian inference with sequential frequency transfer

The in situ absorption characteristics of sound-absorbing samples are generally estimated inversely from sound field measurements above the sample’s surface. However, the entire measurement process is subject to numerous sources of uncertainty, which cannot be accounted for using deterministic inference methods. This paper introduces a Bayesian framework, including sequential frequency transfer to infer point estimates and uncertainty ranges of the frequency-dependent surface admittance, the corresponding sound absorption coefficient and the associated measurement and model uncertainty. The inference is conducted in two phases: in the initial phase, the absorption characteristics and the measurement and model uncertainty at a predefined initial frequency are inferred based on a weakly informative prior. Extensive prior predictive checks are conducted to find the best trade-off between a weak level of informativeness of the prior and physically meaningful predictions of the sound field above the absorber. In the sequential phase, the posterior mean at an analyzed frequency serves as the prior mean at the subsequent frequency. The framework is applied to simulated as well as real free-field measurements of the specific impedance above two different rock wool samples. The measurements are conducted with a combined pressure-particle velocity probe and the samples are assumed to exhibit local reaction behavior. For the investigated samples, a minimum number of four individual measurements of the specific impedance already allows for an accurate estimation of the frequency-dependent absorption characteristics and the associated measurement and model uncertainty, particularly for frequencies above 600 Hz. Due to the proposed sequential frequency transfer, the computational effort of the sampling-based Bayesian inference at a discrete frequency is significantly reduced. This paves the way for efficient inference of frequency-dependent model parameters while overcoming the curse of dimensionality associated with the inference of stochastic processes. The proposed framework lays the foundation for accurate and efficient inference of frequency-dependent parameters in models beyond the field of acoustics.

A Bayesian approach to differential edges with probabilistic interactions: applications in association and classification

AbstractMotivationDifferential network (D-Net) analysis has attracted great attention in systems biology for its ability to identify genetic variations in response to different conditions. Current approaches either estimate the condition-specific networks separately followed by post-procedures to determine the differential edges or estimate the D-Net directly. Both types of analysis overlook the probabilistic inference and can only provide deterministic inference of the edges.ResultsHere, we propose a Bayesian solution and translate the probabilistic estimation in the regression model to an inferential D-Net analysis for genetic association and classification studies. The proposed PRobabilistic Interaction for Differential Edges (PRIDE) focuses on inferring the D-Net with uncertainty so that the existence of the differential edges can be evaluated with probability and even prioritized if comparison among these edges is of interest. The performance of the proposed model is compared with state-of-the-art methods in simulations and is demonstrated in glioblastoma and breast cancer studies. The proposed PRIDE performs comparably to or outperforms most existing tools under deterministic evaluation criteria. Additionally, it offers the unique advantages, including prioritizing the differential edges with probabilities, highlighting the relative importance of hub nodes, and identifying potential sub-networks in a D-Net.Availability and implementationAll the data analyzed in this research can be downloaded at https://xenabrowser.net/datapages/. The R code for implementing PRIDE is available at https://github.com/YJGene0806/PRIDE_Code.

Probabilistic programming with stochastic variational message passing

Stochastic approximation methods for variational inference have recently gained popularity in the probabilistic programming community since these methods are amenable to automation and allow online, scalable, and universal approximate Bayesian inference. Unfortunately, common Probabilistic Programming Languages (PPLs) with stochastic approximation engines lack the efficiency of message passing-based inference algorithms with deterministic update rules such as Belief Propagation (BP) and Variational Message Passing (VMP). Still, Stochastic Variational Inference (SVI) and Conjugate-Computation Variational Inference (CVI) provide principled methods to integrate fast deterministic inference techniques with broadly applicable stochastic approximate inference. Unfortunately, implementation of SVI and CVI necessitates manually driven variational update rules, which does not yet exist in most PPLs. In this paper, we cast SVI and CVI explicitly in a message passing-based inference context. We provide an implementation for SVI and CVI in ForneyLab, which is an automated message passing-based probabilistic programming package in the open source Julia language. Through a number of experiments, we demonstrate how SVI and CVI extends the automated inference capabilities of message passing-based probabilistic programming.

Gaussian Belief Propagation Solvers for Nonsymmetric Systems of Linear Equations

In this paper, we argue for the utility of deterministic inference in the classical problem of numerical linear algebra, that of solving a linear system. We show how the Gaussian belief propagation solver, known to work for symmetric matrices can be modified to handle nonsymmetric matrices. Furthermore, we introduce a new algorithm for matrix inversion that corresponds to the generalized belief propagation derived from the cluster variation method (or Kikuchi approximation). We relate these algorithms to LU and block LU decompositions and provide certain guarantees based on theorems from the theory of belief propagation. All proposed algorithms are compared with classical solvers (e.g., Gauss-Seidel, BiCGSTAB) with application to linear elliptic equations. We also show how the Gaussian belief propagation can be used as multigrid smoother, resulting in a substantially more robust solver than the one based on the Gauss-Seidel iterative method.

Deterministic Inference of Topic Models via Maximal Latent State Replication

Probabilistic topic models, such as latent Dirichlet allocation (LDA), are often used to discover hidden semantic structure of a collection of documents. In recent years, various inference algorithms have been developed to cope with learning of topic models, among which Gibbs sampling methods remain a popular choice. In this paper, we aim to improve the inference of topic models based on the Gibbs sampling framework. We extend a state augmentation based Gibbs sampling method by maximizing the replications of latent states, and propose a new generic deterministic inference method, named maximal latent state replication (MAX), for learning of a family of probabilistic topic models. One key benefit of the proposed method lies in the deterministic nature for inference, which may help to improve its running efficiency as well as predictive perplexity. We have conducted extensive experiments on real-life publicly available datasets, and the results have validated that our proposed method MAX significantly outperforms state-of-the-art baselines for inference of existing well-known topic models.

Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior

Although neural machine translation models reached high translation quality, the autoregressive nature makes inference difficult to parallelize and leads to high translation latency. Inspired by recent refinement-based approaches, we propose LaNMT, a latent-variable non-autoregressive model with continuous latent variables and deterministic inference procedure. In contrast to existing approaches, we use a deterministic inference algorithm to find the target sequence that maximizes the lowerbound to the log-probability. During inference, the length of translation automatically adapts itself. Our experiments show that the lowerbound can be greatly increased by running the inference algorithm, resulting in significantly improved translation quality. Our proposed model closes the performance gap between non-autoregressive and autoregressive approaches on ASPEC Ja-En dataset with 8.6x faster decoding. On WMT'14 En-De dataset, our model narrows the gap with autoregressive baseline to 2.0 BLEU points with 12.5x speedup. By decoding multiple initial latent variables in parallel and rescore using a teacher model, the proposed model further brings the gap down to 1.0 BLEU point on WMT'14 En-De task with 6.8x speedup.

Description and spatial inference of soil drainage using matrix soil colours in the Lower Hunter Valley, New South Wales, Australia.

Soil colour is often used as a general purpose indicator of internal soil drainage. In this study we developed a necessarily simple model of soil drainage which combines the tacit knowledge of the soil surveyor with observed matrix soil colour descriptions. From built up knowledge of the soils in our Lower Hunter Valley, New South Wales study area, the sequence of well-draining → imperfectly draining → poorly draining soils generally follows the colour sequence of red → brown → yellow → grey → black soil matrix colours. For each soil profile, soil drainage is estimated somewhere on a continuous index of between 5 (very well drained) and 1 (very poorly drained) based on the proximity or similarity to reference soil colours of the soil drainage colour sequence. The estimation of drainage index at each profile incorporates the whole-profile descriptions of soil colour where necessary, and is weighted such that observation of soil colour at depth and/or dominantly observed horizons are given more preference than observations near the soil surface. The soil drainage index, by definition disregards surficial soil horizons and consolidated and semi-consolidated parent materials. With the view to understanding the spatial distribution of soil drainage we digitally mapped the index across our study area. Spatial inference of the drainage index was made using Cubist regression tree model combined with residual kriging. Environmental covariates for deterministic inference were principally terrain variables derived from a digital elevation model. Pearson’s correlation coefficients indicated the variables most strongly correlated with soil drainage were topographic wetness index (−0.34), mid-slope position (−0.29), multi-resolution valley bottom flatness index (−0.29) and vertical distance to channel network (VDCN) (0.26). From the regression tree modelling, two linear models of soil drainage were derived. The partitioning of models was based upon threshold criteria of VDCN. Validation of the regression kriging model using a withheld dataset resulted in a root mean square error of 0.90 soil drainage index units. Concordance between observations and predictions was 0.49. Given the scale of mapping, and inherent subjectivity of soil colour description, these results are acceptable. Furthermore, the spatial distribution of soil drainage predicted in our study area is attuned with our mental model developed over successive field surveys. Our approach, while exclusively calibrated for the conditions observed in our study area, can be generalised once the unique soil colour and soil drainage relationship is expertly defined for an area or region in question. With such rules established, the quantitative components of the method would remain unchanged.

Semiparametric Regression Analysis via Infer.NET

We provide several examples of Bayesian semiparametric regression analysis via the Infer.NET package for approximate deterministic inference in Bayesian models. The examples are chosen to encompass a wide range of semiparametric regression situations. Infer.NET is shown to produce accurate inference in comparison with Markov chain Monte Carlo via the BUGS package, but to be considerably faster. Potentially, this contribution represents the start of a new era for semiparametric regression, where large and complex analyses are performed via fast Bayesian inference methodology and software, mainly being developed within Machine Learning.

Simultaneous Sparse Bayesian Learning With Partially Shared Supports

Many simultaneous sparse estimation approaches focus on joint estimation of multiple sparse vectors with a common support from given linear observations, which is however too strict in some real applications. In this letter, instead of forcing a common support, a general joint-sparse model is considered where sparse vectors just have partially shared supports. This letter provides a Bayesian approach that extends the sparse Bayesian learning for such a joint-sparse model. The proposed Bayesian framework is composed of a number of parametric models that correspond to distinct patterns of partially shared supports, and an efficient deterministic Bayesian inference algorithm is developed. The proposed method is characterized as a tuning parameter-free approach, which can effectively infer the underlying sparse structure and also the noise level. Experimental results show the superiority of the proposed approach.

Time-Varying Mixtures of Markov Chains: An Application to Road Traffic Modeling

Time-varying mixture models are useful for representing complex, dynamic distributions. Components in the mixture model can appear and disappear, and persisting components can evolve. This allows great flexibility in streaming data applications where the model can be adjusted as new data arrives. Fitting a mixture model with computational guarantees which can meet real-time requirements is challenging with existing algorithms, especially when the model order can vary with time. Existing approximate inference methods may require multiple restarts to search for a good local solution. Monte-Carlo methods can be used to jointly estimate the model order and model parameters, but when the distribution of each mixand has a high-dimensional parameter space, they suffer from the curse of dimensionality and and from slow convergence. This paper proposes a generative model for time-varying mixture models, tailored for mixtures of discrete-time Markov chains. A novel, deterministic inference procedure is introduced and is shown to be suitable for applications requiring real-time estimation, and the method is guaranteed to converge at each time step. As a motivating application, we model and predict traffic patterns in a transportation network. Experiments illustrate the performance of the scheme and offer insights regarding tuning of the algorithm parameters. The experiments also investigate the predictive power of the proposed model compared to less complex models and demonstrate the superiority of the mixture model approach for prediction of traffic routes in real data.

A Survey of Stochastic Simulation and Optimization Methods in Signal Processing

Modern signal processing (SP) methods rely very heavily on probability and statistics to solve challenging SP problems. SP methods are now expected to deal with ever more complex models, requiring ever more sophisticated computational inference techniques. This has driven the development of statistical SP methods based on stochastic simulation and optimization. Stochastic simulation and optimization algorithms are computationally intensive tools for performing statistical inference in models that are analytically intractable and beyond the scope of deterministic inference methods. They have been recently successfully applied to many difficult problems involving complex statistical models and sophisticated (often Bayesian) statistical inference techniques. This survey paper offers an introduction to stochastic simulation and optimization methods in signal and image processing. The paper addresses a variety of high-dimensional Markov chain Monte Carlo (MCMC) methods as well as deterministic surrogate methods, such as variational Bayes, the Bethe approach, belief and expectation propagation and approximate message passing algorithms. It also discusses a range of optimization methods that have been adopted to solve stochastic problems, as well as stochastic methods for deterministic optimization. Subsequently, areas of overlap between simulation and optimization, in particular optimization-within-MCMC and MCMC-driven optimization are discussed.

Deterministic inference for stochastic systems using multiple shooting and a linear noise approximation for the transition probabilities.

Estimating model parameters from experimental data is a crucial technique for working with computational models in systems biology. Since stochastic models are increasingly important, parameter estimation methods for stochastic modelling are also of increasing interest. This study presents an extension to the ‘multiple shooting for stochastic systems (MSS)’ method for parameter estimation. The transition probabilities of the likelihood function are approximated with normal distributions. Means and variances are calculated with a linear noise approximation on the interval between succeeding measurements. The fact that the system is only approximated on intervals which are short in comparison with the total observation horizon allows to deal with effects of the intrinsic stochasticity. The study presents scenarios in which the extension is essential for successfully estimating the parameters and scenarios in which the extension is of modest benefit. Furthermore, it compares the estimation results with reversible jump techniques showing that the approximation does not lead to a loss of accuracy. Since the method is not based on stochastic simulations or approximative sampling of distributions, its computational speed is comparable with conventional least‐squares parameter estimation methods.

Bayesian Group-Sparse Modeling and Variational Inference

In this paper, we present a general class of multivariate priors for group-sparse modeling within the Bayesian framework. We show that special cases of this class correspond to multivariate versions of several classical priors used for sparse modeling. Hence, this general prior formulation is helpful in analyzing the properties of different modeling approaches and their connections. We derive the estimation procedures with these priors using variational inference for fully Bayesian estimation. In addition, we discuss the differences between the proposed inference and deterministic inference approaches with these priors. Finally, we show the flexibility of this modeling by considering several extensions such as multiple measurements, within-group correlations, and overlapping groups.

In the News

Recent experimental AI-enhanced applications focus on putting the user in control of human-computer interface systems. Examples include microsearch engines, which use reasoning techniques such as probabilistic inference, deterministic inference, optimization, estimation, and configuration to fine tune their results, and an Australian intelligent grid system that lets homeowners decide how to optimize their energy use. Researchers are also exploring utilizing user input to help email users separate junk from important communications and investigating more interactive videogame nonplayer characters. Other AI apps in the works include intelligent transportation systems and applying AI methodologies to sports analysis.

Autonomic Computing for a Complex Problem of Experimental Physics

Standard experimental data analysis is based mainly on conventional, deterministic inference. The complexity of modern physics problems has become so large that new ideas in the field are received with the highest of appreciation. In this paper, the author has analyzed the problem of contemporary high-energy physics concerning the estimation of some parameters of the observed complex phenomenon. This article confronts the Natural and Artificial Networks performance with the standard statistical method of the data analysis and minimization. The general concept of the relations between CI and standard (external) classical and modern informatics was realized and studied by utilizing of Natural Neural Networks (NNN), Artificial Neural Networks (ANN) and MINUIT minimization package from CERN. The idea of Autonomic Computing was followed by using brains of high school students involved in the Roland Maze Project. Some preliminary results of the comparison are given and discussed.

Bayesian bivariate meta‐analysis of diagnostic test studies using integrated nested Laplace approximations

For bivariate meta-analysis of diagnostic studies, likelihood approaches are very popular. However, they often run into numerical problems with possible non-convergence. In addition, the construction of confidence intervals is controversial. Bayesian methods based on Markov chain Monte Carlo (MCMC) sampling could be used, but are often difficult to implement, and require long running times and diagnostic convergence checks. Recently, a new Bayesian deterministic inference approach for latent Gaussian models using integrated nested Laplace approximations (INLA) has been proposed. With this approach MCMC sampling becomes redundant as the posterior marginal distributions are directly and accurately approximated. By means of a real data set we investigate the influence of the prior information provided and compare the results obtained by INLA, MCMC, and the maximum likelihood procedure SAS PROC NLMIXED. Using a simulation study we further extend the comparison of INLA and SAS PROC NLMIXED by assessing their performance in terms of bias, mean-squared error, coverage probability, and convergence rate. The results indicate that INLA is more stable and gives generally better coverage probabilities for the pooled estimates and less biased estimates of variance parameters. The user-friendliness of INLA is demonstrated by documented R-code.

Implementation of multistep fuzzy reasoning shell

AbstractRecently, there have been many trials conducted to develop fuzzy inference systems that can handle the subjective decision of the expert. A problem in the fuzzy control system is that it is an inference composed of only a single step, and cannot be applied to the decision‐making assist or diagnosis, where the inference is made in multisteps. Another point is that the rules of the inference cannot be described for each knowledge source, which has been the conventional inference shell (deterministic inference shell).This paper proposes a fuzzy inference shell which can realize the multistep inference using the blackboard model. The proposed shell uses its own rule description language to represent the knowledge, and the description can be made by combining the new method with the conventional deterministic inference.Using the divided knowledge blocks, the multi‐step fuzzy inference can be realized using the blackboard model. The search function also is provided that can track the inference path in the multistep inference. This paper describes the basic configuration of such a shell.