Evidence For Model Research Articles

Learning Bayesian Network Classifiers to Minimize the Class Variable Parameters

This study proposes and evaluates a new Bayesian network classifier (BNC) having an I-map structure with the fewest class variable parameters among all structures for which the class variable has no parent. Moreover, a new learning algorithm to learn our proposed model is presented. The proposed method is guaranteed to obtain the true classification probability asymptotically. Moreover, the method has lower computational costs than those of exact learning BNC using marginal likelihood. Comparison experiments have demonstrated the superior performance of the proposed method.

Scores for Learning Discrete Causal Graphs with Unobserved Confounders

Structural learning is arguably one of the most challenging and pervasive tasks found throughout the data sciences. There exists a growing literature that studies structural learning in non-parametric settings where conditional independence constraints are taken to define the equivalence class. In the presence of unobserved confounders, it is understood that non-conditional independence constraints are imposed over the observational distribution, including certain equalities and inequalities between functionals of the joint distribution. In this paper, we develop structural learning methods that leverage additional constraints beyond conditional independences. Specifically, we first introduce a score for arbitrary graphs combining Watanabe's asymptotic expansion of the marginal likelihood and new bounds over the cardinality of the exogenous variables. Second, we show that the new score has desirable properties in terms of expressiveness and computability. In terms of expressiveness, we prove that the score captures distinct constraints imprinted in the data, including Verma's and inequalities'. In terms of computability, we show properties of score equivalence and decomposability, which allows, in principle, to break the problem of structural learning in smaller and more manageable pieces. Third, we implement this score using an MCMC sampling algorithm and test its properties in several simulation scenarios.

Disentangled Partial Label Learning

Partial label learning (PLL) induces a multi-class classifier from training examples each associated with a set of candidate labels, among which only one is valid. The formation of real-world data typically arises from heterogeneous entanglement of series latent explanatory factors, which are considered intrinsic properties for discriminating between different patterns. Though learning disentangled representation is expected to facilitate label disambiguation for partial-label (PL) examples, few existing works were dedicated to addressing this issue. In this paper, we make the first attempt towards disentangled PLL and propose a novel approach named TERIAL, which makes predictions according to derived disentangled representation of instances and label embeddings. The TERIAL approach formulates the PL examples as an undirected bipartite graph where instances are only connected with their candidate labels, and employs a tailored neighborhood routing mechanism to yield disentangled representation of nodes in the graph. Specifically, the proposed routing mechanism progressively infers the explanatory factors that contribute to the edge between adjacent nodes and augments the representation of the central node with factor-aware embedding information propagated from specific neighbors simultaneously via iteratively analyzing the promising subspace clusters formed by the node and its neighbors. The estimated labeling confidence matrix is also introduced to accommodate unreliable links owing to the inherent ambiguity of PLL. Moreover, we theoretically prove that the neighborhood routing mechanism will converge to the point estimate that maximizes the marginal likelihood of observed PL training examples. Comprehensive experiments over various datasets demonstrate that our approach outperforms the state-of-the-art counterparts.

Pairwise Likelihood Estimation of the 2PL Model with Locally Dependent Item Responses

The local independence assumption is crucial for the consistent estimation of item parameters in item response theory models. This article explores a pairwise likelihood estimation approach for the two-parameter logistic (2PL) model that treats the local dependence structure as a nuisance in the optimization function. Hence, item parameters can be consistently estimated without explicit modeling assumptions of the dependence structure. Two simulation studies demonstrate that the proposed pairwise likelihood estimation approach allows nearly unbiased and consistent item parameter estimation. Our proposed method performs similarly to the marginal maximum likelihood and pairwise likelihood estimation approaches, which also estimate the parameters for the local dependence structure.

Accommodating spatio-temporal dependency in airline demand modeling

The objective of the current study is to examine monthly air passenger departures at the airport level considering spatial interactions between airports. In this study, we develop a novel spatial grouped generalized ordered probit (SGGOP) model system of monthly air passenger departures at the airport level. Specifically, we estimate two variants of spatial models including spatial lag model and spatial error model. In the presence of repeated demand measures for the airports, we also consider temporal variations of spatial correlation effects among proximally located airports by employing space and time-based weight matrix. The proposed model is estimated using monthly air passenger departures for five years for 369 airports across the US. The proposed spatial model is implemented using composite marginal likelihood (CML) approach that offers a computationally feasible framework. From the estimation results, it is evident that air passenger departures at the airport level are influenced by different factors including MSA specific demographic characteristics, built environment characteristics, airport specific factors, spatial factors, and temporal factors. Moreover, spatial autocorrelation parameter is found to be significant validating our hypothesis of the presence of common unobserved factors associated with the spatial unit of analysis. In this study, we also perform a validation analysis to examine the predictive performance of the proposed spatial models. The results highlight the superiority of spatial error model compared to spatial lag model and the independent model that ignores the spatial interactions. Finally, we undertake an elasticity analysis to quantify the impact of the independent variables.

Performance of an approximate Bayes factor based detector for environmentally informed active sonar

Developed here are a space of approximate Bayes Factor (BF) inference processors for high frequency broadband active sonar with short vertical arrays operating in shallow water ocean waveguides that exploit relatively depth invariant modes of propagation. The relevant information regarding the refractive media, rough surface and volume reverberation are incorporated to build the marginal likelihoods under each of the composite hypotheses of null and alternative. Acoustic scattering from a mobile object of interest under depth uncertainty characterizes the compound alternative hypothesis. Approximations are presented and inferences regarding the presence of the mobile body of interest are determined against a composite null hypothesis of reverberation and ambient acoustic noise. The approximate BF processor is shown to be a time-varying quadratic form of array observations over the beam-delay space. We demonstrate the sub-space processing of depth invariant modes at range and illustrate the BF inferential approach on a few representative waveguides. Performance in classic terms of probability of detection as a function of false alarm rate are presented. [This work supported by the Office of Naval Research.]

Sub‐Model Aggregation for Scalable Eigenvector Spatial Filtering: Application to Spatially Varying Coefficient Modeling

This study proposes a method for aggregating/synthesizing global and local sub‐models for fast and flexible spatial regression modeling. Eigenvector spatial filtering (ESF) was used to model spatially varying coefficients and spatial dependence in the residuals by sub‐model, while the generalized product‐of‐experts method was used to aggregate these sub‐models. The major advantages of the proposed method are as follows: (i) it is highly scalable for large samples in terms of accuracy and computational efficiency; (ii) it is easily implemented by estimating sub‐models independently first and aggregating/averaging them thereafter; and (iii) likelihood‐based inference is available because the marginal likelihood is available in closed‐form. The accuracy and computational efficiency of the proposed method are confirmed using Monte Carlo simulation experiments. This method was then applied to residential land price analysis in Japan. The results demonstrate the usefulness of this method for improving the interpretability of spatially varying coefficients. The proposed method is implemented in an R package spmoran.

Robust Gaussian Process Regression With Input Uncertainty: A PAC-Bayes Perspective.

The Gaussian process (GP) algorithm is considered as a powerful nonparametric-learning approach, which can provide uncertainty measurements on the predictions. The standard GP requires clearly observed data, unexpected perturbations in the input may lead to learned regression model mismatching. Besides, GP also suffers from the lack of good generalization performance guarantees. To deal with data uncertainty and provide a numerical generalization performance guarantee on the unknown data distribution, this article proposes a novel robust noisy input GP (NIGP) algorithm based on the probably approximately correct (PAC) Bayes theory. Furthermore, to reduce the computational complexity, we develop a sparse NIGP algorithm, and then develop a sparse PAC-Bayes NIGP approach. Compared with NIGP algorithms, instead of maximizing the marginal log likelihood, one can optimize the PAC-Bayes bound to pursue a tighter generalization error upper bound. Experiments verify that the NIGP algorithms can attain greater accuracy. Besides, the PAC-NIGP algorithms proposed herein can achieve both robust performance and improved generalization error upper bound in the face of both uncertain input and output data.

The flaw of averages: Bayes factors as posterior means of the likelihood ratio.

As an alternative to the Frequentist p-value, the Bayes factor (or ratio of marginal likelihoods) has been regarded as one of the primary tools for Bayesian hypothesis testing. In recent years, several researchers have begun to re-analyze results from prominent medical journals, as well as from trials for FDA-approved drugs, to show that Bayes factors often give divergent conclusions from those of p-values. In this paper, we investigate the claim that Bayes factors are straightforward to interpret as directly quantifying the relative strength of evidence. In particular, we show that for nested hypotheses with consistent priors, the Bayes factor for the null over the alternative hypothesis is the posterior mean of the likelihood ratio. By re-analyzing 39 results previously published in the New England Journal of Medicine, we demonstrate how the posterior distribution of the likelihood ratio can be computed and visualized, providing useful information beyond the posterior mean alone.

Multifidelity adaptive sequential Monte Carlo for geophysical inversion

SUMMARY In the context of Bayesian inversion, we consider sequential Monte Carlo (SMC) methods that provide an approximation of the posterior probability density function and the evidence (marginal likelihood). These particle approaches build a sequence of importance sampling steps between gradually tempered distributions evolving from the prior to the posterior PDF. To automate the definition of the tempering schedule, adaptive SMC (ASMC) allows tuning the temperature increments on-the-go. One general challenge in Bayesian inversions is the computational burden associated with expensive, high-fidelity forward solvers. Lower-fidelity surrogate models are interesting in this context as they can emulate the response of expensive forward solvers at a fraction of their cost. We consider surrogate modelling within ASMC and introduce first an approach involving surrogate modelling only, in which either prior samples are used to train the surrogate, or the surrogate model is retrained by updating the training set during the inversion. In our implementation, we rely on polynomial chaos expansions for surrogate modelling, principal component analysis for model parametrization and a ground-penetrating radar cross-hole tomography problem with either an eikonal or finite-difference time-domain solver as high-fidelity solver. We find that the method based on retraining the surrogate during the inversion outperforms the results obtained when only considering prior samples. We then introduce a computationally more expensive multifidelity approach including a transition to the high-fidelity forward solver at the end of the surrogate-based ASMC run leading to even more accurate results. Both methods result in speed-ups that are larger than one order of magnitude compared to standard high-fidelity ASMC inversion.

Variable selection through adaptive elastic net for proportional odds model

In this paper, we propose a method for fitting the proportional odds model by maximizing the marginal likelihood while incorporating an elastic net penalty. We assign adaptive weights to different coefficients, allowing important variables to receive smaller penalties and be more protectively retained in the final model, while unimportant variables receive larger penalties and are more likely to be eliminated. This approach combines the strengths of adaptively weighted LASSO shrinkage and quadratic regularization, resulting in optimal large sample performance and the ability to effectively handle collinearity. We also present a computational algorithm for the proposed method and compare its performance to that of LASSO, elastic net, and adaptive LASSO through simulation studies and applications to real datasets. The results demonstrate that the proposed method tends to perform better than existing approaches.

A unifying view for the mixture model of sparse Gaussian processes

The mixture of Gaussian processes (MGP) is generally accurate but slow in multimodal probabilistic regression. To reduce the computational burden of MGP, a mixture of sparse Gaussian processes (MSGP) with a fully independent training conditional (FITC) approximation was designed. However, MSGP with FITC is not always accurate. In this paper, we propose MSGP with a variational inference (VI) approximation to overcome this problem and establish a unifying view of MSGPs with FITC and VI based on their marginal likelihoods to explore the relationship between two MSGPs. Experiments on synthetic and real-world datasets show that MSGP with VI is more favorable than MSGP with FITC in certain behaviors, such as lower heteroscedasticity of noise, less clumping behavior of inducing inputs, more effective recovery of the full MGP and higher probabilistic regression accuracy at similar efficiency.

Linking Frequentist and Bayesian Change-Point Methods

We show that the two-stage minimum description length (MDL) criterion widely used to estimate linear change-point (CP) models corresponds to the marginal likelihood of a Bayesian model with a specific class of prior distributions. This allows results from the frequentist and Bayesian paradigms to be bridged together. Thanks to this link, one can rely on the consistency of the number and locations of the estimated CPs and the computational efficiency of frequentist methods, and obtain a probability of observing a CP at a given time, compute model posterior probabilities, and select or combine CP methods via Bayesian posteriors. Furthermore, we adapt several CP methods to take advantage of the MDL probabilistic representation. Based on simulated data, we show that the adapted CP methods can improve structural break detection compared to state-of-the-art approaches. Finally, we empirically illustrate the usefulness of combining CP detection methods when dealing with long time series and forecasting.

A kernel-based PEM estimator for forward models

This paper addresses the problem of learning the impulse responses characterizing forward models by means of a regularized kernel-based Prediction Error Method (PEM). The common approach to accomplish that is to approximate the system with a high-order stable ARX model. However, such choice induces a certain undesired prior information in the system that we want to estimate. To overcome this issue, we propose a new kernel-based paradigm which is formulated directly in terms of the impulse responses of the forward model and leading to the identification of a high-order MAX model. The most challenging step is the estimation of the kernel hyperparameters optimizing the marginal likelihood. The latter, indeed, does not admit a closed form expression. We propose a method for evaluating the marginal likelihood which makes possible the hyperparameters estimation. Finally, some numerical results showing the effectiveness of the method are presented.

The irtQ R package: a user-friendly tool for item response theory-based test data analysis and calibration.

Computerized adaptive testing (CAT) has become a widely adopted test design for high-stakes licensing and certification exams, particularly in the health professions in the United States, due to its ability to tailor test difficulty in real time, reducing testing time while providing precise ability estimates. A key component of CAT is item response theory (IRT), which facilitates the dynamic selection of items based on examinees' ability levels during a test. Accurate estimation of item and ability parameters is essential for successful CAT implementation, necessitating convenient and reliable software to ensure precise parameter estimation. This paper introduces the irtQ R package, which simplifies IRT-based analysis and item calibration under unidimensional IRT models. While it does not directly simulate CAT, it provides essential tools to support CAT development, including parameter estimation using marginal maximum likelihood estimation via the expectation-maximization algorithm, pretest item calibration through fixed item parameter calibration and fixed ability parameter calibration methods, and examinee ability estimation. The package also enables users to compute item and test characteristic curves and information functions necessary for evaluating the psychometric properties of a test. This paper illustrates the key features of the irtQ package through examples using simulated datasets, demonstrating its utility in IRT applications such as test data analysis and ability scoring. By providing a user-friendly environment for IRT analysis, irtQ significantly enhances the capacity for efficient adaptive testing research and operations. Finally, the paper highlights additional core functionalities of irtQ, emphasizing its broader applicability to the development and operation of IRT-based assessments.

Identifying Latent Groups in Spatial Panel Data Using a Markov Random Field Constrained Product Partition Model

Understanding the heterogeneity over spatial locations is an important problem that has been widely studied in many applications such as economics and environmental science. In this paper, we focus on regression models for spatial panel data analysis, where repeated measurements are collected over time at various spatial locations. We propose a novel class of nonparametric priors that combines Markov random field (MRF) with the product partition model (PPM), and show that the resulting prior, called by MRF-PPM, is capable of identifying the latent group structure among the spatial locations while efficiently utilizing the spatial dependence information. We derive a closed-form conditional distribution for the proposed prior and introduce a new way to compute the marginal likelihood that renders efficient Bayesian inference. We further study the theoretical properties of the proposed MRF-PPM prior and show a clustering consistency result for the posterior distribution. We demonstrate the excellent empirical performance of our method via extensive simulation studies and applications to a US precipitation data and a California median household income data study.

Estimation Method of Covariance Matrix in Atmospheric Inversion of CO2 Emissions

Atmospheric inversion of CO2 Emissions is based on the correction of prior carbon dioxide flux inventories using concentration monitoring data and atmospheric transport models to obtain posterior carbon dioxide flux. In atmospheric inversion studies, fixed covariance functions are commonly used to generate covariance matrices, and the hyperparameters in the covariance functions are empirically estimated. In this study, we design and implement an ideal experiment based on meteorological data from the central urban area of Zhengzhou, using WRF-STILT to generate sensitivity matrices and construct real carbon emission inventories and prior inventories. Based on the real carbon emission inventories and sensitivity matrices of monitoring stations, simulated observation concentration values are generated. Firstly, based on the observed concentration values, sensitivity matrices of monitoring stations, prior inventories, and constructed covariance matrices, the values of hyperparameters are determined based on maximum marginal likelihood estimation. Then, the influence of different prior covariance functions on the inversion results is tested, and it is found that the prior covariance matrix generated by the balgovind covariance function is most suitable for the experimental data.

Monetary policy in practice: do central banks respond to movements in exchange rate and credit growth?

PurposeThis study examines the importance of exchange rate and credit growth fluctuations when designing monetary policy in Thailand. To this end, the authors construct a small open economy New Keynesian dynamic stochastic general equilibrium (DSGE) model. The model encompasses several essential characteristics, including incomplete financial markets, incomplete exchange rate pass-through, deviations from the law of one price and a banking sector. The authors consider generalized Taylor rules, in which policymakers adjust policy rates in response to output, inflation, credit growth and exchange rate fluctuations. The marginal likelihoods are then employed to investigate whether the central bank responds to fluctuations in the exchange rate and credit growth.Design/methodology/approach This study constructs a small open economy DSGE model and then estimates the model using Bayesian methods.Findings The authors demonstrate that the monetary authority does target exchange rates, whereas there is no evidence in favor of incorporating credit growth into the policy rules. These findings survive various robustness checks. Furthermore, the authors demonstrate that domestic shocks contribute significantly to domestic business cycles. Although the terms of trade shock plays a minor role in business cycles, it explains the most significant proportion of exchange rate fluctuations, followed by the country risk premium shock.Originality/value This study is the first attempt at exploring the relevance of exchange rate and credit growth fluctuations when designing monetary policy in Thailand.

A comparison between marginal likelihood and data augmented MCMC algorithms for Gaussian hidden Markov models

We provide a comparison between marginal likelihood and data augmented Markov chain Monte Carlo (MCMC) algorithms for Bayesian estimation of hidden Markov models. In particular, we focus on a specification of these models based on a multivariate Gaussian distribution for the response variables. We evaluate the performance of both methods on the basis of simulated samples and an empirical application, where we compare these approaches in terms of standard measures of estimation and time efficiency. The results show that the augmented algorithm is preferable, obtaining optimal convergence properties and producing larger values of the effective sample size. Instead of using variable dimensional techniques for model selection, we suggest a method based on the so-called parallel sampling for this aim.

Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery.

In real-world situations, multidimensional data may appear on large-scale tests or psychological surveys. The purpose of this study was to investigate the effects of the quantity and magnitude of cross-loadings and model specification on item parameter recovery in multidimensional Item Response Theory (MIRT) models, especially when the model was misspecified as a simple structure, ignoring the quantity and magnitude of cross-loading. A simulation study that replicated this scenario was designed to manipulate the variables that could potentially influence the precision of item parameter estimation in the MIRT models. Item parameters were estimated using marginal maximum likelihood, utilizing the expectation-maximization algorithms. A compensatory two-parameter logistic-MIRT model with two dimensions and dichotomous item-responses was used to simulate and calibrate the data for each combination of conditions across 500 replications. The results of this study indicated that ignoring the quantity and magnitude of cross-loading and model specification resulted in inaccurate and biased item discrimination parameter estimates. As the quantity and magnitude of cross-loading increased, the root mean square of error and bias estimates of item discrimination worsened.