Evidence For Model Research Articles

Unobserved components with stochastic volatility: Simulation‐based estimation and signal extraction

SummaryThe unobserved components time series model with stochastic volatility has gained much interest in econometrics, especially for the purpose of modelling and forecasting inflation. We present a feasible simulated maximum likelihood method for parameter estimation from a classical perspective. The method can also be used for evaluating the marginal likelihood function in a Bayesian analysis. We show that our simulation‐based method is computationally feasible, for both univariate and multivariate models. We assess the performance of the method in a Monte Carlo study. In an empirical study, we analyse U.S. headline inflation using different univariate and multivariate model specifications.

A probabilistic transcranial magnetic stimulation localization method

Objective. Transcranial magnetic stimulation (TMS) can be used to safely and noninvasively activate brain tissue. However, the characteristic parameters of the neuronal activation have been largely unclear. In this work, we propose a novel neuronal activation model and develop a method to infer its parameters from measured motor evoked potential signals. Approach. The connection between neuronal activation due to an induced electric field and a measured motor threshold is modeled. The posterior distribution of the model parameters are inferred from measurement data using Bayes’ formula. The measurements are the active motor thresholds obtained with multiple stimulating coil locations, and the parameters of the model are the location, preferred direction of activation, and threshold electric field value of the activation site. The posterior distribution is sampled using a Markov chain Monte Carlo method. We quantify the plausibility of the model by calculating the marginal likelihood of the measured thresholds. The method is validated with synthetic data and applied to motor threshold measurements from the first dorsal interosseus muscle in five healthy participants. Main results. The method produces a probability distribution for the activation location, from which a minimal volume where the activation occurs with 95% probability can be derived. For eight or nine stimulating coil locations, the smallest such a volume obtained was approximately 100 mm3. The 95% probability volume intersected the pre-central gyral crown and the anterior wall of the central sulcus, and the preferred direction was perpendicular to the central sulcus, both findings being consistent with the literature. Furthermore, it was not possible to rule out if the activation occurred either in the white or grey matter. In one participant, two distinct activations sites were found while others exhibited a unique site. Significance. The method is both generic and robust, and it lays a foundation for a framework that enables accurate analysis and characterization of TMS activation mechanisms.

Dark Matter Constraints from a Unified Analysis of Strong Gravitational Lenses and Milky Way Satellite Galaxies

Abstract Joint analyses of small-scale cosmological structure probes are relatively unexplored and promise to advance measurements of microphysical dark matter properties using heterogeneous data. Here, we present a multidimensional analysis of dark matter substructure using strong gravitational lenses and the Milky Way (MW) satellite galaxy population, accounting for degeneracies in model predictions and using covariances in the constraining power of these individual probes for the first time. We simultaneously infer the projected subhalo number density and the half-mode mass describing the suppression of the subhalo mass function in thermal relic warm dark matter (WDM), M hm, using the semianalytic model Galacticus to connect the subhalo population inferred from MW satellite observations to the strong lensing host halo mass and redshift regime. Combining MW satellite and strong lensing posteriors in this parameter space yields M hm < 107.0 M ⊙ (WDM particle mass m WDM > 9.7 keV) at 95% confidence and disfavors M hm = 107.4 M ⊙ (m WDM = 7.4 keV) with a 20:1 marginal likelihood ratio, improving limits on m WDM set by the two methods independently by ∼30%. These results are marginalized over the line-of-sight contribution to the strong lensing signal, the mass of the MW host halo, and the efficiency of subhalo disruption due to baryons and are robust to differences in the disruption efficiency between the MW and strong lensing regimes at the ∼10% level. This work paves the way for unified analyses of next-generation small-scale structure measurements covering a wide range of scales and redshifts.

Rapid parameter estimation of a two-component neutron star model with spin wandering using a Kalman filter

Abstract The classic, two-component, crust-superfluid model of a neutron star can be formulated as a noise-driven, linear dynamical system, in which the angular velocities of the crust and superfluid are tracked using a Kalman filter applied to electromagnetic pulse timing data and gravitational wave data, when available. Here it is shown how to combine the marginal likelihood of the Kalman filter and nested sampling to estimate full posterior distributions of the six model parameters, extending previous analyses based on a maximum-likelihood approach. The method is tested across an astrophysically plausible parameter domain using Monte Carlo simulations. It recovers the injected parameters to ≲ 10 per cent for time series containing ∼103 samples, typical of long-term pulsar timing campaigns. It runs efficiently in $\mathcal {O}(1)$ CPU-hr for data sets of the above size. In a present-day observational scenario, when electromagnetic data are available only, the method accurately estimates three parameters: the relaxation time, the ensemble-averaged spin-down of the system, and the amplitude of the stochastic torques applied to the crust. In a future observational scenario, where gravitational wave data are also available, the method also estimates the ratio between the moments of inertia of the crust and the superfluid, the amplitude of the stochastic torque applied to the superfluid, and the crust-superfluid lag. These empirical results are consistent with a formal identifiability analysis of the linear dynamical system.

Item Parameter Estimation in Multistage Designs: A Comparison of Different Estimation Approaches for the Rasch Model

There is some debate in the psychometric literature about item parameter estimation in multistage designs. It is occasionally argued that the conditional maximum likelihood (CML) method is superior to the marginal maximum likelihood method (MML) because no assumptions have to be made about the trait distribution. However, CML estimation in its original formulation leads to biased item parameter estimates. Zwitser and Maris (2015, Psychometrika) proposed a modified conditional maximum likelihood estimation method for multistage designs that provides practically unbiased item parameter estimates. In this article, the differences between different estimation approaches for multistage designs were investigated in a simulation study. Four different estimation conditions (CML, CML estimation with the consideration of the respective MST design, MML with the assumption of a normal distribution, and MML with log-linear smoothing) were examined using a simulation study, considering different multistage designs, number of items, sample size, and trait distributions. The results showed that in the case of the substantial violation of the normal distribution, the CML method seemed to be preferable to MML estimation employing a misspecified normal trait distribution, especially if the number of items and sample size increased. However, MML estimation using log-linear smoothing lea to results that were very similar to the CML method with the consideration of the respective MST design.

Modulating scalable Gaussian processes for expressive statistical learning

For a learning task, Gaussian process (GP) is interested in learning the statistical relationship between inputs and outputs, since it offers not only the prediction mean but also the associated variability. The vanilla GP however is hard to learn complicated distribution with the property of, e.g., heteroscedastic noise, multi-modality and non-stationarity, from massive data due to the Gaussian marginal and the cubic complexity. To this end, this article studies new scalable GP paradigms including the non-stationary heteroscedastic GP, the mixture of GPs and the latent GP, which introduce additional latent variables to modulate the outputs or inputs in order to learn richer, non-Gaussian statistical representation. Particularly, we resort to different variational inference strategies to arrive at analytical or tighter evidence lower bounds (ELBOs) of the marginal likelihood for efficient and effective model training. Extensive numerical experiments against state-of-the-art GP and neural network (NN) counterparts on various tasks verify the superiority of these scalable modulated GPs, especially the scalable latent GP, for learning diverse data distributions.

Bayesian Evidence for a Nonlinear Damping Model for Coronal Loop Oscillations

Abstract Recent observational and theoretical studies indicate that the damping of solar coronal loop oscillations depends on the oscillation amplitude. We consider two mechanisms: linear resonant absorption and a nonlinear damping model. We confront theoretical predictions from these models with observed data in the plane of observables defined by the damping ratio and the oscillation amplitude. The structure of the Bayesian evidence in this plane displays a clear separation between the regions where each model is more plausible relative to the other. There is qualitative agreement between the regions of high marginal likelihood and Bayes factor for the nonlinear damping model and the arrangement of observed data. A quantitative application to 101 loop oscillation cases observed with Solar Dynamics Observatory/Atmospheric Imaging Assembly (SDO/AIA) results in the marginal likelihood for the nonlinear model being larger in the majority of them. The cases with conclusive evidence for the nonlinear damping model outnumber considerably those in favor of linear resonant absorption.

Unification of sparse Bayesian learning algorithms for electromagnetic brain imaging with the majorization minimization framework

Methods for electro- or magnetoencephalography (EEG/MEG) based brain source imaging (BSI) using sparse Bayesian learning (SBL) have been demonstrated to achieve excellent performance in situations with low numbers of distinct active sources, such as event-related designs. This paper extends the theory and practice of SBL in three important ways. First, we reformulate three existing SBL algorithms under the majorization-minimization (MM) framework. This unification perspective not only provides a useful theoretical framework for comparing different algorithms in terms of their convergence behavior, but also provides a principled recipe for constructing novel algorithms with specific properties by designing appropriate bounds of the Bayesian marginal likelihood function. Second, building on the MM principle, we propose a novel method called LowSNR-BSI that achieves favorable source reconstruction performance in low signal-to-noise-ratio (SNR) settings. Third, precise knowledge of the noise level is a crucial requirement for accurate source reconstruction. Here we present a novel principled technique to accurately learn the noise variance from the data either jointly within the source reconstruction procedure or using one of two proposed cross-validation strategies. Empirically, we could show that the monotonous convergence behavior predicted from MM theory is confirmed in numerical experiments. Using simulations, we further demonstrate the advantage of LowSNR-BSI over conventional SBL in low-SNR regimes, and the advantage of learned noise levels over estimates derived from baseline data. To demonstrate the usefulness of our novel approach, we show neurophysiologically plausible source reconstructions on averaged auditory evoked potential data.

Equivalence class selection of categorical graphical models

Learning the structure of dependence relations between variables is a pervasive issue in the statistical literature. A directed acyclic graph (DAG) can represent a set of conditional independencies, but different DAGs may encode the same set of relations and are indistinguishable using observational data. Equivalent DAGs can be collected into classes, each represented by a partially directed graph known as essential graph (EG). Structure learning directly conducted on the EG space, rather than on the allied space of DAGs, leads to theoretical and computational benefits. Still, the majority of efforts has been dedicated to Gaussian data, with less attention to methods designed for multivariate categorical data. A Bayesian methodology for structure learning of categorical EGs is then proposed. Combining a constructive parameter prior elicitation with a graph-driven likelihood decomposition, a closed-form expression for the marginal likelihood of a categorical EG model is derived. Asymptotic properties are studied, and an MCMC sampler scheme developed for approximate posterior inference. The methodology is evaluated on both simulated scenarios and real data, with appreciable performance in comparison with state-of-the-art methods.

Nested sampling with any prior you like

ABSTRACT Nested sampling is an important tool for conducting Bayesian analysis in Astronomy and other fields, both for sampling complicated posterior distributions for parameter inference, and for computing marginal likelihoods for model comparison. One technical obstacle to using nested sampling in practice is the requirement (for most common implementations) that prior distributions be provided in the form of transformations from the unit hyper-cube to the target prior density. For many applications – particularly when using the posterior from one experiment as the prior for another – such a transformation is not readily available. In this letter, we show that parametric bijectors trained on samples from a desired prior density provide a general purpose method for constructing transformations from the uniform base density to a target prior, enabling the practical use of nested sampling under arbitrary priors. We demonstrate the use of trained bijectors in conjunction with nested sampling on a number of examples from cosmology.

Approximate pairwise likelihood inference in SGLM models with skew normal latent variables

Spatial generalized linear mixed models are commonly employed for modeling discrete spatial responses that are acquired on a continuous area. A standard assumption in these models is that the latent variables are normally distributed, however skewed residuals appear in some spatial generalized linear mixed models. In this study, we consider a closed skew Gaussian random field for the spatial latent variables in the spatial generalized linear mixed models and present a new approximate pairwise likelihood approach to estimate parameters. In order to introduce a new algorithm to obtain the pairwise maximum likelihood estimates for the parameters, we use a linearization method in the composite marginal likelihood and EM algorithm. Also, techniques to calculate parameter estimates and spatial prediction in this class of skew models are proposed. The performance of the proposed model and method are illustrated through a simulation study, and applied the Tehran air quality index data set.

Comparing Asset Pricing Factor Models under Multivariate t-Distribution: Evidence from China

Factor models provide a cornerstone for understanding financial asset pricing; however, research on China’s stock market risk premia is still limited. Motivated by this, this paper proposes a four-factor model for China’s stock market that includes a market factor, a size factor, a value factor, and a liquidity factor. We compare our four-factor model with a set of prominent factor models based on newly developed likelihood-ratio tests and Bayesian methods. Along with the comparison, we also find supporting evidence for the alternative t-distribution assumption for empirical asset pricing studies. Our results show the following: (1) distributional tests suggest that the returns of factors and stock return anomalies are fat-tailed and therefore are better captured by t-distributions than by normality; (2) under t-distribution assumptions, our four-factor model outperforms a set of prominent factor models in terms of explaining the factors in each other, pricing a comprehensive list of stock return anomalies, and Bayesian marginal likelihoods; (3) model comparison results vary across normality and t-distribution assumptions, which suggests that distributional assumptions matter for asset pricing studies. This paper contributes to the literature by proposing an effective asset pricing factor model and providing factor model comparison tests under non-normal distributional assumptions in the context of China.

The Pearson Bayes factor: An analytic formula for computing evidential value from minimal summary statistics

Summary In Bayesian hypothesis testing, evidence for a statistical model is quantified by the Bayes factor, which represents the relative likelihood of observed data under that model compared to another competing model. In general, computing Bayes factors is difficult, as computing the marginal likelihood of data under a given model requires integrating over a prior distribution of model parameters. In this paper, I capitalize on a particular choice of prior distribution that allows the Bayes factor to be expressed without integral representation, and I develop a simple formula – the Pearson Bayes factor – that requires only minimal summary statistics as commonly reported in scientific papers, such as the t or F score and the degrees of freedom. In addition to presenting this new result, I provide several examples of its use and report a simulation study validating its performance. Importantly, the Pearson Bayes factor gives applied researchers the ability to compute exact Bayes factors from minimal summary data, and thus easily assess the evidential value of any data for which these summary statistics are provided, even when the original data is not available.

Efficient Bayesian inference of general Gaussian models on large phylogenetic trees

Phylogenetic comparative methods correct for shared evolutionary history among a set of nonindependent organisms by modeling sample traits as arising from a diffusion process along the branches of a possibly unknown history. To incorporate such uncertainty, we present a scalable Bayesian inference framework under a general Gaussian trait evolution model that exploits Hamiltonian Monte Carlo (HMC). HMC enables efficient sampling of the constrained model parameters and takes advantage of the tree structure for fast likelihood and gradient computations, yielding algorithmic complexity linear in the number of observations. This approach encompasses a wide family of stochastic processes, including the general Ornstein–Uhlenbeck (OU) process, with possible missing data and measurement errors. We implement inference tools for a biologically relevant subset of all these models into the BEAST phylogenetic software package and develop model comparison through marginal likelihood estimation. We apply our approach to study the morphological evolution in the superfamily of Musteloidea (including weasels and allies) as well as the heritability of HIV virulence. This second problem furnishes a new measure of evolutionary heritability that demonstrates its utility through a targeted simulation study.

SDCRKL-GP: Scalable deep convolutional random kernel learning in gaussian process for image recognition

Deep convolutional neural networks have shown great potential in image recognition tasks. However, the fact that the mechanism of deep learning is difficult to explain hinders its development. It involves a large amount of parameter learning, which results in high computational complexity. Moreover, deep convolutional neural networks are often limited by overfitting in regimes in which the number of training samples is limited. Conversely, kernel learning methods have a clear mathematical theory, fewer parameters, and can contend with small sample sizes; however, they are not able to handle high-dimensional data, e.g., images. It is important to achieve a performance and complexity trade-off in complicated tasks. In this paper, we propose a novel scalable deep convolutional random kernel learning in Gaussian process architecture called SDCRKL-GP, which is characterized by excellent performance and low complexity. First, we successfully incorporated the deep convolutional architecture into kernel learning by implementing the random Fourier feature transform for Gaussian processes, which can effectively capture hierarchical and local image-level features. This approach enabled the kernel method to effectively handle image processing problems. Second, we optimized the parameters of deep convolutional filters and Gaussian kernels by stochastic variational inference. Then, we derived the lower variational bound of the marginal likelihood. Finally, we explored the model architecture design space selection method to determine the appropriate network architecture for different datasets. The design space consists of the number of layers, the channels per layer, and so on. Different design space selections improved the scalability of the SDCRKL-GP architecture. We evaluated SDCRKL-GP on the MNIST, FMNIST, CIFAR10, and CALTECH4 benchmark datasets. Taking MNIST as an example, the error rate of classification is 0.60%, and the number of parameters, number of computations and memory access cost of the architecture are 19.088k, 0.984M, and 1.057M, respectively. The experimental results verified that the proposed SDCRKL-GP method outperforms several state-of-the-art algorithms in both accuracy and speed in image recognition tasks. The code is available at https://github.com/w-tingting/deep-rff-pytorch.

Is euro area lowflation here to stay? Insights from a time‐varying parameter model with survey data

SummaryWe build a time‐varying parameter model that jointly explains the dynamics of euro area inflation and inflation expectations. Our goal is to explain the weak inflation during the post‐financial crisis economic recovery of 2013–2019. We find that the inclusion of survey data leads to a more muted decline of trend inflation in recent years and more economic slack. Moreover, the impact of economic slack and import prices on inflation has recently strengthened, and survey respondents updated their beliefs more actively over the financial crisis period. Our model compares well against restricted specifications in terms of forecast performance and marginal likelihood.

Learning Adjustment Sets from Observational and Limited Experimental Data

Estimating causal effects from observational data is not always possible due to confounding. Identifying a set of appropriate covariates (adjustment set) and adjusting for their influence can remove confounding bias; however, such a set is often not identifiable from observational data alone. Experimental data allow unbiased causal effect estimation, but are typically limited in sample size and can therefore yield estimates of high variance. Moreover, experiments are often performed on a different (specialized) population than the population of interest. In this work, we introduce a method that combines large observational and limited experimental data to identify adjustment sets and improve the estimation of causal effects for a target population. The method scores an adjustment set by calculating the marginal likelihood for the experimental data given an observationally-derived causal effect estimate, using a putative adjustment set. The method can make inferences that are not possible using constraint-based methods. We show that the method can improve causal effect estimation, and can make additional inferences when compared to state-of-the-art methods.

Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression

High-dimensional prediction with multiple data types needs to account for potentially strong differences in predictive signal. Ridge regression is a simple model for high-dimensional data that has challenged the predictive performance of many more complex models and learners, and that allows inclusion of data type-specific penalties. The largest challenge for multi-penalty ridge is to optimize these penalties efficiently in a cross-validation (CV) setting, in particular for GLM and Cox ridge regression, which require an additional estimation loop by iterative weighted least squares (IWLS). Our main contribution is a computationally very efficient formula for the multi-penalty, sample-weighted hat-matrix, as used in the IWLS algorithm. As a result, nearly all computations are in low-dimensional space, rendering a speed-up of several orders of magnitude. We developed a flexible framework that facilitates multiple types of response, unpenalized covariates, several performance criteria and repeated CV. Extensions to paired and preferential data types are included and illustrated on several cancer genomics survival prediction problems. Moreover, we present similar computational shortcuts for maximum marginal likelihood and Bayesian probit regression. The corresponding R-package, multiridge, serves as a versatile standalone tool, but also as a fast benchmark for other more complex models and multi-view learners. Supplementary materials for this article are available online.

Bayes analysis of some important lifetime models using MCMC based approaches when the observations are left truncated and right censored

The paper considers the Bayes analysis of important lifetime models such as the Weibull, the gamma, and the lognormal distributions when the available data are left truncated and right-censored. Weakly informative prior distributions are employed for the purpose. Two well-known Markov chain Monte Carlo based approaches, namely, the Metropolis algorithm and the Hamiltonian Monte Carlo technique are used to draw samples from analytically intractable posterior distributions. Besides, the paper does a comparative study of the three entertained models using Bayes factor. The paper has considered calculating the marginal likelihood using bridge sampler algorithm for evaluating the necessary Bayes factor. Finally, a numerical illustration based on a real dataset compares the two algorithms and draws relevant conclusions appropriately.

Fast Bayesian Compressed Sensing Algorithm via Relevance Vector Machine for LASAR 3D Imaging

Because of the three-dimensional (3D) imaging scene’s sparsity, compressed sensing (CS) algorithms can be used for linear array synthetic aperture radar (LASAR) 3D sparse imaging. CS algorithms usually achieve high-quality sparse imaging at the expense of computational efficiency. To solve this problem, a fast Bayesian compressed sensing algorithm via relevance vector machine (FBCS–RVM) is proposed in this paper. The proposed method calculates the maximum marginal likelihood function under the framework of the RVM to obtain the optimal hyper-parameters; the scattering units corresponding to the non-zero optimal hyper-parameters are extracted as the target-areas in the imaging scene. Then, based on the target-areas, we simplify the measurement matrix and conduct sparse imaging. In addition, under low signal to noise ratio (SNR), low sampling rate, or high sparsity, the target-areas cannot always be extracted accurately, which probably contain several elements whose scattering coefficients are too small and closer to 0 compared to other elements. Those elements probably make the diagonal matrix singular and irreversible; the scattering coefficients cannot be estimated correctly. To solve this problem, the inverse matrix of the singular matrix is replaced with the generalized inverse matrix obtained by the truncated singular value decomposition (TSVD) algorithm to estimate the scattering coefficients correctly. Based on the rank of the singular matrix, those elements with small scattering coefficients are extracted and eliminated to obtain more accurate target-areas. Both simulation and experimental results show that the proposed method can improve the computational efficiency and imaging quality of LASAR 3D imaging compared with the state-of-the-art CS-based methods.