Likelihood Estimation Research Articles

Metropolis–Hastings via Classification

This article develops a Bayesian computational platform at the interface between posterior sampling and optimization in models whose marginal likelihoods are difficult to evaluate. Inspired by contrastive learning and Generative Adversarial Networks (GAN), we reframe the likelihood function estimation problem as a classification problem. Pitting a Generator, who simulates fake data, against a Classifier, who tries to distinguish them from the real data, one obtains likelihood (ratio) estimators which can be plugged into the Metropolis–Hastings algorithm. The resulting Markov chains generate, at a steady state, samples from an approximate posterior whose asymptotic properties we characterize. Drawing upon connections with empirical Bayes and Bayesian misspecification, we quantify the convergence rate in terms of the contraction speed of the actual posterior and the convergence rate of the Classifier. Asymptotic normality results are also provided which justify the inferential potential of our approach. We illustrate the usefulness of our approach on examples which have challenged for existing Bayesian likelihood-free approaches. Supplementary materials for this article are available online.

Distribution-free stochastic model updating of dynamic systems with parameter dependencies

This work proposes a distribution-free stochastic model updating framework to calibrate the joint probabilistic distribution of the multivariate correlated parameters. In this framework, the marginal distributions are defined as the staircase density functions and the correlation structure is described by the Gaussian copula function. The first four moments of the staircase density functions and the correlation coefficients are updated by an approximate Bayesian computation, in which the Bhattacharyya distance-based metric is proposed to define an approximate likelihood that is capable of capturing the stochastic discrepancy between model outputs and observations. The feasibility of the framework is demonstrated on two illustrative examples and a followed engineering application to the updating of a nonlinear dynamic system using observed time signals. The results demonstrate the capability of the proposed updating procedure in the very challenging condition where the prior knowledge about the distribution of the parameters is extremely limited (i.e., no information on the marginal distribution families and correlation structure is available).

How to optimally combine pre-reconstruction full shape and post-reconstruction BAO signals

We review the different approaches for combining the cosmological information from the full shape of the pre-reconstructed power spectrum — usually referred as redshift-space distortion (RSD) analysis — and from the baryon acoustic oscillation (BAO) peak position in the post-reconstructed power spectrum with the aim of finding the optimal procedure. We focus on combining the pre- and post-reconstructed derived quantities at different compression levels: 1) the two-point summary statistics, the power spectrum multipoles, P(ℓ)(k); 2) the compressed BAO variables, α ∥,⊥; and 3) an hybrid approach between 1) and 2). We apply these methods to the publicly available eBOSS Luminous Red Galaxy catalogues, for both data and synthetic EZ-mocks. We find that the three approaches result in very consistent posteriors when the appropriate covariance matrix estimator is used. On average, the combination at P(ℓ)(k) level retrieves 5-10% tighter constraints than the other two approaches, demonstrating that the standard approach of combining at the level of the BAO variables is nearly optimal. We conclude that combining both BAO post-reconstructed and full shape pre-reconstructed signals for the one single data realization at the level of the summary statistics is faster, as it does not requirerunning the whole pipeline on the individual mocks, and brings a moderate 10% improvement, with respect to the other two studied methods. Moreover, we check for potential systematics, such as, the way the matrix is built and the effect of the finite number of mocks on the likelihood estimator and find none of these have a significant impact in the final results. Combining the pre- and post-reconstruction signals at the level of the summary statistics is an attractive, faster and accurate method to be used in future and on-going spectroscopic surveys.

Maximum Doppler Frequency Detection Based on Likelihood Estimation With Theoretical Thresholds

Maximum Doppler Frequency Detection Based on Likelihood Estimation With Theoretical Thresholds

Rüzgar enerji uygulamaları için Weibull dağılım parametrelerinin tahmini: Türkiye’nin Dinar bölgesi için bir çalışma

In this study, the wind energy characteristics of Dinar region were researched. Required wind data was provided by the Dinar Meteorology Station of the Turkish State Meteorology Service. Wind data was recorded hourly between June 2015 and May 2020. The two variable Weibull probability distribution function was used to research wind characteristics of the region. Six different methods were used to calculate shape (k) and scale (c) variables of the probability distribution function. These are moment, graph, Justus empirical, energy trend, energy pattern and maximum likehood estimation methods. The performance of these methods were compared with four different statistical error analysis methods. These are root mean square error (RMSE), coefficient of determination (R2), chi-square error (X2) and mean absolute error (MAE). According to the results, maximum likehood estimation and energy trend methods have been the most successful methods in the calculation of Weibull parameters. In addition, this study is an important preliminary study in determining the wind energy characteristic of the region.

A nonparametric Bayesian analysis for meningococcal disease counts based on integer-valued threshold time series models

To better describe the characteristics of time series of counts showing over-dispersion, asymmetry and structural change features, this paper considers a class of random coefficient integer-valued threshold autoregressive processes that properly capture flexible asymmetric and nonlinear responses without assuming the distributions for the errors. By the Taylor expansion, a normally distributed working likelihood is obtained and the Bayesian empirical likelihood inference is conducted for the model parameters. Through the normalized approximation of the nonparametric likelihood, the originally complicated Bayesian calculation has been greatly simplified. Finally, the proposed method is verified via some simulations and an empirical analysis of the meningococcal disease data set in Germany.

Approximate Likelihood Inference in Generalized Linear Models with Censored Covariates

In many surveys and clinical trials, we obtain measurements on covariates or biomarkers that are left-censored due to the limit of detection. In such cases, it is necessary to correct for the left-censoring when studying covariate effects in regression models. The expectation-maximization (EM) algorithm is widely used for the likelihood inference in generalized linear models with censored covariates. The EM method, however, requires intensive computation involving high-dimensional integration with respect to the covariates when the dimension of the censored covariates is large. To reduce such computational difficulties, we propose and explore a Monte Carlo EM method based on the Metropolis algorithm. The finite-sample properties of the proposed estimators are studied using Monte Carlo simulations. An application is also provided using actual data obtained from a health and nutrition examination survey. Journal of Statistical Research 2021, Vol. 55, No. 2, pp. 359-375

A Bayesian paradigm in a large class of Lévy-driven CARMA models for high frequency data

Continuous-time time series are widely used for modeling the realizations of those phenomena where it is theoretically possible to have observation at any point of the sampling domain. However, technical restrictions cause to see the realizations of such processes as discrete-time sample paths. We project time-domain observations into frequency-domain periodograms and employ the Whittle’s likelihood approximation to make the inference about the parameters of CARMA processes. The inference is given under the Bayesian paradigm and some scenarios about the prior distributions are discussed. We show that our proposed estimator is robust for a large family of Lévy-driven distributed noises under some mild conditions. The Bayes estimates of parameters are obtained using a fast MCMC algorithm and they are examined numerically for the non-informative priors. The mean squared errors of the presented estimates are also compared to two other likelihood-based estimators, numerically. We also examine the performance of this Bayes estimate in prediction procedure of exchange rate data as a real dataset.

Mixed-type multivariate response regression with covariance estimation.

We propose a new method for multivariate response regression and covariance estimation when elements of the response vector are of mixed types, for example some continuous and some discrete. Our method is based on a model which assumes the observable mixed‐type response vector is connected to a latent multivariate normal response linear regression through a link function. We explore the properties of this model and show its parameters are identifiable under reasonable conditions. We impose no parametric restrictions on the covariance of the latent normal other than positive definiteness, thereby avoiding assumptions about unobservable variables which can be difficult to verify in practice. To accommodate this generality, we propose a novel algorithm for approximate maximum likelihood estimation that works “off‐the‐shelf” with many different combinations of response types, and which scales well in the dimension of the response vector. Our method typically gives better predictions and parameter estimates than fitting separate models for the different response types and allows for approximate likelihood ratio testing of relevant hypotheses such as independence of responses. The usefulness of the proposed method is illustrated in simulations; and one biomedical and one genomic data example.

A novel particle filter for extended target tracking with random hypersurface model

In the random hypersurface model for extended target tracking problem, the scaling factor in the measurement equation brings difficulty for existing particle filter to calculate the likelihood in the weighting update stage. In this paper, we firstly simplify the existing approximate likelihood function where the distribution of the scaling factor is approximated by Gaussian one. Then, by directly dealing with the distribution of the scaling factor whose square has uniform distribution, we propose a novel explicit formula of the logarithm of likelihood. Based on this formula, a feasible weighting scheme is obtained and a novel particle filtering algorithm (NPFA) is proposed. Simulation shows that NPFA improves estimation accuracy compared with the existing unscented Kalman filter and particle filter for the tracking problem under discussion.

A mixture distribution approach for assessing genetic impact from twin study.

It is challenging to evaluate the genetic impacts on a biologic feature and separate them from environmental impacts. This is usually achieved through twin studies by assessing the collective genetic impact defined by the differential correlation in monozygotic twins vs dizygotic twins. Since the underlying order in a twin, determined by latent genetic factors, is unknown, the observed twin data are unordered. Conventional methods for correlation are not appropriate. To handle the missing order, we model twin data by a mixture bivariate distribution and estimate under two likelihood functions: the likelihood over the monozygotic and dizygotic twins separately, and the likelihood over the two twin types combined. Both likelihood estimators are consistent. More importantly, the combined likelihood overcomes the drawback of mixture distribution estimation, namely, the slow convergence. It yields correlation coefficient estimator of root-n consistency and allows effective statistical inference on the collective genetic impact. The method is demonstrated by a twin study on immune traits.

Asylum Flows in the EU Context: Lessons from Gravity

The present paper aims to identify the impact of geographical, institutional, and sociopolitical factors as regards the magnitude and the direction of asylum seekers in the European Union between 2000 and 2018. The approach is based on the application of gravity model using Ordinary Least Squares (OLS) and Poisson Pseudo-Maximum Likelihood (PPML) estimators. The analysis incorporates a set of institutional variables, in order to assess the impact of the gradual EU enlargement process as well as the differentiated policies on granting asylum among the EU members. The strong presence of refugees in destination countries can be interpreted as an indication of various favorable conditions for granting asylum to persons of the same nationality. The results suggest that the role of migration networks tends to substitute the lack of an integrated EU immigration policy. Finally, either in geographical or institutional terms, E.U. appears as a non-homogeneous space for asylum seekers.

Anatomy of Research Performance from a Bottom-Up Approach: Examination of Researchers’ Perspective

Performance-based research policies and programmes have fundamentally altered both organisational and individual behaviours and expectations, putting immense pressure on researchers. The soundness of research, originality, valorisation potential, and societal and economic impacts are highly valued and expected characteristics of research. Yet, our understanding of the effects of various systemic and organisational factors on research performance is limited. In an exploratory, single-country case, this paper aimed to develop and examine different models of research performance as perceived by researchers themselves using a large cross-disciplinary sample of 553 researchers from 72 public research organisations in Romania. A pre-tested questionnaire was self-administered online, comprising seven scales: (1) recruitment and selection, (2) research recognition and value, (3) participation in research projects and teams, (4) work incentives, (5) job payment and salary, (6) career development opportunities, and (7) leadership effectiveness. Maximum likelihood and Bayesian estimators were used to test three structural models: (M1) mono-factor; (M2) intercorrelated dimensions, and (M3) the dimensions are indicators of a general construct. Additionally, a path analysis was carried out to study the relationships among the dimensions. We found that M2 and M3 fit the empirical data better. The results showed that career development programmes and opportunities gain centrality in achieving research performance by directly influencing participation and research projects and teams and mediating the effect of job payment. Revealingly, powerful work incentives within research organisations are international mobilities or appreciation awards. When informing evidence-based policies, the models we propose could serve the goal of improving research performance through talent development as the main proxy.

Assessing the Lifetime Performance Index with Digital Inferences of Power Hazard Function Distribution Using Progressive Type-II Censoring Scheme.

This paper deals with estimating the lifetime performance index. The maximum likelihood (ML) and Bayesian estimators for lifetime performance index CLX where LX is the lower specification limit are derived based on progressive type-II censored (Prog-Type-II-C) sample from two-parameter power hazard function distribution (PHFD). Knowing the lower specification limit, the MLE of CLX is applied to construct a new hypothesis testing procedure. Bayesian estimator of CLX is also utilized to develop a credible interval. Also, the relationship between the CLX and the conforming rate of products is investigated. Moreover, the Bayesian test to evaluate the lifetime performance of units is proposed. A simulation study and illustrative example based on a real dataset are discussed to evaluate the performance of the two tests.

Simulation-based Inference of Reionization Parameters from 3D Tomographic 21 cm Light-cone Images

Abstract Tomographic three-dimensional 21 cm images from the epoch of reionization contain a wealth of information about the reionization of the intergalactic medium by astrophysical sources. Conventional power spectrum analysis cannot exploit the full information in the 21 cm data because the 21 cm signal is highly non-Gaussian due to reionization patchiness. We perform a Bayesian inference of the reionization parameters where the likelihood is implicitly defined through forward simulations using density estimation likelihood-free inference (DELFI). We adopt a trained 3D convolutional neural network (CNN) to compress the 3D image data into informative summaries (DELFI-3D CNN). We show that this method recovers accurate posterior distributions for the reionization parameters. Our approach outperforms earlier analysis based on two-dimensional 21 cm images. In contrast, a Monte Carlo Markov Chain analysis of the 3D light-cone-based 21 cm power spectrum alone and using a standard explicit likelihood approximation results in less accurate credible parameter regions than inferred by the DELFI-3D CNN, both in terms of the location and shape of the contours. Our proof-of-concept study implies that the DELFI-3D CNN can effectively exploit more information in the 3D 21 cm images than a 2D CNN or power spectrum analysis. This technique can be readily extended to include realistic effects and is therefore a promising approach for the scientific interpretation of future 21 cm observation data.

On the Stratonovich Estimator for the Itô Diffusion

For the parameter appearing non-linearly in the drift coefficient of homogeneous Itô stochastic differential equation having a stationary ergodic solution, the paper obtains the strong consistency of an approximate maximum likelihood estimator based on Stratonovich type approximation of the continuous Girsanov likelihood, under some regularity conditions, when the corresponding diffusion is observed at equally spaced dense time points over a long time interval in the high frequency regime. Pathwise convergence of stochastic integral approximations and their connection to discrete drift estimators is studied. Often it is shown that discrete drift estimators converge in probability. We obtain convergence of the estimator with probability one. Ornstein-Uhlenbeck process is considered as an example.

Approximate profile likelihood estimation for Cox regression with covariate measurement error.

In nutritional epidemiology, measurement error in covariates is a well-known problem since dietary intakes are usually assessed through self-reporting. In this article, we consider an additive error model in which error variables are highly correlated, and propose a new method called approximate profile likelihood estimation (APLE) for covariates measured with error in the Cox regression. Asymptotic normality of this estimator is established under regularity conditions, and simulation studies are conducted to examine the finite sample performance of the proposed estimator empirically. Moreover, the popular correction method called regression calibration is shown to be a special case of APLE. We then apply APLE to deal with measurement error in some nutrients of interest in the EPIC-InterAct Study under a sensitivity analysis framework.

Evaluating the belt and road initiative effects on trade and migration: Evidence from the East African community

AbstractThis paper uses the new panel data structural gravity equations to examine the impact of China's Belt and Road Initiative in two dimensions, namely trade and migration on participating countries. By leveraging the Poisson pseudo‐maximum likelihood estimator with fixed‐effects to address both endogeneity and omitted variables, our results highlighted the importance of accounting for the regional economic integration in Africa, Europe, and Asia. The model assesses 65 countries, including 30 sub‐Saharan economies from the period 2000–2018. The findings of our study indicate that migration has a significant impact on the flow of bilateral trade. We also observed that the East African Community regional trade agreements have a significant negative effect on intra‐blocs trade under the Belt and Road Initiative. Therefore, a comprehensive policy agenda needs to be developed to enhance migration and trade‐promoting policies among the participating economies of Africa to engender African growth from a win‐win perspective.

A joint model for longitudinal outcomes with potential ceiling and floor effects and survival times, with applications to analysis of quality of life data from a cancer clinical trial

Longitudinal data on patient‐reported outcomes (PROs), such as quality of life of patients, are frequently collected in clinical trials and other medical studies. Joint analysis of these data with survival times may improve the accuracy of statistical inferences, especially when PRO measurements may be missing after the death of patients. Classical linear mixed models are often used as the models for the longitudinal measurements in a joint analysis, but it may not be suitable for longitudinal PRO measurements with potential ceiling and floor effects caused by a large portion of patients who report either a maximum or minimum score. In this paper, we introduce a new joint model that uses a longitudinal Tobit model for the longitudinal outcomes with potential ceiling and floor effects and a Cox proportional hazard model for survival time with a random effect connecting these two models. An estimation procedure based on the partial likelihood and Laplace approximation is developed to estimate the parameters in both models, and a random weighting method is proposed to calculate the variances of these parameter estimators. Performances of the proposed procedures are evaluated through simulation studies and an application to the analysis of quality of life data from a clinical trial.

Maximum approximate likelihood estimation of general continuous-time state-space models

Continuous-time state-space models (SSMs) are flexible tools for analysing irregularly sampled sequential observations that are driven by an underlying state process. Corresponding applications typically involve restrictive assumptions concerning linearity and Gaussianity to facilitate inference on the model parameters via the Kalman filter. In this contribution, we provide a general continuous-time SSM framework, allowing both the observation and the state process to be non-linear and non-Gaussian. Statistical inference is carried out by maximum approximate likelihood estimation, where multiple numerical integration within the likelihood evaluation is performed via a fine discretization of the state process. The corresponding reframing of the SSM as a continuous-time hidden Markov model, with structured state transitions, enables us to apply the associated efficient algorithms for parameter estimation and state decoding. We illustrate the modelling approach in a case study using data from a longitudinal study on delinquent behaviour of adolescents in Germany, revealing temporal persistence in the deviation of an individual's delinquency level from the population mean.