Likelihood Research Articles

Efficient likelihood estimation of Heston model for novel climate-related financial contracts valuation

We propose novel Bitcoin-denominated derivatives contracts on carbon bonds. We consider a futures contract on carbon bonds where its price is expressed in terms of bitcoins. Then, we put forward options on a futures contract of the former type. Governments can use such contracts to hedge climate change and influence the prices of carbon bonds and cryptocurrencies. We show how these derivatives transfer volatility to the bitcoin market without a negative effect in the carbon bonds market. Since the aforementioned options are not yet traded in the market, we price them by assuming that the underlying is driven by the well-known Heston model, where the model parameters are estimated by a novel method based on Shannon wavelets. Heston model belongs to the class of stochastic volatility (SV) models. The discrete observations from the SV model can be seen as a state-space model, that is, a stochastic model in discrete-time which contains two sets of equations, the state equation and the observation equation. While the first describes the transition of a latent process in time, the second shows how an observer measures the latent process at each time period. We infer the properties of the latent variable by means of a filtering algorithm, and we estimate the parameters of the model via maximum likelihood. The evaluation of the likelihood function is a time-consuming task that involves updating and prediction steps of the state variable, leading to the computation of complicated integrals. We calculate these integrals by means of an integration method based on Shannon wavelets, and compare the root mean square error (RMSE) of the estimation with state-of-the-art methods. The results show that the RSME is dramatically reduced in a short CPU time with the use of wavelets.

Conditional independence in a binary choice experiment

Experimental and behavioral economists, as well as psychologists, commonly assume conditional independence of choices when constructing likelihood functions for structural estimation of choice functions. I test this assumption using data from a new experiment designed for this purpose. Within the limits of the experiment’s identifying restriction and designed power to detect deviations from conditional independence, conditional independence is not rejected. In naturally occurring data, concerns about violations of conditional independence are certainly proper and well-taken (for well-known reasons). However, when an experimenter employs the particular experimental mechanisms and designs used here, the findings suggest that conditional independence is an acceptable assumption for analyzing data so generated.

Maximum likelihood weight estimation for partial domain adaptation

Partial Domain Adaptation (PDA) aims to generalize a classification model from a labeled source domain to an unlabeled target domain, where the source label space contains the target label space. There are two main challenges in PDA that weaken the model's classification performance in the target domain: (i) the joint distribution of the source domain is related but different from that of the target domain, and (ii) the source outlier data, whose labels do not belong to the target label space, have a negative impact on learning the target classification model. To tackle these challenges, we propose a Maximum Likelihood Weight Estimation (MLWE) approach to estimate a weight function for the source domain. The weight function matches the joint source distribution of the relevant part to the joint target distribution, and reduces the negative impact of the source outlier data. To be specific, our approach estimates the weight function by maximizing a likelihood function, and the estimation leads to a nice convex optimization problem that has a global optimal solution. In the experiments, our approach demonstrates superior performance on popular benchmark datasets. Intro video and PyTorch code are available at https://github.com/sentaochen/Maximum-Likelihood-Weight-Estimation.

Monitoring of group-structured high-dimensional processes via sparse group LASSO

In a general high-dimensional process, a large number of process parameters or quality characteristics is found to be featured through their dependencies and relevance. The features that have similar characteristics or behaviors in the process operation can be categorized into multiple groups. Thus, when a few quality characteristics in the process change, it is highly probable that the process shift would have occurred in a few relevant groups. Recently, several advanced statistical process control techniques are developed to monitor the changes in high-dimensional processes under sparsity. However, monitoring schemes that utilize the grouped pattern of the quality characteristics are sparse. This paper proposes a new method to monitor the high-dimensional process when the grouped structure of the process data is observed. The proposed method identifies the potentially changed groups and individual variables within the groups based on a modified sparse group LASSO (MSGL) model. Then, a monitoring statistic is obtained using MSGL-based likelihood function to test abnormality of the process. Extensive numerical studies are conducted to demonstrate the effectiveness and efficiency of the proposed method. In addition, a real-life application of a liquefied natural gas process is presented to illustrate the proposed method.

A modified EM-type algorithm to estimate semi-parametric mixtures of non-parametric regressions

Semi-parametric Gaussian mixtures of non-parametric regressions (SPGMNRs) are a flexible extension of Gaussian mixtures of linear regressions (GMLRs). The model assumes that the component regression functions (CRFs) are non-parametric functions of the covariate(s) whereas the component mixing proportions and variances are constants. Unfortunately, the model cannot be reliably estimated using traditional methods. A local-likelihood approach for estimating the CRFs requires that we maximize a set of local-likelihood functions. Using the Expectation-Maximization (EM) algorithm to separately maximize each local-likelihood function may lead to label-switching. This is because the posterior probabilities calculated at the local E-step are not guaranteed to be aligned. The consequence of this label-switching is wiggly and non-smooth estimates of the CRFs. In this paper, we propose a unified approach to address label-switching and obtain sensible estimates. The proposed approach has two stages. In the first stage, we propose a model-based approach to address the label-switching problem. We first note that each local-likelihood function is a likelihood function of a Gaussian mixture model (GMM). Next, we reformulate the SPGMNRs model as a mixture of these GMMs. Lastly, using a modified version of the Expectation Conditional Maximization (ECM) algorithm, we estimate the mixture of GMMs. In addition, using the mixing weights of the local GMMs, we can automatically choose the local points where local-likelihood estimation takes place. In the second stage, we propose one-step backfitting estimates of the parametric and non-parametric terms. The effectiveness of the proposed approach is demonstrated on simulated data and real data analysis.

Efficient estimation of true fixed-effects stochastic frontier models

Fixed-effects modeling has become the method of choice in several panel data settings, including models for stochastic frontier analysis. A notable instance of stochastic frontier panel data models is the true fixed-effects model, which allows disentangling unit heterogeneity from efficiency evaluations. While such a model is theoretically appealing, its estimation is hampered by incidental parameters. This note proposes a simple and rather general estimation approach where the unit-specific intercepts are integrated out of the likelihood function. We apply the theory of composite group families to the model of interest and demonstrate that the resulting integrated likelihood is a marginal likelihood with desirable inferential properties. The derivation of the result is provided in full, along with some connections with the existing literature and computational details. The method is illustrated for three notable models, given by the normal-half normal model, the heteroscedastic exponential model, and the normal-gamma model. The results of simulation experiments highlight the properties of the methodology.

Probabilistic Prediction of Floating Offshore Wind Turbine Platform Motions via Uncertainty Quantification and Information Integration

The accurate prediction of short-term platform motions in a real environment is crucial for the safe design, operation, and maintenance of floating offshore wind turbines (FOWTs). Numerical simulations of motions are typically associated with high uncertainties due to abstracted theoretical models, empirical parameters, initial environment parameters, etc. Therefore, it is necessary to integrate other sources of information associated with less uncertainty, e.g., monitoring data, for accurate predictions. In this paper, we propose a probabilistic prediction based on the Bayesian approach that logically integrates motion monitoring data with simulated motion predictions of FOWTs, considering uncertainties in the environment model, structural properties, motion prediction method, monitoring data, etc. The approach consists of constructing a prior probability density function (PDF) of a random variable (which characterizes the largest value of the initial motion response) via numerical simulations and a likelihood function based on platform motion monitoring data and deriving a posterior PDF of the random variable by Bayesian updating. Then, posterior distributions of short-term extreme motion responses are derived using the posterior PDF of the random variable, representing lower uncertainty and improved accuracy. A Metropolis–Hastings algorithm is adopted to obtain PDFs of complex probability distributions. The effectiveness of the approach is demonstrated on a real FOWT platform in Scotland. The proposed probabilistic prediction approach results in posterior distributions of short-term extreme platform motions associated with less uncertainty and higher accuracy, which is attributed to integrating prior knowledge with monitoring data.

THE UNINFORMATIVE PRIOR OF JEFFREYS’ DISTRIBUTION IN BAYESIAN GEOGRAPHICALLY WEIGHTED REGRESSION

When using the Bayesian method for estimating parameters in a geographically weighted regression model, the choice of the prior distribution directly impacts the posterior distribution. The distribution known as the Jeffreys prior is an uninformative type of prior distribution and is invariant to reparameterization. In cases where information about the parameter is not available, the Jeffreys' prior is utilized. The data was fitted with an uninformative Jeffreys' prior distribution, which yielded a posterior distribution that was utilized for estimating parameters. This study aims to derive the prior and marginal posterior distributions of the Jeffreys' and in Bayesian geographically weighted regression (BGWR). The marginal posterior distributions of and can be obtained by integrating the other parameters of a common posterior distribution. Based on the results and discussion, the Jeffreys prior in BGWR with the likelihood function is . On the other hand, the marginal posterior distribution of follows a normal multivariate distribution, that is, , while the marginal posterior distribution of follows an inverse gamma distribution, that is, . As further research, it is necessary to follow up on several limitations of the results of this research, namely numerical simulations and application to a particular case that related to the results of the analytical studies that we have carried out.

A Robust Mismatch Removal Method for Image Matching Based on the Fusion of the Local Features and the Depth

Feature point matching is a fundamental task in computer vision such as vision simultaneous localization and mapping (VSLAM) and structure from motion (SFM). Due to the similarity or interference of features, mismatches are often unavoidable. Therefore, how to eliminate mismatches is important for robust matching. Smoothness constraint is widely used to remove mismatch, but it cannot effectively deal with the issue in the rapidly changing scene. In this paper, a novel LCS-SSM (Local Cell Statistics and Structural Similarity Measurement) mismatch removal method is proposed. LCS-SSM integrates the motion consistency and structural similarity of a local image block as the statistical likelihood of matched key points. Then, the Random Sampling Consensus (RANSAC) algorithm is employed to preserve the isolated matches that do not satisfy the statistical likelihood. Experimental and comparative results on the public dataset show that the proposed LCS-SSM can effectively and reliably differentiate true and false matches compared with state-of-the-art methods, and can be used for robust matching in scenes with fast motion, blurs, and clustered noise.

Analysis of the Likelihood Function and Cutoff Threshold in the GLUE Procedure for Calibration of the Resistance Parameters of Mountain Rivers

Analysis of the Likelihood Function and Cutoff Threshold in the GLUE Procedure for Calibration of the Resistance Parameters of Mountain Rivers

Optimal parameter estimation for uncertain structural systems under unknown random excitations

An optimal parameter estimation method for uncertain structural systems under unknown random excitations is proposed, which combines the Bayesian inference and stochastic dynamics using geometrically averaged likelihood and optimal response estimation. The general description of optimal parameter estimation problems for uncertain dynamic systems under only stochastic responses measured is presented. The posterior probability density conditional on measured responses is expressed by the likelihood function conditional on system parameters based on Bayes’ theorem. For finite time processes, the parameter estimation problem as the probability integral of conditional means is converted into the optimization problem expressed as maximizing the posterior probability density. A geometrically averaged likelihood function is defined and used for calculating the logarithmic posterior probability density. This estimation can avoid the numerical singularity of the likelihood function and reduce the effects of incomplete posterior probability density and inaccurate prior statistics of unknown random excitations, and then it will be more reasonable and effective. Furthermore, the differential equations for system response means and covariances are derived and solved based on stochastic dynamics theory. The means and covariances conditional on responses at the present instant are expressed by the previous statistics based on optimal response estimation. By combining two results, the analytical expressions of the averaged likelihood function and logarithmic posterior probability density are obtained which will be more reliable and accurate. The proposed optimal estimation method is verified by numerical results for a five-storey frame structure under base random excitation. The estimated results are not affected by the prior statistics errors of random excitation as a factor. For noisy observation, the Kalman filtering is incorporated in the estimation method and hence the estimation is more accurate and robust even for low signal noise ratios. The optimal estimation method has the potential for application to general uncertain systems.

Integrated deviance information criterion for spatial autoregressive models with heteroskedasticity

In this study, we introduce the integrated deviance information criterion (DIC) for nested and non-nested model selection problems in heteroskedastic spatial autoregressive models. In a Bayesian estimation setting, we assume that the idiosyncratic error terms of our spatial autoregressive model have a scale mixture of normal distributions, where the scale mixture variables are latent variables that induce heteroskedasticity. We first derive the integrated likelihood function by analytically integrating out the scale mixture variables from the complete-data likelihood function. We then use the integrated likelihood function to formulate the integrated DIC measure. We investigate the finite sample performance of the integrated DIC in selecting the true model in a simulation study. The simulation results show that the integrated DIC performs satisfactorily and can be useful for selecting the correct model in specification search exercises. Finally, in a spatially augmented economic growth model, we use the integrated DIC to choose the spatial weights matrix that leads to better predictive accuracy.

Bayesian inference for the onset time and epidemiological characteristics of emerging infectious diseases.

Emerging infectious diseases pose a significant threat to global public health. Timely detection and response are crucial in mitigating the spread of such epidemics. Inferring the onset time and epidemiological characteristics is vital for accelerating early interventions, but accurately predicting these parameters in the early stages remains challenging. We introduce a Bayesian inference method to fit epidemic models to time series data based on state-space modeling, employing a stochastic Susceptible-Exposed-Infectious-Removed (SEIR) model for transmission dynamics analysis. Our approach uses the particle Markov chain Monte Carlo (PMCMC) method to estimate key epidemiological parameters, including the onset time, the transmission rate, and the recovery rate. The PMCMC algorithm integrates the advantageous aspects of both MCMC and particle filtering methodologies to yield a computationally feasible and effective means of approximating the likelihood function, especially when it is computationally intractable. To validate the proposed method, we conduct case studies on COVID-19 outbreaks in Wuhan, Shanghai and Nanjing, China, respectively. Using early-stage case reports, the PMCMC algorithm accurately predicted the onset time, key epidemiological parameters, and the basic reproduction number. These findings are consistent with empirical studies and the literature. This study presents a robust Bayesian inference method for the timely investigation of emerging infectious diseases. By accurately estimating the onset time and essential epidemiological parameters, our approach is versatile and efficient, extending its utility beyond COVID-19.

Hierarchical multinomial processing tree models for meta-analysis of diagnostic accuracy studies

Abstract Meta-analysis represents a widely accepted approach for evaluating the accuracy of diagnostic tools in clinical and psychological investigations. This article investigates the applicability of multinomial tree models recently suggested in the literature under a fixed-effects formulation for assessing the accuracy of binary classification tools, where the study specific disease prevalences are taken into account. The model proposed in this article extends previous results to a hierarchical structure accounting for the variability between the studies included in the meta-analysis. Interestingly, by exploiting the parameter separability of the complete likelihood function, the resulting hierarchical multinomial tree model is shown to coincide, in its interest parameter component, with the well-known bivariate random-effects model under an exact within-study distribution for the number of true positives and true negatives subjects. The proposal is in line with a latent-trait approach, where inference follows a frequentist point of view. The applicability of the proposed model and its performance with respect to its fixed-effects counterpart and to the approximate bivariate random-effects model based on normality assumptions commonly used in the literature is evaluated in a series of simulation studies. Methods are applied to a real meta-analysis about the accuracy of the confusion assessment method as delirium screening tool.

Modeling Bellman-error with logistic distribution with applications in reinforcement learning

In modern Reinforcement Learning (RL) approaches, optimizing the Bellman error is a critical element across various algorithms, notably in deep Q-Learning and related methodologies. Traditional approaches predominantly employ the mean-squared Bellman error (MSELoss) as the standard loss function. However, the assumption of Bellman errors following the Gaussian distribution may oversimplify the nuanced characteristics of RL applications. In this work, we revisit the distribution of Bellman error in RL training, demonstrating that it tends to follow the Logistic distribution rather than the commonly assumed Normal distribution. We propose replacing MSELoss with a Logistic maximum likelihood function (LLoss) and rigorously test this hypothesis through extensive numerical experiments across diverse online and offline RL environments. Our findings consistently show that integrating the Logistic correction into the loss functions of various baseline RL methods leads to superior performance compared to their MSE counterparts. Additionally, we employ Kolmogorov–Smirnov tests to substantiate that the Logistic distribution offers a more accurate fit for approximating Bellman errors. This study also offers a novel theoretical contribution by establishing a clear connection between the distribution of Bellman error and the practice of proportional reward scaling, a common technique for performance enhancement in RL. Moreover, we explore the sample-accuracy trade-off involved in approximating the Logistic distribution, leveraging the Bias–Variance decomposition to mitigate excessive computational resources. The theoretical and empirical insights presented in this study lay a significant foundation for future research, potentially advancing methodologies, and understanding in RL, particularly in the distribution-based optimization of Bellman error.

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

BackgroundA Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.MethodsWe compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.ResultsFor most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.ConclusionsWe recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

Parameter Estimation in Spatial Autoregressive Models with Missing Data and Measurement Errors

This study addresses the problem of parameter estimation in spatial autoregressive models with missing data and measurement errors in covariates. Specifically, a corrected likelihood estimation approach is employed to rectify the bias in the log-maximum likelihood function induced by measurement errors. Additionally, a combination of inverse probability weighting (IPW) and mean imputation is utilized to mitigate the bias caused by missing data. Under several mild conditions, it is demonstrated that the proposed estimators are consistent and possess oracle properties. The efficacy of the proposed parameter estimation process is assessed through Monte Carlo simulation studies. Finally, the applicability of the proposed method is further substantiated using the Boston Housing Dataset.

A constrained maximum likelihood approach to developing well-calibrated models for predicting binary outcomes

The added value of candidate predictors for risk modeling is routinely evaluated by comparing the performance of models with or without including candidate predictors. Such comparison is most meaningful when the estimated risk by the two models are both unbiased in the target population. Very often data for candidate predictors are sourced from nonrepresentative convenience samples. Updating the base model using the study data without acknowledging the discrepancy between the underlying distribution of the study data and that in the target population can lead to biased risk estimates and therefore an unfair evaluation of candidate predictors. To address this issue assuming access to a well-calibrated base model, we propose a semiparametric method for model fitting that enforces good calibration. The central idea is to calibrate the fitted model against the base model by enforcing suitable constraints in maximizing the likelihood function. This approach enables unbiased assessment of model improvement offered by candidate predictors without requiring a representative sample from the target population, thus overcoming a significant practical challenge. We study theoretical properties for model parameter estimates, and demonstrate improvement in model calibration via extensive simulation studies. Finally, we apply the proposed method to data extracted from Penn Medicine Biobank to inform the added value of breast density for breast cancer risk assessment in the Caucasian woman population.

Approximate inferences for Bayesian hierarchical generalised linear regression models

SummaryGeneralised linear mixed regression models are fundamental in statistics. Modelling random effects that are shared by individuals allows for correlation among those individuals. There are many methods and statistical packages available for analysing data using these models. Most require some form of numerical or analytic approximation because the likelihood function generally involves intractable integrals over the latents. The Bayesian approach avoids this issue by iteratively sampling the full conditional distributions for various blocks of parameters and latent random effects. Depending on the choice of the prior, some full conditionals are recognisable while others are not. In this paper we develop a novel normal approximation for the random effects full conditional, establish its asymptotic correctness and evaluate how well it performs. We make the case for hierarchical binomial and Poisson regression models with canonical link functions, for hierarchical gamma regression models with log link and for other cases. We also develop what we term a sufficient reduction (SR) approach to the Markov Chain Monte Carlo algorithm that allows for making inferences about all model parameters by replacing the full conditional for the latent variables with a considerably reduced dimensional function of the latents. We expect that this approximation could be quite useful in situations where there are a very large number of latent effects, which may be occurring in an increasingly ‘Big Data’ world. In the sequel, we compare our methods with INLA, which is a particularly popular method and which has been shown to be excellent in terms of speed and accuracy across a variety of settings. Our methods appear to be comparable to theirs in terms of accuracy, while INLA was faster, for the settings we considered. In addition, we note that our methods and those of others that involve Gibbs sampling trivially handle parameters that are functions of multiple parameters, while INLA approximations do not. Our primary illustration is for a three‐level hierarchical binomial regression model for data on health outcomes for patients who are clustered within physicians who are clustered within particular hospitals or hospital systems.

On maximizing the likelihood function of general geostatistical models

AbstractGeneral geostatistical models are powerful tools for analyzing spatial datasets. A two‐step estimation based on the likelihood function is widely used by researchers, but several theoretical and computational challenges remain to be addressed. First, it is unclear whether there is a unique global maximizer of the log‐likelihood function, a seemingly simple but theoretically challenging question. The second challenge is the convexity of the log‐likelihood function. Besides these two challenges in maximizing the likelihood function, we also study the theoretical property of the two‐step estimation. Unlike many previous works, our results can apply to the non‐twice differentiable covariance functions. In the simulation studies, three optimization algorithms are evaluated in terms of maximizing the log‐likelihood functions.