Presence Of Nuisance Parameters Research Articles

The Agreement between the GeneralizedpValue and Bayesian Evidence in the One-Sided Testing Problem

In the problem of testing one-sided hypotheses, a frequentist may measure evidence against the null hypothesis by thepvalue, while a Bayesian may measure it by the posterior probability that the null hypothesis is true. In this paper, we consider the relationship between the generalizedpvalue and the Bayesian evidence in testing one-sided hypotheses in the presence of nuisance parameters. The sufficient conditions for the agreement between these two kinds of evidence are given. Some examples are provided to show the agreement of Bayesian and frequentist evidence in many classical testing problems. This is an illustration of reconcilability of evidence in a general framework where the nuisance parameters are present.

Minimax estimation in sparse canonical correlation analysis

Canonical correlation analysis is a widely used multivariate statistical technique for exploring the relation between two sets of variables. This paper considers the problem of estimating the leading canonical correlation directions in high-dimensional settings. Recently, under the assumption that the leading canonical correlation directions are sparse, various procedures have been proposed for many high-dimensional applications involving massive data sets. However, there has been few theoretical justification available in the literature. In this paper, we establish rate-optimal nonasymptotic minimax estimation with respect to an appropriate loss function for a wide range of model spaces. Two interesting phenomena are observed. First, the minimax rates are not affected by the presence of nuisance parameters, namely the covariance matrices of the two sets of random variables, though they need to be estimated in the canonical correlation analysis problem. Second, we allow the presence of the residual canonical correlation directions. However, they do not influence the minimax rates under a mild condition on eigengap. A generalized sin-theta theorem and an empirical process bound for Gaussian quadratic forms under rank constraint are used to establish the minimax upper bounds, which may be of independent interest.

Irreconcilability of P-value and Bayesian measure in two-sided hypothesis: Pareto distribution with the presence of nuisance parameter

This article is concerned with the comparison of Bayesian and classical testing of a point null hypothesis for the Pareto distribution when there is a nuisance parameter. In the first stage, using a fixed prior distribution, the posterior probability is obtained and compared with the P-value. In the second case, lower bounds of the posterior probability of H0, under a reasonable class of prior distributions, are compared with the P-value. It has been shown that even in the presence of nuisance parameters for the model, these two approaches can lead to different results in statistical inference.

Approximate Non-Similar critical values based tests vs Maximized Monte Carlo tests

Testing in the presence of nuisance parameters is a problem often faced by researchers; consequently, a number of ways are suggested in the literature to manage this situation. Among these, Maximized Monte Carlo (MMC) tests or asymptotically valid MMC (AMMC) tests are becoming popular. The MMC type tests have certain advantages as well as disadvantages. This paper introduces a simple way to obtain Approximate Non-Similar (ANS) critical values using a global optimizer called Simulated Annealing (SA). All three methods are applied in the dynamic linear regression model context. As expected the AMMC approach is certainly less time consuming than the MMC approach. Overall the AMMC approach seems best in terms of power properties; however the ANS approach takes negligible time compared to its competitors. Though the ANS approach controls the sizes well it can be slightly less powerful than its competitors.

Higher-order Bayesian Approximations for Pseudo-posterior Distributions

The theory of higher-order asymptotics provides accurate approximations to posterior distributions for a scalar parameter of interest, and to the corresponding tail area, for practical use in Bayesian analysis. The aim of this article is to extend these approximations to pseudo-posterior distributions, e.g., posterior distributions based on a pseudo-likelihood function and a suitable prior, which are proved to be particularly useful when the full likelihood is analytically or computationally infeasible. In particular, from a theoretical point of view, we derive the Laplace approximation for a pseudo-posterior distribution, and for the corresponding tail area, for a scalar parameter of interest, also in the presence of nuisance parameters. From a computational point of view, starting from these higher-order approximations, we discuss the higher-order tail area (HOTA) algorithm useful to approximate marginal posterior distributions, and related quantities. Compared to standard Markov chain Monte Carlo methods, the main advantage of the HOTA algorithm is that it gives independent samples at a negligible computational cost. The relevant computations are illustrated by two examples.

Non Null Asymptotic Distributions of Some Criteria in Censored Exponential Regression

In this article, we consider the class of censored exponential regression models which is very useful for modeling lifetime data. Under a sequence of Pitman alternatives, the asymptotic expansions up to order n− 1/2 of the non null distribution functions of the likelihood ratio, Wald, Rao score, and gradient statistics are derive in this class of models. The non null asymptotic distribution functions of these statistics are obtained for testing a composite null hypothesis in the presence of nuisance parameters. The power of all four tests, which are equivalent to first order, are compared based on these non null asymptotic expansions. Furthermore, in order to compare the finite-sample performance of these tests in this class of models, we consider Monte Carlo simulations. We also present an empirical application for illustrative purposes.

Approximate Bayesian computation with modified log-likelihood ratios

The aim of this contribution is to discuss approximate Bayesian computation based on the asymptotic theory of modified likelihood roots and log-likelihood ratios. Results on third-order approximations for univariate posterior distributions, also in the presence of nuisance parameters, are reviewed and the computation of asymptotic credible sets for a vector parameter of interest is illustrated. All these approximations are available at little additional computational cost over simple first-order approximations. Some illustrative examples are discussed, with particular attention to the use of matching priors.

Likelihood inference in some finite mixture models

Parametric mixture models are commonly used in applied work, especially empirical economics, where these models are often employed to learn for example about the proportions of various types in a given population. This paper examines the inference question on the proportions (mixing probability) in a simple mixture model in the presence of nuisance parameters when sample size is large. It is well known that likelihood inference in mixture models is complicated due to (1) lack of point identification, and (2) parameters (for example, mixing probabilities) whose true value may lie on the boundary of the parameter space. These issues cause the profiled likelihood ratio (PLR) statistic to admit asymptotic limits that differ discontinuously depending on how the true density of the data approaches the regions of singularities where there is lack of point identification. This lack of uniformity in the asymptotic distribution suggests that confidence intervals based on pointwise asymptotic approximations might lead to faulty inferences. This paper examines this problem in details in a finite mixture model and provides possible fixes based on the parametric bootstrap. We examine the performance of this parametric bootstrap in Monte Carlo experiments and apply it to data from Beauty Contest experiments. We also examine small sample inferences and projection methods.

Marginal Posterior Simulation via Higher-order Tail Area Approximations

We explore the use of higher-order tail area approximations for Bayesian simulation. These approximations give rise to an alternative simulation scheme to MCMC for Bayesian computation of marginal posterior distributions for a scalar parameter of interest, in the presence of nuisance parameters. Its advantage over MCMC methods is that samples are drawn independently with lower computational time and the implementation requires only standard maximum likelihood routines. The method is illustrated by a genetic linkage model, a normal regression with censored data and a logistic regression model.

THE EFFICIENCY OF EMPIRICAL LIKELIHOOD WITH NUISANCE PARAMETERS

In this study we check the asymptotic efficiency of empirical likelihood in the presence of nuisance parameters combined with augmented moment conditions. We show that in the presence of nuisance parameters, the asymptotic efficiency of the empirical likelihood estimator of the parameter of interest will increase by adding more moment conditions, in the sense of the positive semidefiniteness of the difference of information matrices. As a by product, we point out a necessary condition for the asymptotic efficiency to be increased when more moment condition are added.

Change-Point Analysis of Survival Data with Application in Clinical Trials

Effects of many medical procedures appear after a time lag, when a significant change occurs in subjects’ failure rate. This paper focuses on the detection and estimation of such changes which is important for the evaluation and comparison of treatments and prediction of their effects. Unlike the classical change-point model, measurements may still be identically distributed, and the change point is a parameter of their common survival function. Some of the classical change-point detection techniques can still be used but the results are different. Contrary to the classical model, the maximum likelihood estimator of a change point appears consistent, even in presence of nuisance parameters. However, a more efficient procedure can be derived from Kaplan-Meier estimation of the survival function followed by the least-squares estimation of the change point. Strong consistency of these estimation schemes is proved. The finite-sample properties are examined by a Monte Carlo study. Proposed methods are applied to a recent clinical trial of the treatment program for strong drug dependence.

A Practical Procedure to Find Matching Priors for Frequentist Inference

We present a practical way to find matching priors via the use of saddlepoint approximations and obtain p-values of tests of an interest parameter in the presence of nuisance parameters. The advantages of our procedure are the flexibility in choosing different initial conditions so that one may adjust the performance of a test, and the less intensive computational efforts compared to a Markov Chain Monto Carlo method.

Information on parameters of interest decreases under transformations

An important property of Fisher information is that it decreases weakly under transformation of random variables. Kagan and Rao (2003) [A. Kagan, C.R. Rao, Some properties and applications of the efficient Fisher score, J. Statist. Plann. Inference 116 (2003) 343–352] showed that, in the presence of nuisance parameters, Fisher information on the interest parameters decreases similarly. We prove here a general algebraic result on partitioned positive-definite matrices, and use it to show that the decrease in Fisher information on the interest parameters is bounded below by the conditional Fisher information on the interest parameters. A consequence is that standard large-sample confidence regions for parameters of interest based on the deviance, score and Wald statistics become asymptotically ‘wider’ under transformations, both in the context of independent identically distributed random variables and for the binomial detectability models of Fewster and Jupp (2009) [R.M. Fewster, P.E. Jupp, Inference on population size in binomial detectability models, Biometrika 96 (2009) 805–820]. One implication is that models that combine different data sources for inference on the interest parameters are asymptotically more efficient than models for any of the individual data sources, despite the possible need for further nuisance parameters when combining the sources.

Comments on "A Technique for Orthogonal Frequency Division Multiplexing Frequency Offset Correction"

This comment corrects a few errors found in the derivation of the maximum likelihood estimate of differential phase in the paper, "A Technique for Orthogonal Frequency Division Multiplexing Frequency Offset Correction." We show that the problem of differential phase estimation can be considered as an estimation problem in the presence of nuisance parameters, which does not satisfy strong ancillarity. The approach in the above paper to solve this problem can be understood as conditioning for elimination of nuisance parameters but without taking proper steps. After making corrections on the proof, it is demonstrated that the estimator in the above paper is inherently suboptimal and thus prior knowledge on the nuisance parameters, if available, can be utilized to further improve the estimation performance.

Data-dependent probability matching priors for likelihood ratio and adjusted likelihood ratio statistics

We consider likelihood ratio statistics based on the usual profile likelihood and the standard adjustments thereof proposed in the literature in the presence of nuisance parameters. The role of data-dependent priors in ensuring approximate frequentist validity of posterior credible regions based on the inversion of these statistics is investigated. Unlike what happens with data-free priors, it is seen that the resulting probability matching conditions readily admit solutions which entail approximate frequentist validity of the highest posterior density region as well.

Gradient statistic: Higher-order asymptotics and Bartlett-type correction

We obtain an asymptotic expansion for the null distribution function of the gradient statistic for testing composite null hypotheses in the presence of nuisance parameters. The expansion is derived using a Bayesian route based on the shrinkage argument described in [10]. Using this expansion, we propose a Bartlett-type corrected gradient statistic with chi-square distribution up to an error of order $o(n^{-1})$ under the null hypothesis. Further, we also use the expansion to modify the percentage points of the large sample reference chi-square distribution. Monte Carlo simulation experiments and various examples are presented and discussed.

Objective Bayesian higher-order asymptotics in models with nuisance parameters

A higher-order approximation to the marginal posterior distribution for a scalar parameter of interest in the presence of nuisance parameters is proposed. The approximation is obtained using a matching prior. The procedure improves the normal first-order approximation and has several advantages. It does not require the elicitation on the nuisance parameters, neither numerical integration nor Monte Carlo simulation, and it enables us to perform accurate Bayesian inference even for small sample sizes. Numerical illustrations are given for models of practical interest, such as linear non-normal models and logistic regression. Finally, it is shown how the proposed approximation can routinely be applied in practice using results from likelihood asymptotics and the R package bundle hoa.

Marginal Likelihood-based One-sided LR Test for Testing Higher Order Autocorrelation in Presence of Nuisance Parameters-A Distance Based Approach

In most of the cases, only a subvector of the parameters is tested in a model. The remaining parameters arise in the tests as nuisance parameters. The presence of nuisance parameters causes biases in key estimates used in the tests. So inferences made on the presence of nuisance parameters may lead to less accurate conclusions. Even the presence of nuisance parameters can destroy the test. Thus in eliminating the influence of nuisance parameters from the test can improve the tests' performance. The effect of the nuisance parameters can be eliminated by the marginal likelihood, conditional likelihood, canonical likelihood, profile likelihood and Bayesian tests. This paper is concerned with marginal likelihood-based test for eliminating the influence of nuisance parameters. In general, existing one-sided and two-sided tests for autocorrelation are tested only autocorrelation coefficients but not the regression coefficients in the model. So we proposed a distance-based marginal likelihood one-sided Likelihood Ratio (DMLR) test in eliminating the influence of nuisance parameters for testing higher order autocorrelation with one-sided alternatives in linear regression model using marginal likelihood and distance-based approach. Monte Carlo simulations are conducted to compare power properties of the proposed DMLR test with their respective conventional counterparts. It is found that the DMLR test shows substantially improved power for most of cases considered.

On the identifiability problem in the presence of random nuisance parameters

This paper concerns with the identifiability of a vector of unknown deterministic parameters. In many practical applications, the data model is affected by additional random parameters whose estimation is not strictly required, the so-called nuisance parameters. In these cases, the classical definition of identifiability, which requires the calculation of the Fisher Information Matrix (FIM) and of its rank, is often difficult or impossible to perform. Instead, the Modified Fisher Information Matrix (MFIM) can be computed. We generalize the main results on the identifiability problem to take the presence of random nuisance parameters into account. We provide an alternative definition of identifiability that can be always applied but that is weaker than the classical definition, and we investigate the relationships between the identifiability condition and the MFIM. Finally, we apply the obtained results to the identifiability in presence of nuisance parameters to the relative grid-locking problem for netted radar system.

Likelihood-Based Selection and Sharp Parameter Estimation

In high-dimensional data analysis, feature selection becomes one effective means for dimension reduction, which proceeds with parameter estimation. Concerning accuracy of selection and estimation, we study nonconvex constrained and regularized likelihoods in the presence of nuisance parameters. Theoretically, we show that constrained L 0 likelihood and its computational surrogate are optimal in that they achieve feature selection consistency and sharp parameter estimation, under one necessary condition required for any method to be selection consistent and to achieve sharp parameter estimation. It permits up to exponentially many candidate features. Computationally, we develop difference convex methods to implement the computational surrogate through prime and dual subproblems. These results establish a central role of L 0 constrained and regularized likelihoods in feature selection and parameter estimation involving selection. As applications of the general method and theory, we perform feature selection in linear regression and logistic regression, and estimate a precision matrix in Gaussian graphical models. In these situations, we gain a new theoretical insight and obtain favorable numerical results. Finally, we discuss an application to predict the metastasis status of breast cancer patients with their gene expression profiles. This article has online supplementary material.