Unknown Nuisance Parameters Research Articles

Asymptotic Form of the Covariance Matrix of Likelihood-Based Estimator in Multidimensional Linear System Model for the Case of Infinity Number of Nuisance Parameters

This article is devoted to the synthesis and analysis of the quality of the statistical estimate of parameters of a multidimensional linear system (MLS) with one input and m outputs. A nontrivial case is investigated when the one-dimensional input signal of MLS is a deterministic process, the values of which are unknown nuisance parameters. The estimate is based only on observations of MLS output signals distorted by random Gaussian stationary m-dimensional noise with a known spectrum. It is assumed that the likelihood function of observations of the output signals of MLS satisfies the conditions of local asymptotic normality. The n-consistency of the estimate is established. Under the assumption of asymptotic normality of an objective function, the limiting covariance matrix of the estimate is calculated for case where the number of observations tends to infinity.

Comparison of known spawner abundance from fence counts to visual counts for simplified spawner estimation methods

Many salmon species are monitored by visual counts of spawners in streams; however, there are few data sets where abundance is known and compared to estimates derived from visual counts. We used spawner fences to obtain known kokanee ( Oncorhynchus nerka) spawner abundance (14 stream-years) on streams that are monitored with annual visual surveys (7 to 9 counts per year) and incorporated similar published data from pink salmon ( Oncorhynchus gorbuscha) (11 stream-years). We investigated the performance of several simplified expansion factor estimation methods with survey life and observer efficiency as unknown nuisance parameters. All visual indices of kokanee and pink salmon spawners from live ground counts were highly correlated to abundance from fence counts ( r2 ≥ 0.96 and 0.89, respectively). Application of cross-validation on out-of-sample data for both species showed that mean% error could range from 13% to 53% on a previously unsampled stream depending on the species, counting method, and visual index used. Predictive performance metrics were less sensitive to counting frequency than observer efficiency and associated variability, which was influenced by the counting method (aerial versus ground surveys).

Specification tests for GARCH processes with nuisance parameters on the boundary

This paper develops tests for the correct specification of the conditional variance function in GARCH models when the true parameter may lie on the boundary of the parameter space. The test statistics considered are of Kolmogorov-Smirnov and Cramér-von Mises type, and are based on empirical processes marked by centered squared residuals. The limiting distributions of the test statistics depend on unknown nuisance parameters in a non-trivial way, making the tests difficult to implement. We therefore introduce a novel bootstrap procedure which is shown to be asymptotically valid under general conditions, irrespective of the presence of nuisance parameters on the boundary. The proposed bootstrap approach is based on shrinking of the parameter estimates used to generate the bootstrap sample toward the boundary of the parameter space at a proper rate. It is simple to implement and fast in applications, as the associated test statistics have simple closed form expressions. Although the bootstrap test is designed for a data generating process with fixed parameters (i.e., independent of the sample size n), we also discuss how to obtain valid inference for sequences of DGPs with parameters approaching the boundary at the rate. A simulation study demonstrates that the new tests: (i) have excellent finite sample behaviour in terms of empirical rejection probabilities under the null as well as under the alternative; (ii) provide a useful complement to existing procedures based on Ljung-Box type approaches. Two data examples illustrate the implementation of the proposed tests in applications.

Community Detection With Known, Unknown, or Partially Known Auxiliary Latent Variables

Empirical observations suggest that in practice, community membership does not completely explain the dependency between the edges of an observation graph. The residual dependence of the graph edges are modeled in this paper, to first order, by auxiliary node latent variables that affect the statistics of the graph edges but <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">carry no information about the communities of interest.</i> We then study community detection in graphs obeying the stochastic block model and censored block model with auxiliary latent variables. We analyze the conditions for exact recovery when these auxiliary latent variables are unknown, representing unknown nuisance parameters or model mismatch. We also analyze exact recovery when these secondary latent variables have been either fully or partially revealed. Finally, we propose a semidefinite programming algorithm for recovering the desired labels when the secondary labels are either known or unknown. We show that exact recovery is possible by semidefinite programming down to the respective maximum likelihood exact recovery threshold.

Testing equivalence of clustering

In this paper, we test whether two data sets measured on the same set of subjects share a common clustering structure. As a leading example, we focus on comparing clustering structures in two independent random samples from two deterministic two-component mixtures of multivariate Gaussian distributions. Mean parameters of these Gaussian distributions are treated as potentially unknown nuisance parameters and are allowed to differ. Assuming knowledge of mean parameters, we first determine the phase diagram of the testing problem over the entire range of signal-to-noise ratios by providing both lower bounds and tests that achieve them. When nuisance parameters are unknown, we propose tests that achieve the detection boundary adaptively as long as ambient dimensions of the data sets grow at a sublinear rate with the sample size.

Estimation of location parameter within pre-specified error bound with second-order efficient two-stage procedure

This paper develops a general approach for constructing a confidence interval for a parameter of interest with a specified confidence coefficient and a specified width. This is done assuming known a positive lower bound for the unknown nuisance parameter and independence of suitable statistics. Under mild conditions, we develop a modified two-stage procedure which enjoys attractive optimality properties including a second-order efficiency property and asymptotic consistency property. We extend this work for finding a confidence interval for the location parameter of the inverse Gaussian distribution. As an illustration, we developed a modified mean absolute deviation-based procedure in the supplementary section for finding a fixed-width confidence interval for the normal mean.

Unified Bayesian theory of sparse linear regression with nuisance parameters

We study frequentist asymptotic properties of Bayesian procedures for high-dimensional Gaussian sparse regression when unknown nuisance parameters are involved. Nuisance parameters can be finite-, high-, or infinite-dimensional. A mixture of point masses at zero and continuous distributions is used for the prior distribution on sparse regression coefficients, and appropriate prior distributions are used for nuisance parameters. The optimal posterior contraction of sparse regression coefficients, hampered by the presence of nuisance parameters, is also examined and discussed. It is shown that the procedure yields strong model selection consistency. A Bernstein-von Mises-type theorem for sparse regression coefficients is also obtained for uncertainty quantification through credible sets with guaranteed frequentist coverage. Asymptotic properties of numerous examples are investigated using the theory developed in this study.

Optimal two-stage sampling for mean estimation in multilevel populations when cluster size is informative

To estimate the mean of a quantitative variable in a hierarchical population, it is logistically convenient to sample in two stages (two-stage sampling), i.e. selecting first clusters, and then individuals from the sampled clusters. Allowing cluster size to vary in the population and to be related to the mean of the outcome variable of interest (informative cluster size), the following competing sampling designs are considered: sampling clusters with probability proportional to cluster size, and then the same number of individuals per cluster; drawing clusters with equal probability, and then the same percentage of individuals per cluster; and selecting clusters with equal probability, and then the same number of individuals per cluster. For each design, optimal sample sizes are derived under a budget constraint. The three optimal two-stage sampling designs are compared, in terms of efficiency, with each other and with simple random sampling of individuals. Sampling clusters with probability proportional to size is recommended. To overcome the dependency of the optimal design on unknown nuisance parameters, maximin designs are derived. The results are illustrated, assuming probability proportional to size sampling of clusters, with the planning of a hypothetical survey to compare adolescent alcohol consumption between France and Italy.

Distribution Approximation of Shrinkage Estimate in Censored Regression Model via Randomly Weighting Method

Censored regression (“Tobit”) model is a special case of limited dependent variable regression model, and plays an important role in econometrics. Based on this model, all kinds of methods for variable or group variable selection have been developed and the corresponding shrinkage parameter estimates are widely studied. However, asymptotic distributions of the shrinkage estimates involve unknown nuisance parameters, such as density function of error term. To avoid estimating nuisance parameters, this paper presents a randomly weighting method to approximate to the asymptotic distribution of the shrinkage estimate. A computation procedure of random approximation is provided and asymptotic properties of the randomly weighting estimates are also obtained. The proposed methods are evaluated with extensively numerical studies and a women labor supply example.

Robust estimation in single-index models when the errors have a unimodal density with unknown nuisance parameter

This paper develops a robust profile estimation method for the parametric and nonparametric components of a single-index model when the errors have a strongly unimodal density with unknown nuisance parameter. We derive consistency results for the link function estimators as well as consistency and asymptotic distribution results for the single-index parameter estimators. Under a log-Gamma model, the sensitivity to anomalous observations is studied using the empirical influence curve. We also discuss a robust K-fold cross-validation procedure to select the smoothing parameters. A numerical study carried on with errors following a log-Gamma model and for contaminated schemes shows the good robustness properties of the proposed estimators and the advantages of considering a robust approach instead of the classical one. A real data set illustrates the use of our proposal.

The Risk-Unbiased Cramér–Rao Bound for Non-Bayesian Multivariate Parameter Estimation

How accurately can one estimate a deterministic parameter subject to other unknown deterministic model nuisance parameters? The most popular answer to this question is given by the Cramer–Rao bound (CRB). The main assumption behind the derivation of the CRB is local unbiased estimation of all model parameters. The foundations of this paper rely on doubting this assumption. Generally, in multivariate parameter estimation, each parameter in its turn can be treated as a single parameter of interest, whereas the other model parameters are treated as nuisance, as their misknowledge interferes with the estimation of the parameter of interest. This approach is utilized in this paper to provide a fresh look at deterministic parameter estimation. A new Cramer–Rao (CR) type bound is derived without assuming unbiased estimation of the nuisance parameters. Rather than that, we apply Lehmann's concept of unbiasedness for a risk that measures the distance between the estimator and the locally best unbiased estimator, which assumes perfect knowledge of the nuisance parameters. The proposed risk-unbiased CRB (RUCRB) is proven to be asymptotically attainable by the maximum likelihood estimator while being tighter than the conventional CRB. Furthermore, simulations verify the asymptotic achievability of the RUCRB by the maximum likelihood estimator for an array processing problem.

Sensor system optimization to meet reliability targets

In this work, we show the influence of sensor system measurement uncertainties to sensor system reliability and ways to meet reliability targets. A general model to handle measurement uncertainties is defined and the according influence to reliability is presented, which is defined as probability of meeting specification requirements. Initial step is to optimize sensor systems concerning lowest influences of sensor system parameter fluctuations to the measurement uncertainty using statistical optimization methodologies. In case the influence of unknown nuisance parameters cannot be sufficiently suppressed, such parameters may be additionally measured in order to further reduce measurement uncertainties. The remaining uncertainties are again addressed using statistical optimization methodologies. Finally, measurement uncertainty also affects the reliability of such a system. For sensor systems in safety critical applications it may thus be required to include measures such as redundancy. This is also included in the investigations. Further examples for explained optimization methodologies of measurement uncertainty reduction are presented.

A perspective on recent methods on testing predictability of asset returns

This paper highlights some recent developments in testing predictability of asset returns with focuses on linear mean regressions, quantile regressions and nonlinear regression models. For these models, when predictors are highly persistent and their innovations are contemporarily correlated with dependent variable, the ordinary least squares estimator has a finite-sample bias, and its limiting distribution relies on some unknown nuisance parameter, which is not consistently estimable. Without correcting these issues, conventional test statistics are subject to a serious size distortion and generate a misleading conclusion in testing predictability of asset returns in real applications. In the past two decades, sequential studies have contributed to this subject and proposed various kinds of solutions, including, but not limit to, the bias-correction procedures, the linear projection approach, the IVX filtering idea, the variable addition approaches, the weighted empirical likelihood method, and the double-weight robust approach. Particularly, to catch up with the fast-growing literature in the recent decade, we offer a selective overview of these methods. Finally, some future research topics, such as the econometric theory for predictive regressions with structural changes, and nonparametric predictive models, and predictive models under a more general data setting, are also discussed.

Minimum Hellinger distance estimation for a semiparametric location-shifted mixture model

ABSTRACTIn this article, we propose a minimum Hellinger distance estimation (MHDE) for a semiparametric two-component mixture model where the two components are unknown location-shifted symmetric distributions and . In the construction of MHDE, an appropriate estimation of the unknown nuisance parameter f is required. We propose to use the inversion formula given in Bordes et al. to estimate f based on current available sample from the mixture. To obtain the MHDE, an algorithm is presented to ease the numerical calculation. We also propose a simple but intuitive and robust initial estimator of the parameters. To assess its performance, we carry out a simulation study with comparison with a minimum profile Hellinger distance estimator (MPHDE) given in Wu et al. We use the proposed estimator to analyse the Old Faithful Geyser data in order to demonstrate its application. Through the numerical studies, we observe that our proposed MHDE for this semiparametric mixture model inherits the desired robustness and efficiency properties of that for parametric models. The proposed MHDE is very competitive with the MPHDE when there is no data contamination, whereas it performs better than the MPHDE in terms of bias when data is contaminated with outliers. Moreover, the MHDE reduces significantly the computing time of the MPHDE.

Facilitating the Calculation of the Efficient Score Using Symbolic Computing

ABSTRACTThe score statistic continues to be a fundamental tool for statistical inference. In the analysis of data from high-throughput genomic assays, inference on the basis of the score usually enjoys greater stability, considerably higher computational efficiency, and lends itself more readily to the use of resampling methods than the asymptotically equivalent Wald or likelihood ratio tests. The score function often depends on a set of unknown nuisance parameters which have to be replaced by estimators, but can be improved by calculating the efficient score, which accounts for the variability induced by estimating these parameters. Manual derivation of the efficient score is tedious and error-prone, so we illustrate using computer algebra to facilitate this derivation. We demonstrate this process within the context of a standard example from genetic association analyses, though the techniques shown here could be applied to any derivation, and have a place in the toolbox of any modern statistician. We further show how the resulting symbolic expressions can be readily ported to compiled languages, to develop fast numerical algorithms for high-throughput genomic analysis. We conclude by considering extensions of this approach. The code featured in this report is available online as part of the supplementary material.

A Two-Stage Latent Variable Estimation Procedure for Time-Censored Accelerated Degradation Tests

Parallel constant-stress accelerated degradation testing (PCSADT) is widely used to assess the reliability of highly reliable products in a timely manner when the products’ degradation can be measured. Under a time-censored PCSADT, several groups of units are tested simultaneously, but under different stress levels, until a prespecified censoring time is reached. At this time, degradation values from the censored units, and failure times of the failed units are obtained. When the degradation follows a Wiener process where the parameters depend on the stress level through a life-stress model containing an unknown nuisance parameter, estimating this parameter often biases the maximum likelihood and least-squares estimators of the lifetime parameters. In this paper, we propose a two-stage procedure to address this problem. In the first stage, we transform the data under the different stress levels of a PCSADT so that the resulting data can be considered to have been obtained under normal stress. In the second stage, we introduce a latent variable for the unobserved degradation after the failure time for each failed unit to obtain a pseudodegradation value at the censoring time. We then use all degradation values (pseudo or observed) at the censoring time to develop latent variable estimators for all model parameters. Unlike other existing estimators, the proposed estimators are shown to be s-consistent, have closed-form expressions, and are easy to interpret. We use a real example of light-emitting diodes to illustrate the proposed method. In addition to proving s-consistencies, we conduct a simulation study to demonstrate that the proposed estimators also perform well in finite samples.

Parameter Estimation in Sensor Networks Under Probabilistic Censoring

We consider the problem of estimating an unknown parameter that is observed indirectly by a sensor network. Each of the sensors exhibits probabilistic nonlinear behavior causing censoring (clipping) of the signals at random. The signals then get transmitted over fading wireless channels to the fusion center. We develop three algorithms for estimating the unknown parameter: first is based on Pearson's Method of Moments, which involves the calculation of the statistical moments of the model that can be derive exactly; second is based on the Maximum-Likelihood Estimator , which results in an intractable likelihood function. To evaluate the likelihood function, we develop a novel approximation via a nonparametric probability density estimator that is based on series expansions of the Gram–Charlier family of basis functions; and third is the Marginalized Least Squares in which we solve the least squares problem by marginalizing out the random unknown nuisance parameters of the model. All three algorithms enjoy a low computational complexity, and only involve solving a one-dimensional optimization problem that is simple to implement in practice. We compare the performance of the proposed algorithms under various system configurations and show various tradeoffs between the algorithms as a function of system parameters (e.g., number of sensors, frame length, signal-to-noise ratio, sensing quality etc.).

The Bayesian D-optimal design for simple beta regression model in two cases (both known and unknown nuisance parameter)

ABSTRACTAccording to investigated topic in the context of optimal designs, various methods can be used to obtain optimal design, of which Bayesian method is one. In this paper, considering the model and the features of the information matrix, this method (Bayesian optimality criterion) has been used for obtaining optimal designs which due to the variation range of the model parameters, prior distributions such as Uniform, Normal and Exponential have been used and the results analysed.

Goodness-of-Fit Test in a Structural Errors-in-Variables Model Based on a Score Function

A polynomial structural measurement error model is considered. A goodness-of-fit test is constructed based on the quasi-likelihood estimator, which is asymptotically optimal in a large class of estimators. The power of the test is discussed. The test for the linear model with unknown nuisance parameters is studied in more detail. Similar test can be applied to much more general situation, where the estimator is constructed based on a score function.

Aligned rank tests in measurement error model

Aligned rank tests are introduced in the linear regression model with possible measurement errors. Unknown nuisance parameters are estimated first and then classical rank tests are applied on the residuals. Two situations are discussed: testing about an intercept in the linear regression model considering the slope parameter as nuisance and testing of parallelism of several regression lines, i.e. whether the slope parameters of all lines are equal. Theoretical results are derived and the simulation study is also made to illustrate good performance of introduced tests.