Canonical Link Research Articles

Instrumental Variable Estimation in Generalized Linear Measurement Error Models

Abstract Instrumental variable estimation in generalized linear measurement error models are studied. For models with canonical link functions, unbiased estimating equations are derived. The maximum likelihood estimator for the normal theory, structural linear instrumental variable model is shown to be a solution to the estimating equations derived herein. Logistic regression is studied in detail. An example is given and a simulation study described for the logistic model based on the Framingham Heart Study data.

Monte Carlo Exact Conditional Tests for Log-Linear and Logistic Models

SUMMARY The form of the exact conditional distribution of a sufficient statistic for the interest parameters, given a sufficient statistic for the nuisance parameters, is derived for a generalized linear model with canonical link. General results for log-linear and logistic models are given. A Gibbs sampling approach for generating from the conditional distribution is proposed, which enables Monte Carlo exact conditional tests to be performed. Examples include tests for goodness of fit of the all-two-way interaction model for a 28-table and of a simple logistic model. Tests against non-saturated alternatives are also considered.

Practical Small-Sample Asymptotics for Regression Problems

Abstract Saddlepoint approximations are derived for sums of independent, but not necessarily identically distributed random variables, along with generalizations to estimating equations and multivariate problems. These results are particularly useful for accurately approximating the distribution of regression coefficients. General formulas are given for the distribution of the coefficients arising out of a generalized linear model with both canonical and noncanonical link functions. We illustrate the case of logistic regression with a real data example and show how the Gibbs sampler may be used to obtain confidence sets for each regression parameter based on the saddlepoint approximation.

Practical Small-Sample Asymptotics for Regression Problems

Abstract Saddlepoint approximations are derived for sums of independent, but not necessarily identically distributed random variables, along with generalizations to estimating equations and multivariate problems. These results are particularly useful for accurately approximating the distribution of regression coefficients. General formulas are given for the distribution of the coefficients arising out of a generalized linear model with both canonical and noncanonical link functions. We illustrate the case of logistic regression with a real data example and show how the Gibbs sampler may be used to obtain confidence sets for each regression parameter based on the saddlepoint approximation.

Testing the Fit of a Regression Model Via Score Tests in Random Effects Models

This paper considers testing the goodness-of-fit of regression models. Emphasis is on a goodness-of-fit test for generalized linear models with canonical link function and known dispersion parameter. The test is based on the score test for extra variation in a random effects model. By choosing a suitable form for the dispersion matrix, a goodness-of-fit test statistic is obtained which is quite similar to test statistics based on non-parametric kernel methods. We consider the distribution of the test statistic and discuss the choice of the dispersion matrix. The testing method can handle models with continuous and discrete covariates. Corrections for bias when parameters are estimated are available and extensions to models with unknown dispersion parameters, and more general nonlinear models are discussed. The proposed goodness-of-fit method is demonstrated in a simulation study and on real data of bone marrow transplant patients. The individual contributions of observations to the test statistic are used to perform residual analyses.

Bias Correction in Generalised Linear Mixed Models with a Single Component of Dispersion

General expressions are derived for the asymptotic biases in three approximate estimators of regression coefficients and variance component, for small values of the variance component, in generalised linear mixed models with canonical link function and a single source of extraneous variation. The estimators involve first and second order Laplace expansions of the integrated likelihood and a related procedure known as penalised quasi-likelihood. Numerical studies of a series of matched pairs of binary outcomes show that the first order estimators of the variance component are seriously biased. Easily computed correction factors produce satisfactory estimators of small variance components, comparable to those obtained with a second order Laplace expansion, and markedly improve the asymptotic performance for larger values. For a series of matched pairs of binomial observations, the variance correction factors rapidly approach one as the binomial denominators increase. These results greatly extend the range of parameter values for which the approximate estimation procedures have satisfactory asymptotic properties.

Bias correction in generalised linear mixed models with a single component of dispersion

General expressions are derived for the asymptotic biases in three approximate estimators of regression coefficients and variance component, for small values of the variance component, in generalised linear mixed models with canonical link function and a single source of extraneous variation. The estimators involve first and second order Laplace expansions of the integrated likelihood and a related procedure known as penalised quasi-likelihood. Numerical studies of a series of matched pairs of binary outcomes show that the first order estimators of the variance component are seriously biased. Easily computed correction factors produce satisfactory estimators of small variance components, comparable to those obtained with a second order Laplace expansion, and markedly improve the asymptotic performance for larger values. For a series of matched pairs of binomial observations, the variance correction factors rapidly approach one as the binomial denominators increase. These results greatly extend the range of parameter values for which the approximate estimation procedures have satisfactory asymptotic properties.

Adapting for the Missing Link

We consider the fitting of generalized linear models in which the link function is assumed to be unknown, and propose the following computational method: First, estimate regression coefficients using the canonical link. Then, estimate the link via a kernel smoother, treating the direction in the predictor space determined by the regression coefficients as known. Then reestimate the direction using the estimated link and alternate between these two steps. We show that under fairly general conditions, $n^{1/2}$-consistent estimates of the direction are obtained. A small Monte Carlo study is presented.

Graduation by dynamic regression methods.

This paper extends the theory of graduation by parametric formulae to include dynamic estimation methods. This is an application of the Kaiman filter and allows the parameters of the curve fitted to vary with age. The amount of variation is determined by the amount of smoothing required, and the method can be regarded as a combination of curve fitting and sequential smoothing, each of which has been used separately for performing graduations. In practice, a dynamic straight line can always be used for the graduation and the method has a sensible logical interpretation. these papers concentrate on the use of parametric modelling methods to obtain a reasonably close fit to actual experience, giving due weight to model parsimony and smoothness. In doing this, it is usual to fit a curve (a straight line, cubic, quartic, etc.) which is assumed to apply to the data over the whole range of ages considered. It is sometimes the case that a straight line produces an adequate fit (to the transformations of the death rate considered in this paper), but greater flexibility can be obtained by using higher order polynomials, non-linear regression functions or, within the context of generalised linear models, non- canonical link functions. This paper proposes a different method of obtaining greater flexibility than can be obtained by simply fitting a straight line. The technique proposed has greater heuristic appeal than fitting higher order polynomials, and the form of the flexibility used appears to be very natural in the context of graduation. It involves a sophisticated smoothing procedure (retaining the parametric form of the regression), which can also be related to more basic methods of smoothing which seem natural for graduation, and have been used in the past. When, for example,

Small sample intervals for generalized linear regression

Small sample confidence intervals are constructed for inference in a generalized linear model. Intervals for a real parameter θ are calculated using the empirical distribution of the observed values of the influence curve for θ. Assumptions about adequacy of the experimental design lead to robust behaviour of the confidence intervals. The model considered is that of an I×J table of counts. These can be viewed as many J-dimensional multinomial responses or as IJ poisson observations with row totals fixed. Maximum likelihood estimates are used to estimate the parameters β for multinomial response with canonical link. The small sample intervals which are constructed for the model are not symmehc about the parameterestimate. Confidence intervals for linear combinations of components of β are compared with some asymptotic intervals, obtained from the normal approximation to the signed root likelihood. Comparisons are also made with the simpler asymptotic intervals obtained from the asymptotic normality of the ...

Posterior mode estimation for the generalized linear model

Posterior mode estimators are proposed, which arise from simply expressed prior opinion about expected outcomes, roughly as follows: a conjugate family of prior distributions is determined by a given variance function. Using a conjugate prior, a posterior mode estimator and its estimated (co-)variances are obtained through conventional maximum likelihood computations, by means of small alterations to the observed outcomes and/or to the modelled variance function. Within the conjugate family, for purposes of inference about the regression vector, a reference prior is proposed for a given choice of linear design of the canonical link. The resulting approximate reference inferences approximate the Bayesian inferences which arise from a “minimally informative” reference prior. A set of subjective prior upper and lower percentage points for the expected outcomes can be used to determine a conjugate family member. Alternatively, a set of subjective prior means and standard deviations determines a member. The subfamily of priors determinable by percentage points either includes or approximates the proposed reference prior.

The compactification of generalized linear models

The main purpose of this paper is to unify and extend the existing theory of ‘estimated zeroes’ in log‐linear and logit models. To this end it is shown that every generalized linear model (GLM) can be embedded in a larger model with a compact parameter space and a continuous likelihood (a ‘CGLM’). Clearly in a CGLM the maximum likelihood estimate (MLE) always exists, easing a major data analysis problem. In the mean‐value parametrization, the construction of the CGLM is remarkably simple; except in a rather pathological and rare case, the estimated expected values are always finite., In the β‐parametrization however, the compactification is more complex; the MLE need not correspond with a finite β, as is well known for estimated zeros in log‐linear models. The boundary distributions of CGLMs are classified in four categories: ‘Inadmissible’, ‘degenerate’, ‘Chentsov’, and ‘constrained’. For a large class of GLMs, including all GLMs with canonical link functions and probit models, the MLE in the corresponding CGLM exists and is unique. Even stronger, the likelihood has no other local maxima. We give equivalent algebraic and geometric conditions (in the vein of Haberman (1974, 1977) and Albert and Anderson (1984) respectively), necessary for the existence of the MLE in the GLM corresponding to a finite β. For a large class of GLMs these conditions are also sufficient. Even for log‐linear models this seams to be a new result.

On Bayesian Analysis of Generalized Linear Models Using Jeffreys's Prior

Abstract Generalized linear models (GLM's) have proved suitable for modeling various kinds of data consisting of exponential family response variables with covariates. Bayesian analysis of such data requires specification of a prior for the regression parameters in the model used. Uniform priors are very commonly used as conventional noninformative priors. We show, however, that uniform priors for GLM's can lead to improper posterior distributions thus making them undesirable. Alternative reference priors may be constructed from Jeffreys's rule. In this article, we give two theorems that support the use of Jeffreys's priors for GLM's with intrinsically fixed or known scale parameters. These theorems provide (i) sufficient and (ii) necessary and sufficient conditions for the propriety of the (i) posterior and (ii) prior distributions as well as for the existence of moments. Implications of these theorems for some commonly used GLM's are discussed. Finally, an illustrative example is given for the binomial model with canonical link (i.e., logistic regression), and results using Jeffreys's priors are compared with those based on other non-informative and informative priors.

Bootstrap for generalized linear models

We consider the distribution of the (standardized) ML-estimator of the unknown parameter vector in a Generalized Linear Model with canonical link function. It will be shown that its (parametric) Bootstrap estimator is consistent under the same assumptions needed by Fahrmeir & Kaufmann (1985, 1986) to show its asymptotic normality.

Unbiased estimation of a nonlinear function a normal mean with application to measurement err oorf models

Let W be a normal random variable with mean μand known variance σ2. Conditions on the function f(·) are given under which there exists an unbiased estimator, f(W), of f(μ) for all real μ. In particular it is shown that f(·) must be an entire function over the complex plane. Infinite series solutions for F(·) are obtained which are shown to be valid under growth conditions of the derivatives, fk( ·), of f(·). Approximate solutions are given for the cases in which no exact solution exists. The theory is applied to nonlinear measurement-error models as a means of finding unbiased score functions when measurement error is normally distributed. Relative efficiencies comparing the proposed method to the use of conditional scores (Stefanski and Carroll, 1987) are given for the Poisson regression model with canonical link.

Approximate Conditional Inference in Generalized Linear Models

SUMMARY Easily calculated and accurate approximations are developed to the conditional densities and distributions of sufficient statistics in generalized linear models with canonical link functions. They enable conditional inferences based on modified profile log-likelihoods and tail probabilities to be made using only the deviance and the variance matrix estimate based on fitted models. Examples are given in binary logistic regression and a log-linear model, and the results are applied to added variable tests of model adequacy.

Covariate measurement error in generalized linear models

SUMMARY The EM algorithm is used to obtain estimators of regression coefficients for generalized linear models with canonical link when normally distributed covariates are masked by normally distributed measurement errors. By casting the true covariates as 'missing data', the EM procedure suggests an iterative scheme in which each cycle consists of an E-step, requiring the computation of approximate first and second conditional moments of the true covariates given the observed data, followed by an M-step in which regression parameters are updated by iteratively reweighted least squares based on these approximations. The proposed procedure is compared numerically with the exact maximum likelihood solution, obtained by using Gaussian quadrature instead of the approximations in the E-step of the EM algorithm, and with alternative estimators for simple logistic regression with measurement error. The results for the proposed procedure are encouraging. A procedure is proposed in this paper for estimating regression coefficients in generalized linear models when one or more of the covariates is measured with error. This work is related to the paper by Carroll et al. (1984) in which maximum likelihood estimates are considered for the structural logistic regression model. These authors provide estimates for the computationally simpler probit regression model and suggest that the structural logistic regression can be done in principle. This paper follows up on that suggestion but differs in detail from similar work by Armstrong (1985). Other procedures with related aims have been suggested by Stefanski & Carroll (1985), Wolter & Fuller (1982) and Prentice (1982). For a quick introduction to the proposed method it is convenient to display the naive estimator which ignores measurement error in its standard computational form. If yi is an observed response variable and xi is the observed covariate, then, for the model which equates the canonical parameter of the distribution of yi to xtf3, the maximum likelihood estimator of f3 can be obtained by iteratively reweighted least-squares. The estimate of f3 after (s +1) cycles is given by

Communications and Radar Receiver Gains for Minimum Average Cost of Excluding Randomly Fluctuating Signals in Random Noise

The problem of automatic gain control is approached from a statistical point of view. A simple generic equation is found whose solution yields the required receiver gain or attenuation for minimum average cost of excluding (from the receiver's limited dynamic range) randomly fluctuating signals in random noise. A canonical phase-incoherent link is considered and the resulting transcendental equation is solved using an iterative technique. The analysis and the results obtained apply to both linear and nonlinear incoherent receivers including those of the logarithmic or lin-log type and to a range of fluctuation models including Rician, Rayleigh, and nonfluctuating cases. It is shown that the optimum receiver gain is relatively insensitive to the ratio of costs of saturation at the upper and lower dynamic range bounds, differing at most by about 3 dB from the optimum for the equal cost (minimum exclusion probability) case for typical parameters. The effect of noise introduced by the gain adjustment cascade itself is discussed. The results, presented in concise normalized form, are applicable to a wide range of signal, noise, and channel conditions and have important implications for communications through fading channels as well as for radar observation of fluctuating targets.