Expected Information Matrix Research Articles

The information matrix of the bivariate extended skew-normal distribution

AbstractFor the extended skew-normal distribution, which represents an extension of the normal (or Gaussian) distribution, we focus on the properties of the log-likelihood function and derived quantities in the bivariate case. Specifically, we derive explicit expressions for the score function and the information matrix, in the observed and the expected form; these do not appear to have been examined before in the literature. Ensuing numerical work illustrates some relevant aspects of the expected information matrix.

Issues with the expected information matrix of linear mixed models provided by popular statistical packages under missingness at random dropout.

Likelihood-based methods ignoring missingness at random (MAR) produce consistent estimates provided that the whole likelihood model is correct. However, the expected information matrix (EIM) depends on the missingness mechanism. It has been shown that calculating the EIM by considering the missing data pattern as fixed (naive EIM) is incorrect under MAR, but the observed information matrix (OIM) is valid under any MAR missingness mechanism. In longitudinal studies, linear mixed models (LMMs) are routinely applied, often without any reference to missingness. However, most popular statistical packages currently provide precision measures for the fixed effects by inverting only the corresponding submatrix of the OIM (naive OIM), which is effectively equivalent to the naive EIM. In this paper, we analytically derive the correct form of the EIM of LMMs under MAR dropout to compare its differences with the naive EIM, which clarifies why the naive EIM fails under MAR. The asymptotic coverage rate of the naive EIM is numerically calculated for two parameters (population slope and slope difference between two groups) under various dropout mechanisms. The naive EIM can severely underestimate the true variance, especially when the degree of MAR dropout is high. Similar trends emerge under misspecified covariance structure, where, even the full OIM may lead to incorrect inferences and sandwich/bootstrap estimators are generally required. Results from simulation studies and application to real data led to similar conclusions. In LMMs, the full OIM should be preferred to the naive EIM/OIM, though if misspecified covariance structure is suspected, robust estimators should be used.

Likelihood-Based Inference for the Asymmetric Exponentiated Bimodal Normal Model

Asymmetric probability distributions have been widely studied by various authors in recent decades, who have introduced new families of flexible distributions in terms of skewness and kurtosis than the classical distributions known in statistical theory. Most of the new distributions fit unimodal data, others fit bimodal data, however, in the bimodal, singularity problems have been found in their information matrices in most of the proposals presented. In contrast, in this paper an extension of the family of alpha-power distributions was developed, which has a non-singular information matrix, based on the bimodal-normal and bimodal elliptic-skew-normal probability distributions. These new extensions model asymmetric bimodal data commonly found in various areas of scientific interest. The properties of these new probabilistic distributions were also studied in detail and the respective statistical inference process was carried out to estimate the parameters of these new models. The stochastic convergence for the vector of maximum likelihood estimators could be found due to the non-singularity of the expected information matrix in the corresponding support.

MLE-Based Parameter Estimation for Four-Parameter Exponential Gamma Distribution and Asymptotic Variance of Its Quantiles

The choice of a probability distribution function and confidence interval of estimated design values have long been of interest in flood frequency analysis. Although the four-parameter exponential gamma (FPEG) distribution has been developed for application in hydrology, its maximum likelihood estimation (MLE)-based parameter estimation method and asymptotic variance of its quantiles have not been well documented. In this study, the MLE method was used to estimate the parameters and confidence intervals of quantiles of the FPEG distribution. This method entails parameter estimation and asymptotic variances of quantile estimators. The parameter estimation consisted of a set of four equations which, after algebraic simplification, were solved using a three dimensional Levenberg-Marquardt algorithm. Based on sample information matrix and Fisher’s expected information matrix, derivatives of the design quantile with respect to the parameters were derived. The method of estimation was applied to annual precipitation data from the Weihe watershed, China and confidence intervals for quantiles were determined. Results showed that the FPEG was a good candidate to model annual precipitation data and can provide guidance for estimating design values.

Power Calculations for ZIP and ZINB Models

We present power calculations for zero-inflated Poisson (ZIP) and zero-inflated negative-binomial (ZINB) models. We detail direct computations for a ZIP model based on a two-sample Wald test using the expected information matrix. We also demonstrate how Lyles, Lin, and Williamson’s method (2006) of power approximation for categorical and count outcomes can be extended to both zero-inflated models. This method can be used for power calculations based on the Wald test (via the observed information matrix) and the likelihood ratio test, and can accommodate both categorical and continuous covariates. All the power calculations can be conducted when covariates are used in the modeling of both the count data and the “excess zero” data, or in either part separately. We present simulations to detail the performance of the power calculations. Analysis of a malaria study is used for illustration.

The Kumaraswamy Generalized Half-Normal Distribution for Skewed Positive Data

For the first time, we propose and study the Kumaraswamy generalized half-normal distribution for modeling skewed positive data. The half-normal and generalized half-normal (Cooray and Ananda, 2008) distributions are special cases of the new model. Various of its structural properties are derived, including explicit expressions for the density function, moments, generating and quantile functions, mean deviations and moments of the order statistics. We investigate maximum likelihood estimation of the parameters and derive the expected information matrix. The proposed model is modified to open the possibility that long-term survivors may be presented in the data. Its applicability is illustrated by means of four real data sets.

The Asymmetric Power-Student-t Model for Censored and Truncated Data.

In this paper, we propose the power Student-t regression model for censored (limited) observations which extends the Student-t censored regression model. This extension is based on the asymmetric and heavy-tailed power Student-t distribution. The score functions and expected information matrix are given as well as the process for estimating the parameters in the model is discussed by using the likelihood approach. Two simulation studies are conducted to evaluate parameter recovery and properties of the model and finally, two applications to a real data set are reported to demonstrate the usefulness of this new methodology.

Chi-square Difference Tests for Comparing Nested Models: An Evaluation with Non-normal Data

ABSTRACT The relative fit of two nested models can be evaluated using a chi-square difference statistic. We evaluate the performance of five robust chi-square difference statistics in the context of confirmatory factor analysis with non-normal continuous outcomes. The mean and variance corrected difference statistics performed adequately across all conditions investigated. In contrast, the mean corrected difference statistics required larger samples for the p-values to be accurate. Sample size requirements for the mean corrected difference statistics increase as the degrees of freedom for difference testing increase. We recommend that the mean and variance corrected difference testing be used whenever possible. When performing mean corrected difference testing, we recommend that the expected information matrix is used (i.e., choice MLM), as the use of the observed information matrix (i.e., choice MLR) requires larger samples for p-values to be accurate. Supplementary materials for applied researchers to implement difference testing in their own research are provided.

A bivariate Kumaraswamy-exponential distribution with application

Abstract In this paper, we introduce a new bivariate Kumaraswamy exponential distribution, whose marginals are univariate Kumaraswamy exponential. Some probabilistic properties of this bivariate distribution are derived, such as joint density function, marginal density functions, conditional density functions, moments and stress-strength reliability. Also, we provide the expected information matrix with its elements in a closed form. Estimation of the parameters is investigated by the maximum likelihood, Bayesian and least squares estimation methods. A simulation study is carried out to compare the performance of the estimators by estimation methods. Further, one data set have been analyzed to show how the proposed distribution works in practice.

The Azzalini Skew-t Information Matrix Evaluation and Use for Standard Error Calculations

The Azzalini skew-t distributions are popular because of their theoretical foundation and the availability of computational methods in the R package sn. One difficulty with this skew-t family is that the elements of the expected information matrix do not have closed form analytic formulas. Thus, we developed a numerical integration method of computing the expected information matrix in the R package skewtInfo. The accuracy of our expected information matrix calculation method was confirmed by comparing the result with that obtained using an observed information matrix for a very large sample size. A Monte Carlo study to evaluate the accuracy of the finite-sample standard errors obtained with our expected information matrix calculation method, for the case of three realistic skew-t parameter vectors, indicates that use of the expected information matrix results in standard errors as accurate as, and sometimes a little more accurate than, use of an observed information matrix.

A New Class of Beta-Complementary Exponential Power Series Distributions

Abstract In this paper, we introduce a new class of beta-complementary exponential power series distributions, which is obtained by replacing the cumulative distribution function of the complementary exponential power series distributions in the logit of a beta random variable. This new class of distributions contains some new submodels, such as beta-complementary exponential geometric, beta-complementary exponential Poisson, beta-complementary exponential binomial, beta-complementary exponential logarithmic, and complementary exponential power series distributions. A general class of distributions is presented and some various properties are obtained in this paper. We characterize the beta-complementary exponential geometric distribution as one of the most applicable distributions in this class. Some mathematical properties of the beta-complementary exponential geometric distribution are reached in terms of the corresponding properties of the complementary exponential geometric and beta exponential distributions. We present expressions for the probability density function, cumulative distribution function, moment generating function, and moments. The estimation of parameters is approached by the maximum likelihood estimation procedure, and the expected information matrix is derived. The flexibility of the distribution is illustrated in application of a real data set.

Optimal design for experiments with possibly incomplete observations

Missing responses occur in many industrial or medical experiments, for example in clinical trials where slow acting treatments are assessed. Finding efficient designs for such experiments can be problematic since it is not known at the design stage which observations will be missing. The design literature mainly focuses on assessing robustness of designs for missing data scenarios, rather than finding designs which are optimal in this situation. Imhof, Song and Wong (2002) propose a framework for design search, based on the expected information matrix. We develop a new approach which includes Imhof, Song and Wong (2002)'s method as special case and justifies its use retrospectively. Our method is illustrated through a simulation study based on real data from an Alzheimer's disease trial.

Testing for sub-models of the skew t-distribution

The skew t-distribution includes both the skew normal and the normal distributions as special cases. Inference for the skew t-model becomes problematic in these cases because the expected information matrix is singular and the parameter corresponding to the degrees of freedom takes a value at the boundary of its parameter space. In particular, the distributions of the likelihood ratio statistics for testing the null hypotheses of skew normality and normality are not asymptotically $$\chi ^2$$ . The asymptotic distributions of the likelihood ratio statistics are considered by applying the results of Self and Liang (J Am Stat Assoc 82:605–610, 1987) for boundary-parameter inference in terms of reparameterizations designed to remove the singularity of the information matrix. The Self–Liang asymptotic distributions are mixtures, and it is shown that their accuracy can be improved substantially by correcting the mixing probabilities. Furthermore, although the asymptotic distributions are non-standard, versions of Bartlett correction are developed that afford additional accuracy. Bootstrap procedures for estimating the mixing probabilities and the Bartlett adjustment factors are shown to produce excellent approximations, even for small sample sizes.

Profile-likelihood Confidence Intervals in Item Response Theory Models

ABSTRACTConfidence intervals (CIs) are fundamental inferential devices which quantify the sampling variability of parameter estimates. In item response theory, CIs have been primarily obtained from large-sample Wald-type approaches based on standard error estimates, derived from the observed or expected information matrix, after parameters have been estimated via maximum likelihood. An alternative approach to constructing CIs is to quantify sampling variability directly from the likelihood function with a technique known as profile-likelihood confidence intervals (PL CIs). In this article, we introduce PL CIs for item response theory models, compare PL CIs to classical large-sample Wald-type CIs, and demonstrate important distinctions among these CIs. CIs are then constructed for parameters directly estimated in the specified model and for transformed parameters which are often obtained post-estimation. Monte Carlo simulation results suggest that PL CIs perform consistently better than Wald-type CIs for both non-transformed and transformed parameters.

Maximum Likelihood Estimation of Structural Equation Models for Continuous Data: Standard Errors and Goodness of Fit

Classical accounts of maximum likelihood (ML) estimation of structural equation models for continuous outcomes involve normality assumptions: standard errors (SEs) are obtained using the expected information matrix and the goodness of fit of the model is tested using the likelihood ratio (LR) statistic. Satorra and Bentler (1994) introduced SEs and mean adjustments or mean and variance adjustments to the LR statistic (involving also the expected information matrix) that are robust to nonnormality. However, in recent years, SEs obtained using the observed information matrix and alternative test statistics have become available. We investigate what choice of SE and test statistic yields better results using an extensive simulation study. We found that robust SEs computed using the expected information matrix coupled with a mean- and variance-adjusted LR test statistic (i.e., MLMV) is the optimal choice, even with normally distributed data, as it yielded the best combination of accurate SEs and Type I errors.

The Kumaraswamy Laplace Distribution

A generalized Laplace distribution using the Kumaraswamy distribution is introduced. Different structural properties of this new distribution are derived, including the moments, and the moment generating function. We discuss maximum likelihood estimation of the model parameters and obtain the observed and expected information matrix. A real data set is used to compare the new model with widely known distributions.

Extended arcsine distribution to proportional data: Properties and applications

We propose a new two-parameter continuous model called the extended arcsine distribution restricted to the unit interval. It is a very competitive model to the beta and Kumaraswamy distributions for modeling percentages, rates, fractions and proportions. We provide a mathematical treatment of the new distribution including explicit expressions for the ordinary and incomplete moments, mean deviations, Bonferroni and Lorenz curves, generating and quantile functions, Shannon entropy and order statistics. Maximum likelihood is used to estimate the model parameters and the expected information matrix is determined. We demonstrate by means of two applications to proportional data that it can give consistently a better fit than other important statistical models.

Approximate Methods for Maximum Likelihood Estimation of Multivariate Nonlinear Mixed-Effects Models

Multivariate nonlinear mixed-effects models (MNLMM) have received increasing use due to their flexibility for analyzing multi-outcome longitudinal data following possibly nonlinear profiles. This paper presents and compares five different iterative algorithms for maximum likelihood estimation of the MNLMM. These algorithmic schemes include the penalized nonlinear least squares coupled to the multivariate linear mixed-effects (PNLS-MLME) procedure, Laplacian approximation, the pseudo-data expectation conditional maximization (ECM) algorithm, the Monte Carlo EM algorithm and the importance sampling EM algorithm. When fitting the MNLMM, it is rather difficult to exactly evaluate the observed log-likelihood function in a closed-form expression, because it involves complicated multiple integrals. To address this issue, the corresponding approximations of the observed log-likelihood function under the five algorithms are presented. An expected information matrix of parameters is also provided to calculate the standard errors of model parameters. A comparison of computational performances is investigated through simulation and a real data example from an AIDS clinical study.

Global Influence Diagnostics in Gaussian Spatial Linear Model with Multiple Repetitions

Repeated measures of data have been widely analyzed in many fields, such as biology and medicine, but we can find a few works that study repeated measures in the geostatistical field. In this work, we present global diagnostics techniques for case deletion to assess the influence of observations on Gaussian spatial linear models with multiple replications. We present Cook's distance likelihood-based and Q-function-based with one-step approximation. Moreover, we propose to use the expected information matrix and the expectation of the second derivative of the Q-function as part of this measure. An application to real data illustrates the methodology developed. The results show that the methods are effective on the detection of influential observations.

The beta log-logistic distribution

A new continuous distribution, so-called the beta log-logistic distribution, that extends the log-logistic distribution and some other distributions is proposed and studied. The new model is quite flexible to analyze positive data. Various structural properties of the new distribution are derived, including explicit expressions for the moments, mean deviations and Renyi and Shannon entropies. The score function is derived and the estimation of the model parameters is performed by maximum likelihood. We also determine the expected information matrix. The usefulness of the new model is illustrated by means of two real data sets. We hope that the new distribution proposed here will serve as an alternative model to other models available in the literature for modeling positive real data in many areas.