Expected Fisher Information Matrix Research Articles

Multiple arbitrarily inflated Poisson regression analysis

In this paper, a new flexible count regression analysis is proposed. For this purpose, a new modification of the Poisson distribution is introduced which generalizes the Poisson, zero-inflated Poisson, zero-one inflated Poisson, and zero-one-two inflated Poisson distributions. Some distributional properties are discussed for the proposed distribution. The Fisher scoring and EM algorithms are derived to attain the maximum likelihood estimates of the unknown parameters. An expected Fisher information matrix is provided to construct an approximate confidence interval for the parameters. Using the modified Poisson distribution, an arbitrary multiply inflated counting regression model is proposed. The performance of the maximum likelihood methodology is investigated with a simulation study for the distribution and count regression model. Finally, two practical data sets are analyzed and the superiority of the proposed model is demonstrated among others.

Scalable Physics-Based Maximum Likelihood Estimation Using Hierarchical Matrices

Physics-based covariance models provide a systematic way to construct covariance models that are consistent with the underlying physical laws in Gaussian process analysis. The unknown parameters in the covariance models can be estimated using maximum likelihood estimation, but direct construction of the covariance matrix and classical strategies of computing with it require physical model runs, storage complexity, and computational complexity. To address such challenges, we propose to approximate the discretized covariance function using hierarchical matrices. By utilizing randomized range sketching for individual off-diagonal blocks, the construction process of the hierarchical covariance approximation requires physical model applications and the maximum likelihood computations require effort per iteration. We propose a new approach to compute exactly the trace of products of hierarchical matrices which results in the expected Fisher information matrix being computable in as well. The construction is totally matrix-free and the derivatives of the covariance matrix can then be approximated in the same hierarchical structure by differentiating the whole process. Numerical results are provided to demonstrate the effectiveness, accuracy, and efficiency of the proposed method for parameter estimations and uncertainty quantification.

Estimation of Multicomponent Stress-strength Reliability under Inverse Topp-Leone Distribution

In this article, the reliability inference for a multicomponent stress-strength (MSS) model, when both stress and strength random variables follow inverse Topp-Leone distributions, was studied. The maximum likelihood and uniformly minimum variance unbiased estimates for the reliability of MSS model were obtained explicitly. The exact Bayes estimate of MSS reliability was derived the under squared error loss function. Also, the Bayes estimate was obtained using the Monte Carlo Markov Chain method for comparison with the aforementioned exact estimate. The asymptotic confidence interval was determined under the expected Fisher information matrix. Furthermore, the highest probability density credible interval was established through using Gibbs sampling method. Monte Carlo simulations were implemented to compare the different proposed methods. Finally, a real life example was presented in support of the suggested procedures.

Optimal design of experiments for optimization-based model calibration using Fisher information matrix

Statistical model calibration to infer unknown model parameters and model bias has been widely developed through comparison between simulation response and experimental data. Bayesian-based model calibration typically represented as Kennedy and O'Hagan (KOH) framework and optimization-based model calibration have been proposed, but efforts on optimization of experimental design to reduce the epistemic uncertainty minimizing experimental resources are still limited. Furthermore, when an unknown model parameter has natural and uncontrollable variability, the estimation may be much more difficult since both aleatory and epistemic uncertainties exist. In this paper, we have developed a framework to find the optimal design of experiments (DoE) satisfying the target information gain for inference of unknown model parameters based on optimization-based model calibration. The expected Fisher information matrix is approximated to quantify the expected information gain for the maximum likelihood estimation (MLE). Namely, the necessary number of experiments at each experimental design can be obtained to attain desired precision on estimators while minimizing the overall experimental cost. The numerical study verifies the feasibility of the proposed framework, which means that there is certainly a dominant DoE that gives more information to the inference on specific model parameters.

Reliability analysis of Weibull multicomponent system with stress‐dependent parameters from accelerated life data

AbstractIn this paper, reliability estimation of multicomponent system under a multilevel accelerated life testing. When the lifetime of components follows Weibull distribution, the problem of point and interval estimates are discussed from different perspectives. Under a general life‐stress assumption that there are multiple nonconstant and stress‐dependent scale and shape parameters, the maximum likelihood estimates of unknown parameters along with associated existence and uniqueness are established. Approximate confidence intervals are constructed correspondingly via expected Fisher information matrix. Furthermore, some pivotal quantities are constructed and alternative generalized point and interval estimates are also proposed for comparison. In addition, predictive intervals for the lifetime of the multicomponent system are discussed under classical and generalized pivotal approaches, respectively. The results show that the proposed generalized estimates are superior to the conventional likelihood approach in terms of the accuracy. A real data example is carried out to illustrate the implementations of the proposed methods.

Statistical inference for mixture GARCH models with financial application

In this paper we consider mixture generalized autoregressive conditional heteroskedastic models, and propose a new iteration algorithm of type EM for the estimation of model parameters. The maximum likelihood estimates are shown to be consistent, and their asymptotic properties are investigated. More precisely, we derive simple expressions in closed form for the asymptotic covariance matrix and the expected Fisher information matrix of the ML estimator. Finally, we study the model selection and propose testing procedures. A simulation study and an application to financial real-series illustrate the results.

On a progressively censored competing risks data from Gompertz distribution

We study a competing risks model using Gompertz distribution under progressive Type-II censoring when probability distributions of failure causes are identically distributed with common scale and different shape parameters. Maximum likelihood estimates (MLEs) of these parameters are obtained and their uniqueness and existence behavior are also discussed. The asymptotic intervals are derived from the observed Fisher information matrix. We compare the performance of all the estimators numerically using simulations. Analysis of a real data set is presented as well. We further determine optimal censoring scheme using expected Fisher information matrix. The design parameters are selected based on suitable measures like cost-based and variance-based criteria functions. Finally, we discuss single and multi-objective optimization approaches to find optimal censoring schemes.

Inference of progressively type-II censored competing risks data from Chen distribution with an application

In this paper, the estimation of unknown parameters of Chen distribution is considered under progressive Type-II censoring in the presence of competing failure causes. It is assumed that the latent causes of failures have independent Chen distributions with the common shape parameter, but different scale parameters. From a frequentist perspective, the maximum likelihood estimate of parameters via expectation–maximization (EM) algorithm is obtained. Also, the expected Fisher information matrix based on the missing information principle is computed. By using the obtained expected Fisher information matrix of the MLEs, asymptotic 95% confidence intervals for the parameters are constructed. We also apply the bootstrap methods (Bootstrap-p and Bootstrap-t) to construct confidence intervals. From Bayesian aspect, the Bayes estimates of the unknown parameters are computed by applying the Markov chain Monte Carlo (MCMC) procedure, the average length and coverage rate of credible intervals are also carried out. The Bayes inference is based on the squared error, LINEX, and general entropy loss functions. The performance of point estimators and confidence intervals is evaluated by a simulation study. Finally, a real-life example is considered for illustrative purposes.

Statistical inference for Markov chains with applications to credit risk

The focus of this paper is on the derivation of confidence and credibility intervals for Markov chains when discrete-time, continuous-time or discretely observed continuous-time data are available. Thereby, our contribution is threefold: First, we discuss and compare multinomial confidence regions for the rows of discrete-time Markov transition matrices in the light of empirical characteristics of credit rating migrations. Second, we derive an analytical expression for the expected Fisher information matrix of a continuous-time Markov chain which is used to construct credibility intervals using a non-informative Jeffreys prior distribution and a Metropolis-Hastings Algorithm. Third, we concretize profile and estimated/pseudo likelihood based confidence intervals in the continuous-time data settings, which in contrast to asymptotic normality based intervals explicitly consider non-negativity constraints for the parameters. Furthermore, we illustrate the described methods by Moody’s corporate ratings data with exact continuous-time transitions.

Inference on a progressive type I interval-censored truncated normal distribution

In this paper, we consider the problem of making statistical inference for a truncated normal distribution under progressive type I interval censoring. We obtain maximum likelihood estimators of unknown parameters using the expectation-maximization algorithm and in sequel, we also compute corresponding midpoint estimates of parameters. Estimation based on the probability plot method is also considered. Asymptotic confidence intervals of unknown parameters are constructed based on the observed Fisher information matrix. We obtain Bayes estimators of parameters with respect to informative and non-informative prior distributions under squared error and linex loss functions. We compute these estimates using the importance sampling procedure. The highest posterior density intervals of unknown parameters are constructed as well. We present a Monte Carlo simulation study to compare the performance of proposed point and interval estimators. Analysis of a real data set is also performed for illustration purposes. Finally, inspection times and optimal censoring plans based on the expected Fisher information matrix are discussed.

Scalable Gaussian Process Computations Using Hierarchical Matrices

We present a kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes. The proposed approximation provides natural and scalable stochastic estimators for its gradient and Hessian, as well as the expected Fisher information matrix, that are computable in quasilinear complexity for a large range of models. To accomplish this, we (i) choose a specific hierarchical approximation for covariance matrices that enables the computation of their exact derivatives and (ii) use a stabilized form of the Hutchinson stochastic trace estimator. Since both the observed and expected information matrices can be computed in quasilinear complexity, covariance matrices for maximum likelihood estimators (MLEs) can also be estimated efficiently. In this study, we demonstrate the scalability of the method, show how details of its implementation effect numerical accuracy and computational effort, and validate that the resulting MLEs and confidence intervals based on the inverse Fisher information matrix faithfully approach those obtained by the exact likelihood. Supplementary materials for this article are available online.

Statistical Inference of the Rayleigh Distribution Based on Progressively Type II Censored Competing Risks Data

A competing risks model under progressively type II censored data following the Rayleigh distribution is considered. We establish the maximum likelihood estimation for unknown parameters and compute the observed information matrix and the expected Fisher information matrix to construct the asymptotic confidence intervals. Moreover, we obtain the Bayes estimation based on symmetric and non-symmetric loss functions, that is, the squared error loss function and the general entropy loss function, and the highest posterior density intervals are also derived. In addition, a simulation study is presented to assess the performances of different methods discussed in this paper. A real-life data set analysis is provided for illustration purposes.

Statistical Inference for Type I Multiply Left-Censored Samples from Weibull Distribution*

Left-censored data with one or more detection limits occur frequently in many application areas. In this paper, the computational procedure for calculation of maximum likelihood estimates of the parameters for type I multiply left-censored data from underlying Weibull distribution is suggested and used considering various numbers of detection limits. The expected Fisher information matrix is analytically determined, and its performance is compared with sample (observed) Fisher information matrix using simulations. Simulations are focused primarily on the properties of estimators for small sample sizes. Real data illustration is included.

Estimating the parameters of a truncated normal distribution under progressive type II censoring

The problem of making statistical inference for a truncated normal distribution is considered under progressive type II censoring. Maximum likelihood and Bayesian approaches are used to obtain point and interval estimates of unknown parameters. Bayes estimates are derived with respect to informative and non-informative prior distributions when the loss function is squared error. Monte Carlo simulations and real data analysis are presented to study the performance of proposed methods. Finally, optimal censoring plans based on the expected Fisher information matrix are discussed under different optimality criteria.

Analysis and validation of macromolecular B values.

This paper describes a global analysis of macromolecular B values. It is shown that the distribution of B values generally follows the shifted inverse-gamma distribution (SIGD). The parameters of the SIGD are estimated using the Fisher scoring technique with the expected Fisher information matrix. It is demonstrated that a contour plot based on the parameters of the SIGD can play a role in the validation of macromolecular structures. The dependence of the peak-height distribution on resolution and atomic B values is also analysed. It is demonstrated that the B-value distribution can have a dramatically different effect on peak heights at different resolutions. Consequently, a comparative analysis of the B values of neighbouring atoms must account for resolution. A combination of the SIGD, peak-height distribution and outlier detection was used to identify a number of entries from the PDB that require attention. It is also shown that the presence of a multimodal B-value distribution often indicates that some loops or parts of the molecule have either been mismodelled or have dramatically different mobility, depending on their environment within the crystal. These distributions can also indicate the level of sharpening/blurring used before atomic structure refinement. It is recommended that procedures such as sharpening/blurring should be avoided during refinement, although they can play important roles in map visualization and model building.

Robust designs in longitudinal studies accounting for parameter and model uncertainties – application to count data

ABSTRACTNonlinear mixed effect models (NLMEMs) are widely used for the analysis of longitudinal data. To design these studies, optimal designs based on the expected Fisher information matrix (FIM) can be used. A method evaluating the FIM using Monte-Carlo Hamiltonian Monte-Carlo (MC-HMC) has been proposed and implemented in the R package MIXFIM using Stan. This approach, however, requires a priori knowledge of models and parameters, which leads to locally optimal designs. The objective of this work was to extend this MC-HMC-based method to evaluate the FIM in NLMEMs accounting for uncertainty in parameters and in models. When introducing uncertainty in the population parameters, we evaluated the robust FIM as the expectation of the FIM computed by MC-HMC over the distribution of these parameters. Then, the compound D-optimality criterion (CD optimality), corresponding to a weighted product of the D-optimality criteria of several candidate models, was used to find a common CD-optimal design for the set of candidate models. Finally, a compound DE-criterion (CDE optimality), corresponding to a weighted product of the normalized determinants of the robust FIMs of all the candidate models accounting for uncertainty in parameters, was calculated to find the CDE-optimal design which was robust on both parameters and model. These methods were applied in a longitudinal Poisson count model. We assumed prior distributions on the population parameters, as well as several candidate models describing the relationship between the logarithm of the event rate parameter and the dose. We found that assuming uncertainty in parameters could lead to different optimal designs, and misspecification of models could induce designs with low efficiencies. The CD- or CDE-optimal designs therefore provided a good compromise for different candidate models. Finally, the proposed approach allows for the first time optimization of designs for repeated discrete data accounting for parameter and model uncertainties.

Left-Censored Samples from Skewed Distributions: Statistical Inference and Applications

Left‑censored data occur frequently in many areas. At present, researchers pay attention to skewed censored distributions more frequently. This paper deals with statistical inference of type I multiply left‑censored Weibull and exponential distributions. It suggests a computational procedure for calculation of maximum likelihood estimates of the parameters. The expected Fisher information matrix for estimation of variances of estimated parameters is introduced. The estimates are then used for construction of confidence intervals for the expectation using the maximum likelihood method. Asymptotic tests for comparison of distributions (expectations respectively) of two independent left‑censored Weibull samples are proposed. Furthermore, asymptotic tests for assessing suitability of reduction of the Weibull distribution to the exponential distribution are introduced. Finally, the left‑censored exponential distribution is briefly described. Methods derived in this paper are illustrated on elemental carbon measurements, and can be applied in analysis of real environmental and/or chemical data.

PFIM 4.0, an extended R program for design evaluation and optimization in nonlinear mixed-effect models

Background and ObjectiveNonlinear mixed-effect models (NLMEMs) are increasingly used for the analysis of longitudinal studies during drug development. When designing these studies, the expected Fisher information matrix (FIM) can be used instead of performing time-consuming clinical trial simulations. The function PFIM is the first tool for design evaluation and optimization that has been developed in R. In this article, we present an extended version, PFIM 4.0, which includes several new features. MethodsCompared with version 3.0, PFIM 4.0 includes a more complete pharmacokinetic/pharmacodynamic library of models and accommodates models including additional random effects for inter-occasion variability as well as discrete covariates. A new input method has been added to specify user-defined models through an R function. Optimization can be performed assuming some fixed parameters or some fixed sampling times. New outputs have been added regarding the FIM such as eigenvalues, conditional numbers, and the option of saving the matrix obtained after evaluation or optimization. Previously obtained results, which are summarized in a FIM, can be taken into account in evaluation or optimization of one-group protocols. This feature enables the use of PFIM for adaptive designs. The Bayesian individual FIM has been implemented, taking into account a priori distribution of random effects. Designs for maximum a posteriori Bayesian estimation of individual parameters can now be evaluated or optimized and the predicted shrinkage is also reported. It is also possible to visualize the graphs of the model and the sensitivity functions without performing evaluation or optimization. ResultsThe usefulness of these approaches and the simplicity of use of PFIM 4.0 are illustrated by two examples: (i) an example of designing a population pharmacokinetic study accounting for previous results, which highlights the advantage of adaptive designs; (ii) an example of Bayesian individual design optimization for a pharmacodynamic study, showing that the Bayesian individual FIM can be a useful tool in therapeutic drug monitoring, allowing efficient prediction of estimation precision and shrinkage for individual parameters. ConclusionPFIM 4.0 is a useful tool for design evaluation and optimization of longitudinal studies in pharmacometrics and is freely available at http://www.pfim.biostat.fr.

On Testing Reduction of Left-Censored Weibull Distribution to Exponential Submodel

When analyzing environmental or chemical data, it is often necessary to deal with left-censoredobservations. Since the distribution of the observed variable is often asymmetric, the exponential or the Weibulldistribution can be used. This paper summarizes statistical model of a multiply left-censored Weibull distribution,and estimation of its parameters and their variances using the expected Fisher information matrix. Since inmany situations the Weibull distribution is unnecessarily complicated for data modelling, statistical tests (theLagrange multiplier test, the likelihood ratio test, the Wald test) for assessing suitability of replacement ofthe censored Weibull distribution with the exponential submodel are introduced and their power functions areanalyzed using simulations.

Optimum simple accelerated life tests based on progressively Type-I hybrid censoring

This paper presents the process of the optimum test design for simple constant-stress and step-stress accelerated life tests under the progressively Type-I hybrid censoring data. Considering the constant-stress and step-stress accelerated models, the explicit conditional density functions of order statistics under progressively Type-I hybrid censoring scheme are given to obtain the expected Fisher information matrix. By minimizing the asymptotic variance of the mean life under the use stress, the optimum test design is completed by determining the test units allocated to each stress in constant-stress accelerated life test and the changing time to severer accelerated stress in step-stress accelerated life test. Finally, an example is shown to illustrate the optimum test design. The numerical results show that the step-stress accelerated life test is a better choice.