Computation Of Maximum Likelihood Estimates Research Articles

Improving ERGM starting values using simulated annealing

Much of the theory of estimation for exponential family models, which include exponential-family random graph models (ERGMs) as a special case, is well-established and maximum likelihood estimates (MLEs) in particular enjoy many desirable properties. However, in the case of many ERGMs, direct calculation of MLEs is impossible and therefore methods for approximating MLEs and/or alternative estimation methods must be employed. Many MLE approximation algorithms require an alternative estimate as a starting point. The maximum pseudo-likelihood estimator (MPLE) is frequently taken as this starting point. Here, we discuss a potentially large class of such alternatives based on the fact that, unlike the MLE, the MPLE fails to satisfy the so-called “likelihood principle”. This means that different networks may have different MPLEs even if they have the same sufficient statistics. We exploit this fact here to search for improved starting values for approximation-based MLE methods. The method we propose has shown its merit in producing an MLE for a network dataset and model that had defied estimation using all other known methods.

The multivariate component zero‐inflated Poisson model for correlated count data analysis

SummaryMultivariate zero‐inflated Poisson (ZIP) distributions are important tools for modelling and analysing correlated count data with extra zeros. Unfortunately, existing multivariate ZIP distributions consider only the overall zero‐inflation while the component zero‐inflation is not well addressed. This paper proposes a flexible multivariate ZIP distribution, called the multivariate component ZIP distribution, in which both the overall and component zero‐inflations are taken into account. Likelihood‐based inference procedures including the calculation of maximum likelihood estimates of parameters in the model without and with covariates are provided. Simulation studies indicate that the performance of the proposed methods on the multivariate component ZIP model is satisfactory. The Australia health care utilisation data set is analysed to demonstrate that the new distribution is more appropriate than the existing multivariate ZIP distributions.

Computing maximum likelihood estimates for Gaussian graphical models with Macaulay2

We introduce the package "GraphicalModelsMLE" for computing the maximum likelihood estimates (MLEs) of a Gaussian graphical model in the computer algebra system Macaulay2. This package allows the computation of MLEs for the class of loopless mixed graphs. Additional functionality allows the user to explore the underlying algebraic structure of the model, such as its maximum likelihood degree and the ideal of score equations.

An Automated Profile-Likelihood-Based Algorithm for Fast Computation of the Maximum Likelihood Estimate in a Statistical Model for Crash Data

Numerical computation of maximum likelihood estimates (MLE) is one of the most common problems encountered in applied statistics. Even if there exist many algorithms considered as performing, they can suffer in some cases for one or many of the following criteria: global convergence (capacity of an algorithm to converge to the true unknown solution from all starting guesses), numerical stability (ascent property), implementation feasibility (for example, algorithms requiring matrix inversion cannot be implemented when the involved matrices are not invertible), low computation time, low computational complexity, and capacity to handle high dimensional problems. The reality is that, in practice, no algorithm is perfect, and for each problem, it is necessary to find the most performing of all existing algorithms or even develop new ones. In this paper, we consider the computing of the maximum likelihood estimate of the vector parameter of a statistical model of crash frequencies. We split the parameter vector, and we develop a new estimation algorithm using the profile likelihood principle. We provide an automatic starting guess for which convergence and numerical stability are guaranteed. We study the performance of our new algorithm on simulated data by comparing it to some of the most famous and modern optimization algorithms. The results suggest that our proposed algorithm outperforms these algorithms.

A gentle tutorial on accelerated parameter and confidence interval estimation for hidden Markov models using Template Model Builder.

A very common way to estimate the parameters of a hidden Markov model (HMM) is the relatively straightforward computation of maximum likelihood (ML) estimates. For this task, most users rely on user-friendly implementation of the estimation routines via an interpreted programming language such as the statistical software environment R. Such an approach can easily require time-consuming computations, in particular for longer sequences of observations. In addition, selecting a suitable approach for deriving confidence intervals for the estimated parameters is not entirely obvious, and often the computationally intensive bootstrap methods have to be applied. In this tutorial, we illustrate how to speed up the computation of ML estimates significantly via the R package TMB. Moreover, this approach permits simple retrieval of standard errors at the same time. We illustrate the performance of our routines using different data sets: first, two smaller samples from a mobile application for tinnitus patients and a well-known data set of fetal lamb movements with 87 and 240 data points, respectively. Second, we rely on larger data sets of simulated data of sizes 2000 and 5000 for further analysis. This tutorial is accompanied by a collection of scripts, which are all available in the Supporting Information. These scripts allow any user with moderate programming experience to benefit quickly from the computational advantages of TMB.

Dirichlet composition distribution for compositional data with zero components: An application to fluorescence in situ hybridization (FISH) detection of chromosome.

Zeros in compositional data are very common and can be classified into rounded and essential zeros. The rounded zero refers to a small proportion or below detection limit value, while the essential zero refers to the complete absence of the component in the composition. In this article, we propose a new framework for analyzing compositional data with zero entries by introducing a stochastic representation. In particular, a new distribution, namely the Dirichlet composition distribution, is developed to accommodate the possible essential‐zero feature in compositional data. We derive its distributional properties (e.g., its moments). The calculation of maximum likelihood estimates via the Expectation‐Maximization (EM) algorithm will be proposed. The regression model based on the new Dirichlet composition distribution will be considered. Simulation studies are conducted to evaluate the performance of the proposed methodologies. Finally, our method is employed to analyze a dataset of fluorescence in situ hybridization (FISH) for chromosome detection.

Estimation in multivariate linear mixed models for longitudinal data with multiple outputs: Application to PBCseq data analysis

In many biomedical studies or clinical trials, we have data with more than one response variable on the same subject repeatedly measured over time. In analyzing such data, we adopt a multivariate linear mixed-effects longitudinal model. On the other hand, in longitudinal data, we often find features that do not impact modeling the response variable and are eliminated from the study. In this paper, we consider the problem of simultaneous variable selection and estimation in a multivariate t linear mixed-effects model (MtLMM) for analyzing longitudinally measured multioutcome data. This work's motivation comes from a cohort study of patients with primary biliary cirrhosis. The interest is eliminating insignificant variables using the smoothly clipped and absolute deviation penalty function in the MtLMM. The proposed penalized model offers robustness and flexibility to accommodate fat tails. An expectation conditional maximization algorithm is employed for the computation of maximum likelihood estimates of parameters. The calculation of standard errors is affected by an information-based method. The methodology is illustrated by analyzing Mayo Clinic Primary Biliary Cirrhosis sequential (PBCseq) data and a simulation study. We found drugs and sex can be eliminated from the PBCseq analysis, and over time the diseaseprogresses.

Scale and shape mixtures of matrix variate extended skew normal distributions

In this paper, we propose a matrix extension of the scale and shape mixtures of multivariate skew normal distributions and present some particular cases of this new class. We also present several formal properties of this class, such as the marginal distributions, the moment generating function, the distribution of linear and quadratic forms, and the selection and stochastic representations. In addition, we introduce the matrix variate tail conditional expectation measure and derive this risk measure for the scale and shape mixtures of matrix variate extended skew normal distributions. We present an efficient EM-type algorithm for the computation of maximum likelihood estimates of parameters in some special cases of the proposed class. Finally, we conduct a small simulation study and fit various special cases of the new class to a real dataset.

A robust class of multivariate fatigue distributions based on normal mean-variance mixture model

The Birnbaum–Saunders (BS) distribution, introduced in 1969, is a popular univariate fatigue life distribution which has been widely used to model right-skewed lifetime and reliability data. In this paper, a new class of generalized multivariate BS distributions is proposed based on mean-variance mixture models to accommodate strongly skewed and heavy tailed multivariate lifetime data. Some special cases of this class as well as their properties are then discussed. We present a hierarchical representation which facilitates an efficient EM-type algorithm for the computation of maximum likelihood estimates. Empirical results from a simulation study and real data analyses show that this class of distributions outperforms many existing extensions of the BS distribution in modeling lifetime data.

AdequacyModel: An R package for probability distributions and general purpose optimization.

Several lifetime distributions have played an important role to fit survival data. However, for some of these models, the computation of maximum likelihood estimators is quite difficult due to presence of flat regions in the search space, among other factors. Several well-known derivative-based optimization tools are unsuitable for obtaining such estimates. To circumvent this problem, we introduce the AdequacyModel computational library version 2.0.0 for the R statistical environment with two major contributions: a general optimization technique based on the Particle Swarm Optimization (PSO) method (with a minor modification of the original algorithm) and a set of statistical measures for assessment of the adequacy of the fitted model. This library is very useful for researchers in probability and statistics and has been cited in various papers in these areas. It serves as the basis for the Newdistns library (version 2.1) published in an impact journal in the area of computational statistics, see https://CRAN.R-project.org/package=Newdistns. It is also the basis of the Wrapped library (version 2.0), see https://CRAN.R-project.org/package=Wrapped. A third package making use of the AdequacyModel library can be found in https://CRAN.R-project.org/package=sglg. In addition, the proposed library has proved to be very useful for maximizing log-likelihood functions with complex search regions. The library provides a greater control of the optimization process by introducing a stop criterion based on a minimum number of iterations and the variance of a given proportion of optimal values. We emphasize that the new library can be used not only in statistics but in physics and mathematics as proved in several examples throughout the paper.

A joint model for multiple dynamic processes and clinical endpoints: Application to Alzheimer's disease.

As other neurodegenerative diseases, Alzheimer's disease, the most frequent dementia in the elderly, is characterized by multiple progressive impairments in the brain structure and in clinical functions such as cognitive functioning and functional disability. Until recently, these components were mostly studied independently because no joint model for multivariate longitudinal data and time to event was available in the statistical community. Yet, these components are fundamentally interrelated in the degradation process toward dementia and should be analyzed together. We thus propose a joint model to simultaneously describe the dynamics of multiple correlated components. Each component, defined as a latent process, is measured by one or several continuous markers (not necessarily Gaussian). Rather than considering the associated time to diagnosis as in standard joint models, we assume diagnosis corresponds to the passing above a covariate-specific threshold (to be estimated) of a pathological process that is modeled as a combination of the component-specific latent processes. This definition captures the clinical complexity of diagnoses such as dementia diagnosis but also benefits from simplifications for the computation of maximum likelihood estimates. We show that the model and estimation procedure can also handle competing clinical endpoints. The estimation procedure, implemented in an R package, is validated by simulations and the method is illustrated on a large French population-based cohort of cerebral aging in which we focused on the dynamics of three clinical manifestations and the associated risk of dementia and death before dementia.

Statistical Inference for Type I Multiply Left-Censored Samples from Weibull Distribution*

Left-censored data with one or more detection limits occur frequently in many application areas. In this paper, the computational procedure for calculation of maximum likelihood estimates of the parameters for type I multiply left-censored data from underlying Weibull distribution is suggested and used considering various numbers of detection limits. The expected Fisher information matrix is analytically determined, and its performance is compared with sample (observed) Fisher information matrix using simulations. Simulations are focused primarily on the properties of estimators for small sample sizes. Real data illustration is included.

Computation of maximum likelihood estimates in cyclic structural equation models

Software for computation of maximum likelihood estimates in linear structural equation models typically employs general techniques from nonlinear optimization, such as quasi-Newton methods. In practice, careful tuning of initial values is often required to avoid convergence issues. As an alternative approach, we propose a block-coordinate descent method that cycles through the considered variables, updating only the parameters related to a given variable in each step. We show that the resulting block update problems can be solved in closed form even when the structural equation model comprises feedback cycles. Furthermore, we give a characterization of the models for which the block-coordinate descent algorithm is well defined, meaning that for generic data and starting values all block optimization problems admit a unique solution. For the characterization, we represent each model by its mixed graph (also known as path diagram), which leads to criteria that can be checked in time that is polynomial in the number of considered variables.

Maximum likelihood estimation of Gaussian copula models for geostatistical count data

This work investigates the computation of maximum likelihood estimators in Gaussian copula models for geostatistical count data. This is a computationally challenging task because the likelihood function is only expressible as a high dimensional multivariate normal integral. Two previously proposed Monte Carlo methods are reviewed, the Genz–Bretz and Geweke–Hajivassiliou–Keane simulators, and a new method is investigated. The new method is based on the so–called data cloning algorithm, which uses Markov chain Monte Carlo algorithms to approximate maximum likelihood estimators and their (asymptotic) variances in models with computationally challenging likelihoods. A simulation study is carried out to compare the statistical and computational efficiencies of the three methods. It is found that the three methods have similar statistical properties, but the Geweke–Hajivassiliou–Keane simulator requires the least computational effort. Hence, this is the method we recommend. A data analysis of Lansing Woods tree counts is used to illustrate the methods.

Left-Censored Samples from Skewed Distributions: Statistical Inference and Applications

Left‑censored data occur frequently in many areas. At present, researchers pay attention to skewed censored distributions more frequently. This paper deals with statistical inference of type I multiply left‑censored Weibull and exponential distributions. It suggests a computational procedure for calculation of maximum likelihood estimates of the parameters. The expected Fisher information matrix for estimation of variances of estimated parameters is introduced. The estimates are then used for construction of confidence intervals for the expectation using the maximum likelihood method. Asymptotic tests for comparison of distributions (expectations respectively) of two independent left‑censored Weibull samples are proposed. Furthermore, asymptotic tests for assessing suitability of reduction of the Weibull distribution to the exponential distribution are introduced. Finally, the left‑censored exponential distribution is briefly described. Methods derived in this paper are illustrated on elemental carbon measurements, and can be applied in analysis of real environmental and/or chemical data.

Facile semi-automated forensic body fluid identification by multiplex solution hybridization of NanoString® barcode probes to specific mRNA targets.

A DNA profile from the perpetrator does not reveal, per se, the circumstances by which it was transferred. Body fluid identification by mRNA profiling may allow extraction of contextual 'activity level' information from forensic samples. Here we describe the development of a prototype multiplex digital gene expression (DGE) method for forensic body fluid/tissue identification based upon solution hybridization of color-coded NanoString(®) probes to 23 mRNA targets. The method identifies peripheral blood, semen, saliva, vaginal secretions, menstrual blood and skin. We showed that a simple 5 min room temperature cellular lysis protocol gave equivalent results to standard RNA isolation from the same source material, greatly enhancing the ease-of-use of this method in forensic sample processing. We first describe a model for gene expression in a sample from a single body fluid and then extend that model to mixtures of body fluids. We then describe calculation of maximum likelihood estimates (MLEs) of body fluid quantities in a sample, and we describe the use of likelihood ratios to test for the presence of each body fluid in a sample. Known single source samples of blood, semen, vaginal secretions, menstrual blood and skin all demonstrated the expected tissue-specific gene expression for at least two of the chosen biomarkers. Saliva samples were more problematic, with their previously identified characteristic genes exhibiting poor specificity. Nonetheless the most specific saliva biomarker, HTN3, was expressed at a higher level in saliva than in any of the other tissues. Crucially, our algorithm produced zero false positives across this study's 89 unique samples. As a preliminary indication of the ability of the method to discern admixtures of body fluids, five mixtures were prepared. The identities of the component fluids were evident from the gene expression profiles of four of the five mixtures. Further optimization of the biomarker 'CodeSet' will be required before it can be used in casework, particularly with respect to increasing the signal-to-noise ratio of the saliva biomarkers. With suitable modifications, this simplified protocol with minimal hands on requirement should facilitate routine use of mRNA profiling in casework laboratories.

Testing homogeneity of proportion ratios for stratified correlated bilateral data in two-arm randomized clinical trials.

Stratified data analysis is an important research topic in many biomedical studies and clinical trials. In this article, we develop five test statistics for testing the homogeneity of proportion ratios for stratified correlated bilateral binary data based on an equal correlation model assumption. Bootstrap procedures based on these test statistics are also considered. To evaluate the performance of these statistics and procedures, we conduct Monte Carlo simulations to study their empirical sizes and powers under various scenarios. Our results suggest that the procedure based on score statistic performs well generally and is highly recommended. When the sample size is large, procedures based on the commonly used weighted least square estimate and logarithmic transformation with Mantel-Haenszel estimate are recommended as they do not involve any computation of maximum likelihood estimates requiring iterative algorithms. We also derive approximate sample size formulas based on the recommended test procedures. Finally, we apply the proposed methods to analyze a multi-center randomized clinical trial for scleroderma patients.

Variable Selection for Naive Bayes Semisupervised Learning

This article deals with a semisupervised learning based on naive Bayes assumption. A univariate Gaussian mixture density is used for continuous input variables whereas a histogram type density is adopted for discrete input variables. The EM algorithm is used for the computation of maximum likelihood estimators of parameters in the model when we fix the number of mixing components for each continuous input variable. We carry out a model selection for choosing a parsimonious model among various fitted models based on an information criterion. A common density method is proposed for the selection of significant input variables. Simulated and real datasets are used to illustrate the performance of the proposed method.

Asymptotic Properties of Maximum Likelihood Estimation: Parameterized Diffusion in a Manifold

This paper studies maximum likelihood estimation for a parameterised elliptic diffusion in a manifold. The focus is on asymptotic properties of maximum likelihood estimates obtained from continuous time observation. These are well known when the underlying manifold is a Euclidean space. However, no systematic study exists in the case of a general manifold. The starting point is to write down the likelihood function and equation. This is achieved using the tools of stochastic differential geometry. Consistency, asymptotic normality and asymptotic optimality of maximum likelihood estimates are then proved, under regularity assumptions. Numerical computation of maximum likelihood estimates is briefly discussed.

Computation of maximum likelihood estimates for multiresponse generalized linear mixed models with non-nested, correlated random effects

Estimation of generalized linear mixed models (GLMMs) with non-nested random effects structures requires the approximation of high-dimensional integrals. Many existing methods are tailored to the low-dimensional integrals produced by nested designs. We explore the modifications that are required in order to adapt an EM algorithm with first-order and fully exponential Laplace approximations to a non-nested, multiple response model. The equations in the estimation routine are expressed as functions of the first four derivatives of the conditional likelihood of an arbitrary GLMM, providing a template for future applications. We apply the method to a joint Poisson–binary model for ranking sporting teams, and discuss the estimation of a correlated random effects model designed to evaluate the sensitivity of value-added models for teacher evaluation to assumptions about the missing data process. Source code in R is provided in the online supplementary material (see Appendix C).