Penalized Maximum Likelihood Estimation Research Articles

Accelerated Life Testing With Semiparametric Modeling of Stress Effects

Accelerated life testing has been widely applied to obtain the reliability information of an asset (component) at use conditions when the component is highly reliable. In accelerated life testing, tested components are exposed to stress levels more severe than usual so that more component failures can be observed in a short period of time. Most existing statistical models assume a known parametric relationship between the lifetime (or lifetime characteristic) and accelerating stresses. In this paper, we propose semiparametric modeling for the relationship between the lifetime characteristic and accelerating stresses. The proposed model is suitable for both single and multiple accelerating stress cases. A maximum penalized likelihood estimation method is developed to estimate the model parameters. A simulation study is implemented to illustrate the performance of the proposed model, and a real-world case study is conducted to demonstrate the proposed model.

Some theoretical results regarding the polygonal distribution

ABSTRACTPolygonal distributions are a class of distributions that can be defined via the mixture of triangular distributions over the unit interval. We demonstrate that the densities of polygonal distributions are dense in the class of continuous and concave densities with bounded second derivatives. Furthermore, we prove that polygonal density functions provide O(g− 2) approximations (where g is the number of triangular distribution components), in the supremum distance, to any density function from the hypothesized class. Parametric consistency and Hellinger consistency results for the maximum likelihood (ML) estimator are obtained. A result regarding model selection via penalized ML estimation is proved.

A data-adaptive maximum penalized likelihood estimation for the generalized extreme value distribution

A data-adaptive maximum penalized likelihood estimation for the generalized extreme value distribution

Sample selection models for count data in R

We provide a detailed hands-on tutorial for the R package SemiParSampleSel (version 1.5). The package implements selection models for count responses fitted by penalized maximum likelihood estimation. The approach can deal with non-random sample selection, flexible covariate effects, heterogeneous selection mechanisms and varying distributional parameters. We provide an overview of the theoretical background and then demonstrate how SemiParSampleSel can be used to fit interpretable models of different complexity. We use data from the German Socio-Economic Panel survey (SOEP v28, 2012. doi:10.5684/soep.v28) throughout the tutorial.

A Poisson Log-Normal Model for Constructing Gene Covariation Network Using RNA-seq Data.

Constructing expression networks using transcriptomic data is an effective approach for studying gene regulation. A popular approach for constructing such a network is based on the Gaussian graphical model (GGM), in which an edge between a pair of genes indicates that the expression levels of these two genes are conditionally dependent, given the expression levels of all other genes. However, GGMs are not appropriate for non-Gaussian data, such as those generated in RNA-seq experiments. We propose a novel statistical framework that maximizes a penalized likelihood, in which the observed count data follow a Poisson log-normal distribution. To overcome the computational challenges, we use Laplace's method to approximate the likelihood and its gradients, and apply the alternating directions method of multipliers to find the penalized maximum likelihood estimates. The proposed method is evaluated and compared with GGMs using both simulated and real RNA-seq data. The proposed method shows improved performance in detecting edges that represent covarying pairs of genes, particularly for edges connecting low-abundant genes and edges around regulatory hubs.

Non-concave penalization in linear mixed-effect models and regularized selection of fixed effects

Mixed-effect models are very popular for analyzing data with a hierarchical structure, e.g. repeated observations within subjects in a longitudinal design, patients nested within centers in a multicenter design. However, recently, due to the medical advances, the number of fixed effect covariates collected from each patient can be quite large, e.g. data on gene expressions of each patient, and all of these variables are not necessarily important for the outcome. So, it is very important to choose the relevant covariates correctly for obtaining the optimal inference for the overall study. On the other hand, the relevant random effects will often be low-dimensional and pre-specified. In this paper, we consider regularized selection of important fixed effect variables in linear mixed-effects models along with maximum penalized likelihood estimation of both fixed and random effect parameters based on general non-concave penalties. Asymptotic and variable selection consistency with oracle properties are proved for low-dimensional cases as well as for high-dimensionality of non-polynomial order of sample size (number of parameters is much larger than sample size). We also provide a suitable computationally efficient algorithm for implementation. Additionally, all the theoretical results are proved for a general non-convex optimization problem that applies to several important situations well beyond the mixed model set-up (like finite mixture of regressions etc.) illustrating the huge range of applicability of our proposal.

SeqCNV: a novel method for identification of copy number variations in targeted next-generation sequencing data

BackgroundTargeted next-generation sequencing (NGS) has been widely used as a cost-effective way to identify the genetic basis of human disorders. Copy number variations (CNVs) contribute significantly to human genomic variability, some of which can lead to disease. However, effective detection of CNVs from targeted capture sequencing data remains challenging.ResultsHere we present SeqCNV, a novel CNV calling method designed to use capture NGS data. SeqCNV extracts the read depth information and utilizes the maximum penalized likelihood estimation (MPLE) model to identify the copy number ratio and CNV boundary. We applied SeqCNV to both bacterial artificial clone (BAC) and human patient NGS data to identify CNVs. These CNVs were validated by array comparative genomic hybridization (aCGH).ConclusionsSeqCNV is able to robustly identify CNVs of different size using capture NGS data. Compared with other CNV-calling methods, SeqCNV shows a significant improvement in both sensitivity and specificity.

Penalized maximum likelihood estimation and effective dimension

Cet article generalise certains resultats statistiques importants dont le Theoreme de Fisher et le phenomene de Wilks a l’estimation du maximum de vraisemblance penalisee de facon quadratique. Il apparait que des developpements precis pour l’EMV penalisee $\widetilde{\boldsymbol{\theta}}_{G}$ et le maximum de vraisemblance penalise peuvent etre obtenus sans arguments asymptotiques, les resultats reposent alors seulement sur la regularite et les proprietes de la fonction de log-vraisemblance. L’erreur d’estimation est specifiee en fonction de la dimension effective $\mathtt{p}_{G}$ de l’ensemble des parametres qui peut etre beaucoup plus petite que la veritable dimension et permet ainsi de considerer un cas infini dimensionnel. Dans le cas i.i.d., le developpement de Fisher pour l’EMV penalisee peut etre etabli sous la contrainte « $\mathtt{p}_{G}^{2}/n$ est petit » tandis que le reste dans le resultat de Wilks est d’ordre $\sqrt{\mathtt{p}_{G}^{3}/n}$.

Non parametric hazard estimation with dependent censoring using penalized likelihood and an assumed copula

ABSTRACTThis article introduces a novel non parametric penalized likelihood hazard estimation when the censoring time is dependent on the failure time for each subject under observation. More specifically, we model this dependence using a copula, and the method of maximum penalized likelihood (MPL) is adopted to estimate the hazard function. We do not consider covariates in this article. The non negatively constrained MPL hazard estimation is obtained using a multiplicative iterative algorithm. The consistency results and the asymptotic properties of the proposed hazard estimator are derived. The simulation studies show that our MPL estimator under dependent censoring with an assumed copula model provides a better accuracy than the MPL estimator under independent censoring if the sign of dependence is correctly specified in the copula function. The proposed method is applied to a real dataset, with a sensitivity analysis performed over various values of correlation between failure and censoring times.

Log-symmetric regression models under the presence of non-informative left- or right-censored observations

In this paper, an extension to allow the presence of non-informative left- or right-censored observations in log-symmetric regression models is addressed. Under such models, the log-lifetime distribution belongs to the symmetric class and its location and scale parameters are described by semi-parametric functions of explanatory variables, whose nonparametric components are approximated using natural cubic splines or P-splines. An iterative process of parameter estimation by the maximum penalized likelihood method is presented. The large sample properties of the maximum penalized likelihood estimators are studied analytically and by simulation experiments. Diagnostic methods such as deviance-type residuals and local influence measures are derived. The package ssym, which includes an implementation in the computational environment R of the methodology addressed in this paper, is also discussed. The proposed methodology is illustrated by the analysis of a real data set.

Maximum Penalized Likelihood Estimation for the Reliability Function

Maximum Penalized Likelihood Estimation for the Reliability Function

Semiparametric Multinomial Ordinal Model to Analyze Spatial Patterns of Child Birth Weight in Nigeria.

Background: Birth weight is an important health parameter for obstetricians and gynaecologists. It is a good health indicator of a child-bearing mother and a strong predictor of infant morbidity and mortality. Methods: This paper utilizes data on 28,647 children born between 2003–2008 obtained from the 2008 Nigeria Demographic and Health Survey (NDHS). For a simple epidemiological convenience, the occurrence of a newborn weight can intuitively be considered to be categorical in nature and the thresholds can be put on a continuous scale. In survey reporting, the mothers frequently estimate their infant’s birth weight and make a classification in ordinal category (low, normal, large) instead of actual birth weight. The study fits a multinomial regression model to analyze the relationships between the polytomous response and different kind of covariates in a unified manner. We estimate the fixed effects of bio-social covariates parametrically and the non-linear effect modeled using P-spline. The spatial component was modeled using conditional autoregressive error. A penalized maximum likelihood estimation was performed to estimate the model parameters. Results: We found risk factors that are positively associated with low birth weight, which include multiple birth, short birth interval, death of sibling, childhood diarrhea, fever, mother’s smoking, firewood/dung cooking and poor household. Results further showed that iron syrup supplementation, antenatal attendance, mother literacy and household wealth had significant association with low probability of low birth weight. The finding also showed spatial patterns, which are not captured by the underlying determinants, and we produced probability predictive maps of the spatial residual effects. Conclusions: In addition to the statistical relevance of our method, the generated spatial maps identify highly endemic areas of low birth weight that can assist government agency to channel scarce health resources. A comprehensive approach which institutes a combination of interventions to improve the overall health care of the women is needed.

Evaluation of the Addition of Firth’s Penalty Term to the Bradley-Terry Likelihood

A major shortcoming of the Bradley-Terry model is that the maximum likelihood estimates are infinite-valued in the presence of separation and may be unreliable when data are nearly separated. A well-known solution consists of the addition of Firth' s penalty term to the log-likelihood function, and solve this penalized likelihood through logistic regression.The maximum likelihood estimates with and without Firth's penalty are compared in a large and heterogeneous population of table-tennis players. We additionally show that exact penalized maximum likelihood estimates can be reasonably approximated using a well-chosen Minorization-Maximization (MM) algorithm.

Semiparametric models for multilevel overdispersed count data with extra zeros.

This study proposes semiparametric models for analysis of hierarchical count data containing excess zeros and overdispersion simultaneously. The methods discussed in this paper handle nonlinear covariate effects through flexible semiparametric multilevel regression techniques. This is performed by providing a comprehensive comparison of semiparametric multilevel zero-inflated negative binomial and semiparametric multilevel zero-inflated generalized Poisson models under the real and simulated data. An EM algorithm based on Newton-Raphson equations for maximum penalized likelihood estimation approach is developed. The performance of the proposed models is assessed by using a Monte Carlo simulation study. We also illustrated the methods by the analysis of decayed, missing, and filled teeth of children aged 5-14 years old.

Simultaneous pose estimation and patient-specific model reconstruction from single image using maximum penalized likelihood estimation (MPLE)

Pose estimation and shape reconstruction are two common problems in pattern recognition, which oftentimes are tackled separately. But in some medical applications, both pose and shape of a target anatomy are crucial and have to be estimated from intra-operative two-dimensional images. As pose estimation and shape reconstruction are two coupled problems, previous feature-based methods solved the problems in consecutive stages utilizing statistical shape models (SSMs). Only the mean shape of SSM is used to estimate the pose by finding paired correspondences in the first stage, based on which SSM-regularized surface deformations are performed in the following stages. Such a strategy heavily depends on the paired correspondences. In this paper, bypassing correspondence establishment, a novel method is proposed to simultaneously optimize pose and shape by formulating the coupled problems as a maximum penalized likelihood estimation (MPLE). It models oriented object contours as a mixture of von Mises–Fisher Gaussian distributions, and solves the MPLE effectively using a global optimizer. It utilize the entire knowledge of SSM in both solving pose and reconstructing shape, providing robustness to large offsets in initializations. Leave-one-out cross-validations on 19 dry cadaveric femurs were performed using simulated X-ray images with accurate ground-truth, under various initial conditions. Our method achieved sub-degree rotational and sub-millimeter in-plane translational pose estimation errors, and an approximately one millimeter average mean surface-to-surface distance in shape reconstruction. The reconstruction accuracy is comparable to those reported in the literature using two or more images. The experiment results are encouraging and indicate that an accurate simultaneous 3D–2D pose estimation and surface reconstruction is achievable from one single image.

A comparative study of methods for logistic regression with separated or nearly separated data

Objective To introduce a comparative study of methods for Logistic regression with separated or nearly separated data. Methods Human immunodeficiency virus (HIV) infection and its influential factors, 75 drug users were analyzed by maximum Likelihood estimate, exact Logistic regression and Firth's penalized maximum Likelihood estimate, then results of the three methods were compared. Results Both of penalized maximum Likelihood estimation and exact Logistic regression produced valid parameter estimates and confidence interval of the latter was wider. The results of penalized maximum Likelihood estimation and exact Logistic regression showed that race was significantly associated with HIV infection, and Yi people with HIV infection was higher than Han people. Conclusions The maximum Likelihood estimate for separated or nearly separated data is invalid. However, exact Logistic regression and Firth's penalized maximum Likelihood estimate can get valid estimates. Since exact Logistic regression have problems of complex calculations, over-conditioning and conditional distributions degenerated, Firth's penalized maximum Likelihood estimate is recommended to Logistic regression with separated or nearly separated data.

Nonparametric estimation of characteristics of the interspike interval distribution

We address the problem of non-parametric estimation of the probability density function as a description of the probability distribution of noncorrelated interspike intervals (ISI) in records of neuronal activity. We also continue our previous effort [1,2] to propose alternative estimators of the variability measures. Kernel density estimators are probably the most frequently used non-parametric estimators of the probability distribution. However, there are also other non-parametric approaches. We focus on non-parametric methods based on a principle of extrema of the Fisher information. Specifically, we focus on the maximum penalized likelihood estimation of the probability density function proposed by Good and Gaskins [3], which can be understood as a kernel estimator with a particular kernel function [4]. Other non-parametric approach we would like to address is the spline interpolation proposed by Huber [5] which can uniquely estimate the ISI distribution.

Unsupervised learning of regression mixture models with unknown number of components

ABSTRACTWe propose a new unsupervised learning algorithm to fit regression mixture models with unknown number of components. The developed approach consists in a penalized maximum likelihood estimation carried out by a robust expectation–maximization (EM)-like algorithm. We derive it for polynomial, spline, and B-spline regression mixtures. The proposed learning approach is unsupervised: (i) it simultaneously infers the model parameters and the optimal number of the regression mixture components from the data as the learning proceeds, rather than in a two-fold scheme as in standard model-based clustering using afterward model selection criteria, and (ii) it does not require accurate initialization unlike the standard EM for regression mixtures. The developed approach is applied to curve clustering problems. Numerical experiments on simulated and real data show that the proposed algorithm performs well and provides accurate clustering results, and confirm its benefit for practical applications.

Multivariate spline analysis for multiplicative models: Estimation, testing and application to climate change

This paper presents multiplicative, or bi-additive, models with some spline-type regularity for a rectangular array of data, for example in space and time. The one-dimensional smoothing spline model is extended to this multiplicative model including regularity in each dimension. For estimation, we prove the existence of the maximum penalized likelihood estimates (MPLEs), and introduce a numerical algorithm that converges in a weak sense to a critical point of the penalized likelihood. Explicit MPLEs are given in two important particular cases. With regard to hypothesis testing, we focus on the “no effect” test and prove that the null distribution of the penalized likelihood ratio test (PLRT) does not depend on the nuisance parameters under H 0 , leading to easy Monte-Carlo techniques. Numerical results are presented for both simulated data and climate data. For simulated data, our estimation algorithm is shown to have a good behavior. The application to climate data illustrates how multivariate spline analysis for multiplicative models may be of interest in climate change detection.

Maximum penalized likelihood estimation in semiparametric mark-recapture-recovery models.

We discuss the semiparametric modeling of mark-recapture-recovery data where the temporal and/or individual variation of model parameters is explained via covariates. Typically, in such analyses a fixed (or mixed) effects parametric model is specified for the relationship between the model parameters and the covariates of interest. In this paper, we discuss the modeling of the relationship via the use of penalized splines, to allow for considerably more flexible functional forms. Corresponding models can be fitted via numerical maximum penalized likelihood estimation, employing cross-validation to choose the smoothing parameters in a data-driven way. Our contribution builds on and extends the existing literature, providing a unified inferential framework for semiparametric mark-recapture-recovery models for open populations, where the interest typically lies in the estimation of survival probabilities. The approach is applied to two real datasets, corresponding to gray herons (Ardea cinerea), where we model the survival probability as a function of environmental condition (a time-varying global covariate), and Soay sheep (Ovis aries), where we model the survival probability as a function of individual weight (a time-varying individual-specific covariate). The proposed semiparametric approach is compared to a standard parametric (logistic) regression and new interesting underlying dynamics are observed in both cases.