Penalized Maximum Likelihood Estimation Research Articles

Penalized Likelihood Estimation for Current Status Data with Informative Censoring

Current status data occurs when failure time of subjects in a survival study is only known to be either less or greater than the censoring time. Thus, the failure time is either left – or right – censored. Analyzing data of this structure under the Cox Proportional Hazards model with dependent censoring assumption can be challenging. To address this, a Penalized Maximum Likelihood Estimation (PMLE) approach was proposed. The unknown baseline cumulative hazard functions for both the failure time and the censoring time were estimated using splines. The advantage of penalized approach over unpenalized method is that that the desired smoothness level of the functions are controlled by their respective penalty terms. The possible dependence between the failure and censoring times was accounted for using the gamma-frailty model. An easy to implement hybrid computational algorithm is proposed to estimate the PMLEs and the Bayesian technique was employed for the estimation of the variances of the parameters. Extensive simulation studies were conducted to assess the statistical properties of the PMLEs. It was observed that the realized estimators were not only consistent, asymptotically normal and efficient, but also, were robust to the number of knots chosen, the proportion of dependent censoring used and the frailty distribution employed. The proposed PMLE method was further applied to real data obtained from tumorigenicity experiment.

Barriers of adopting sustainable soil management practices for organic and conventional farming systems

There is a general consensus among stakeholders about the benefits of sustainable soil management practices for soil health, yet their implementation lags behind expectations. The aim of this paper is to analyse factors supporting or hindering the implementation of sustainable soil management practices in conventional and organic farming systems. This is achieved using a Logit model, based on penalized maximum likelihood estimation on survey data by 76 farmers from 10 federal states in Germany. Affected factors were categorized in two main groups: agricultural soil management practices, and barriers to their implementation. The measures for soil improving agricultural management included (I) structural landscape elements, (II) organic fertilization, (III) diversified crop rotation, (IV) permanent soil cover (V) conservation tillage, (VI) reduced weight pressure, and (VII) optimized timing of wheeling. Results show differences in preferences between conventional and organic farmers for key sustainable soil management practices, including conservation tillage, optimized wheeling timing, reduced weight pressure, and diversified crop rotation. Economic constraints decrease crop rotation diversification in conventional systems by 14%. Conservation tillage raises the chance of a farm being conventional by 16%, while optimizing wheeling timing reduces it by 60%, highlighting soil compaction concerns among organic farmers. Marginal effects confirm that economic factors, not knowledge, are the main barriers to sustainable soil management. Inconsistent policies, lack of support, and insufficient financial incentives from government institutions can hinder the willingness of farmers to adopt these practices, further exacerbating the barriers to adoption.

Heterogeneity-aware integrative regression for ancestry-specific association studies.

Ancestry-specific proteome-wide association studies (PWAS) based on genetically predicted protein expression can reveal complex disease etiology specific to certain ancestral groups. These studies require ancestry-specific models for protein expression as a function of SNP genotypes. In order to improve protein expression prediction in ancestral populations historically underrepresented in genomic studies, we propose a new penalized maximum likelihood estimator for fitting ancestry-specific joint protein quantitative trait loci models. Our estimator borrows information across ancestral groups, while simultaneously allowing for heterogeneous error variances and regression coefficients. We propose an alternative parameterization of our model that makes the objective function convex and the penalty scale invariant. To improve computational efficiency, we propose an approximate version of our method and study its theoretical properties. Our method provides a substantial improvement in protein expression prediction accuracy in individuals of African ancestry, and in a downstream PWAS analysis, leads to the discovery of multiple associations between protein expression and blood lipid traits in the African ancestry population.

Effect of a correlated competing risk on marginal survival estimation in an accelerated failure time model

In problems with a time to event outcome, subjects may experience competing events, which censor the outcome of interest. Cox’s partial likelihood estimator treating competing events as independent censoring is commonly used to examine group differences in clinical trials but fails to adjust for omitted covariates and can bias the assessment of marginal benefit. A bivariate normal linear model generating latent data with dependent censoring is used to assess this bias. Our R-package bnc provides maximum penalized likelihood (MPL) parameter estimation using a novel EM algorithm. Using bnc, we study the properties of such MPL estimation. Simulation results for two-sample survival comparisons of time to an event of interest, with independent censoring accompanied by censoring from a correlated competing risk, are presented. Key parameters—means, hazard ratios, and correlation—are estimated. These results demonstrated that, despite ill-conditioning in models generating correlated competing risks, estimates of marginal effects are reliable. Bivariate normal models were fitted in a trial of head and neck cancer. Model fits help with clinical interpretation while also supplementing other standard methods for follow-up that are terminated by intervening risks.

Local Influence for the Thin-Plate Spline Generalized Linear Model

Thin-Plate Spline Generalized Linear Models (TPS-GLMs) are an extension of Semiparametric Generalized Linear Models (SGLMs), because they allow a smoothing spline to be extended to two or more dimensions. This class of models allows modeling a set of data in which it is desired to incorporate the non-linear joint effects of some covariates to explain the variability of a certain variable of interest. In the spatial context, these models are quite useful, since they allow the effects of locations to be included, both in trend and dispersion, using a smooth surface. In this work, we extend the local influence technique for the TPS-GLM model in order to evaluate the sensitivity of the maximum penalized likelihood estimators against small perturbations in the model and data. We fit our model through a joint iterative process based on Fisher Scoring and weighted backfitting algorithms. In addition, we obtained the normal curvature for the case-weight perturbation and response variable additive perturbation schemes, in order to detect influential observations on the model fit. Finally, two data sets from different areas (agronomy and environment) were used to illustrate the methodology proposed here.

Time series clustering based on latent volatility mixture modeling with applications in finance

Modeling financial time series data poses a significant challenge in the realm of time series analysis. The Autoregressive Conditional Heteroskedasticity (ARCH) model stands out as a potent tool for capturing time-varying volatility and heteroskedasticity in financial data. However, conventional ARCH models display sensitivity to departures from normality, leading to the development of extensions employing more flexible distributions. In this context, we propose a robust enhancement to the mixture of ARCH (MoARCH) model by integrating normal mean–variance mixture (NMVM) distributions to model component errors. The stochastic representation of the proposed model allows for a straightforward implementation of an Expectation Conditional Maximization Either (ECME) algorithm for obtaining Maximum Penalized Likelihood estimates (MPL). To thoroughly evaluate the model, we conduct four simulation studies to explore finite-sample properties, assess MPL estimators, scrutinize model robustness, and evaluate the accuracy of our proposal in fitting, clustering, and forecasting. Practical applications further highlight the effectiveness of our methodology, showcasing successful implementations across diverse real datasets.

Prediction model for anabolic androgenic steroid positivity in forensic autopsy cases – a new tool to the autopsy room

Non-prescription use of anabolic androgenic steroids (AAS) is associated with an increased risk of premature death. However, these substances are seldom screened in connection with forensic cause-of-death investigation, unless the forensic pathologist specifically suspects use, often based on a positive AAS use history. Since AAS use is often concealed from others, this practice may lead to mistargeting of these analyses and significant underestimation of the true number of AAS positive cases undergoing forensic autopsy. Thus, more accurate diagnostic tools are needed to identify these cases. The main objective of this study was to determine, whether a multivariable model could predict AAS urine assay positivity in forensic autopsies. We analyzed retrospectively the autopsy reports of all cases that had been screened for AAS during forensic cause-of-death investigation between 2016–2019 at the Finnish Institute for Health and Welfare forensic units (n = 46). Binary logistic regression with penalized maximum likelihood estimation was used to generate a nine-variable model combining circumferential and macroscopic autopsy-derived variables. The multivariable model predicted AAS assay positivity significantly better than a “conventional” model with anamnestic information about AAS use only (area under the receiver operating characteristic curve [AUC] = 0.968 vs. 0.802, p = 0.005). Temporal validation was conducted in an independent sample of AAS screened cases between 2020–2022 (n = 31), where the superiority of the multivariable model was replicated (AUC = 0.856 vs. 0.644, p = 0.004). Based on the model, a calculator predicting AAS assay positivity is released as a decision-aiding tool for forensic pathologists working in the autopsy room.

A novel predictive model of lymphovascular space invasion in early-stage endometrial cancer.

To predict lymphovascular space invasion (LVSI) positivity in early-stage (stage 1-2) endometrial cancer (EC) using a predictive model with prognostic factors of EC. We included 461 patients who underwent total hysterectomy and bilateral salpingo-oophorectomy with pelvic-paraaortic lymphadenectomy as the primary treatment for presumed early-stage EC at our clinic between 2010 and 2020. Moreover, all surgical specimens were examined histopathologically for the positivity or negativity of LVSI, and the patients were divided into two groups based on these pathologic outcomes. Age, menopausal status, histological type (type 1-2), histological grade (grades 1-2-3), depth of myometrial invasion, and peritoneal cytology results were recorded and analyzed as clinicopathological and demographic characteristics of the patients. The Loess algorithm determined the relationship between the observed and predicted outcomes. The distinction between the algorithms was evaluated by calculating the C-index. LVSI positivity was significantly associated with advanced age, menopause, type 2 EC, advanced histological grade, malignant peritoneal cytology, cervical involvement, and a tumor exceeding 50% of the myometrial depth (p<0.001, respectively). Remarkably, LVSI was most strongly associated with three explanatory variables: 1- More than 50% myometrial invasion [odds ratio (OR): 3.78; 95% confidence interval (CI): 1.80-7.60], 2- Advanced histological grade [OR=1.98 (1.20-3.20) 95% CI], 3- Malignant peritoneal cytology [OR= 3.06 (1.40-6.30) 95% CI]. The penalized maximum likelihood estimation model correctly classified 87% of the included patients (C-index: 0.876). Our predictive model may help predict LVSI based on different prognostic factors. The prognostic factors included in the nomogram were significantly associated with LVSI, particularly myometrial invasion depth of more than 50%, advanced histological grade, and malignant peritoneal cytology.

A unifying switching regime regression framework with applications in health economics

Motivated by three health economics-related case studies, we propose a unifying and flexible regression modeling framework that involves regime switching. The proposal can handle the peculiar distributional shapes of the considered outcomes via a vast range of marginal distributions, allows for a wide variety of copula dependence structures and permits to specify all model parameters (including the dependence parameters) as flexible functions of covariate effects. The algorithm is based on a computationally efficient and stable penalized maximum likelihood estimation approach. The proposed modeling framework is employed in three applications in health economics, that use data from the Medical Expenditure Panel Survey, where novel patterns are uncovered. The framework has been incorporated in the R package GJRM, hence allowing users to fit the desired model(s) and produce easy-to-interpret numerical and visual summaries.

Modeling Tenant’s Credit Scoring Using Logistic Regression

This study implements the multivariable logistic regression to develop a credit scoring model based on tenants’ characteristics. The credit history of tenant is not considered. Rental information of tenants was collected from a landlord company in Malaysia. Parameters of the multivariable logistic regression were estimated by using the penalized maximum likelihood estimation with ridge regression since separation in training data was detected. The initial factors considered that affect tenants’ credit score were their gender, age, marital status, monthly income, household income, expense-to-income ratio, number of dependents, previous monthly rent, and number of months late payment. However, the marital status factor was then excluded from the logistic regression model due to its low significance to the model. Meanwhile, a tenant’s credit scoring model was generated by calculating the tenant’s probability of defaulting. The main factors of the tenant’s credit score are the number of months late payment, the expense-to-income ratio, gender, previous monthly rent, and age. There is no underfitting or overfitting in the proposed credit scoring model which means the model’s bias and variance are low.

Adaptive LASSO estimation for functional hidden dynamic geostatistical models

We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hidden dynamic geostatistical models (f-HDGM). These models employ a classic mixed-effect regression structure with embedded spatiotemporal dynamics to model georeferenced data observed in a functional domain. Thus, the regression coefficients are functions. The algorithm simultaneously selects the relevant spline basis functions and regressors that are used to model the fixed effects. In this way, it automatically shrinks to zero irrelevant parts of the functional coefficients or the entire function for an irrelevant regressor. The algorithm is based on an adaptive LASSO penalty function, with weights obtained by the unpenalised f-HDGM maximum likelihood estimators. The computational burden of maximisation is drastically reduced by a local quadratic approximation of the log-likelihood. A Monte Carlo simulation study provides insight in prediction ability and parameter estimate precision, considering increasing spatiotemporal dependence and cross-correlations among predictors. Further, the algorithm behaviour is investigated when modelling air quality functional data with several weather and land cover covariates. Within this application, we also explore some scalability properties of our algorithm. Both simulations and empirical results show that the prediction ability of the penalised estimates are equivalent to those provided by the maximum likelihood estimates. However, adopting the so-called one-standard-error rule, we obtain estimates closer to the real ones, as well as simpler and more interpretable models.

Maximum softly-penalized likelihood for mixed effects logistic regression

Maximum likelihood estimation in logistic regression with mixed effects is known to often result in estimates on the boundary of the parameter space. Such estimates, which include infinite values for fixed effects and singular or infinite variance components, can cause havoc to numerical estimation procedures and inference. We introduce an appropriately scaled additive penalty to the log-likelihood function, or an approximation thereof, which penalizes the fixed effects by the Jeffreys’ invariant prior for the model with no random effects and the variance components by a composition of negative Huber loss functions. The resulting maximum penalized likelihood estimates are shown to lie in the interior of the parameter space. Appropriate scaling of the penalty guarantees that the penalization is soft enough to preserve the optimal asymptotic properties expected by the maximum likelihood estimator, namely consistency, asymptotic normality, and Cramér-Rao efficiency. Our choice of penalties and scaling factor preserves equivariance of the fixed effects estimates under linear transformation of the model parameters, such as contrasts. Maximum softly-penalized likelihood is compared to competing approaches on two real-data examples, and through comprehensive simulation studies that illustrate its superior finite sample performance.

Detecting and Correcting for Separation in Strategic Choice Models

AbstractSeparation or “perfect prediction” is a common problem in discrete choice models that, in practice, leads to inflated point estimates and standard errors. Standard statistical packages do not provide clear advice on how to correct these problems. Furthermore, separation can go completely undiagnosed in fitting advanced models that optimize a user-supplied log-likelihood rather than relying on pre-programmed estimation procedures. In this paper, we both describe the problems that separation can cause and address the issue of detecting it in empirical models of strategic interaction. We then consider several solutions based on penalized maximum likelihood estimation. Using Monte Carlo experiments and a replication study, we demonstrate that when separation is detected in the data, the penalized methods we consider are superior to ordinary maximum likelihood estimators.

The sparse estimation of the semiparametric linear transformation model with dependent current status data

In this paper, we study the sparse estimation under the semiparametric linear transformation models for the current status data, also called type I interval-censored data. For the problem, the failure time of interest may be dependent on the censoring time and the association parameter between them is left unspecified. To address this, we employ the copula model to describe the dependence between them and a two-stage estimation procedure to estimate both the association parameter and the regression parameter. In addition, we propose a penalized maximum likelihood estimation procedure based on the broken adaptive ridge regression, and Bernstein polynomials are used to approximate the nonparametric functions involved. The oracle property of the proposed method is established and the numerical studies suggest that the method works well for practical situations. Finally, the method is applied to an Alzheimer's disease study that motivated this investigation.

Modeling climate extremes using the four-parameter kappa distribution for [formula omitted]-largest order statistics

Accurate estimation of the T-year return levels of climate extremes using statistical distribution is a critical step in the projection of future climate and in engineering design for disaster response. We show how the estimation of such quantities can be improved by fitting the four-parameter kappa distribution for r-largest order statistics (rK4D), which was developed in this study. The rK4D is an extension of the generalized extreme value distribution for r-largest order statistics (rGEVD), similar to the four-parameter kappa distribution (K4D), which is an extension of the generalized extreme value distribution (GEVD). This new distribution (rK4D) can be useful not only for fitting data when three parameters in the GEVD are not sufficient to capture the variability of the extreme observations, but also in reducing the estimation uncertainty by making use of the r-largest extreme observations instead of only the block maxima. We derive a joint probability density function (PDF) of rK4D and the marginal and conditional cumulative distribution functions and PDFs. To estimate the parameters, the maximum likelihood estimation and the maximum penalized likelihood estimation methods were considered. The usefulness and practical effectiveness of the rK4D are illustrated by the Monte Carlo simulation and by an application to the Bangkok extreme rainfall data. A few new distributions for r-largest order statistics are also derived as special cases of the rK4D, such as the r-largest logistic, the r-largest generalized logistic, and the r-largest generalized Gumbel distributions. These distributions for r-largest order statistics would be useful in modeling extreme values for many research areas, including hydrology and climatology.

Joint variable selection of both fixed and random effects for Gaussian process-based spatially varying coefficient models

Spatially varying coefficient (SVC) models are a type of regression models for spatial data where covariate effects vary over space. If there are several covariates, a natural question is which covariates have a spatially varying effect and which not. We present a new variable selection approach for Gaussian process-based SVC models. It relies on a penalized maximum likelihood estimation and allows joint variable selection both with respect to fixed effects and Gaussian process random effects. We validate our approach in a simulation study as well as a real world data set. In the simulation study, the penalized maximum likelihood estimation correctly identifies zero fixed and random effects, while the penalty-induced bias of non-zero estimates is negligible. In the real data application, our proposed penalized maximum likelihood estimation yields sparser SVC models and achieves a smaller information criterion than classical maximum likelihood estimation. In a cross-validation study applied on the real data, we show that our proposed penalized maximum likelihood estimation consistently yields the sparsest SVC models while achieving similar predictive performance compared to other SVC modeling methodologies.

This too in moderation: Evaluating the effect of delegate dissatisfaction on ambitious party group switching in the European Parliament

The literature on party group switching in the European Parliament claims that members who are ideological outliers have the highest odds of changing their group label, but is it true that the most incongruent legislators are also the most successful at switching groups? This article seeks to determine whether or not voicing dissatisfaction by casting votes against the party line is associated with an increased probability of party group switching. Using logistic regression with a penalized maximum likelihood estimator, the analysis of ambitious switching (1979–2009) uses loyalty and policy distance variables to show that moderate, not extreme, outliers have the highest odds of exiting their group. These findings revise what we know about the relationship between legislative voting behavior and party switching, and they have important implications for examining the effect of policy-seeking on party switching in national parliaments.

Estimating parameters of mixtures of multivariate [formula omitted]-populations and application to classification of observations

The problem of classifying an observation into mixtures of two multivariate t-distributions is studied when the location parameters and covariance matrices are unknown. For mixtures of two multivariate t-distributions, we propose classification rules based on the maximum likelihood estimators and Bayes estimators of the parameters. The maximum penalized likelihood estimators of the parameters are derived using some informative penalty function. We derive shrinkage estimators of the covariance matrices using a regularized parameter and propose the corresponding classification rule. The kernel density-based rule is proposed considering the diagonal matrix as a bandwidth parameter. A simulation study is carried out to compare the rules in terms of the expected probability of misclassification. Applications of the rules are described using real data sets arising in clinical studies and stock market.

Penalized maximum likelihood estimator for finite multivariate skew normal mixtures

In practice, multivariate skew normal mixture (MSNM) models provide a more flexible framework than multivariate normal mixture models, especially for heterogeneous and asymmetric data. For MSNM models, the maximum likelihood estimator often leads to a statistical inference referred to as “badness” under certain properties, because of the unboundedness of the likelihood function and the divergence of shape parameters. We consider two penalties for the log-likelihood function to counter these issues simultaneously in MSNM models. We show that the penalized maximum likelihood estimator is strongly consistent when the putative order of the mixture is equal to or larger than the true order. We also provide penalized expectation-maximization-type algorithms to compute penalized estimates. Finite sample performance is examined through simulations, real data applications, and comparison with existing methods.

Predictor Factors in the Success of Slow-Release Dinoprostone Used for Cervical Ripening in Pregnancies with Premature Rupture of Membranes.

The study aimed to evaluate the factors affecting successful vaginal delivery in induction with slow-release dinoprostone at term pregnancy with premature rupture of membranes. Pregnancies between 370/7 and 416/7 gestation weeks with premature rupture of membranes in which slow-release dinoprostone was used for cervical ripening were sought for inclusion in the study. Pregnancies with previous uterine surgery, multiple fetal gestations, chorioamnionitis, non-cephalic presentation, fetal distress at the time of admission, HIV positivity, and estimated fetal weight >4500 on ultrasonographic evaluation were excluded. The primary outcome of measures were factors affecting the success of vaginal delivery including maternal age, gestational weeks at delivery, initial Bishop score, parity, induction time, and induction-delivery time interval. To reduce the risk of overfitting in the study, penalized maximum likelihood estimation was performed instead of traditional logistic regression in the statistical analysis. A total of 1266 participants who met the study criteria were included in the study. Among the parameters evaluated for the prediction of successful vaginal delivery in cases with premature rupture of membranes, maternal age (P < .001), Bishop score (P < .001), parity (P=.01), induction time (P < .001), and induction-delivery time interval (P < .001) had an impact on success. The mean gestational week of the participants who had cesarean deliveries was lower than in those who had vaginal deliveries (P=.03); however, this was not a predictor factor of penalized maximum likelihood estimation (P=.70). Basic parameters such as maternal age, induction time, parity, and Bishop score can be used to predict successful vaginal birth following dinoprostone slow-release vaginal insert administration.