Penalized Maximum Likelihood Estimation Research Articles

Local Influence for the Thin-Plate Spline Generalized Linear Model

Thin-Plate Spline Generalized Linear Models (TPS-GLMs) are an extension of Semiparametric Generalized Linear Models (SGLMs), because they allow a smoothing spline to be extended to two or more dimensions. This class of models allows modeling a set of data in which it is desired to incorporate the non-linear joint effects of some covariates to explain the variability of a certain variable of interest. In the spatial context, these models are quite useful, since they allow the effects of locations to be included, both in trend and dispersion, using a smooth surface. In this work, we extend the local influence technique for the TPS-GLM model in order to evaluate the sensitivity of the maximum penalized likelihood estimators against small perturbations in the model and data. We fit our model through a joint iterative process based on Fisher Scoring and weighted backfitting algorithms. In addition, we obtained the normal curvature for the case-weight perturbation and response variable additive perturbation schemes, in order to detect influential observations on the model fit. Finally, two data sets from different areas (agronomy and environment) were used to illustrate the methodology proposed here.

Time series clustering based on latent volatility mixture modeling with applications in finance

Modeling financial time series data poses a significant challenge in the realm of time series analysis. The Autoregressive Conditional Heteroskedasticity (ARCH) model stands out as a potent tool for capturing time-varying volatility and heteroskedasticity in financial data. However, conventional ARCH models display sensitivity to departures from normality, leading to the development of extensions employing more flexible distributions. In this context, we propose a robust enhancement to the mixture of ARCH (MoARCH) model by integrating normal mean–variance mixture (NMVM) distributions to model component errors. The stochastic representation of the proposed model allows for a straightforward implementation of an Expectation Conditional Maximization Either (ECME) algorithm for obtaining Maximum Penalized Likelihood estimates (MPL). To thoroughly evaluate the model, we conduct four simulation studies to explore finite-sample properties, assess MPL estimators, scrutinize model robustness, and evaluate the accuracy of our proposal in fitting, clustering, and forecasting. Practical applications further highlight the effectiveness of our methodology, showcasing successful implementations across diverse real datasets.

Prediction model for anabolic androgenic steroid positivity in forensic autopsy cases - a new tool to the autopsy room.

Non-prescription use of anabolic androgenic steroids (AAS) is associated with an increased risk of premature death. However, these substances are seldom screened in connection with forensic cause-of-death investigation, unless the forensic pathologist specifically suspects use, often based on a positive AAS use history. Since AAS use is often concealed from others, this practice may lead to mistargeting of these analyses and significant underestimation of the true number of AAS positive cases undergoing forensic autopsy. Thus, more accurate diagnostic tools are needed to identify these cases. The main objective of this study was to determine, whether a multivariable model could predict AAS urine assay positivity in forensic autopsies. We analyzed retrospectively the autopsy reports of all cases that had been screened for AAS during forensic cause-of-death investigation between 2016-2019 at the Finnish Institute for Health and Welfare forensic units (n = 46). Binary logistic regression with penalized maximum likelihood estimation was used to generate a nine-variable model combining circumferential and macroscopic autopsy-derived variables. The multivariable model predicted AAS assay positivity significantly better than a "conventional" model with anamnestic information about AAS use only (area under the receiver operating characteristic curve [AUC] = 0.968 vs. 0.802, p = 0.005). Temporal validation was conducted in an independent sample of AAS screened cases between 2020-2022 (n = 31), where the superiority of the multivariable model was replicated (AUC = 0.856 vs. 0.644, p = 0.004). Based on the model, a calculator predicting AAS assay positivity is released as a decision-aiding tool for forensic pathologists working in the autopsy room.

A novel predictive model of lymphovascular space invasion in early-stage endometrial cancer.

To predict lymphovascular space invasion (LVSI) positivity in early-stage (stage 1-2) endometrial cancer (EC) using a predictive model with prognostic factors of EC. We included 461 patients who underwent total hysterectomy and bilateral salpingo-oophorectomy with pelvic-paraaortic lymphadenectomy as the primary treatment for presumed early-stage EC at our clinic between 2010 and 2020. Moreover, all surgical specimens were examined histopathologically for the positivity or negativity of LVSI, and the patients were divided into two groups based on these pathologic outcomes. Age, menopausal status, histological type (type 1-2), histological grade (grades 1-2-3), depth of myometrial invasion, and peritoneal cytology results were recorded and analyzed as clinicopathological and demographic characteristics of the patients. The Loess algorithm determined the relationship between the observed and predicted outcomes. The distinction between the algorithms was evaluated by calculating the C-index. LVSI positivity was significantly associated with advanced age, menopause, type 2 EC, advanced histological grade, malignant peritoneal cytology, cervical involvement, and a tumor exceeding 50% of the myometrial depth (p<0.001, respectively). Remarkably, LVSI was most strongly associated with three explanatory variables: 1- More than 50% myometrial invasion [odds ratio (OR): 3.78; 95% confidence interval (CI): 1.80-7.60], 2- Advanced histological grade [OR=1.98 (1.20-3.20) 95% CI], 3- Malignant peritoneal cytology [OR= 3.06 (1.40-6.30) 95% CI]. The penalized maximum likelihood estimation model correctly classified 87% of the included patients (C-index: 0.876). Our predictive model may help predict LVSI based on different prognostic factors. The prognostic factors included in the nomogram were significantly associated with LVSI, particularly myometrial invasion depth of more than 50%, advanced histological grade, and malignant peritoneal cytology.

A unifying switching regime regression framework with applications in health economics

Motivated by three health economics-related case studies, we propose a unifying and flexible regression modeling framework that involves regime switching. The proposal can handle the peculiar distributional shapes of the considered outcomes via a vast range of marginal distributions, allows for a wide variety of copula dependence structures and permits to specify all model parameters (including the dependence parameters) as flexible functions of covariate effects. The algorithm is based on a computationally efficient and stable penalized maximum likelihood estimation approach. The proposed modeling framework is employed in three applications in health economics, that use data from the Medical Expenditure Panel Survey, where novel patterns are uncovered. The framework has been incorporated in the R package GJRM, hence allowing users to fit the desired model(s) and produce easy-to-interpret numerical and visual summaries.

Modeling Tenant’s Credit Scoring Using Logistic Regression

This study implements the multivariable logistic regression to develop a credit scoring model based on tenants’ characteristics. The credit history of tenant is not considered. Rental information of tenants was collected from a landlord company in Malaysia. Parameters of the multivariable logistic regression were estimated by using the penalized maximum likelihood estimation with ridge regression since separation in training data was detected. The initial factors considered that affect tenants’ credit score were their gender, age, marital status, monthly income, household income, expense-to-income ratio, number of dependents, previous monthly rent, and number of months late payment. However, the marital status factor was then excluded from the logistic regression model due to its low significance to the model. Meanwhile, a tenant’s credit scoring model was generated by calculating the tenant’s probability of defaulting. The main factors of the tenant’s credit score are the number of months late payment, the expense-to-income ratio, gender, previous monthly rent, and age. There is no underfitting or overfitting in the proposed credit scoring model which means the model’s bias and variance are low.

Adaptive LASSO estimation for functional hidden dynamic geostatistical models

We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hidden dynamic geostatistical models (f-HDGM). These models employ a classic mixed-effect regression structure with embedded spatiotemporal dynamics to model georeferenced data observed in a functional domain. Thus, the regression coefficients are functions. The algorithm simultaneously selects the relevant spline basis functions and regressors that are used to model the fixed effects. In this way, it automatically shrinks to zero irrelevant parts of the functional coefficients or the entire function for an irrelevant regressor. The algorithm is based on an adaptive LASSO penalty function, with weights obtained by the unpenalised f-HDGM maximum likelihood estimators. The computational burden of maximisation is drastically reduced by a local quadratic approximation of the log-likelihood. A Monte Carlo simulation study provides insight in prediction ability and parameter estimate precision, considering increasing spatiotemporal dependence and cross-correlations among predictors. Further, the algorithm behaviour is investigated when modelling air quality functional data with several weather and land cover covariates. Within this application, we also explore some scalability properties of our algorithm. Both simulations and empirical results show that the prediction ability of the penalised estimates are equivalent to those provided by the maximum likelihood estimates. However, adopting the so-called one-standard-error rule, we obtain estimates closer to the real ones, as well as simpler and more interpretable models.

Maximum softly-penalized likelihood for mixed effects logistic regression

Maximum likelihood estimation in logistic regression with mixed effects is known to often result in estimates on the boundary of the parameter space. Such estimates, which include infinite values for fixed effects and singular or infinite variance components, can cause havoc to numerical estimation procedures and inference. We introduce an appropriately scaled additive penalty to the log-likelihood function, or an approximation thereof, which penalizes the fixed effects by the Jeffreys’ invariant prior for the model with no random effects and the variance components by a composition of negative Huber loss functions. The resulting maximum penalized likelihood estimates are shown to lie in the interior of the parameter space. Appropriate scaling of the penalty guarantees that the penalization is soft enough to preserve the optimal asymptotic properties expected by the maximum likelihood estimator, namely consistency, asymptotic normality, and Cramér-Rao efficiency. Our choice of penalties and scaling factor preserves equivariance of the fixed effects estimates under linear transformation of the model parameters, such as contrasts. Maximum softly-penalized likelihood is compared to competing approaches on two real-data examples, and through comprehensive simulation studies that illustrate its superior finite sample performance.

Detecting and Correcting for Separation in Strategic Choice Models

AbstractSeparation or “perfect prediction” is a common problem in discrete choice models that, in practice, leads to inflated point estimates and standard errors. Standard statistical packages do not provide clear advice on how to correct these problems. Furthermore, separation can go completely undiagnosed in fitting advanced models that optimize a user-supplied log-likelihood rather than relying on pre-programmed estimation procedures. In this paper, we both describe the problems that separation can cause and address the issue of detecting it in empirical models of strategic interaction. We then consider several solutions based on penalized maximum likelihood estimation. Using Monte Carlo experiments and a replication study, we demonstrate that when separation is detected in the data, the penalized methods we consider are superior to ordinary maximum likelihood estimators.

The sparse estimation of the semiparametric linear transformation model with dependent current status data

In this paper, we study the sparse estimation under the semiparametric linear transformation models for the current status data, also called type I interval-censored data. For the problem, the failure time of interest may be dependent on the censoring time and the association parameter between them is left unspecified. To address this, we employ the copula model to describe the dependence between them and a two-stage estimation procedure to estimate both the association parameter and the regression parameter. In addition, we propose a penalized maximum likelihood estimation procedure based on the broken adaptive ridge regression, and Bernstein polynomials are used to approximate the nonparametric functions involved. The oracle property of the proposed method is established and the numerical studies suggest that the method works well for practical situations. Finally, the method is applied to an Alzheimer's disease study that motivated this investigation.

Modeling climate extremes using the four-parameter kappa distribution for [formula omitted]-largest order statistics

Accurate estimation of the T-year return levels of climate extremes using statistical distribution is a critical step in the projection of future climate and in engineering design for disaster response. We show how the estimation of such quantities can be improved by fitting the four-parameter kappa distribution for r-largest order statistics (rK4D), which was developed in this study. The rK4D is an extension of the generalized extreme value distribution for r-largest order statistics (rGEVD), similar to the four-parameter kappa distribution (K4D), which is an extension of the generalized extreme value distribution (GEVD). This new distribution (rK4D) can be useful not only for fitting data when three parameters in the GEVD are not sufficient to capture the variability of the extreme observations, but also in reducing the estimation uncertainty by making use of the r-largest extreme observations instead of only the block maxima. We derive a joint probability density function (PDF) of rK4D and the marginal and conditional cumulative distribution functions and PDFs. To estimate the parameters, the maximum likelihood estimation and the maximum penalized likelihood estimation methods were considered. The usefulness and practical effectiveness of the rK4D are illustrated by the Monte Carlo simulation and by an application to the Bangkok extreme rainfall data. A few new distributions for r-largest order statistics are also derived as special cases of the rK4D, such as the r-largest logistic, the r-largest generalized logistic, and the r-largest generalized Gumbel distributions. These distributions for r-largest order statistics would be useful in modeling extreme values for many research areas, including hydrology and climatology.

Joint variable selection of both fixed and random effects for Gaussian process-based spatially varying coefficient models

Spatially varying coefficient (SVC) models are a type of regression models for spatial data where covariate effects vary over space. If there are several covariates, a natural question is which covariates have a spatially varying effect and which not. We present a new variable selection approach for Gaussian process-based SVC models. It relies on a penalized maximum likelihood estimation and allows joint variable selection both with respect to fixed effects and Gaussian process random effects. We validate our approach in a simulation study as well as a real world data set. In the simulation study, the penalized maximum likelihood estimation correctly identifies zero fixed and random effects, while the penalty-induced bias of non-zero estimates is negligible. In the real data application, our proposed penalized maximum likelihood estimation yields sparser SVC models and achieves a smaller information criterion than classical maximum likelihood estimation. In a cross-validation study applied on the real data, we show that our proposed penalized maximum likelihood estimation consistently yields the sparsest SVC models while achieving similar predictive performance compared to other SVC modeling methodologies.

This too in moderation: Evaluating the effect of delegate dissatisfaction on ambitious party group switching in the European Parliament

The literature on party group switching in the European Parliament claims that members who are ideological outliers have the highest odds of changing their group label, but is it true that the most incongruent legislators are also the most successful at switching groups? This article seeks to determine whether or not voicing dissatisfaction by casting votes against the party line is associated with an increased probability of party group switching. Using logistic regression with a penalized maximum likelihood estimator, the analysis of ambitious switching (1979–2009) uses loyalty and policy distance variables to show that moderate, not extreme, outliers have the highest odds of exiting their group. These findings revise what we know about the relationship between legislative voting behavior and party switching, and they have important implications for examining the effect of policy-seeking on party switching in national parliaments.

Estimating parameters of mixtures of multivariate [formula omitted]-populations and application to classification of observations

The problem of classifying an observation into mixtures of two multivariate t-distributions is studied when the location parameters and covariance matrices are unknown. For mixtures of two multivariate t-distributions, we propose classification rules based on the maximum likelihood estimators and Bayes estimators of the parameters. The maximum penalized likelihood estimators of the parameters are derived using some informative penalty function. We derive shrinkage estimators of the covariance matrices using a regularized parameter and propose the corresponding classification rule. The kernel density-based rule is proposed considering the diagonal matrix as a bandwidth parameter. A simulation study is carried out to compare the rules in terms of the expected probability of misclassification. Applications of the rules are described using real data sets arising in clinical studies and stock market.

Penalized maximum likelihood estimator for finite multivariate skew normal mixtures

In practice, multivariate skew normal mixture (MSNM) models provide a more flexible framework than multivariate normal mixture models, especially for heterogeneous and asymmetric data. For MSNM models, the maximum likelihood estimator often leads to a statistical inference referred to as “badness” under certain properties, because of the unboundedness of the likelihood function and the divergence of shape parameters. We consider two penalties for the log-likelihood function to counter these issues simultaneously in MSNM models. We show that the penalized maximum likelihood estimator is strongly consistent when the putative order of the mixture is equal to or larger than the true order. We also provide penalized expectation-maximization-type algorithms to compute penalized estimates. Finite sample performance is examined through simulations, real data applications, and comparison with existing methods.

Predictor Factors in the Success of Slow-Release Dinoprostone Used for Cervical Ripening in Pregnancies with Premature Rupture of Membranes.

The study aimed to evaluate the factors affecting successful vaginal delivery in induction with slow-release dinoprostone at term pregnancy with premature rupture of membranes. Pregnancies between 370/7 and 416/7 gestation weeks with premature rupture of membranes in which slow-release dinoprostone was used for cervical ripening were sought for inclusion in the study. Pregnancies with previous uterine surgery, multiple fetal gestations, chorioamnionitis, non-cephalic presentation, fetal distress at the time of admission, HIV positivity, and estimated fetal weight >4500 on ultrasonographic evaluation were excluded. The primary outcome of measures were factors affecting the success of vaginal delivery including maternal age, gestational weeks at delivery, initial Bishop score, parity, induction time, and induction-delivery time interval. To reduce the risk of overfitting in the study, penalized maximum likelihood estimation was performed instead of traditional logistic regression in the statistical analysis. A total of 1266 participants who met the study criteria were included in the study. Among the parameters evaluated for the prediction of successful vaginal delivery in cases with premature rupture of membranes, maternal age (P < .001), Bishop score (P < .001), parity (P=.01), induction time (P < .001), and induction-delivery time interval (P < .001) had an impact on success. The mean gestational week of the participants who had cesarean deliveries was lower than in those who had vaginal deliveries (P=.03); however, this was not a predictor factor of penalized maximum likelihood estimation (P=.70). Basic parameters such as maternal age, induction time, parity, and Bishop score can be used to predict successful vaginal birth following dinoprostone slow-release vaginal insert administration.

Confidence intervals that utilize sparsity

We consider a linear regression model with orthogonal regressors and Gaussian random errors with known variance, in the low‐dimensional setting that the length of the regression parameter vector does not exceed the length of the response vector. We suppose that we have uncertain prior information that sparsity holds, that is, that many of the components of the regression parameter vector are zero. Our aim is to construct confidence intervals for the components of this vector with both the desired coverage probability and attractive expected length properties, particularly when sparsity holds. It is known that for fixed‐width confidence intervals centered on penalized maximum likelihood estimators, such as hard‐thresholding, LASSO and SCAD estimators, the achievement of the desired minimum coverage necessarily results in very unattractive expected length properties. We present confidence intervals, with data‐based widths, which achieve our aim. These confidence intervals dominate the usual confidence intervals with the same coverage probability, whenever the degree of sparsity exceeds a known value.

A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models

Mixture of experts (MoE) are a popular class of statistical and machine learning models that have gained attention over the years due to their flexibility and efficiency. In this work, we consider Gaussian-gated localized MoE (GLoME) and block-diagonal covariance localized MoE (BLoME) regression models to present nonlinear relationships in heterogeneous data with potential hidden graph-structured interactions between high-dimensional predictors. These models pose difficult statistical estimation and model selection questions, both from a computational and theoretical perspective. This paper is devoted to the study of the problem of model selection among a collection of GLoME or BLoME models characterized by the number of mixture components, the complexity of Gaussian mean experts, and the hidden block-diagonal structures of the covariance matrices, in a penalized maximum likelihood estimation framework. In particular, we establish non-asymptotic risk bounds that take the form of weak oracle inequalities, provided that lower bounds for the penalties hold. The good empirical behavior of our models is then demonstrated on synthetic and real datasets.

A semiparametric mixed-effects model for censored longitudinal data.

In longitudinal studies involving laboratory-based outcomes, repeated measurements can be censored due to assay detection limits. Linear mixed-effects (LMEs) models are a powerful tool to model the relationship between a response variable and covariates in longitudinal studies. However, the linear parametric form of linear mixed-effect models is often too restrictive to characterize the complex relationship between a response variable and covariates. More general and robust modeling tools, such as nonparametric and semiparametric regression models, have become increasingly popular in the last decade. In this article, we use semiparametric mixed models to analyze censored longitudinal data with irregularly observed repeated measures. The proposed model extends the censored linear mixed-effect model and provides more flexible modeling schemes by allowing the time effect to vary nonparametrically over time. We develop an Expectation-Maximization (EM) algorithm for maximum penalized likelihood estimation of model parameters and the nonparametric component. Further, as a byproduct of the EM algorithm, the smoothing parameter is estimated using a modified linear mixed-effects model, which is faster than alternative methods such as the restricted maximum likelihood approach. Finally, the performance of the proposed approaches is evaluated through extensive simulation studies as well as applications to data sets from acquired immune deficiency syndrome studies.

Post percutaneous coronary intervention hemoglobin levels predict in-hospital mortality in patients with STEMI treated with primary percutaneous coronary intervention.

In this study, we aimed to determine whether admission hemoglobin versus post-percutaneous coronary intervention (PCI) hemoglobin level at 24 hours is a predictor of in-hospital mortality for patients with ST elevation myocardial infarction (STEMI) without evidence of clinical hemorrhage who underwent primary PCI. In this study, we included 1,444 consecutive patients with STEMI who underwent primary PCI at a tertiary heart hospital. The primary outcome of the study was the in-hospital all-cause mortality. We used the penalized maximum likelihood estimation (PMLE) logistic regression method to examine the relationship between primary outcome and candidate predictors. In total, 172 (11.9%) patients died during the in-hospital course. According to a PMLE logistic regression analysis, age, KILLIP class ≥2, pre-PCI thrombolysis in myocardial infarction (TIMI) flow <3, systolic blood pressure, creatinine, glycoprotein IIb/IIIa inhibitor use, and post-PCI hemoglobin levels at 24 hours were predictors of in-hospital mortality. The relative importance of post-PCI hemoglobin at 24 hours (contributing 6% of the explainable outcome in the model) was significantly higher than admission hemoglobin (contributing only 0.1% of the explainable outcome in the model). This study demonstrated that post-PCI hemoglobin levels were independently associated with in-hospital survival in patients with STEMI without evidence of bleeding following primary PCI. In addition, post-PCI hemoglobin was a better predictor of in-hospital mortality than admission hemoglobin for patients with STEMI who underwent primary PCI.