Penalized Likelihood Approach Research Articles

On variable selection in a semiparametric AFT mixture cure model.

In clinical studies, one often encounters time-to-event data that are subject to right censoring and for which a fraction of the patients under study never experience the event of interest. Such data can be modeled using cure models in survival analysis. In the presence of cure fraction, the mixture cure model is popular, since it allows to model probability to be cured (called the incidence) and the survival function of the uncured individuals (called the latency). In this paper, we develop a variable selection procedure for the incidence and latency parts of a mixture cure model, consisting of a logistic model for the incidence and a semiparametric accelerated failure time model for the latency. We use a penalized likelihood approach, based on adaptive LASSO penalties for each part of the model, and we consider two algorithms for optimizing the criterion function. Extensive simulations are carried out to assess the accuracy of the proposed selection procedure. Finally, we employ the proposed method to a real dataset regarding heart failure patients with left ventricular systolic dysfunction.

Intramedullary Spinal Cord Tumors: Whole-Genome Sequencing to Assist Management and Prognosis.

Intramedullary spinal cord tumors (IMSCTs) harbor unique genetic mutations which may play a role in prognostication and management. To this end, we present the largest cohort of IMSCTs with genetic characterization in the literature from our multi-site institutional registry. A total of 93 IMSCT patient records were reviewed from the years 1999 to 2020. Out of these, 61 complied with all inclusion criteria, 14 of these patients had undergone genetic studies with 8 undergoing whole-genomic sequencing. Univariate analyses were used to assess any factors associated with progression-free survival (PFS) using the Cox proportional hazards model. Firth's penalized likelihood approach was used to account for the low event rates. Fisher's exact test was performed to compare whole-genome analyses and specific gene mutations with progression. PFS (months) was given as a hazard ratio. Only the absence of copy neutral loss of heterozygosity (LOH) was shown to be significant (0.05, p = 0.008). Additionally, higher risk of recurrence/progression was associated with LOH (p = 0.0179). Our results suggest LOH as a genetic predictor of shorter progression-free survival, particularly within ependymoma and glioblastoma tumor types. Further genomic research with larger multi-institutional datasets should focus on these mutations as possible prognostic factors.

Loneliness and firearm acquisition among low-income U.S. veterans: Findings from the National Veteran Homeless and Other Poverty Experiences (NV-HOPE) study

RationaleThere are theoretical reasons to expect loneliness related to firearm purchasing. For instance, loneliness might amplify social isolation and feelings of insecurity, anxiety, perceived danger, and need for self-reliance. Purchasing a firearm might be seen as a way to bolster one's security and gain a sense of control when faced with potential threats, yet there is a lack of research assessing this possibility. ObjectiveThis study aimed to examine the relationship between loneliness and purchasing a firearm among a sample of low-income U.S. military veterans. MethodsData are from a national survey of 1,004 low-income U.S. veterans collected in December 2022 and January 2023. Firth logistic regression—a rare event logistic regression model to address small-sample bias stemming from rare outcomes through a penalized likelihood approach—was used to estimate the adjusted association between loneliness and purchasing a firearm in the past year. ResultsApproximately 5.4% reported the purchase of a new firearm in the past year, and respondents reported an average loneliness score of 1.82 (standard deviation = 1.96; range = 0–6). Results of Firth Logit regression analyses found that net of control variables for demographic, socioeconomic, and household characteristics, loneliness was positively associated with purchasing a firearm in the past year (OR = 1.167, 95% CI = 1.006, 1.352). ConclusionThe findings indicate that loneliness among U.S. military veterans is associated with firearm purchasing. Considering the risks posed by both loneliness and firearm ownership for veteran suicide, the findings suggest the need for greater focus and synergistic strategies to address loneliness and safe firearm practices among U.S. veterans.

On semiparametric accelerated failure time models with time-varying covariates: A maximum penalised likelihood estimation.

The accelerated failure time (AFT) model offers an important and useful alternative to the conventional Cox proportional hazards model, particularly when the proportional hazards assumption for a Cox model is violated. Since an AFT model is basically a log-linear model, meaningful interpretations of covariate effects on failure times can be made directly. However, estimation of a semiparametric AFT model imposes computational challenges even when it only has time-fixed covariates, and the situation becomes much more complicated when time-varying covariates are included. In this paper, we propose a penalised likelihood approach to estimate the semiparametric AFT model with right-censored failure time, where both time-fixed and time-varying covariates are permitted. We adopt the Gaussian basis functions to construct a smooth approximation to the nonparametric baseline hazard. This model fitting method requires a constrained optimisation approach. A comprehensive simulation study is conducted to demonstrate the performance of the proposed method. An application of our method to a motor neuron disease data set is provided.

Simultaneous selection and incorporation of consistent external aggregate information.

Interest has grown in synthesizing participant level data of a study with relevant external aggregate information. Several efficient and flexible procedures have been developed under the assumption that the internal study and the external sources concern the same population. This homogeneity condition, albeit commonly being imposed, is hard to check due to limitedly available external information in aggregate data forms. Bias may be introduced when the assumption is violated. In this article, we propose a penalized likelihood approach that avoids undesirable bias by simultaneously selecting and synthesizing consistent external aggregate information. The proposed approach provides a general framework which incorporate consistent external information from heterogeneous study populations as long as the conditional distribution of the dependent variable under investigation is same and differences in the independent variable distributions are properly accounted for via a semi-parametric density ratio model. The proposed approach also properly accounts for the sampling errors in the external information. A two-step estimator and an optimization algorithm are proposed for computation. We establish the selection and estimation consistency and the asymptotic normality of the two-step estimator. The proposed approach is illustrated with an analysis of gestational weight gain management studies.

Identification of sources of DIF using covariates in patient-reported outcome measures: a simulation study comparing two approaches based on Rasch family models.

When analyzing patient-reported outcome (PRO) data, sources of differential item functioning (DIF) can be multiple and there may be more than one covariate of interest. Hence, it could be of great interest to disentangle their effects. Yet, in the literature on PRO measures, there are many studies where DIF detection is applied separately and independently for each covariate under examination. With such an approach, the covariates under investigation are not introduced together in the analysis, preventing from simultaneously studying their potential DIF effects on the questionnaire items. One issue, among others, is that it may lead to the detection of false-positive effects when covariates are correlated. To overcome this issue, we developed two new algorithms (namely ROSALI-DIF FORWARD and ROSALI-DIF BACKWARD). Our aim was to obtain an iterative item-by-item DIF detection method based on Rasch family models that enable to adjust group comparisons for DIF in presence of two binary covariates. Both algorithms were evaluated through a simulation study under various conditions aiming to be representative of health research contexts. The performance of the algorithms was assessed using: (i) the rates of false and correct detection of DIF, (ii) the DIF size and form recovery, and (iii) the bias in the latent variable level estimation. We compared the performance of the ROSALI-DIF algorithms to the one of another approach based on likelihood penalization. For both algorithms, the rate of false detection of DIF was close to 5%. The DIF size and form influenced the rates of correct detection of DIF. Rates of correct detection was higher with increasing DIF size. Besides, the algorithm fairly identified homogeneous differences in the item threshold parameters, but had more difficulties identifying non-homogeneous differences. Over all, the ROSALI-DIF algorithms performed better than the penalized likelihood approach. Integrating several covariates during the DIF detection process may allow a better assessment and understanding of DIF. This study provides valuable insights regarding the performance of different approaches that could be undertaken to fulfill this aim.

IDENTIFICATION OF IMMUNE RESPONSE COMBINATIONS ASSOCIATED WITH HETEROGENEOUS INFECTION RISK IN THE IMMUNE CORRELATES ANALYSIS OF HIV VACCINE STUDIES.

In HIV vaccine/prevention research, probing into the vaccine-induced immune responses that can help predict the risk of HIV infection provides valuable information for the development of vaccine regimens. Previous correlate analysis of the Thai vaccine trial aided the discovery of interesting immune correlates related to the risk of developing an HIV infection. The present study aimed to identify the combinations of immune responses associated with the heterogeneous infection risk. We explored a "change-plane" via combination of a subset of immune responses that could help separate vaccine recipients into two heterogeneous subgroups in terms of the association between immune responses and the risk of developing infection. Additionally, we developed a new variable selection algorithm through a penalized likelihood approach to investigate a parsimonious marker combination for the change-plane. The resulting marker combinations can serve as candidate correlates of protection and can be used for predicting the protective effect of the vaccine against HIV infection. The application of the proposed statistical approach to the Thai trial has been presented, wherein the marker combinations were explored among several immune responses and antigens.

Network Inference Using the Hub Model and Variants

Statistical network analysis primarily focuses on inferring the parameters of an observed network. In many applications, especially in the social sciences, the observed data is the groups formed by individual subjects. In these applications, the network is itself a parameter of a statistical model. Zhao and Weko propose a model-based approach, called the hub model, to infer implicit networks from grouping behavior. The hub model assumes that each member of the group is brought together by a member of the group called the hub. The set of members which can serve as a hub is called the hub set. The hub model belongs to the family of Bernoulli mixture models. Identifiability of Bernoulli mixture model parameters is a notoriously difficult problem. This article proves identifiability of the hub model parameters and estimation consistency under mild conditions. Furthermore, this article generalizes the hub model by introducing a model component that allows hubless groups in which individual nodes spontaneously appear independent of any other individual. We refer to this additional component as the null component. The new model bridges the gap between the hub model and the degenerate case of the mixture model—the Bernoulli product. Identifiability and consistency are also proved for the new model. In addition, a penalized likelihood approach is proposed to estimate the hub set when it is unknown. Supplementary materials for this article are available online.

Variable selection using the EM and CEM algorithms in mixtures of linear mixed models

In order to deal with data of great complexity, the need to develop new methods for variable selection has emerged. Among the new methods, methods based on penalty functions have received great attention. These methods can be used in complex data problems since they shrink a subset of coefficient estimates to zero, thus removing the associated variables from the model, which allows the identification of the subset of the most relevant explanatory variables and, consequently, drastically reduces the computational burden. In this work we analyse the problem of variable selection in mixtures of linear mixed models. In order to do it, we compare the performance of a penalized likelihood approach for variable selection via the Expectation-Maximization and the Classification Expectation-Maximization algorithms through a simulation study and a real-world application.

Cox models with time-varying covariates and partly-interval censoring-A maximum penalised likelihood approach.

Time-varying covariates can be important predictors when model based predictions are considered. A Cox model that includes time-varying covariates is usually referred to as an extended Cox model. When only right censoring is presented in the observed survival times, the conventional partial likelihood method is still applicable to estimate the regression coefficients of an extended Cox model. However, if there are interval-censored survival times, then the partial likelihood method is not directly available unless an imputation, such as the middle point imputation, is used to replaced the left- and interval-censored data. However, such imputation methods are well known for causing biases. This paper considers fitting of the extended Cox models using the maximum penalised likelihood method allowing observed survival times to be partly interval censored, where a penalty function is used to regularise the baseline hazard estimate. We present simulation studies to demonstrate the performance of our proposed method, and illustrate our method with applications to two real datasets from medical research.

Robust Multivariate Lasso Regression with Covariance Estimation

Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking account the covariance structure of the response variables. MRCE uses a penalized likelihood approach to simultaneously estimate the regression coefficients and the inverse covariance matrix so that prediction accuracy can be significantly improved. However, traditional likelihood-based methods such as MRCE can produce very misleading results in the presence of outliers. In this work, we propose an extension of MRCE, namely, a robust multivariate lasso regression with covariance estimation (RMLC) to handle potential outliers within the data. By using Huber’s loss or Tukey’s biweight loss, RMLC can be resistant to outliers in the responses or in both the responses and the covariates. A novel optimization algorithm that incorporates a 2-fold accelerated proximal gradient (APG) algorithm is developed to solve RMLC efficiently. We also demonstrate that our proposed RMLC enjoys the oracle property. Our simulation study shows that RMLC produces very reliable results for both the regression coefficients and the correlation structure of the responses, even if the data are contaminated. A real analysis on hyperspectral data further demonstrates the utility of RMLC. Supplementary materials for this article are available online.

Data-driven spatiotemporal assessment of the event-size distribution of the Groningen extraction-induced seismicity catalogue

For induced seismicity, the non-stationary, heterogeneous character of subsurface stress perturbations can be a source of spatiotemporal variations in the scaling of event sizes; one of the critical parameters controlling seismic hazard and risk. We demonstrate and test a systematic, statistical, penalized-likelihood approach to analysing both spatial and temporal variations in event size distributions. The methodology used is transferable to the risk analysis of any subsurface operation, especially for small earthquake catalogues. We explore the whole solution space and circumvent conventional, arbitrary choices that require a priori knowledge of these variations. We assess the effect of possible bias in the derivation, e.g., due to tapering of the earthquake-size distribution, correlation between the b-value and the magnitude of completeness and correlation between the b-value and the largest magnitude observed. We analyse the spatiotemporal variations in the earthquake-size distribution of the Groningen induced seismicity catalogue (December 1991–November 16, 2021). We find statistically significant spatial variations without any compelling, statistical evidence of a temporal variation. Furthermore, we find that the largest magnitudes observed are inconsistent with the sampling statistics of an unconstrained earthquake-size distribution. Current risk assessment models likely overestimate the probability of larger magnitude events (M ≥ 3.0) and thus the risk posed.

On estimation for accelerated failure time models with small or rare event survival data

BackgroundSeparation or monotone likelihood may exist in fitting process of the accelerated failure time (AFT) model using maximum likelihood approach when sample size is small and/or rate of censoring is high (rare event) or there is at least one strong covariate in the model, resulting in infinite estimates of at least one regression coefficient.MethodsThis paper investigated the properties of the maximum likelihood estimator (MLE) of the regression parameters of the AFT models for small sample and/or rare-event situation and addressed the problems by introducing a penalized likelihood approach. The penalized likelihood function and the corresponding score equation is derived by adding a penalty term to the existing likelihood function, which was originally proposed by Firth (Biometrika, 1993) for the exponential family models. Further, a post-hoc adjustment of intercept and scale parameters is discussed keeping them out of penalization to ensure accurate prediction of survival probability. The penalized method was illustrated for the widely used log-location-scale family models such as Weibull, Log-normal and Log-logistic distributions and compared the models and methods uisng an extensive simulation study.ResultsThe simulation study, performed separately for each of the log-location-scale models, showed that Firth’s penalized likelihood succeeded to solve the problem of separation and achieve convergence, providing finite estimates of the regression coefficients, which are not often possible by the MLE. Furthermore, the proposed penalized method showed substantial improvement over MLE by providing smaller amount of bias, mean squared error (MSE), narrower confidence interval and reasonably accurate prediction of survival probabilities. The methods are illustrated using prostate cancer data with existence of separation, and results supported the simulation findings.ConclusionWhen sample size is small (≤ 50) or event is rare (i.e., censoring proportion is high) and/or there is any evidence of separation in the data, we recommend to use Firth’s penalized likelihood method for fitting AFT model.

Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models.

Cognitive Diagnosis Models (CDMs) are a special family of discrete latent variable models that are widely used in educational and psychological measurement. A key component of CDMs is the Q-matrix characterizing the dependence structure between the items and the latent attributes. Additionally, researchers also assume in many applications certain hierarchical structures among the latent attributes to characterize their dependence. In most CDM applications, the attribute-attribute hierarchical structures, the item-attribute Q-matrix, the item-level diagnostic models, as well as the number of latent attributes, need to be fully or partially pre-specified, which however may be subjective and misspecified as noted by many recent studies. This paper considers the problem of jointly learning these latent and hierarchical structures in CDMs from observed data with minimal model assumptions. Specifically, a penalized likelihood approach is proposed to select the number of attributes and estimate the latent and hierarchical structures simultaneously. An expectation-maximization (EM) algorithm is developed for efficient computation, and statistical consistency theory is also established under mild conditions. The good performance of the proposed method is illustrated by simulation studies and real data applications in educational assessment.

A penalized likelihood approach for dealing with separation in count data regression model

Separation or monotone likelihood can be observed in fitting process of both Poisson or generalized Poisson (GP) regression, particularly in case of small and/or sparse count data, using maximum likelihood estimation (MLE) when one or more regression coefficients diverge to infinity. The study investigates the consequence of separation in the MLE based standard Poisson or GP model and addressed the problems by introducing a penalized likelihood approach. The penalized likelihood function is derived by adding a penalty term to the standard maximum likelihood function, which was originally proposed by Firth (1993) for reducing first order bias in MLE. The corresponding penalized likelihood score equation has shown to achieve convergence and provide finite estimate of the regression coefficient, which was not possible for the maximum likelihood method score equation. The simulation study, with different forms of separation, showed that penalized Poisson model or penalized GP outperform the standard Poisson and even the Zero-inflated Poisson (ZIP) in the presence of complete or quasi-complete separation by achieving convergence and providing finite estimate of the regression coefficients. Even in the presence of near-to-quasi-complete separation, which is very common in practice, the penalized method showed better results than the standard Poisson, GP and ZIP in all simulation scenarios. The method was illustrated using antenatal care data extracted from database of the Bangladesh demographic health survey 2018.

On estimating a constrained bivariate random effects model for meta-analysis of test accuracy studies.

Tailored meta-analysis uses setting-specific knowledge for the test positive rate and disease prevalence to constrain the possible values for a test's sensitivity and specificity. The constrained region is used to select those studies relevant to the setting for meta-analysis using an unconstrained bivariate random effects model (BRM). However, sometimes there may be no studies to aggregate, or the summary estimate may lie outside the plausible or “applicable” region. Potentially these shortcomings may be overcome by incorporating the constraints in the BRM to produce a constrained model. Using a penalised likelihood approach we developed an optimisation algorithm based on co-ordinate ascent and Newton-Raphson iteration to fit a constrained bivariate random effects model (CBRM) for meta-analysis. Using numerical examples based on simulation studies and real datasets we compared its performance with the BRM in terms of bias, mean squared error and coverage probability. We also determined the ‘closeness’ of the estimates to their true values using the Euclidian and Mahalanobis distances. The CBRM produced estimates which in the majority of cases had lower absolute mean bias and greater coverage probability than the BRM. The estimated sensitivities and specificity for the CBRM were, in general, closer to the true values than the BRM. For the two real datasets, the CBRM produced estimates which were in the applicable region in contrast to the BRM. When combining setting-specific data with test accuracy meta-analysis, a constrained model is more likely to yield a plausible estimate for the sensitivity and specificity in the practice setting than an unconstrained model.

Variable Selection for Robust Mixture Regression Model with Skew Scale Mixtures of Normal Distributions

In this paper, we propose a robust mixture regression model based on the skew scale mixtures of normal distributions (RMR-SSMN) which can accommodate asymmetric, heavy-tailed and contaminated data better. For the variable selection problem, the penalized likelihood approach with a new combined penalty function which balances the SCAD and l2 penalty is proposed. The adjusted EM algorithm is presented to get parameter estimates of RMR-SSMN models at a faster convergence rate. As simulations show, our mixture models are more robust than general FMR models and the new combined penalty function outperforms SCAD for variable selection. Finally, the proposed methodology and algorithm are applied to a real data set and achieve reasonable results.

Sparse assortment personalization in high dimensions

The data-driven conditional multinomial logit choice model with customer features performs well in the assortment personalization problem when the low-rank structure of the parameter matrix is considered. However, despite recent theoretical and algorithmic advances, parameter estimation in the choice model still poses a challenging task, especially when there are more predictors than observations. For this reason, we suggest a penalized likelihood approach based on a feature matrix to recover the sparse structure from populations and products toward the assortment. Our proposed method considers simultaneously low-rank and sparsity structures, which can further reduce model complexity and improve its estimation and prediction accuracy. A new algorithm, sparse factorial gradient descent (SFGD), was proposed to estimate the parameter matrix, which has high interpretability and efficient computing performance. As a first-order method, the SFGD works well in high-dimensional scenarios because of the absence of the Hessian matrix. Simulation studies show that the SFGD algorithm outperforms state-of-the-art methods in terms of estimation, sparsity recovery, and average regret. We also demonstrate the effectiveness of our proposed method using advertising behavior data analysis.

Estimating the number of components in finite mixture models via the Group-Sort-Fuse procedure

Estimation of the number of components (or order) of a finite mixture model is a long standing and challenging problem in statistics. We propose the Group-Sort-Fuse (GSF) procedure—a new penalized likelihood approach for simultaneous estimation of the order and mixing measure in multidimensional finite mixture models. Unlike methods which fit and compare mixtures with varying orders using criteria involving model complexity, our approach directly penalizes a continuous function of the model parameters. More specifically, given a conservative upper bound on the order, the GSF groups and sorts mixture component parameters to fuse those which are redundant. For a wide range of finite mixture models, we show that the GSF is consistent in estimating the true mixture order and achieves the n−1/2 convergence rate for parameter estimation up to polylogarithmic factors. The GSF is implemented for several univariate and multivariate mixture models in the R package GroupSortFuse. Its finite sample performance is supported by a thorough simulation study, and its application is illustrated on two real data examples.

Prevalence of and Risk Factors for Sagittal Posture Abnormalities in Children Born With Esophageal Atresia: A Prospective Cohort Study.

Introduction: Scoliosis is a well-described complication of esophageal atresia (EA) caused by the associated spine malformations and/or thoracotomy. However, the sagittal posture abnormalities in patients with EA have not been described. The aim of this study was to evaluate the prevalence of and risk factors for sagittal posture abnormalities at the age of 6 years in patients operated on for EA.Methods: A prospective cohort of 123 patients with EA was examined by the same rehabilitation doctor at the time of a multidisciplinary visit scheduled at the age of 6 years. Children presenting with scoliosis (n = 4) or who missed the consultation (n = 33) were excluded. Univariate and multivariate logistic regression models with Firth's penalized-likelihood approach were used to identify risk factors associated with sagittal posture anomalies. Candidate risk factors included neonatal characteristics, associated malformations, atresia type, postoperative complications, psychomotor development retardation, orthopedic abnormalities, and neurological hypotonia.Results: The prevalence rates of sagittal posture abnormalities were 25.6% (n = 22; 95% CI, 16.7–36.1%). Multivariate analysis showed that minor orthopedic abnormalities (OR: 4.02, 95% CI: 1.29–13.43, P = 0.021), and VACTERL (OR: 3.35, 95% CI: 1.09–10.71, P = 0.042) were significant risk factors for sagittal posture abnormalities.Conclusion: This study shows that sagittal posture anomalies occur frequently in children operated on at birth for EA and are not directly linked to the surgical repair. These children should be screened and treated using postural physiotherapy, especially those with VACTERL and minor orthopedic abnormalities.