Penalized Quasi-likelihood Research Articles

A simulation study to assess the impact missing values on the performance of different statistical methods for analysis of binary repeated measures data with an additional hierarchical structure

The primary objective of the study was to assess the impact of missing values on the analy- sis of binary repeated measures data with an additional hierarchical structure. One motivat- ing example for the present study was records of high somatic cell counts in milk samples obtained by approximately monthly sampling throughout the lactations of cows in dairy herds. Random effects models with autocorrelated (ρ = 1, 0.9 or 0.5) subject-level ran- dom effects were behind the simulated data. In general, the settings of the simulation were chosen to reflect a real somatic cell count dataset (scc40), except that the within-cow time series length was set to 8-time points for each cow. The estimation procedures consid- ered were: Ordinary Logistic Regression (OLR), Alternating Logistic Regression (ALR), Weighted Generalized Estimating Equations (WGEE), Penalized Quasi Likelihood (PQL), Maximum likelihood via numerical integration (ML) and Bayesian Markov chain Monte Carlo (MCMC). Multiple scenarios of simulated incomplete datasets were considered and include: a scenario corresponded to a combination of missingness patterns present in the scc40 dataset (scc40 scenario) The remaining scenarios involved only drop-outs, and corre- sponded to either moderate or high percentages of values either missing at random (MAR) or not missing at random (NMAR), respectively. In the scc40 scenario, all estimation procedures except OLR performed well and produced estimates with small relative bias (generally less than 5%) for levels of missingness that roughly corresponded to the scc40 data. In MAR missingness scenarios, some biases were found for ALR, WGEE and PQL procedures, whereas the likelihood-based procedures were largely unaffected by the miss- ing values. In NMAR scenarios, all procedures experienced similar and strong biases in the time coefficient; however, fixed effects estimates at the subject and cluster levels were relatively unaffected. Journal of Statistical Research 2023, Vol 57, No.1-2, pp.35-67

The Use Penalized Quasi Likelihood (PQL) to Estimate Multilevel Binary Logistic Model to Identify the Factors of Anemia

Multilevel logistic regression analysis is used in many fields, including health, medical, geography, social and educational, where the researchers were interested in this analysis to identify and study the nature of the relationship between the behavior of the vocabulary or units of study, and the social, environmental and economic variables in different environments in which they live and belong, and in such hierarchical data. In this research, multilevel data was used, Data were taken on anemia in children, as the dependent variable represents anemia infections (infected - uninfected) from four hospitals affiliated to the Babylon Health Department, namely (Al-Hah General Teaching Hospital - Imam Al-Sadiq Hospital (PBUH) - Marjan General Teaching Hospital - Babel Women's and Children's Hospital ) which represents the third level in the analysis and according to the type of lobby (public - private), which represents the second level. (50) cases of anemia were taken from the general ward from the general ward and (20) cases from the private ward, and from Imam al-Sadiq Hospital ( P) Peace, (75) sick cases were taken from the general ward and (20) sick cases from the private ward. From Marjan General Teaching Hospital, (30) sick cases were taken from the private ward and (20) sick cases from the general ward, and (40) sick cases were taken from the general ward and (25) sick cases from the general ward, and these sick cases represent the level Third, in the multi-level analysis, so that the total number of disease cases is (290) cases, and the independent variables that can affect anemia were taken (sex, age, weight, occupation, marital status, smoking, academic achievement, place of residence, infection with other diseases, blood pressure). That is, the first level contains (10) independent variables. The logistic analysis of the multilevel binary was carried out using the NCSS 2022 program, using the method of possibility semi-penalty. has been reached Significance of the multi-level binary logistic regression model, as the p-value = 0.0015, which is less than the level of significance 1%, and achieved a high odds ratio of (0.7767). The variables (age - occupation - smoking - blood pressure) are not significant and the variables (weight - water source - place of residence - other diseases - wealth index) are significant.

SpatialMap: Spatial Mapping of Unmeasured Gene Expression Profiles in Spatial Transcriptomic Data Using Generalized Linear Spatial Models.

Recent advances in various single-cell RNA sequencing (scRNA-seq) technologies have enabled profiling the gene expression level with the whole transcriptome at a single-cell resolution. However, it lacks the spatial context of tissues. The image-based transcriptomics in situ studies (e.g., MERFISH and seqFISH) maintain the cell spatial context at individual cell levels but can only measure a limited number of genes or transcripts (up to roughly 1,000 genes). Therefore, integrating scRNA-seq data and image-based transcriptomics data can potentially gain the complementary benefits of both. Here, we develop a computational method, SpatialMap, to bridge the gap, which primarily facilitates spatial mapping of unmeasured gene profiles in spatial transcriptomic data via integrating with scRNA-seq data from the same tissue. SpatialMap directly models the count nature of spatial gene expression data through generalized linear spatial models, which accounts for the spatial correlation among spatial locations using conditional autoregressive (CAR) prior. With a newly developed computationally efficient penalized quasi-likelihood (PQL)-based algorithm, SpatialMap can scale up to performing large-scale spatial mapping analysis. Finally, we applied the SpatialMap to four publicly available tissue-paired studies (i.e., scRNA-seq studies and image-based transcriptomics studies). The results demonstrate that the proposed method can accurately predict unmeasured gene expression profiles across various spatial and scRNA-seq dataset pairs of different species and technologies.

A STUDY OF GENERALIZED LINEAR MIXED MODEL FOR COUNT DATA USING HIERARCHICAL BAYES METHOD

Poisson Log-Normal Model is one of the hierarchical mixed models that can be used for count data. Several estimation methods can be used to estimate the model parameters. The first objective of this study was to examine the performance of the parameter estimator and model built using the Hierarchical Bayes method via Markov Chain Monte Carlo (MCMC) with simulation. The second objective was applied the Poisson Log-Normal model to the West Java illiteracy Cases data which is sourced from the Susenas data on March 2019. In 2019, the incidence of illiteracy is a very rare occurrence in West Java Province. So that, it is suitable as an application case in this study. The simulation results showed that the Hierarchical Bayes parameter estimator through MCMC has the smallest Root Mean Squared Error of Prediction (RMSEP) value and the absolute bias is relatively mostly similar when compared to the Maximum Likelihood (ML) and Penalized Quasi-Likelihood (PQL) methods. Meanwhile, the empirical results showed that the fixed variable is the number of respondents who have a maximum education of elementary school have the greatest risk of illiteracy. Also, the diversity of census blocks significantly affects illiteracy cases in West Java 2019.

A comparison of statistical methods for the analysis of binary repeated measures data with additional hierarchical structure

The objective of the study was to compare statistical methods for the analysis of binary repeated measures data with an additional hierarchical level. Random effects true models with autocorrelated ($\rho=1$, 0.9 or 0.5) subject random effects were used in this simulation study. The settings of the simulation were chosen to reflect a real veterinary somatic cell count dataset, except that the within--subject time series were balanced, complete and of fixed length (4 or 8 time points). Four fixed effects parameters were studied: binary predictors at the subject and cluster levels, respectively, a linear time effect, and the intercept. The following marginal and random effects statistical procedures were considered: ordinary logistic regression (OLR), alternating logistic regression (ALR), generalized estimating equations (GEE), marginal quasi-likelihood (MQL), penalized quasi-likelihood (PQL), pseudo likelihood (REPL), maximum likelihood (ML) estimation and Bayesian Markov chain Monte Carlo (MCMC). The performance of these estimation procedures was compared specifically for the four fixed parameters as well as variance and correlation parameters. The findings of this study indicate that in data generated by random intercept models ($\rho=1$), the ML and MCMC procedures performed well and had fairly similar estimation errors. The PQL regression estimates were attenuated while the variance estimates were less accurate than ML and MCMC, but the direction of the bias depended on whether binomial or extra-binomial dispersion was assumed. In datasets with autocorrelation ($\rho<1$), random effects estimates procedures gave downwards biased estimates, while marginal estimates were little affected by the presence of autocorrelation. The results also indicate that in addition to ALR, a GEE procedure that accounts for clustering at the highest hierarchical level is sufficient.

Laplace approximation, penalized quasi-likelihood, and adaptive Gauss\u2013Hermite quadrature for generalized linear mixed models: towards meta-analysis of binary outcome with sparse data

BackgroundIn meta-analyses of a binary outcome, double zero events in some studies cause a critical methodology problem. The generalized linear mixed model (GLMM) has been proposed as a valid statistical tool for pooling such data. Three parameter estimation methods, including the Laplace approximation (LA), penalized quasi-likelihood (PQL) and adaptive Gauss–Hermite quadrature (AGHQ) were frequently used in the GLMM. However, the performance of GLMM via these estimation methods is unclear in meta-analysis with zero events.MethodsA simulation study was conducted to compare the performance. We fitted five random-effects GLMMs and estimated the results through the LA, PQL and AGHQ methods, respectively. Each scenario conducted 20,000 simulation iterations. The data from Cochrane Database of Systematic Reviews were collected to form the simulation settings. The estimation methods were compared in terms of the convergence rate, bias, mean square error, and coverage probability.ResultsOur results suggested that when the total events were insufficient in either of the arms, the GLMMs did not show good point estimation to pool studies of rare events. The AGHQ method did not show better properties than the LA estimation in terms of convergence rate, bias, coverage, and possibility to produce very large odds ratios. In addition, although the PQL had some advantages, it was not the preferred option due to its low convergence rate in some situations, and the suboptimal point and variance estimation compared to the LA.ConclusionThe GLMM is an alternative for meta-analysis of rare events and is especially useful in the presence of zero-events studies, while at least 10 total events in both arms is recommended when employing GLMM for meta-analysis. The penalized quasi-likelihood and adaptive Gauss–Hermite quadrature are not superior to the Laplace approximation for rare events and thus they are not recommended.

Firth bias correction for estimating variance components of logistics linear mixed model using penalized quasi likelihood method

Firth bias correction originally was applied to correct bias of the variance components estimator that obtained by the maximum likelihood method. Extensive research has shown that Firth bias correction is powerful to reduce bias for normal distributed response model. Questions have been raised about the use of Firth bias correction in binomial distributed response model which has under dispersion problem. The motivation of this study is giving contribution to exploring the Firth bias correction for binomial distributed response model. The binomial distributed response model which is estimated by the maximum likelihood method obtain an under-dispersion estimator. Therefore, the Penalized Quasi-Likelihood (PQL) is used as alternative numerical method to estimate the model. This paper aims to investigate whether the Firth method can reduce bias of the variance components using the PQL technique in longitudinal data.

Sample size issues in multilevel logistic regression models.

Educational researchers, psychologists, social, epidemiological and medical scientists are often dealing with multilevel data. Sometimes, the response variable in multilevel data is categorical in nature and needs to be analyzed through Multilevel Logistic Regression Models. The main theme of this paper is to provide guidelines for the analysts to select an appropriate sample size while fitting multilevel logistic regression models for different threshold parameters and different estimation methods. Simulation studies have been performed to obtain optimum sample size for Penalized Quasi-likelihood (PQL) and Maximum Likelihood (ML) Methods of estimation. Our results suggest that Maximum Likelihood Method performs better than Penalized Quasi-likelihood Method and requires relatively small sample under chosen conditions. To achieve sufficient accuracy of fixed and random effects under ML method, we established ''50/50" and ''120/50" rule respectively. On the basis our findings, a ''50/60" and ''120/70" rules under PQL method of estimation have also been recommended.

Poisson Multilevel Models with Small Samples

Recent methodological studies have investigated the properties of multilevel models with small samples. Previous work has primarily focused on continuous outcomes and little attention has been paid to count outcomes. The estimation of count outcome models can be difficult because the likelihood has no closed-form solution, meaning that approximation methods are required. Although adaptive Gaussian quadrature (AGQ) is generally seen as the gold standard, its comparative performance has been investigated with larger samples. AGQ approximates the full likelihood, a function that is known to produce biased estimates with small samples with continuous outcomes. Conversely, penalized quasi-likelihood (PQL) is considered to be a less desirable approximation; however, it can approximate the restricted likelihood function, a function that is known to perform well with smaller samples with continuous outcomes. The goal of this paper is to compare the small sample bias of full likelihood methods to the linearization bias of PQL with restricted likelihood. Simulation results indicate that the linearization bias of PQL is preferable to the finite sample bias of AGQ with smaller samples.

Estimation of small area counts with the benchmarking property

Estimation of small area totals makes use of auxiliary variables to borrow strength from related areas through a model. Precision of final small area estimates depends on the validity of such a model. To protect against possible model failures, benchmarking procedures make the sum of small area estimates match a design consistent estimate of the total of a larger area. This is also particularly important for national institutes of statistics to ensure coherence between small area estimates and direct estimates produced at higher level planned domains. In this paper we propose a self-benchmarked estimator of small area totals which is based on a unit level logistic mixed model for a binary response. In particular, we use a plug-in approach and add a constraint to the maximum penalized quasi-likelihood (PQL) procedure considered in Saei and Chambers (Working paper M03/15, Southampton Statistical Sciences Research Institute. University of Southampton, 2003) to accommodate benchmarking. An analytic estimator for the mean squared error of the final small area estimator is also proposed following the ad-hoc procedure proposed by Gonzalez-Manteiga et al. (Comput Stat Data Anal 51:2720–2733, 2007). We then compare the performance of the proposed benchmarked estimator with several competing estimators through a set of simulation studies.

Modeling Count Data from Dependent Clusters with Poisson Mixed Models


 
 
 
 Responses collected from dependent clusters are affected by the dependence among clusters and it should be taken into account in modeling such responses. In this study, a new approach was evaluated to incorporate cluster dependence in generalized linear Poisson mixed models for count responses from dependent clusters. Performance of this approach was evaluated by using a simulation process under three different designs and different covariates. The Marginal Generalized Quasi-likelihood (GQL) method was used for estimation of parameters with the cluster dependence. Monte Carlo likelihood (MCL) and Penalized Quasi-likelihood (PQL) estimates also were obtained for the purpose of comparison. Proposed approach was tested with a real data set also.
 The proposed approach, with the incorporation of cluster dependence, gives better estimates for both fixed effects and variance of random effects with low standard errors with compared to the estimates obtained by ignoring the cluster dependence. Therefore, the proposed approach can be used for modeling count responses from a dependent cluster set up.
 
 
 

Spatio-Temporal Variation of HIV Infection in Kenya

Disease mapping is the study of the distribution of disease relative risks or rates in space and time, and normally uses generalized linear mixed models (GLMMs) which includes fixed effects and spatial, temporal, and spatio-temporal random effects. Model fitting and statistical inference are commonly accomplished through the empirical Bayes (EB) and fully Bayes (FB) approaches. The EB approach usually relies on the penalized quasi-likelihood (PQL), while the FB approach, which has increasingly become more popular in the recent past, usually uses Markov chain Monte Carlo (McMC) techniques. However, there are many challenges in conventional use of posterior sampling via McMC for inference. This includes the need to evaluate convergence of posterior samples, which often requires extensive simulation and can be very time consuming. Spatio-temporal models used in disease mapping are often very complex and McMC methods may lead to large Monte Carlo errors if the dimension of the data at hand is large. To address these challenges, a new strategy based on integrated nested Laplace approximations (INLA) has recently been recently developed as a promising alternative to the McMC. This technique is now becoming more popular in disease mapping because of its ability to fit fairly complex space-time models much more quickly than the McMC. In this paper, we show how to fit different spatio-temporal models for disease mapping with INLA using the Leroux CAR prior for the spatial component, and we compare it with McMC using Kenya HIV incidence data during the period 2013-2016.

Joint Selection in Mixed Models using Regularized PQL

ABSTRACTThe application of generalized linear mixed models presents some major challenges for both estimation, due to the intractable marginal likelihood, and model selection, as we usually want to jointly select over both fixed and random effects. We propose to overcome these challenges by combining penalized quasi-likelihood (PQL) estimation with sparsity inducing penalties on the fixed and random coefficients. The resulting approach, referred to as regularized PQL, is a computationally efficient method for performing joint selection in mixed models. A key aspect of regularized PQL involves the use of a group based penalty for the random effects: sparsity is induced such that all the coefficients for a random effect are shrunk to zero simultaneously, which in turn leads to the random effect being removed from the model. Despite being a quasi-likelihood approach, we show that regularized PQL is selection consistent, that is, it asymptotically selects the true set of fixed and random effects, in the setting where the cluster size grows with the number of clusters. Furthermore, we propose an information criterion for choosing the single tuning parameter and show that it facilitates selection consistency. Simulations demonstrate regularized PQL outperforms several currently employed methods for joint selection even if the cluster size is small compared to the number of clusters, while also offering dramatic reductions in computation time. Supplementary materials for this article are available online.

Sufficient Sample Size and Power in Multilevel Ordinal Logistic Regression Models

For most of the time, biomedical researchers have been dealing with ordinal outcome variable in multilevel models where patients are nested in doctors. We can justifiably apply multilevel cumulative logit model, where the outcome variable represents the mild, severe, and extremely severe intensity of diseases like malaria and typhoid in the form of ordered categories. Based on our simulation conditions, Maximum Likelihood (ML) method is better than Penalized Quasilikelihood (PQL) method in three-category ordinal outcome variable. PQL method, however, performs equally well as ML method where five-category ordinal outcome variable is used. Further, to achieve power more than 0.80, at least 50 groups are required for both ML and PQL methods of estimation. It may be pointed out that, for five-category ordinal response variable model, the power of PQL method is slightly higher than the power of ML method.

Multilevel cumulative logistic regression model with random effects: Application to British social attitudes panel survey data

A multilevel model for ordinal data in generalized linear mixed models (GLMM) framework is developed to account for the inherent dependencies among observations within clusters. Motivated by a data set from the British Social Attitudes Panel Survey (BSAPS), the random district effects and respondent effects are incorporated into the linear predictor to accommodate the nested clusterings. The fixed (random) effects are estimated (predicted) by maximizing the penalized quasi likelihood (PQL) function, whereas the variance component parameters are obtained via the restricted maximum likelihood (REML) estimation method. The model is employed to analyze the BSAPS data. Simulation studies are conducted to assess the performance of estimators.

Robustness of Bayesian D-optimal design for the logistic mixed model against misspecification of autocorrelation

In medicine and health sciences mixed effects models are often used to study time-structured data. Optimal designs for such studies have been shown useful to improve the precision of the estimators of the parameters. However, optimal designs for such studies are often derived under the assumption of a zero autocorrelation between the errors, especially for binary data. Ignoring or misspecifying the autocorrelation in the design stage can result in loss of efficiency. This paper addresses robustness of Bayesian D-optimal designs for the logistic mixed effects model for longitudinal data with a linear or quadratic time effect against incorrect specification of the autocorrelation. To find the Bayesian D-optimal allocations of time points for different values of the autocorrelation, under different priors for the fixed effects and different covariance structures of the random effects, a scalar function of the approximate variance---covariance matrix of the fixed effects is optimized. Two approximations are compared; one based on a first order penalized quasi likelihood (PQL1) and one based on an extended version of the generalized estimating equations (GEE). The results show that Bayesian D-optimal allocations of time points are robust against misspecification of the autocorrelation and are approximately equally spaced. Moreover, PQL1 and extended GEE give essentially the same Bayesian D-optimal allocation of time points for a given subject-to-measurement cost ratio. Furthermore, Bayesian optimal designs are hardly affected either by the choice of a covariance structure or by the choice of a prior distribution.

On fitting spatio-temporal disease mapping models using approximate Bayesian inference

Spatio-temporal disease mapping comprises a wide range of models used to describe the distribution of a disease in space and its evolution in time. These models have been commonly formulated within a hierarchical Bayesian framework with two main approaches: an empirical Bayes (EB) and a fully Bayes (FB) approach. The EB approach provides point estimates of the parameters relying on the well-known penalized quasi-likelihood (PQL) technique. The FB approach provides the posterior distribution of the target parameters. These marginal distributions are not usually available in closed form and common estimation procedures are based on Markov chain Monte Carlo (MCMC) methods. However, the spatio-temporal models used in disease mapping are often very complex and MCMC methods may lead to large Monte Carlo errors and a huge computation time if the dimension of the data at hand is large. To circumvent these potential inconveniences, a new technique called integrated nested Laplace approximations (INLA), based on nested Laplace approximations, has been proposed for Bayesian inference in latent Gaussian models. In this paper, we show how to fit different spatio-temporal models for disease mapping with INLA using the Leroux CAR prior for the spatial component, and we compare it with PQL via a simulation study. The spatio-temporal distribution of male brain cancer mortality in Spain during the period 1986-2010 is also analysed.

Generalized Linear Mixed Models for Binary Data: Are Matching Results from Penalized Quasi-Likelihood and Numerical Integration Less Biased?

BackgroundOver time, adaptive Gaussian Hermite quadrature (QUAD) has become the preferred method for estimating generalized linear mixed models with binary outcomes. However, penalized quasi-likelihood (PQL) is still used frequently. In this work, we systematically evaluated whether matching results from PQL and QUAD indicate less bias in estimated regression coefficients and variance parameters via simulation.MethodsWe performed a simulation study in which we varied the size of the data set, probability of the outcome, variance of the random effect, number of clusters and number of subjects per cluster, etc. We estimated bias in the regression coefficients, odds ratios and variance parameters as estimated via PQL and QUAD. We ascertained if similarity of estimated regression coefficients, odds ratios and variance parameters predicted less bias.ResultsOverall, we found that the absolute percent bias of the odds ratio estimated via PQL or QUAD increased as the PQL- and QUAD-estimated odds ratios became more discrepant, though results varied markedly depending on the characteristics of the datasetConclusionsGiven how markedly results varied depending on data set characteristics, specifying a rule above which indicated biased results proved impossible.This work suggests that comparing results from generalized linear mixed models estimated via PQL and QUAD is a worthwhile exercise for regression coefficients and variance components obtained via QUAD, in situations where PQL is known to give reasonable results.

Random effects selection in generalized linear mixed models via shrinkage penalty function

In this paper, we discuss the selection of random effects within the framework of generalized linear mixed models (GLMMs). Based on a reparametrization of the covariance matrix of random effects in terms of modified Cholesky decomposition, we propose to add a shrinkage penalty term to the penalized quasi-likelihood (PQL) function of the variance components for selecting effective random effects. The shrinkage penalty term is taken as a function of the variance of random effects, initiated by the fact that if the variance is zero then the corresponding variable is no longer random (with probability one). The proposed method takes the advantage of a convenient computation for the PQL estimation and appealing properties for certain shrinkage penalty functions such as LASSO and SCAD. We propose to use a backfitting algorithm to estimate the fixed effects and variance components in GLMMs, which also selects effective random effects simultaneously. Simulation studies show that the proposed approach performs quite well in selecting effective random effects in GLMMs. Real data analysis is made using the proposed approach, too.

Conditional Inference Functions for Mixed-Effects Models With Unspecified Random-Effects Distribution

In longitudinal studies, mixed-effects models are important for addressing subject-specific effects. However, most existing approaches assume a normal distribution for the random effects, and this could affect the bias and efficiency of the fixed-effects estimator. Even in cases where the estimation of the fixed effects is robust with a misspecified distribution of the random effects, the estimation of the random effects could be invalid. We propose a new approach to estimate fixed and random effects using conditional quadratic inference functions (QIFs). The new approach does not require the specification of likelihood functions or a normality assumption for random effects. It can also accommodate serial correlation between observations within the same cluster, in addition to mixed-effects modeling. Other advantages include not requiring the estimation of the unknown variance components associated with the random effects, or the nuisance parameters associated with the working correlations. We establish asymptotic results for the fixed-effect parameter estimators that do not rely on the consistency of the random-effect estimators. Real data examples and simulations are used to compare the new approach with the penalized quasi-likelihood (PQL) approach, and SAS GLIMMIX and nonlinear mixed-effects model (NLMIXED) procedures. Supplemental materials including technical details are available online.