Profile Likelihood Confidence Intervals Research Articles

Cox proportional hazards regression in small studies of predictive biomarkers

Predictive biomarkers are essential for personalized medicine since they select the best treatment for a specific patient. However, of all biomarkers that are evaluated, only few are eventually used in clinical practice. Many promising biomarkers may be erroneously abandoned because they are investigated in small studies using standard statistical techniques which can cause small sample bias or lack of power. The standard technique for failure time endpoints is Cox proportional hazards regression with a multiplicative interaction term between binary variables of biomarker and treatment. Properties of this model in small studies have not been evaluated so far, therefore we performed a simulation study to understand its small sample behavior. As a remedy, we applied a Firth correction to the score function of the Cox model and obtained confidence intervals (CI) using a profile likelihood (PL) approach. These methods are generally recommended for small studies of different design. Our results show that a Cox model estimates the biomarker-treatment interaction term and the treatment effect in one of the biomarker subgroups with bias, and overestimates their standard errors. Bias is however reduced and power is increased with Firth correction and PL CIs. Hence, the modified Cox model and PL CI should be used instead of a standard Cox model with Wald based CI in small studies of predictive biomarkers.

Estimating the size of a closed population by modeling latent and observed heterogeneity.

The paper extends the empirical likelihood (EL) approach of Liu etal. to a new and very flexible family of latent class models for capture-recapture data also allowing for serial dependence on previous capture history, conditionally on latent type and covariates. The EL approach allows to estimate the overall population size directly rather than by adding estimates conditional to covariate configurations. A Fisher-scoring algorithm for maximum likelihood estimation is proposed and a more efficient alternative to the traditional EL approach for estimating the non-parametric component is introduced; this allows us to show that the mapping between the non-parametric distribution of the covariates and the probabilities of being never captured is one-to-one and strictly increasing. Asymptotic results are outlined, and a procedure for constructing profile likelihood confidence intervals for the population size is presented. Two examples based on real data are used to illustrate the proposed approach and a simulation study indicates that, when estimating the overall undercount, the method proposed here is substantially more efficient than the one based on conditional maximum likelihood estimation, especially when the sample size is not sufficiently large.

Profile likelihood confidence interval for the prevalence assessed by an imperfect diagnostic test.

We present a new confidence interval for the prevalence of a disease for a situation when sensitivity and specificity of the diagnostic test are estimated from validation samples independent of the study sample. The new interval is based on profile likelihood and incorporates an adjustment improving the coverage probability. Its coverage probability and expected length were assessed by simulation and compared to two other methods for this problem, namely those by Lang and Reiczigel (2014) and Flor et al. (2020). Expected length of the new interval is less than that of the Lang and Reiczigel interval while its coverage is about the same. Comparison to the Flor interval resulted in similar expected length but higher coverage probabilities for the new interval. All in all, the new interval proved to be better than both its competitors.

Abundance of humpback whales on the Abrolhos Bank wintering ground, Brazil

Abundance estimates are presented for the humpback whales wintering off Brazil which ‘visit’ a surveyed area off Abrolhos Bank and display fluke-exposing behaviour. The study is based on photo-identification data collected between 1996 and 2000. Chapman-corrected Petersen estimates for all pairs of data result in estimates from 1,948 individuals up to 3,001 with coefficients of variation around 0.25. A more elaborate closed population multiple-recapture maximum-likelihood estimate is 2,393 with 95% profile-likelihood confidence interval (CI=1,924, 3,060). Replacing the closed-population assumption with a population allowed to grow (or decrease) according to some constant rate over the study period, the maximum-likelihood estimate of population size for the year 2000 becomes 3,871 (CI=2,795, 5,542) associated to an estimated annual growth rate of 31% over the study period and in the surveyed area. Although the inclusion of the growth rate results in a less restrictive assumption about population size, it is unclear at this time how to interpret it since the population around the Abrolhos Bank is some (unknown) fraction of whales wintering off Brazil. Alternatively, a fit of Whitehead’s model allowing for emigration and re-immigration gives an estimate of about 3,000 whales (CI=2,500, 3,650).

Dependence-Robust Confidence Intervals for Capture-Recapture Surveys.

Capture-recapture (CRC) surveys are used to estimate the size of a population whose members cannot be enumerated directly. CRC surveys have been used to estimate the number of Coronavirus Disease 2019 (COVID-19) infections, people who use drugs, sex workers, conflict casualties, and trafficking victims. When k-capture samples are obtained, counts of unit captures in subsets of samples are represented naturally by a contingency table in which one element-the number of individuals appearing in none of the samples-remains unobserved. In the absence of additional assumptions, the population size is not identifiable (i.e., point identified). Stringent assumptions about the dependence between samples are often used to achieve point identification. However, real-world CRC surveys often use convenience samples in which the assumed dependence cannot be guaranteed, and population size estimates under these assumptions may lack empirical credibility. In this work, we apply the theory of partial identification to show that weak assumptions or qualitative knowledge about the nature of dependence between samples can be used to characterize a nontrivial confidence set for the true population size. We construct confidence sets under bounds on pairwise capture probabilities using two methods: test inversion bootstrap confidence intervals and profile likelihood confidence intervals. Simulation results demonstrate well-calibrated confidence sets for each method. In an extensive real-world study, we apply the new methodology to the problem of using heterogeneous survey data to estimate the number of people who inject drugs in Brussels, Belgium.

Designing a small-sided game to elicit attacking tactical behaviour in professional rugby union forwards

ABSTRACT This study aimed to investigate the consistency of attacking tactical and technical behaviour, and physical characteristics, over multiple bouts, and variability across days, of a specific rugby union forwards small-sided game (SSG). Data was collected from 21 professional rugby union forwards during four training sessions. The SSG, consisting of five bouts of work (150-s) interspersed by passive recovery (75-s), aimed to elicit specific attacking tactical behaviour. Tactical behaviour (i.e., regularity of attacking shape [entropy]), and technical (e.g., passes) and physical (e.g., total distance) characteristics were quantified. Results showed that technical characteristics remained consistent, whereas the regularity of width of the attacking shape and two physical characteristics (i.e., total distance, training impulse) varied across bouts. However, these effects had limited practical significance. Technical characteristics were consistent across days, but minimal variability was observed for tactical behaviour and physical characteristics, as shown by their small random effects with 95% profile likelihood confidence intervals (PLCI) including zero (e.g., SD[95%PLCI] = 0.03[0.00, 0.06]). Consequently, consistency of stimulus over bouts and days is achievable for the majority of the variables investigated, thus supporting the use of SSG to elicit consistent attacking behaviour, but also technical and physical characteristics in rugby union forwards.

Statistical Inference for Odds Ratio of Two Proportions in Bilateral Correlated Data

Bilateral correlated data frequently arise in medical clinical studies such as otolaryngology and ophthalmology. Based on an equal correlation coefficient model, this paper mainly aimed to investigate the statistical inference for the odds ratio of two proportions in bilateral correlated data, including not only three test procedures but also four confidence interval (CI) constructions. Through iterative algorithms, all unknown parameters are estimated in order to construct the likelihood ratio, score and Wald-type tests. Furthermore, the profile likelihood CI, score CI, and Wald-type CI are obtained by the bisection root-finding algorithm. We provided another Wald-type CI based on an asymptotic normality property. The performance of the proposed tests were investigated with regard to empirical type I error rate and power, and CI methods were compared in terms of mean coverage probability and mean interval width. Numerical simulations show that the score test is more robust, and has higher power than other tests. The score CI also has a shorter interval width, and its coverage probability is closer to 0.95. A real example is used to illustrate the proposed methods.

High-risk phenotypes of genetic disease in a Neonatal Intensive Care Unit population.

High-risk phenotypes of genetic disease in a Neonatal Intensive Care Unit population.

Confidence intervals for assessing equivalence of two treatments with combined unilateral and bilateral data

Responses from the paired organs are generally highly correlated in bilateral studies, statistical procedures ignoring the correlation could lead to incorrect results. Note the intraclass correlation in the study of combined unilateral and bilateral outcomes; 11 confidence intervals (CIs) including 7 asymptotic CIs and 4 Bootstrap-resampling CIs for assessing the equivalence of 2 treatments are derived under Rosner's correlated binary data model. Performance is evaluated with respect to the empirical coverage probability (ECP), the empirical coverage width (ECW) and the ratio of the mesial non-coverage probability to the non-coverage probability (RMNCP) via simulation studies. Simulation results show that (i) all CIs except for the Wald CI and the bias-corrected Bootstrap percentile CI generally produce satisfactory ECPs and hence are recommended; (ii) all CIs except for the bias-corrected Bootstrap percentile CI provide preferred RMNCPs and are more symmetrical; (iii) as the measurement of the dependence increases, the ECWs of all CIs except for the score CI and the profile likelihood CI show increasing patterns that look like linear, while there is no obvious pattern on the ECPs of all CIs except for the profile likelihood CI. A data set from an otolaryngologic study is used to illustrate the proposed methods.

A robust and efficient algorithm to find profile likelihood confidence intervals

Profile likelihood confidence intervals are a robust alternative to Wald’s method if the asymptotic properties of the maximum likelihood estimator are not met. However, the constrained optimization problem defining profile likelihood confidence intervals can be difficult to solve in these situations, because the likelihood function may exhibit unfavorable properties. As a result, existing methods may be inefficient and yield misleading results. In this paper, we address this problem by computing profile likelihood confidence intervals via a trust-region approach, where steps computed based on local approximations are constrained to regions where these approximations are sufficiently precise. As our algorithm also accounts for numerical issues arising if the likelihood function is strongly non-linear or parameters are not estimable, the method is applicable in many scenarios where earlier approaches are shown to be unreliable. To demonstrate its potential in applications, we apply our algorithm to benchmark problems and compare it with 6 existing approaches to compute profile likelihood confidence intervals. Our algorithm consistently achieved higher success rates than any competitor while also being among the quickest methods. As our algorithm can be applied to compute both confidence intervals of parameters and model predictions, it is useful in a wide range of scenarios.

Computing uncertainty in the optimum nitrogen rate using a generalized cost function

A Python package, “EONR”, was developed for computing the economic optimum nitrogen rate (EONR) and its profile-likelihood confidence intervals (CIs) under economic conditions defined by the user. This work was motivated by the need to improve nitrogen fertilizer recommendations using the maximum return to nitrogen approach, specifically to make it easier for researchers and other practitioners to calculate uncertainty and consider externalities to the cost function while computing the EONR. The “EONR” package fits yield response data to a re-parameterized quadratic-plateau model, which is generally accepted as the most appropriate model for describing yield response to nitrogen in maize (the package also supports the quadratic model). Although grain price and fertilizer cost are typically the only economic factors producers consider for determining the EONR, this package allows the user to also consider variable costs and/or externalities. A general cost function may be desired if the user wishes to consider costs to the farm operation (e.g., equipment, technology, labor, etc.) or environmental costs/penalties that may result from excess fertilizer application (e.g., water treatment or health costs that result from pollution) in addition to the traditional fertilizer to grain price ratio. In addition to the development of the “EONR” Python package, the objectives of this work were to: (i) design an algorithm that utilizes a general cost function for computing the EONR and its profile-likelihood CIs for any crop and (ii) clearly document the methodology and algorithms used. The “EONR” Python package can be downloaded from the Python Package Index (https://pypi.org/), and installation instructions, tutorials, and supplementary background information can be found in the online documentation (https://eonr.readthedocs.io).

Efficient Likelihood Ratio Confidence Intervals using Constrained Optimization

Using constrained optimization, we develop a simple, efficient approach (applicable in both unconstrained and constrained maximum-likelihood estimation problems) to computing profile-likelihood confidence intervals. In contrast to Wald-type or score-based inference, the likelihood ratio confidence intervals use all the information encoded in the likelihood function concerning the parameters, which leads to improved statistical properties. In addition, the method does no suffer from the computational burdens inherent in the bootstrap. In an application to Rust's (1987) bus-engine replacement problem, our approach does better than either the Wald or the bootstrap methods, delivering very accurate estimates of the confidence intervals quickly and efficiently. An extensive Monte Carlo study reveals that in small samples, only likelihood ratio confidence intervals yield reasonable coverage properties, while at the same time discriminating implausible values.

Asymptotic confidence interval construction for proportion ratio based on correlated paired data

ABSTRACTIn ophthalmological and otolaryngology studies, measurements obtained from both organs (e.g., eyes or ears) of an individual are often highly correlated. Ignoring the intraclass correlation between paired measurements may yield biased inferences. In this article, four different confidence interval (CI) construction methods (maximum likelihood estimates based Wald-type CI, profile likelihood CI, asymptotic score CI and an existing method adjusted for correlated bilateral data) are applied to this type of correlated bilateral data to construct CI for proportion ratio, taking the intraclass correlation into consideration. The coverage probabilities and widths of the resulting CIs are compared with each other in a Monte Carlo simulation study to evaluate their performances. A real dataset from an ophthalmologic study is used to illustrate our methodology.

Confidence intervals for the cumulative incidence function via constrained NPMLE.

The cumulative incidence function (CIF) displays key information in the competing risks setting, which is common in medical research. In this article, we introduce two new methods to compute non-parametric confidence intervals for the CIF. First, we introduce non-parametric profile-likelihood confidence intervals. The method builds on constrained non-parametric maximum likelihood estimation (NPMLE), for which we derive closed-form formulas. This method can be seen as an extension of that of Thomas and Grunkemeier (J Am Stat Assoc 70:865-871, 1975) to the competing risks setting, when the CIF is of interest instead of the survival function. Second, we build on constrained NPMLE to introduce constrained bootstrap confidence intervals. This extends an interesting approach introduced by Barber and Jennison (Biometrics 52:430-436, 1999) to the competing risks setting. A simulation study illustrates how these methods can perform as compared to benchmarks implemented in popular software. The results suggest that more accurate confidence intervals than usual Wald-type ones can be obtained in the case of small to moderate sample sizes and few observed events. An application to melanoma data is provided for illustration purpose.

Cleft Palates and Occlusal Outcomes in Pierre Robin Sequence.

To assess the dental class occlusion and lateral cephalometry of children with conservatively treated Pierre Robin sequence (PRS) and to identify associations between these findings and prepalatoplasty cleft palate measurements. Retrospective cohort study. Among 22 patients with PRS, the following data were prospectively collected: demographics and preoperative cleft palate measurements. After patients reached age 6 years, an orthodontist assessed dental occlusion class and performed a lateral cephalometric analysis. PRS cephalometric data were compared with reference population values. Bivariate logistic regression was used to test the association with malocclusion class. Results are presented as odds ratios with 95% profile likelihood confidence intervals. The association between cleft measurements and cephalometric parameters was tested with Spearman's correlation ( rs). All 22 patients had bimaxillary hypoplasia and were prone to hyperdivergency, with a 41% rate of dental class III malocclusion. An increased anterior growth of the still retrusive mandible mostly accounts for the occurrence of the class III malocclusion in PRS (class II SNB = 74.3° vs class III SNB = 77.6°, P = .04). A larger cleft at the time of the cleft repair (mean, 11 months) was associated with increased mandibular retrusion (smaller SNB angle, rs = -0.5, P = .02). The 41% rate of class III malocclusion among these conservatively treated patients needs to be considered in the choice of the initial airway approach. The future impact of early mandibular advancement will have to be determined.

Confidence intervals for proportion ratios of stratified correlated bilateral data

ABSTRACTIn stratified bilateral studies, responses from two paired body parts are correlated. Confidence intervals (CIs), which reveal various features of the data, should take the correlations into account. In this article, five CI methods (sample-size weighted naïve Maximum likelihood estimation (MLE)-based Wald-type CI, complete MLE-based Wald-type CI, profile likelihood CI, MLE-based score CI and pooled MLE-based Wald-type CI) are derived for proportion ratios under the assumption of equal correlation coefficient within each stratum. Monte Carlo simulation shows that the complete MLE-based Wald-type CI approach generally produces the shortest mean interval width and satisfactory empirical coverage probability with close form solution; while the profile likelihood CI and the MLE-based score CI provide preferred ratio of non coverage probability and are more symmetric. Two real examples are used to demonstrate the performance of the proposed methods.

Assessing models for estimation and methods for uncertainty quantification for spatial return levels

AbstractThe return level estimation is an essential topic in studying spatial extremes for environmental data. Recently, various models for spatial extremes have emerged, which generally yield different estimates for return levels, given the same data. In the meantime, several approaches that obtain confidence intervals (CIs) for return levels have arisen, and the results from different approaches can also largely disagree. These pose natural questions for assessing different return level estimation methods and different CI derivation approaches. In this article, we compare an array of popular models for spatial extremes in return level estimation, as well as three approaches in CI derivation, through extensive Monte Carlo simulations. Our results show that in general, max‐stable models yield return level estimates with similar mean squared error, and the spatial generalized extreme value model also provides comparable estimates. The bootstrap method is recommended for max‐stable models to compute the CI, and the profile likelihood CI works well for spatial generalized extreme value. We also evaluate the methods for return level interpolation at unknown spatial locations and find that kriging of marginal return level estimates performs as well as max‐stable models.

Confidence Intervals of Fit Indexes by Inverting a Bootstrap Test

Fit indexes are an important tool in the evaluation of model fit in structural equation modeling (SEM). Currently, the newest confidence interval (CI) for fit indexes proposed by Zhang and Savalei (2016) is based on the quantiles of a bootstrap sampling distribution at a single level of misspecification. This method, despite a great improvement over naive and model-based bootstrap methods, still suffers from unsatisfactory coverage. In this work, we propose a new method of constructing bootstrap CIs for various fit indexes. This method directly inverts a bootstrap test and produces a CI that involves levels of misspecification that would not be rejected in a bootstrap test. Similar in rationale to a parametric CI of root mean square error of approximation (RMSEA) based on a noncentral χ2 distribution and a profile-likelihood CI of model parameters, this approach is shown to have better performance than the approach of Zhang and Savalei (2016), with more accurate coverage and more efficient widths.

Profile-likelihood Confidence Intervals in Item Response Theory Models

ABSTRACTConfidence intervals (CIs) are fundamental inferential devices which quantify the sampling variability of parameter estimates. In item response theory, CIs have been primarily obtained from large-sample Wald-type approaches based on standard error estimates, derived from the observed or expected information matrix, after parameters have been estimated via maximum likelihood. An alternative approach to constructing CIs is to quantify sampling variability directly from the likelihood function with a technique known as profile-likelihood confidence intervals (PL CIs). In this article, we introduce PL CIs for item response theory models, compare PL CIs to classical large-sample Wald-type CIs, and demonstrate important distinctions among these CIs. CIs are then constructed for parameters directly estimated in the specified model and for transformed parameters which are often obtained post-estimation. Monte Carlo simulation results suggest that PL CIs perform consistently better than Wald-type CIs for both non-transformed and transformed parameters.

Evaluating Testing, Profile Likelihood Confidence Interval Estimation, and Model Comparisons for Item Covariate Effects in Linear Logistic Test Models.

The linear logistic test model (LLTM) has been widely applied to investigate the effects of item covariates on item difficulty. The LLTM was extended with random item residuals to account for item differences not explained by the item covariates. This extended LLTM is called the LLTM-R. In this article, statistical inference methods are investigated for these two models. Type I error rates and power are compared via Monte Carlo studies. Based on the simulation results, the use of the likelihood ratio test (LRT) is recommended over the paired-sample t test based on sum scores, the Wald z test, and information criteria, and the LRT is recommended over the profile likelihood confidence interval because of the simplicity of the LRT. In addition, it is concluded that the LLTM-R is the better general model approach. Inferences based on the LLTM while the LLTM-R is the true model appear to be largely biased in the liberal way, while inferences based on the LLTM-R while the LLTM is the true model are only biased in a very minor and conservative way. Furthermore, in the absence of residual variance, Type I error rate and power were acceptable except for power when the number of items is small (10 items) and also the number of persons is small (200 persons). In the presence of residual variance, however, the number of items needs to be large (80 items) to avoid an inflated Type I error and to reach a power level of .90 for a moderate effect.