Probabilistic Indexing Model Research Articles

Covariate-adjusted generalized pairwise comparisons in small samples.

Semiparametric probabilistic index models allow for the comparison of two groups of observations, whilst adjusting for covariates, thereby fitting nicely within the framework of generalized pairwise comparisons (GPC). As with most regression approaches in this setting, the limited amount of data results in invalid inference as the asymptotic normality assumption is not met. In addition, separation issues might arise when considering small samples. In this article, we show that the parameters of the probabilistic index model can be estimated using generalized estimating equations, for which adjustments exist that lead to estimators of the sandwich variance-covariance matrix with improved finite sample properties and that can deal with bias due to separation. In this way, appropriate inference can be performed as is shown through extensive simulation studies. The known relationships between the probabilistic index and other GPC statistics allow to also provide valid inference for example, the net treatment benefit or the success odds.

Simple and flexible sign and rank-based methods for testing for differential abundance in microbiome studies.

Microbiome data obtained with amplicon sequencing are considered as compositional data. It has been argued that these data can be analysed after appropriate transformation to log-ratios, but ratios and logarithms cause problems with the many zeroes in typical microbiome experiments. We demonstrate that some well chosen sign and rank transformations also allow for valid inference with compositional data, and we show how logistic regression and probabilistic index models can be used for testing for differential abundance, while inheriting the flexibility of a statistical modelling framework. The results of a simulation study demonstrate that the new methods perform better than most other methods, and that it is comparable with ANCOM-BC. These methods are implemented in an R-package 'signtrans' and can be installed from Github (https://github.com/lucp9827/signtrans).

Toxicity Index, patient-reported outcomes, and persistence of breast cancer chemotherapy-associated side effects in NRG Oncology/NSABP B-30

Adjuvant chemotherapy improves breast cancer survival but is associated with bothersome short- and long-term toxicity. Factors associated with toxicity, especially subacute toxicity up to 2 years following chemotherapy, have not been fully elucidated. The NRG Oncology/NSABP B-30 clinical trial compared 3 different doxorubicin-, cyclophosphamide-, and docetaxel-based chemotherapy regimens given over 3–6 months. Patients with hormone receptor-positive breast cancer received subsequent adjuvant endocrine therapy. From baseline through 24 months, 2156 patients completed questionnaires serially. We used multivariable probabilistic index models to identify factors associated with acute (>0–12 months) and subacute (>12–24 months) difficulties with pain, cognition, vasomotor symptoms, and vaginal symptoms. For all symptom domains, presence of symptoms prior to chemotherapy initiation were associated with symptoms in the subacute period (all p < 0.001). In addition, different combinations of patient factors and breast cancer treatments were associated with increased likelihood of pain, vasomotor, and vaginal symptoms in the subacute period. Consideration of pre-treatment symptoms and patient factors, as well as treatments for breast cancer, can facilitate identification of groups of patients that may experience symptoms following completion of chemotherapy. This information may be important for treatment-decision-making when alternative regimens are equivalent in benefit.

Simulation study comparing analytical methods for single-item longitudinal patient-reported outcomes data

PurposeEfficient analytical methods are necessary to make reproducible inferences on single-item longitudinal ordinal patient-reported outcome (PRO) data. A thorough simulation study was performed to compare the performance of the semiparametric probabilistic index models (PIM) with a longitudinal analysis using parametric cumulative logit mixed models (CLMM).MethodsIn the setting of a control and intervention arm, we compared the power of the PIM and CLMM to detect differences in PRO adverse event (AE) between these groups using several existing and novel summary scores of PROs. For each scenario, PRO data were simulated using copula multinomial models. Comparisons were also exemplified using clinical trial data.ResultsOn average, CLMM provided substantially greater power than the PIM to detect differences in PRO-AEs between the groups when the baseline-adjusted method was used, and a small advantage in power when using the baseline symptom as a covariate.ConclusionAlthough the CLMM showed the best performance among analytical methods, it relies on assumptions difficult to verify and that might not be fulfilled in the real world, therefore our recommendation is the use of PIM models with baseline symptom as a covariate.

The win odds: statistical inference and regression

ABSTRACT Generalized pairwise comparisons and win statistics (i.e., win ratio, win odds and net benefit) are advantageous in analyzing and interpreting a composite of multiple outcomes in clinical trials. An important limitation of these statistics is their inability to adjust for covariates other than by stratified analysis. Because the win ratio does not account for ties, the win odds, a modification that includes ties, has attracted attention. We review and combine information on the win odds to articulate the statistical inferences for the win odds. We also show alternative variance estimators based on the exact permutation and bootstrap as well as statistical inference via the probabilistic index. Finally, we extend multiple-covariate regression probabilistic index models to the win odds with a univariate outcome. As an illustration we apply the regression models to the data in the CHARM trial.

Evaluation of burnout in physician members of the American Brachytherapy Society

PURPOSETo evaluate the prevalence of burnout among brachytherapy specialists and to identify factors associated with burnout. METHODS AND MATERIALSAn anonymous, online, cross-sectional survey was administered to non-trainee physician members of the American Brachytherapy Society. Burnout was evaluated using the validated Maslach Burnout Inventory-Human Services Survey (MBI-HSS). Demographic and practice-specific questions were collected from respondents. Univariate and multivariable analysis of outcomes were performed using probabilistic index models. RESULTSOverall, 51 of 400 physicians responded (13% response rate). Fifty-seven percent of respondents demonstrated at least one symptom of professional burnout. However, only 6% of respondents met strict criteria for high burnout. Analysis of the individual MBI-HSS subdomains demonstrated higher subscale scores for emotional exhaustion and depersonalization, but also higher scores for personal accomplishment. On multivariable analysis after adjusting for increased feelings of burnout due to the COVID-19 pandemic or total hours of work per week, younger age was associated with both increased subscale scores for emotional exhaustion (p = 0.026) and lower personal accomplishment (p = 0.010). Lastly, nearly half of all respondents (47%) reported increased feelings of burnout due to the COVID-19 pandemic. Respondents from academic facilities were significantly more likely to report increased burnout due to COVID-19 compared to those from non-academic facilities (odds ratio, 7.04; 95% CI 1.60–31.0; p = 0.010). CONCLUSIONSNearly 60% of brachytherapists demonstrated symptoms of professional burnout, which is higher than other radiation oncology groups (academic chairs, program directors, residents). Managing stressors related to workload, COVID and support for junior physicians are potential areas for improving feelings of burnout.

Abstract PD5-04: Risk factors for long-term adjuvant chemotherapy toxicity using pre-treatment host factors and self-rated treatment bother (GP5) in a clinical trial population

Abstract Background: Chemotherapy-induced toxicities are prevalent in patients treated with adjuvant chemotherapy and often impact on activities of daily living and other aspects of function and well-being. Understanding which patients are more likely to experience long-term chemotherapy-induced peripheral neuropathy (CIPN) and hot flashes may help tailor treatment recommendations. Methods: Patients enrolled in the NRG Oncology/NSABP B-30 clinical trial were randomized to doxorubicin and docetaxel (AT), doxorubicin, docetaxel, and cyclophosphamide (ATC), or doxorubicin and cyclophosphamide followed by docetaxel (AC-T) (with increasing docetaxel doses across regimens). 1840 patients completed patient-reported outcomes (PRO) questionnaires including a symptom checklist asking about bother due to “numbness or tingling in hands or feet” and “hot flashes” as well as the FACT-General question 5 (GP5; “I am bothered by side effects of treatment”), assessed using a 5 category response scale. We examined the impact of pre-treatment host factors (demographic and clinical) on development and persistence of CIPN and hot flashes in NSABP B-30 patients. Analyses were performed using the Toxicity Index (TI), which summarizes the severity and frequency of toxicities over time. Univariate and multivariable probabilistic index models, a rank-based method that is a more general version of the Kruskal-Wallis test, for the cumulative TI of toxicity in the acute period (cycle 4 day 1 through 12 months post-randomization) and long-term follow-up (18 and 24 months post-randomization) were performed, and included baseline demographic, clinical, and PRO data. Results: Multivariable analysis revealed a higher probability of CIPN in the acute period for patients randomized to the AC-T treatment regimen and those with pre-existing numbness, higher GP5 at baseline, Black race, and obesity. In the long-term follow-up period, all the above factors except race were statistically significant. Older age at surgery and mastectomy were also associated with persistence of CIPN. GP5 at 12 months was significantly associated with the probability of CIPN in the longer-term period (0.577, p&amp;lt;.001). Multivariable analysis revealed a higher probability of hot flashes in the acute period for patients treated randomized to the ATC treatment regimen and those with pre-existing hot flashes, greater GP5 at baseline, younger age, and obesity. In the longer-term period, all the above factors except baseline GP5 and obesity remained statistically significant. Endocrine therapy was also significantly associated with long-term hot flashes. GP5 at 12 months was statistically significantly associated with the probability of hot flashes in the longer-term period (0.561, p&amp;lt;.001). Conclusions: Most pre-treatment patient characteristics that influence early development of CIPN and hot flashes are also associated with persistence of toxicity beyond 1 year. Persistent bother from side effects of treatment at 12 months (GP5) predicts continued toxicity during the subsequent year. Support: U10CA180868; U10CA180822. Multivariable probabilistic index models for cumulative TIsVariableComparison A&amp;lt;BProbability CIPN - AcuteP valueProbability CIPN - Long-termP valueProbability Hot Flashes - AcuteP valueProbability Hot Flashes - Long-termP valueTreatmentAT &amp;lt; ATC0.537&amp;lt;.0010.519.2450.567&amp;lt;.0010.577&amp;lt;.001AT &amp;lt; AC-T0.688&amp;lt;.0010.591&amp;lt;.0010.502.9050.542.058ATC &amp;lt; AC-T0.655&amp;lt;.0010.573&amp;lt;.0010.435&amp;lt;.0010.465.167Numbness at baselinePer 1-unit increment0.597&amp;lt;.0010.595&amp;lt;.001NANANANAHot flashes at baselinePer 1-unit incrementNANANANA0.601&amp;lt;.0010.592&amp;lt;.001GP5 at baselinePer 1-unit increment0.516.0330.523.0040.521.0040.509.289Age at surgeryPer 1-year increment0.501.0880.503.0010.492&amp;lt;.0010.498.016BMIObesity &amp;lt; Normal0.453.0010.411&amp;lt;.0010.516.3760.510.582Obesity &amp;lt; Overweight0.475.1470.470.0870.536.0420.532.087RaceWhite &amp;lt; Black0.587.0010.523.3630.499.9710.518.547Breast surgeryLumpectomy &amp;lt; Mastectomy0.515.2500.548&amp;lt;.0010.503.8290.507.629Endocrine therapyNo &amp;lt; YesNANA0.490.576NANA0.655&amp;lt;.001 Citation Format: N. Lynn Henry, Sungjin Kim, Ron D. Hays, Marcio A. Diniz, Michael Luu, Mourad Tighiouart, Reena S. Cecchini, Greg Yothers, Andre Rogatko, Patricia A. Ganz. Risk factors for long-term adjuvant chemotherapy toxicity using pre-treatment host factors and self-rated treatment bother (GP5) in a clinical trial population [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr PD5-04.

Evaluating Treatment Tolerability in Cancer Clinical Trials Using the Toxicity Index.

BackgroundThe National Cancer Institute Moonshot research initiative calls for improvements in the analysis and reporting of treatment toxicity to advise key stakeholders on treatment tolerability and inform regulatory and clinical decision making. This study illustrates alternative approaches to toxicity evaluation using the National Surgical Adjuvant Breast and Bowel Project R-04 clinical trial as an example.MethodsNational Surgical Adjuvant Breast and Bowel Project R-04 was a neoadjuvant chemoradiation trial in stage II–III rectal cancer patients. A 2 x 2 factorial design was used to evaluate whether the addition of oxaliplatin (Oxa) to 5-fluorouracil (5FU) or capecitabine (Cape) with radiation therapy improved local-regional tumor control. The toxicity index (TI), which accounts for the frequency and severity of toxicities, was compared across treatments using multivariable probabilistic index models, where Pr A < B indicates the probability that higher values of TI were observed for A when compared with B. Baseline age, sex, performance status, body mass index, surgery type, and stage were evaluated as independent risk factors.ResultsA total of 4560 toxicities from 1558 patients were analyzed. Results from adjusted probabilistic index models indicate that oxaliplatin-containing regimens had statistically significant (P < .001) probability (Pr) for higher TI compared with regimens without oxaliplatin (Pr 5FU < 5FU + Oxa = 0.619, 95% confidence interval [CI] = 0.560 to 0.674; Pr 5FU < Cape + Oxa = 0.627, 95% CI = 0.568 to 0.682; Pr Cape < 5FU + Oxa = 0.587, 95% 0.527 to 0.644; and Pr Cape < Cape + Oxa = 0.596, 95% 0.536 to 0.653). When compared with other existing toxicity analysis methods, TI provided greater power to detect differences between treatments.ConclusionsThis article uses standard data collected in a cancer clinical trial to introduce descriptive and analytic methods that account for the additional burden of multiple toxicities. These methods may provide a more accurate description of a patient’s treatment experience that could lead to individualized dosing for better toxicity control. Future research will evaluate the generalizability of these findings in trials with similar drugs.

A tutorial on probabilistic index models: Regression models for the effect size P(Y1 < Y2).

The probabilistic index (PI), also known as the probability of superiority or the common language effect size, refers to the probability that the outcome of a randomly selected subject exceeds the outcome of another randomly selected subject, conditional on the covariate values of both subjects. This summary measure has a long history, especially for the 2-sample design where the covariate value typically refers to 1 of 2 treatments. Despite some of the attractive features of the PI, it is often not used beyond the 2-sample design. One reason is the lack of a flexible regression framework that embeds the PI and that allows the user to estimate it for more complicated designs. However, Thas, De Neve, Clement, and Ottoy (2012) recently developed such a regression framework, named probabilistic index models (PIMs). In this tutorial we provide an introduction to PIMs where we discuss several theoretical properties, motivate why we think PIMs could be useful for behavioral sciences, and illustrate how it can be used in practice using the R package pim. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Small sample inference for probabilistic index models

Probabilistic index models may be used to generate classical and new rank tests, with the additional advantage of supplementing them with interpretable effect size measures. The popularity of rank tests for small sample inference makes probabilistic index models also natural candidates for small sample studies. However, at present, inference for such models relies on asymptotic theory that can deliver poor approximations of the sampling distribution if the sample size is rather small. A bias-reduced version of the bootstrap and adjusted jackknife empirical likelihood are explored. It is shown that their application leads to drastic improvements in small sample inference for probabilistic index models, justifying the use of such models for reliable and informative statistical inference in small sample studies.

Increasing the power of the Mann-Whitney test in randomized experiments through flexible covariate adjustment.

The Mann-Whitney U test is frequently used to evaluate treatment effects in randomized experiments with skewed outcome distributions or small sample sizes. It may lack power, however, because it ignores the auxiliary baseline covariate information that is routinely collected. Wald and score tests in so-called probabilistic index models generalize the Mann-Whitney U test to enable adjustment for covariates, but these may lack robustness by demanding correct model specification and do not lend themselves to small sample inference. Using semiparametric efficiency theory, we here propose an alternative extension of the Mann-Whitney U test, which increases its power by exploiting covariate information in an objective way and which lends itself to permutation inference. Simulation studies and an application to an HIV clinical trial show that the proposed permutation test attains the nominal Type I error rate and can be drastically more powerful than the classical Mann-Whitney U test.

Goodness-of-Fit Methods for Probabilistic Index Models

A class of semiparametric regression models, called probabilistic index models, has been recently proposed. Because these models are semiparametric, inference is only valid when the proposed model is consistent with the underlying data-generating model. However, no formal goodness-of-fit methods for these probabilistic index models exist yet. We propose a test and a graphical tool for assessing the model adequacy. Simulation results indicate that both methods succeed in detecting lack-of-fit. The methods are also illustrated on a case study.

A probabilistic method for computing term-by-term relationships

This article suggests a probabilistic method to compute the term relationships from relevance information, which complements the studies on a non-probabilistic technique called pseudo-classification. A quadratic ranking function (i.e., a bilinear function) on the components of document and query vectors is derived by incorporating the term-by-term relationships. The conventional probabilistic indexing model, the probabilistic retrieval model, and our earlier generalized model are special cases of the proposed model. By exploring the different views of probability, procedures for estimating the required parameters are provided. © 1993 John Wiley & Sons, Inc.

Dynamically Updating Relevance Judgements in Probabilistic Information Systems Via Users' Feedback

A decision maker's performance relies on the availability of relevant information. In many environments, the relation between the decision maker's informational needs and the information base is complex and uncertain. A fundamental concept of information systems, such as decision support and document retrieval, is the probability that the retrieved information is useful to the decision maker's query. This paper presents a sequential, Bayesian, probabilistic indexing model that explicitly combines expert opinion with data about the system's performance. The expert opinion is encoded into probability statements. These statements are modified by the users' feedback about the relevance of the retrieved information to their queries. The predictive probability that a datum in the information base is applicable to the current query is a logistic function of the expert opinion and the feedback. This feedback enters the computation through a measure of association between the current query-datum pair with previous, relevant query-datum pairs. When this measure is based on the proportional matching of multiple attributes, the predictive probabilities have a recursive formula that makes the model computationally feasible for large information bases.