Greater Net Benefit Research Articles

Comparing the Diagnostic Accuracy of Clinician Judgment to a Novel Host Response Diagnostic for Acute Respiratory Illness.

Difficulty discriminating bacterial from viral infections drives antibacterial misuse. Host gene expression tests discriminate bacterial and viral etiologies, but their clinical utility has not been evaluated. Host gene expression and procalcitonin levels were measured in 582 emergency department participants with suspected infection. We also recorded clinician diagnosis and clinician-recommended treatment. These 4 diagnostic strategies were compared with clinical adjudication as the reference. To estimate the clinical impact of host gene expression, we calculated the change in overall Net Benefit (∆NB; the difference in Net Benefit comparing 1 diagnostic strategy with a reference) across a range of prevalence estimates while factoring in the clinical significance of false-positive and -negative errors. Gene expression correctly classified bacterial, viral, or noninfectious illness in 74.1% of subjects, similar to the other strategies. Clinical diagnosis and clinician-recommended treatment revealed a bias toward overdiagnosis of bacterial infection resulting in high sensitivity (92.6% and 94.5%, respectively) but poor specificity (67.2% and 58.8%, respectively), resulting in a 33.3% rate of inappropriate antibacterial use. Gene expression offered a more balanced sensitivity (79.0%) and specificity (80.7%), which corresponded to a statistically significant improvement in average weighted accuracy (79.9% vs 71.5% for procalcitonin and 76.3% for clinician-recommended treatment; P<.0001 for both). Consequently, host gene expression had greater Net Benefit in diagnosing bacterial infection than clinician-recommended treatment (∆NB=6.4%) and procalcitonin (∆NB=17.4%). Host gene expression-based tests to distinguish bacterial and viral infection can facilitate appropriate treatment, improving patient outcomes and mitigating the antibacterial resistance crisis.

Development and validation of a multiparametric MRI-based radiomics signature for distinguishing between indolent and aggressive prostate cancer.

To develop and validate a non-invasive MRI-based radiomics signature for distinguishing between indolent and aggressive prostate cancer (PCa) prior to therapy. In all, 139 qualified and pathology-confirmed PCa patients were divided into a training set (n = 93) and a validation set (n = 46). A total of 1576 radiomics features were extracted from the T2WI (n = 788) and diffusion-weighted imaging (n = 788) for each patient. The Select K Best and the least absolute shrinkage and selection operator regression algorithm were used to construct a radiomics signature in the training set. The predictive performance of the radiomics signature was assessed in the training set and then validated in the validation set by receiver operating characteristic curve analysis. We computed the calibration curve and the decision curve to evaluate the calibration and clinical usefulness of the signature. Nine radiomics features were identified to form the radiomics signature. The radiomics score (Rad-score) was significantly different between indolent and aggressive PCa (p < 0.001). The radiomics signature exhibited favorable discrimination between the indolent and aggressive PCa groups in the training set (AUC: 0.853, 95% CI: 0.766 to 0.941) and validation set (AUC: 0.901, 95% CI: 0.793 to 1.000). The decision curve analysis showed that a greater net benefit would be obtained when the threshold probability ranged from 20 to 90%. The multiparametric MRI-based radiomics signature can potentially serve as a non-invasive tool for distinguishing between indolent and aggressive PCa prior to therapy. The multiparametric MRI-based radiomics signature has the potential to non-invasively distinguish between the indolent and aggressive PCa, which might aid clinicians in making personalized therapeutic decisions.

The data-collection on adverse effects of anti-HIV drugs (D:A:D) model for predicting cardiovascular events: External validation in a diverse cohort of people living with HIV.

ObjectivesLittle is known about the external validity of the Data‐collection on Adverse Effects of Anti‐HIV Drugs (D:A:D) model for predicting cardiovascular disease (CVD) risk among people living with HIV (PLWH). We aimed to evaluate the performance of the updated D:A:D model for 5‐year CVD risk in a diverse group of PLWH engaged in HIV care.MethodsWe used data from an institutional HIV registry, which includes PLWH engaged in care at a safety‐net HIV clinic. Eligible individuals had a baseline clinical encounter between 1 January 2013 and 31 December 2014, with follow‐up through to 31 December 2019. We estimated 5‐year predicted risks of CVD as a function of the prognostic index and baseline survival of the D:A:D model, which were used to assess model discrimination (C‐index), calibration and net benefit.ResultsOur evaluable population comprised 1029 PLWH, of whom 30% were female, 50% were non‐Hispanic black, and median age was 45 years. The C‐index was 0.70 [95% confidence limits (CL): 0.64–0.75]. The predicted 5‐year CVD risk was 3.0% and the observed 5‐year risk was 8.9% (expected/observed ratio = 0.33, 95% CL: 0.26–0.54). The model had a greater net benefit than treating all or treating none at a risk threshold of 10%.ConclusionsThe D:A:D model was miscalibrated for CVD risk among PLWH engaged in HIV care at an urban safety‐net HIV clinic, which may be related to differences in case‐mix and baseline CVD risk. Nevertheless, the HIV D:A:D model may be useful for decisions about CVD intervention for high‐risk patients.

Spatial cost-benefit analysis of blue restoration and factors driving net benefits globally.

Marine coastal ecosystems, commonly referred to as blue ecosystems, provide valuable services to society but are under increasing threat worldwide due to a variety of drivers, including eutrophication, development, land-use change, land reclamation, and climate change. Ecological restoration is sometimes necessary to facilitate recovery in coastal ecosystems. Blue restoration (i.e., in marine coastal systems) is a developing field, and projects to date have been small scale and expensive, leading to the perception that restoration may not be economically viable. We conducted a global cost-benefit analysis to determine the net benefits of restoring coral reef, mangrove, saltmarsh, and seagrass ecosystems, where the benefit is defined as the monetary value of ecosystem services. We estimated costs from published restoration case studies and used an adjusted-value-transfer method to assign benefit values to these case studies. Benefit values were estimated as the monetary value provided by ecosystem services of the restored habitats. Benefits outweighed costs (i.e., there were positive net benefits) for restoration of all blue ecosystems. Mean benefit:cost ratios for ecosystem restoration were eight to 10 times higher than prior studies of coral reef and seagrass restoration, most likely due to the more recent lower cost estimates we used. Among ecosystems, saltmarsh had the greatest net benefits followed by mangrove; coral reef and seagrass ecosystems had lower net benefits. In general, restoration in nations with middle incomes had higher (eight times higher in coral reefs and 40 times higher in mangroves) net benefits than those with high incomes. Within an ecosystem type, net benefit varied with restoration technique (coral reef and saltmarsh), ecosystem service produced (mangrove and saltmarsh), and project duration (seagrass). These results challenge the perceptions of the low economic viability of blue restoration and should encourage further targeted investment in this field.

MO353MACHINE LEARNING-BASED PREDICTION OF ACUTE KIDNEY INJURY AFTER NEPHRECTOMY IN PATIENTS WITH RENAL CELL CARCINOMA

Abstract Background and Aims The precise prediction of acute kidney injury (AKI) after nephrectomy for renal cell carcinoma (RCC) is an important issue because of its relationship with subsequent kidney dysfunction and high mortality. Herein we addressed whether machine learning algorithms could predict postoperative AKI risk better than conventional logistic regression (LR) models. Method A total of 4,104 RCC patients who had undergone unilateral nephrectomy from January 2003 to December 2017 were reviewed. Machine learning models such as support vector machine, random forest, extreme gradient boosting, and light gradient boosting machine (LightGBM) were developed, and their performance based on the area under the receiver operating characteristic curve, accuracy, and F1 score was compared with that of the LR-based scoring model. Results Postoperative AKI developed in 1,167 patients (28.4%). All the machine learning models had higher performance index values than the LR-based scoring model. Among them, the LightGBM model had the highest value of 0.810 (0.783–0.837). The decision curve analysis demonstrated a greater net benefit of the machine learning models than the LR-based scoring model over all the ranges of threshold probabilities. The LightGBM and random forest models, but not others, were well calibrated. Conclusion The application of machine learning algorithms improves the predictability of AKI after nephrectomy for RCC, and these models perform better than conventional LR-based models.

Improving the stratification of intermediate risk prostate cancer.

Intermediate risk prostate cancer (IR PCa) may exhibit a wide array of phenotypes, from favorable to unfavorable. NCCN criteria help distinguishing between favorable versus unfavorable subgroups. We studied and attempted to improve this classification. Within the SEER database 2010-2016, we identified 19,193 IR PCa patients treated with radical prostatectomy. A multivariable logistic regression model predicting unfavorable IR PCa was developed and externally validated, in addition to a head-to-head comparison with NCCN IR PCa stratification. Model development (development cohort N.=13,436: 3585 unfavorable versus 9851 favorable) rested on age, PSA, clinical T stage, biopsy Gleason Grade Group (GGG) and percentage of positive cores. All were independent predictors of unfavorable IR PCa. In external validation cohort (N.=5757: 1652 unfavorable versus 4105 favorable), NCCN stratification was 61.8% accurate in discriminating between favorable versus unfavorable, compared to 67.6% for nomogram, which exhibited excellent calibration, less pronounced departures from ideal prediction and greater net-benefit in decision curve analyses (DCA) than NCCN stratification. The optimal nomogram cutoff misclassified 312 of 1976 patients (15.8%) versus 598 of 2877 (20.8%) for NCCN stratification. Of NCCN misclassified patients, 90.0% harbored pT3-4 stages versus 84.6% of nomogram. The newly developed, externally validated nomogram discriminates better between favorable versus unfavorable IR PCa, according to overall accuracy, calibration, DCA, and actual numbers and stage distribution of misclassified patients.

Prospective Validation and Comparative Analysis of Coronary Risk Stratification Strategies Among Emergency Department Patients With Chest Pain.

BackgroundCoronary risk stratification is recommended for emergency department patients with chest pain. Many protocols are designed as “rule‐out” binary classification strategies, while others use graded‐risk stratification. The comparative performance of competing approaches at varying levels of risk tolerance has not been widely reported.Methods and ResultsThis is a prospective cohort study of adult patients with chest pain presenting between January 2018 and December 2019 to 13 medical center emergency departments within an integrated healthcare delivery system. Using an electronic clinical decision support interface, we externally validated and assessed the net benefit (at varying risk thresholds) of several coronary risk scores (History, ECG, Age, Risk Factors, and Troponin [HEART] score, HEART pathway, Emergency Department Assessment of Chest Pain Score Accelerated Diagnostic Protocol), troponin‐only strategies (fourth‐generation assay), unstructured physician gestalt, and a novel risk algorithm (RISTRA‐ACS). The primary outcome was 60‐day major adverse cardiac event defined as myocardial infarction, cardiac arrest, cardiogenic shock, coronary revascularization, or all‐cause mortality. There were 13 192 patient encounters included with a 60‐day major adverse cardiac event incidence of 3.7%. RISTRA‐ACS and HEART pathway had the lowest negative likelihood ratios (0.06, 95% CI, 0.03–0.10 and 0.07, 95% CI, 0.04–0.11, respectively) and the greatest net benefit across a range of low‐risk thresholds. RISTRA‐ACS demonstrated the highest discrimination for 60‐day major adverse cardiac event (area under the receiver operating characteristic curve 0.92, 95% CI, 0.91–0.94, P<0.0001).ConclusionsRISTRA‐ACS and HEART pathway were the optimal rule‐out approaches, while RISTRA‐ACS was the best‐performing graded‐risk approach. RISTRA‐ACS offers promise as a versatile single approach to emergency department coronary risk stratification.RegistrationURL: https://www.clinicaltrials.gov; Unique identifier: NCT03286179.

External validation and comparison of magnetic resonance imaging-based predictive models for clinically significant prostate cancer

PurposeSeveral multiparametric magnetic resonance imaging (mpMRI)-based models have been developed with significant improvements in diagnostic accuracy for clinically significant prostate cancer (csCaP), but lack proper external validation. We therefore sought to externally validate and compare all published mpMRI-based csCaP risk prediction models in an independent Asian population. Patients and MethodsA total of 449 men undergoing combined transperineal fusion-targeted/systematic prostate biopsy at our specialist center between 2015 to 2019 were retrospectively analyzed. csCaP was defined as lesions with ISUP (International Society of Urological Pathology) grade group ≥2. The performance of 6 mpMRI-based risk models (MRI-ERSPC-3/4, Distler, Radtke, Mehralivand, van Leeuwen and He) were evaluated in terms of discrimination, calibration and clinical utility, using area under the receiver operating characteristic curve (AUC), calibration curves and decision curve analyses. ResultsA total of 202 (45%) subjects were diagnosed with csCaP. All models demonstrated excellent accuracy with AUCs ranging from 0.75 to 0.86, and most significantly outperformed mpMRI PIRADSv2.0 (Prostate Imaging Reporting and Data System version 2.0) alone. The models by Mehralivand and He showed good calibration to our validation population, with respective intercepts of -0.08 and -0.84. All models were nevertheless recalibrated to the csCaP prevalence in our population for analysis. Decision curve analysis showed that above a threshold probability of 10%, all mpMRI-based models demonstrated superior net benefit compared to mpMRI PIRADSv2.0 or a biopsy-all-men strategy. The van Leeuwen model had the greatest net benefit, avoiding 39% of unnecessary biopsies while missing only 4% of csCaP, at a threshold probability of 15%. ConclusionsThe mpMRI-based risk models demonstrate excellent discrimination and clinical utility and are easy to apply in practice, suggesting that individualized risk-based approaches can be considered over mpMRI alone to avoid unnecessary biopsies.

Cost-effectiveness of Diagnostic Tools to Establish the Presence of Peripheral Arterial Disease in People With Diabetes.

To evaluate the cost-effectiveness of point-of-care duplex ultrasound (PAD-scan) and other bedside tests for the diagnosis of peripheral arterial disease (PAD) in people with diabetes. PAD is a risk factor for cardiovascular disease, diabetic foot ulceration (DFU), and amputation in diabetic patients. Diagnosis enables optimization of therapies to manage these risks. PAD-scan can be performed by frontline staff and has been shown to be the most accurate bedside test. However, its cost-effectiveness has not been investigated. A Markov model was constructed to estimate the health outcomes and costs over 5 years of different testing strategies applied to a cohort of diabetic patients. Bedside tests investigated were PAD-scan, ankle-brachial pressure index, toe-brachial pressure index, audible and visual Doppler, transcutaneous pressure of oxygen, and pulse palpation. Health outcomes were incidence of new DFU, major cardiovascular events, amputation, death, and DFU healing rates. Sensitivity analyses were performed. PAD-scan was the most cost-effective bedside test with an incremental cost-effectiveness ratio of £11,391/quality-adjusted life years. PADscan had the highest probability (78.7%) of having the greatest net benefit at a willingness to pay threshold of £20,000 per quality-adjusted life years. It reduced the number of amputations by 24% and the number of cardiovascular deaths by 10% over 5 years, compared to toe-brachial pressure index (next best alternative). PAD-scans superiority in incremental cost-effectiveness ratio occurred at a PAD prevalence threshold of 0.24. PAD-scan is a cost-effective test for the detection of PAD in patients with diabetes.

Comparative cost-effectiveness of SARS-CoV-2 testing strategies in the USA: a modelling study

BackgroundTo mitigate the COVID-19 pandemic, countries worldwide have enacted unprecedented movement restrictions, physical distancing measures, and face mask requirements. Until safe and efficacious vaccines or antiviral drugs become widely available, viral testing remains the primary mitigation measure for rapid identification and isolation of infected individuals. We aimed to assess the economic trade-offs of expanding and accelerating testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) across the USA in different transmission scenarios.MethodsWe used a multiscale model that incorporates SARS-CoV-2 transmission at the population level and daily viral load dynamics at the individual level to assess eight surveillance testing strategies that varied by testing frequency (from daily to monthly testing) and isolation period (1 or 2 weeks), compared with the status-quo strategy of symptom-based testing and isolation. For each testing strategy, we first estimated the costs (incorporating costs of diagnostic testing and admissions to hospital, and salary lost while in isolation) and years of life lost (YLLs) prevented under rapid and low transmission scenarios. We then assessed the testing strategies across a range of scenarios, each defined by effective reproduction number (Re), willingness to pay per YLL averted, and cost of a test, to estimate the probability that a particular strategy had the greatest net benefit. Additionally, for a range of transmission scenarios (Re from 1·1 to 3), we estimated a threshold test price at which the status-quo strategy outperforms all testing strategies considered.FindingsOur modelling showed that daily testing combined with a 2-week isolation period was the most costly strategy considered, reflecting increased costs with greater test frequency and length of isolation period. Assuming a societal willingness to pay of US$100 000 per YLL averted and a price of $5 per test, the strategy most likely to be cost-effective under a rapid transmission scenario (Re of 2·2) is weekly testing followed by a 2-week isolation period subsequent to a positive test result. Under low transmission scenarios (Re of 1·2), monthly testing of the population followed by 1-week isolation rather than 2-week isolation is likely to be most cost-effective. Expanded surveillance testing is more likely to be cost-effective than the status-quo testing strategy if the price per test is less than $75 across all transmission rates considered.InterpretationExtensive expansion of SARS-CoV-2 testing programmes with more frequent and rapid tests across communities coupled with isolation of individuals with confirmed infection is essential for mitigating the COVID-19 pandemic. Furthermore, resources recouped from shortened isolation duration could be cost-effectively allocated to more frequent testing.FundingUS National Institutes of Health, US Centers for Disease Control and Prevention, and Love, Tito's.

A Comprehensive Way to Access Hospital Death Prediction Model for Acute Mesenteric Ischemia: A Combination of Traditional Statistics and Machine Learning.

PurposeThis study aimed to use traditional statistics and machine learning to develop and validate prediction models for predicting hospital death in patients with AMI and compare these models’ performance.Patients and MethodsData were retrieved from the Medical Information Mart for Intensive Care (MIMIC III) electronic clinical database. A total of 338 eligible AMI patients were divided into a training cohort (n = 238) and a validation cohort (n = 100), and all patients were divided into survival groups and nonsurvival groups according to patients’ hospital outcomes. The performance of the traditional statistics prediction model and the optimal machine learning prediction model was evaluated and compared with respect to discrimination, calibration, and clinical utility in the validation cohort.ResultsUnivariate and multivariate logistic regression analyses identified the following independent risk factors associated with hospital death for AMI in the training cohort, including diastolic blood pressure, blood lactate, blood creatinine, age, blood pH, and red blood cell distribution width. Both the nomogram (AUC = 77.0%, 67.9–86.1%) and optimal machine learning model (AUC = 82.9%, 74.9–91.0%) achieved good discrimination and calibration in the validation cohort. Decision curves analysis showed that the optimal machine learning model has a greater net benefit than that of nomogram in this study.ConclusionThe nomogram achieved a concise and relatively accurate prediction of hospital death in patients with AMI, the machine learning model also has good discrimination and seems to have better clinical utility. Traditional statistics may help infer the relationship between risk factors and hospital death, while machine learning may contribute to a more accurate prediction. Traditional statistics and machine learning are complementary in developing the prediction model for hospital death of AMI. Therefore, a combination of nomogram–machine learning (Nomo-ML) predictive model may improve care and help clinicians make AMI management-related decisions.

Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study

SummaryBackgroundPrognostic models to predict the risk of clinical deterioration in acute COVID-19 cases are urgently required to inform clinical management decisions.MethodsWe developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) among consecutively hospitalised adults with highly suspected or confirmed COVID-19 who were prospectively recruited to the International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study across 260 hospitals in England, Scotland, and Wales. Candidate predictors that were specified a priori were considered for inclusion in the model on the basis of previous prognostic scores and emerging literature describing routinely measured biomarkers associated with COVID-19 prognosis. We used internal–external cross-validation to evaluate discrimination, calibration, and clinical utility across eight National Health Service (NHS) regions in the development cohort. We further validated the final model in held-out data from an additional NHS region (London).Findings74 944 participants (recruited between Feb 6 and Aug 26, 2020) were included, of whom 31 924 (43·2%) of 73 948 with available outcomes met the composite clinical deterioration outcome. In internal–external cross-validation in the development cohort of 66 705 participants, the selected model (comprising 11 predictors routinely measured at the point of hospital admission) showed consistent discrimination, calibration, and clinical utility across all eight NHS regions. In held-out data from London (n=8239), the model showed a similarly consistent performance (C-statistic 0·77 [95% CI 0·76 to 0·78]; calibration-in-the-large 0·00 [–0·05 to 0·05]); calibration slope 0·96 [0·91 to 1·01]), and greater net benefit than any other reproducible prognostic model.InterpretationThe 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19.FundingNational Institute for Health Research (NIHR), UK Medical Research Council, Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, EU Platform for European Preparedness Against (Re-)emerging Epidemics, NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool, NIHR HPRU in Respiratory Infections at Imperial College London.

Predicting Adverse Outcomes for Shiga Toxin–Producing Escherichia coli Infections in Emergency Departments

To assess the performance of a hemolytic uremic syndrome (HUS) severity score among children with Shiga toxin-producing Escherichia coli (STEC) infections and HUS by stratifying them according to their risk of adverse events. The score has not been previously evaluated in a North American acute care setting. We reviewed medical records of children <18years old infected with STEC and treated in 1 of 38 participating emergency departments in North America between 2011 and 2015. The HUS severity score (hemoglobin [g/dL] plus 2-times serum creatinine [mg/dL]) was calculated using first available laboratory results. Children with scores >13 were designated as high-risk. We assessed score performance to predict severe adverse events (ie, dialysis, neurologic complication, respiratory failure, and death) using discrimination and net benefit (ie, threshold probability), with subgroup analyses by age and day-of-illness. A total of 167 children had HUS, of whom 92.8% (155/167) had relevant data to calculate the score; 60.6% (94/155) experienced a severe adverse event. Discrimination was acceptable overall (area under the curve 0.71, 95% CI 0.63-0.79) and better among children <5years old (area under the curve 0.77, 95% CI 0.68-0.87). For children <5years, greatest net benefit was achieved for a threshold probability >26%. The HUS severity score was able to discriminate between high- and low-risk children <5years old with STEC-associated HUS at a statistically acceptable level; however, it did not appear to provide clinical benefit at a meaningful risk threshold.

External Validation of Six Liver Functional Reserve Models to predict Posthepatectomy Liver Failure after Major Resection for Hepatocellular Carcinoma.

Objective: To validate and compare the predictive ability of albumin-bilirubin model (ALBI) with other 5 liver functional reserve models (APRI, FIB4, MELD, PALBI, King's score) for posthepatectomy liver failure (PHLF) in patients with hepatocellular carcinoma (HCC) who underwent major hepatectomy.Methods: Data of patients undergoing major hepatectomy for HCC from 4 hospitals between January 01, 2008 and December 31, 2019 were retrospectively analyzed. PHLF was evaluated according to the definition of the 50-50 criteria. Performances of six liver functional reserve models were determined by the area under the receiver operating characteristic curve (AUC), calibration plot and decision curve analysis.Results: A total of 745 patients with 103 (13.8%) experienced PHLF were finally included in this study. Among six liver functional reserve models, ALBI showed the highest AUC (0.64, 95% CI: 0.58-0.69) for PHLF. All models showed good calibration and greater net benefit than treating all patients at a limit range of threshold probabilities, but the ALBI demonstrated net benefit across the largest range of threshold probabilities. Subgroup analysis also showed ALBI had good predictive performance in cirrhotic (AUC=0.63) or non-cirrhotic (AUC=0.62) patients.Conclusion: Among the six models, the ALBI model shows more accurate predictive ability for PHLF in HCC patients undergoing major hepatectomy, regardless of having cirrhosis or not.

1218. Comparing the Diagnostic Accuracy of Clinician Judgement to a Novel Host Response Diagnostic for Acute Respiratory Illness

BackgroundDiscriminating bacterial and viral infections remains clinically challenging. The resulting antibacterial misuse contributes to antimicrobial resistance. Host gene expression-based tests are a promising strategy to discriminate of bacterial and viral infections, but their potential clinical utility has not yet been evaluated.MethodsA host gene expression biosignature was measured using either qRT-PCR or microarray in 683 ED subjects with suspected infection. Based on chart reviews, we recorded clinical diagnosis as defined both by the provider assessment and by the provider treatment plan. The biosignature, diagnosis, treatment plan, and procalcitonin were compared to clinical adjudication as the reference standard. With this as a baseline, we then calculated average weighted accuracy (AWA) and change in overall net benefit (∆NB), weighting bacterial false negatives four times more seriously than false positives.ResultsGene expression correctly classified the three possible disease etiologies (bacterial, viral, or non-infectious) 76.1% of the time, outperforming provider diagnosis, provider treatment, and procalcitonin (Table 1). Overall accuracy was higher in subjects with bacterial infections (n=278, 83.8% accurate) compared to those with viral (n=234, 76.5%) and non-infectious (n=171, 63.2%) etiologies. Due to a strong sensitivity bias to treat bacterial infections at the expense of diagnostic accuracy and specificity, the provider diagnosis was overall more accurate than the corresponding treatment plan (71.4% accuracy vs. 68.1%), resulting in inappropriate antibiotic use in 41.0% of cases where antibiotics were prescribed. The gene expression test had significantly higher AWA for the diagnosis of bacterial infection than both procalcitonin and provider treatment (82.4% vs. 70.3% and 74.4%, respectively; p < 0.0001). Consequently, the host gene expression test had greater net benefit than provider treatment (∆NBbact = 9.9%), provider diagnosis (∆NBbact = 4.4%), and procalcitonin (∆NBbact = 27.1%).Table 1: Summary of provider, procalcitonin, and host gene expression test performance in a cohort of 683 subjects. ConclusionHost gene expression-based tests to distinguish bacterial and viral infection can facilitate more appropriate treatment, leading to improved patient outcomes and mitigating the antibiotic resistance crisis.Disclosures Geoffrey S. Ginsburg, MD PhD, Predigen, Inc (Shareholder, Other Financial or Material Support) Ephraim L. Tsalik, MD, MHS, PhD, Predigen (Shareholder, Other Financial or Material Support, Founder)

A cost/benefit analysis of clinical trial designs for COVID-19 vaccine candidates.

We compare and contrast the expected duration and number of infections and deaths averted among several designs for clinical trials of COVID-19 vaccine candidates, including traditional and adaptive randomized clinical trials and human challenge trials. Using epidemiological models calibrated to the current pandemic, we simulate the time course of each clinical trial design for 756 unique combinations of parameters, allowing us to determine which trial design is most effective for a given scenario. A human challenge trial provides maximal net benefits-averting an additional 1.1M infections and 8,000 deaths in the U.S. compared to the next best clinical trial design-if its set-up time is short or the pandemic spreads slowly. In most of the other cases, an adaptive trial provides greater net benefits.

A novel nomogram to predict perioperative acute kidney injury following isolated coronary artery bypass grafting surgery with impaired left ventricular ejection fraction

Background and objectiveHeart failure (HF) is a global health issue, and coronary artery bypass graft (CABG) is one of the most effective surgical treatments for HF with coronary artery disease. Unfortunately, the incidence of postoperative acute kidney injury (AKI) is high in HF patients following CABG, and there are few tools to predict AKI after CABG surgery for such patients. The aim of this study is to establish a nomogram to predict the incidence of AKI after CABG in patients with impaired left ventricular ejection fraction (LVEF).MethodsFrom 2012 to 2017, Clinical information of 1208 consecutive patients who had LVEF< 50% and underwent isolated CABG was collected to establish a derivation cohort. A novel nomogram was developed using the logistic regression model to predict postoperative AKI among these patients. According to the same inclusion criteria and the same period, we extracted the data of patients from 6 other large cardiac centers in China (n = 540) from the China Heart Failure Surgery Registry (China-HFSR) database for external validation of the new model. The nomogram was compared with 3 other available models predicting renal failure after cardiac surgery in terms of calibration, discrimination and net benefit.ResultsIn the derivation cohort (n = 1208), 90 (7.45%) patients were diagnosed with postoperative AKI. The nomogram included 7 independent risk factors: female, increased preoperative creatinine(> 2 mg/dL), LVEF< 35%, previous myocardial infarction (MI), hypertension, cardiopulmonary bypass(CPB) used and perioperative blood transfusion. The area under the receiver operating characteristic curve (AUC) was 0.738, higher than the other 3 models. By comparing calibration curves and decision curve analyses (DCA) with other models, the novel nomogram showed better calibration and greater net benefit. Among the 540 patients in the validation cohort, 104 (19.3%) had postoperative AKI, and the novel nomogram performed better with respect to calibration, discrimination and net benefit.ConclusionsThe novel nomogram is a reliable model to predict postoperative AKI following isolated CABG for patients with impaired LVEF.

Modelling the lifetime cost-effectiveness of radical prostatectomy, radiotherapy and active monitoring for men with clinically localised prostate cancer from median 10-year outcomes in the ProtecT randomised trial

BackgroundOptimal management strategies for clinically localised prostate cancer are debated. Using median 10-year data from the largest randomised controlled trial to date (ProtecT), the lifetime cost-effectiveness of three major treatments (radical radiotherapy, radical prostatectomy and active monitoring) was explored according to age and risk subgroups.MethodsA decision-analytic (Markov) model was developed and informed by clinical input. The economic evaluation adopted a UK NHS perspective and the outcome was cost per Quality-Adjusted Life Year (QALY) gained (reported in UK£), estimated using EQ-5D-3L.ResultsCosts and QALYs extrapolated over the lifetime were mostly similar between the three randomised strategies and their subgroups, but with some important differences. Across all analyses, active monitoring was associated with higher costs, probably associated with higher rates of metastatic disease and changes to radical treatments.When comparing the value of the strategies (QALY gains and costs) in monetary terms, for both low-risk prostate cancer subgroups, radiotherapy generated the greatest net monetary benefit (£293,446 [95% CI £282,811 to £299,451] by D’Amico and £292,736 [95% CI £284,074 to £297,719] by Grade group 1). However, the sensitivity analysis highlighted uncertainty in the finding when stratified by Grade group, as radiotherapy had 53% probability of cost-effectiveness and prostatectomy had 43%. In intermediate/high risk groups, using D’Amico and Grade group > = 2, prostatectomy generated the greatest net monetary benefit (£275,977 [95% CI £258,630 to £285,474] by D’Amico and £271,933 [95% CI £237,864 to £287,784] by Grade group). This finding was supported by the sensitivity analysis.Prostatectomy had the greatest net benefit (£290,487 [95% CI £280,781 to £296,281]) for men younger than 65 and radical radiotherapy (£201,311 [95% CI £195,161 to £205,049]) for men older than 65, but sensitivity analysis showed considerable uncertainty in both findings.ConclusionOver the lifetime, extrapolating from the ProtecT trial, radical radiotherapy and prostatectomy appeared to be cost-effective for low risk prostate cancer, and radical prostatectomy for intermediate/high risk prostate cancer, but there was uncertainty in some estimates. Longer ProtecT trial follow-up is required to reduce uncertainty in the model.Trial registrationCurrent Controlled Trials number, ISRCTN20141297: http://isrctn.org (14/10/2002); ClinicalTrials.gov number, NCT02044172: http://www.clinicaltrials.gov (23/01/2014).

Management of Acute Pain From Non-Low Back, Musculoskeletal Injuries : A Systematic Review and Network Meta-analysis of Randomized Trials.

Patients and clinicians can choose from several treatment options to address acute pain from non-low back, musculoskeletal injuries. To assess the comparative effectiveness of outpatient treatments for acute pain from non-low back, musculoskeletal injuries by performing a network meta-analysis of randomized clinical trials (RCTs). MEDLINE, EMBASE, CINAHL, PEDro (Physiotherapy Evidence Database), and Cochrane Central Register of Controlled Trials to 2 January 2020. Pairs of reviewers independently identified interventional RCTs that enrolled patients presenting with pain of up to 4 weeks' duration from non-low back, musculoskeletal injuries. Pairs of reviewers independently extracted data. Certainty of evidence was evaluated by using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. The 207 eligible studies included 32959 participants and evaluated 45 therapies. Ninety-nine trials (48%) enrolled populations with diverse musculoskeletal injuries, 59 (29%) included patients with sprains, 13 (6%) with whiplash, and 11 (5%) with muscle strains; the remaining trials included various injuries ranging from nonsurgical fractures to contusions. Topical nonsteroidal anti-inflammatory agents (NSAIDs) proved to have the greatest net benefit, followed by oral NSAIDs and acetaminophen with or without diclofenac. Effects of these agents on pain were modest (around 1 cm on a 10-cm visual analogue scale, approximating the minimal important difference). Regarding opioids, compared with placebo, acetaminophen plus an opioid improved intermediate pain (1 to 7 days) but not immediate pain (≤2 hours), tramadol was ineffective, and opioids increased the risk for gastrointestinal and neurologic harms (all moderate-certainty evidence). Only English-language studies were included. The number of head-to-head comparisons was limited. Topical NSAIDs, followed by oral NSAIDs and acetaminophen with or without diclofenac, showed the most convincing and attractive benefit-harm ratio for patients with acute pain from non-low back, musculoskeletal injuries. No opioid achieved benefit greater than that of NSAIDs, and opioids caused the most harms. National Safety Council. (PROSPERO: CRD42018094412).

CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?

CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?