Point Estimates Of Sensitivity Research Articles

Multisite external validation of blood test for colorectal neoplasia screening in a majority average-risk screening cohort of 449 subjects.

66 Background: The FirstSight CRC screening blood test was previously shown to have sensitivities of 92.1% and 54.5% for CRC and AA at a specificity of 90.6% for 1,038 subjects from 18 clinical sites. In this report, we describe the independent external validation of the predefined FirstSight CRC screening blood test with new subjects from 16 sites dispersed across the US, 4 of which are new and did not participate in the previous study. Methods: The validation study included subjects from two sources: Average-risk, asymptomatic screening subjects from screening clinics supplemented with diseased-only subjects from surgical oncology centers. Blood was drawn before bowel prep for colonoscopy or treatment. External validation as well as Monte Carlo cross-validation (MCCV) methods were used to evaluate the performance of the pre-defined FirstSight assay, algorithm, and clinical thresholds in this independent external validation set. Results include sensitivity and specificity to detect CRC and AA and 95% confidence intervals. Results: The study cohort (55.2% female; mean age 56.9 yrs.) consisted of 449 subjects (White 61.7%; Black 14.5%; Hispanic 18%; AAPI and Middle Easterners 5.8%). The majority of them, 376 (84%), were asymptomatic, average-risk screening subjects (including 1 CRC, 53 AA, and all 322 negative subjects). An additional 73 (16%) diseased subjects (39 CRC and 34 AA) were enriched due to low disease prevalence. The FirstSight algorithm derives a test score from 0 to 100 as a quantitative measure of AA and CRC risk with a cutpoint to assign a binary low- or high-risk designation. The previously described cutpoint of 47.2 yielded a test specificity of 90.0%, and 90.0% and 52.9% sensitivity for the detection of CRC and AA, respectively. Here we report a higher cutpoint 50 with identical sensitivities but improved specificity to 90.7%. Point estimates of sensitivity and specificity, including CRC staging, and Clopper-Pearson exact 95% confidence intervals based on the external validation results are presented below. Conclusions: An external validation included 449 subjects from a multi-site, majority average-risk CRC screening study successfully validated the FirstSight Blood Test’s high performance as previously established with 1,038 subjects. The results bring forth confidence in the Test’s reproducibility in a large-scale clinical trial. Clinical trial information: NCT05127096 . [Table: see text]

A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well

The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.

The solely A+ CSF Aβ42/40 ratio using Elecsys® assays performs similar to A/T and A/N ratios in predicting amyloid PET positivity

AbstractBackgroundpTau181/Aβ42 (A+/T+ as per A/T/(N) classification) and tTau/Aβ42 (A+/(N)+) ratios using fully automated Elecsys® cerebrospinal‐fluid (CSF) immunoassays have been shown to perform well in distinguishing amyloid‐positivity (A+) by positron emission tomography (PET) among patients with subjective and mild to severe cognitive impairment. Higher performance has also been demonstrated for the CSF Aβ42/40 ratio compared to CSF amyloid 42 (Aβ42) alone. We aimed to further evaluate the performance of the Elecsys® Aβ42/40 ratio in a clinical cohort.MethodsFrozen (‐80°C) CSF samples from 103 patients treated in the Centre for Cognitive Disorders, Technical University of Munich were selected based on availability of visual amyloid‐PET status (n=54 PET+; 49 PET‐) determined during standard treatment around the time of CSF collection. In total, 71 patients were undergoing treatment for mild cognitive impairment (MCI) (n=44) or mild to moderate dementia (n=27) due to Alzheimer’s disease (AD) and 32 patients for non‐AD related fronto‐temporal lobar degeneration (n=17), MCI (n=8), depression (n=5), alcohol dependence (n=1), or subjective cognitive impairment (SCD) (n=1). CSF was measured using Elecsys® assays. CSF status for Aβ42 , pTau181/Aβ42 , and tTau/Aβ42 CSF (positive/negative) was determined based on pre‐defined biomarker cutoffs previously established using a different pre‐analytical protocol in MCI/SCD patients with suspected AD (CSF positive if Aβ42<1000 pg/ml, pTau181/Aβ42>0.024, and tTau/Aβ42>0.028). Sensitivity and specificity were calculated for CSF‐cutoff‐based status and ROC analysis was performed to determine AUC each versus visual amyloid‐PET status.ResultsPre‐defined biomarker thresholds were considered off‐label due to deviation from recommended sample collection protocol and storage parameters. Point estimates for sensitivity and specificity for each biomarker were; CSF Aβ42 alone (sensitivity = 0.93, specificity = 0.57), pTau181/Aβ42 (0.96, 0.69), and tTau/Aβ42 (0.92, 0.69). For AUC point estimates (95% confidence interval) were; CSF Aβ42 alone [0.78 (0.68‐0.88)], pTau181/Aβ42 [0.88 (0.81‐0.95)], tTau/Aβ42 [0.87(0.80‐0.95)], and Aβ42/40 [0.90 (0.83‐0.95)].ConclusionFully automated Elecsys® cerebrospinal fluid (CSF) immunoassay predicted amyloid positivity with high accuracy. In this study, the solely A+ marker CSF Aβ42/40 ratio outperformed Aβ42 alone and performed similarly to Tau ratios. These results support those of previous studies showing high performance for the non‐Tau marker CSF Aβ42/40.

NEW PROGNOSTIC METHOD FOR ESTIMATION OF LIPID DISTRESS SYNDROME DEVELOPMENT IN PATIENTS WITH CHRONIC HEPATITIS C

Objectives - to work out a prognostic method, based on clinical and laboratory data, which can predict the development of lipid distress syndrome in patients with chronic viral hepatitis C. Material and methods. We studied the outcomes of chronic viral hepatitis C with respect to development of lipid distress syndrome in 267 patients who did not receive antiviral therapy during the observation period lasting for 1 to 7 years. Results. Discriminant function analysis of the data acquired after examination of 167 patients helped us to work out a discriminative model of predicting the development of lipid distress syndrome in patients with chronic viral hepatitis C in the absence of antiviral therapy. Crosschecking of prediction accuracy was carried out on the data of examination of other 100 patients. Conclusion. The designed technique for predicting the development of lipid distress syndrome in patients with chronic hepatitis C is characterized by point estimates of sensitivity and prognostic value of the negative result equal to 100%.

Correction for misclassification of caries experience in the absence of internal validation data

To quantify the effects of risk factors and/or determinants on disease occurrence, it is important that the risk factors as well as the variable that measures the disease outcome are recorded with the least error as possible. When investigating the factors that influence a binary outcome, a logistic regression model is often fitted under the assumption that the data are collected without error. However, most categorical outcomes (e.g., caries experience) are accompanied by misclassification and this needs to be accounted for. The aim of this research was to adjust for binary outcome misclassification using an external validation study when investigating factors influencing caries experience in schoolchildren. Data from the Signal Tandmobiel(®) study were used. A total of 500 children from the main and 148 from the validation study were included in the analysis. Regression models (with several covariates) for sensitivity and specificity were used to adjust for misclassification in the main data. The use of sensitivity and specificity modeled as functions of several covariates resulted in a better correction compared to using point estimates of sensitivity and specificity. Age, geographical location of the school to which the child belongs, dentition type, tooth type, and surface type were significantly associated with the prevalence of caries experience. Sensitivity and specificity calculated based on an external validation study may resemble those obtained from an internal study if conditioned on a rich set of covariates. Main data can be corrected for misclassification using information obtained from an external validation study when a rich set of covariates is recorded during calibration.

Effect of Specimen Order on Chlamydia trachomatis and Neisseria gonorrhoeae Test Performance and Adequacy of Papanicolaou Smear

Abstract Study Objective To evaluate the effect of specimen collection order on the performance of diagnostic tests for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (GC), and the specimen adequacy of ThinPrep Papanicolaou (Pap) smears. Design Prospective cohort study. Setting Public adolescent clinic. Population 313 women. Interventions The order of five cervical testing specimens was randomized for (1) ligase chain reaction (LCR) and (2) polymerase chain reaction (PCR) for both CT and GC; (3) transcription-mediated amplification (TMA) for CT; (4) culture for CT; and (5) cytology and LCR for GC and CT performed on ThinPrep Pap specimens. For CT and GC, a reference standard was based on three different tests performed on separate specimens. Generalized estimating equations were used to account for repeated measures. Main Outcome Measure Sensitivity and specificity of diagnostic tests. Results The proportion of inadequate Pap smears was independent of specimen order. As a group, nucleic acid amplification test (NAAT) sensitivity and specificity for GC and CT were similar in the first two (early) and last three (late) swabs. Although point estimates for sensitivity were higher in the early swabs compared to the late swabs for GC LCR (13% difference), GC PCR (13%), and CT TMA (10%), these differences were not statistically significant. Their clinical significance warrants further investigation. Conclusions In clinical settings where both Pap smears and STI testing are performed in adolescents, clinical considerations can influence the order of specimen collection, since neither Pap specimen adequacy nor test performance of NAAT for CT and GC were significantly associated with swab order.

Effect of Specimen Order on Chlamydia trachomatis and Neisseria gonorrhoeae Test Performance and Adequacy of Papanicolaou Smear

Study Objective To evaluate the effect of specimen collection order on the performance of diagnostic tests for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (GC), and the specimen adequacy of ThinPrep Papanicolaou (Pap) smears. Design Prospective cohort study. Setting Public adolescent clinic. Population 313 women. Interventions The order of five cervical testing specimens was randomized for (1) ligase chain reaction (LCR) and (2) polymerase chain reaction (PCR) for both CT and GC; (3) transcription-mediated amplification (TMA) for CT; (4) culture for CT; and (5) cytology and LCR for GC and CT performed on ThinPrep Pap specimens. For CT and GC, a reference standard was based on three different tests performed on separate specimens. Generalized estimating equations were used to account for repeated measures. Main Outcome Measure Sensitivity and specificity of diagnostic tests. Results The proportion of inadequate Pap smears was independent of specimen order. As a group, nucleic acid amplification test (NAAT) sensitivity and specificity for GC and CT were similar in the first two (early) and last three (late) swabs. Although point estimates for sensitivity were higher in the early swabs compared to the late swabs for GC LCR (13% difference), GC PCR (13%), and CT TMA (10%), these differences were not statistically significant. Their clinical significance warrants further investigation. Conclusions In clinical settings where both Pap smears and STI testing are performed in adolescents, clinical considerations can influence the order of specimen collection, since neither Pap specimen adequacy nor test performance of NAAT for CT and GC were significantly associated with swab order.

Screening for Primary Aldosteronism in Essential Hypertension: Diagnostic Accuracy of the Ratio of Plasma Aldosterone Concentration to Plasma Renin Activity

The ratio of plasma aldosterone concentration to plasma renin activity (PRA) is considered the screening test of choice for primary aldosteronism. Uncertainty exists, however, regarding its diagnostic accuracy and the effects of antihypertensive drugs and dietary sodium balance on test characteristics. We measured PRA and aldosterone in 118 white adults [71 men and 47 women; mean (SD) age, 51 (7) years] with previously diagnosed essential hypertension. Measurements were made while individuals were on antihypertensive drug therapy, after a 2-week drug-free period, after 4 days of dietary sodium loading, and after acute furosemide diuresis. We measured 24-h urine aldosterone excretion and PRA on the 4th day of dietary sodium loading to establish the diagnosis of primary aldosteronism. ROC curves were constructed for ratios measured under each clinical condition, and likelihood ratios were determined for individuals on or off antihypertensive drug therapy. Fifteen patients [13%; 95% confidence interval (CI), 7-20%] met the reference standard for primary aldosteronism. The mean (SD) areas under the ROC curves did not differ significantly across conditions of measurement [range, 0.80 (0.10) to 0.85 (0.04); P = 0.72]. When measured on and off antihypertensive drug therapy, the 95% CIs for the optimum cutpoint for the ratio overlapped. Point estimates of sensitivity on and off therapy were 73% (95% CI, 50-96%) and 87% (70-100%), respectively, and specificities were 74% (65-83%) and 75% (66-84%). Under either condition, increased ratios were associated with 2.4- to 13-fold increases of posttest odds above pretest odds. The aldosterone:PRA ratio provides only fair diagnostic accuracy in screening for primary aldosteronism, but concomitant antihypertensive drug therapy or acute variation in dietary sodium balance does not adversely affect test accuracy. Reporting of likelihood ratios associated with ranges of values of the aldosterone:PRA ratio, rather than use of a single "optimum" cutpoint, may enhance the usefulness of the test.

99mTc Sestamibi Breast Imaging for the Examination of Patients with Dense and Fatty Breasts: Multicenter Study

To evaluate the accuracy of scintimammography as an adjunct to physical examination and mammography in the detection of breast cancer in women with dense and fatty breasts. A total of 558 women were prospectively enrolled from 42 centers in North America. Images were interpreted by readers blinded to the subjects' clinical history, mammographic findings, and other test results. The Breast Imaging Reporting and Data System classification was used to describe breast density. Parenchymal patterns of "heterogeneously dense" and "extremely dense" were used to classify breasts as dense, whereas "almost entirely fat" and "numerous vague densities" defined fatty breasts. Between-group differences were evaluated with the 2 test for categorical variables and Student t test for continuous variables. Accuracy of scintimammography was assessed against the core laboratory histopathologic evaluation, the standard. The 95% CIs around point estimates of sensitivity, specificity, and positive and negative predictive values were calculated with the normal approximation to the binomial distribution. The analyses were based on 580 breasts with an abnormality; 276 (48%) breasts were dense and 228 had a malignant lesion. Diagnostic properties for scintimammography of fatty versus dense breasts were, respectively, sensitivity, 72% versus 70%; specificity, 80% versus 78%; positive predictive value, 72% versus 67%; negative predictive value, 81% versus 81%; and accuracy, 77% versus 75% (all not significant). Scintimammography led to similar and significant changes in the posttest likelihood of cancer for both dense and fatty breasts. The diagnostic accuracy of scintimammography is not affected by breast density.

A GENERALIZED LINEAR MODEL FOR ANALYSING RECEIVER OPERATING CHARACTERISTIC CURVES

We present a continuation ratio model for analysing ordinal categorical data and we apply the model to the problem of estimating receiver operating characteristic (ROC) curves. We apply the methods to post-prandial capillary blood glucose measurements as a criterion for a potential screening test for diabetes mellitus. One can obtain point estimates of sensitivity and specificity and their associated standard errors at any value along the observed range of post-prandial capillary blood glucose measurements. Also, in comparison to the models for ROCs described by Tosteson and Begg, ROC curves based on the continuation ratio model have the desired features that allow ROCs to be concave but not necessarily symmetric.

Surface Immunoglobulin Light Chain Restriction in B-Cell Non-Hodgkin’s Malignant Lymphomas

Twenty-four control cases of reactive follicular hyperplasia (RFH) and 28 cases of B-cell non-Hodgkin's malignant lymphoma (ML) were evaluated for surface immunoglobulin (sIg) light chain expression with flow cytometry. Because there was no significant difference between the distribution of the ML kappa/lambda ratios for which kappa/lambda greater than 1 compared with those ML lambda/kappa ratios for which lambda/kappa greater than 1 using the Kolmogorov-Smirnov statistic, the authors treated kappa/lambda and lambda/kappa data symmetrically. By transforming all kappa/lambda ratios to maximum ratios, that is, Rmax = max (kappa/lambda, lambda/kappa), so that all resulting ratios were at least one, the authors defined a new statistic for evaluating sIg light chain restriction. There was a highly significant difference for equality of distribution of the data using the Mann-Whitney test (P less than 0.0001). By choosing cut-points based on the Rmax statistic, the authors were able to separate the RFH and ML cases with sensitivity = 0.96 +/- 0.07 (95% confidence) and with specificity = 0.91 +/- 0.11 (95% confidence). These point estimates of sensitivity and specificity appear to be promising for the clinical utility of the Rmax statistic, even for cases of B-cell ML with discordantly low expression of sIg light chains. The Rmax statistic will facilitate the comparison of data between different laboratories because it does not require prior assignment of cut-points and can be made using a simple nonparametric test.