Sensitivity-specificity Pairs Research Articles

Current Pyuria Cutoffs Promote Inappropriate Urinary Tract Infection Diagnosis in Older Women.

Pre-existing lower urinary tract symptoms (LUTS), cognitive impairment, and the high prevalence of asymptomatic bacteriuria (ASB) complicate the diagnosis of urinary tract infection (UTI) in older women. The presence of pyuria remains the cornerstone of UTI diagnosis. However, >90% of ASB patients have pyuria, prompting unnecessary treatment. We quantified pyuria by automated microscopy and flowcytometry to determine the diagnostic accuracy for UTI and to derive pyuria thresholds for UTI in older women. Women ≥65 years with ≥2 new-onset LUTS and 1 uropathogen ≥104 colony-forming units (CFU)/mL were included in the UTI group. Controls were asymptomatic and classified as ASB (1 uropathogen ≥105 CFU/mL), negative culture, or mixed flora. Patients with an indwelling catheter or antimicrobial pretreatment were excluded. Leukocyte medians were compared and sensitivity-specificity pairs were derived from a receiver operating characteristic curve. We included 164 participants. UTI patients had higher median urinary leukocytes compared with control patients (microscopy: 900 vs 26 leukocytes/µL; flowcytometry: 1575 vs 23 leukocytes/µL; P < .001). Area under the curve was 0.93 for both methods. At a cutoff of 264 leukocytes/µL, sensitivity and specificity of microscopy were 88% (positive and negative likelihood ratio: 7.2 and 0.1, respectively). The commonly used cutoff of 10 leukocytes/µL had a poor specificity (36%) and a sensitivity of 100%. The degree of pyuria can help to distinguish UTI in older women from ASB and asymptomatic controls with pyuria. Current pyuria cutoffs are too low and promote inappropriate UTI diagnosis in older women. Clinical Trials Registration. International Clinical Trials Registry Platform: NL9477 (https://trialsearch.who.int/Trial2.aspx?TrialID=NL9477).

A novel method for interrogating receiver operating characteristic curves for assessing prognostic tests

BackgroundDisease prevalence is rarely explicitly considered in the early stages of the development of novel prognostic tests. Rather, researchers use the area under the receiver operating characteristic (AUROC) as the key metric to gauge and report predictive performance ability. Because this statistic does not account for disease prevalence, proposed tests may not appropriately address clinical requirements. This ultimately impedes the translation of prognostic tests into clinical practice.MethodsA method to express positive- and/or negative predictive value criteria (PPV, NPV) within the ROC space is presented. Equations are derived for so-called equi-PPV (and equi-NPV) lines. Herewith it is possible, for any given prevalence, to plot a series of sensitivity-specificity pairs which meet a specified PPV (or NPV) criterion onto the ROC space.This concept is introduced by firstly reviewing the well-established “mechanics”, strengths and limitations of the ROC analysis in the context of developing prognostic models. Then, the use of PPV (and/or) NPV criteria to augment the ROC analysis is elaborated.Additionally, an interactive web tool was also created to enable people to explore the dynamics of lines of equi-predictive value in function of prevalence. The web tool also allows to gauge what ROC curve shapes best meet specific positive and/or negative predictive value criteria (http://d4ta.link/ppvnpv/).ResultsTo illustrate the merits and implications of this concept, an example on the prediction of pre-eclampsia risk in low-risk nulliparous pregnancies is elaborated.ConclusionsIn risk stratification, the clinical usefulness of a prognostic test can be expressed in positive- and negative predictive value criteria; the development of novel prognostic tests will be facilitated by the possibility to co-visualise such criteria together with ROC curves. To achieve clinically meaningful risk stratification, the development of separate tests to meet either a pre-specified positive value (rule-in) or a negative predictive value (rule-out) criteria should be considered: the characteristics of successful rule-in and rule-out tests may markedly differ.

Three-dimensional saline infusion sonography compared to two-dimensional saline infusion sonography for the diagnosis of focal intracavitary lesions.

Three-dimensional saline infusion sonography compared to two-dimensional saline infusion sonography for the diagnosis of focal intracavitary lesions.

Protein profile analysis of cellular samples from the cervix for the objective diagnosis of cervical cancer using HPLC-LIF

Protein profiles of cytologic samples from the cervix were studied using High Performance Liquid Chromatographic (HPLC) separation combined with ultra-sensitive laser induced fluorescence (LIF) detection. HPLC-LIF protein profiles of samples from clinically normal subjects, individuals suffering from cervical cancer (different stages), and subjects who had other gynecological problems related to cervix, like erosion of cervix and Nabothian cyst, but no malignancy, were subjected to Principal Component Analysis (PCA). The application of HPLC-LIF protein profiling combined with PCA was found to be a highly efficient method for discrimination of different classes of samples with high sensitivity and specificity. Diagnostic accuracy and optimal threshold – decision criterion – for objective discrimination were estimated using sensitivity–specificity pairs and Youden's index ( J) plots.

Posttreatment Recurrence of Malignant Brain Neoplasm: Accuracy of Relative Cerebral Blood Volume Fraction in Discriminating Low from High Malignant Histologic Volume Fraction

To determine the accuracy of relative cerebral blood volume (rCBV) fraction for distinguishing high-grade recurrent neoplasm from treatment-related necrosis (TRN) in enhancing masses identified on surveillance magnetic resonance (MR) images following treatment for primary or secondary brain neoplasm. This institutional review board approved and HIPAA-compliant retrospective study included 30 patients undergoing resection of recurrent enhancing mass appearing after treatment with surgery and radiation, with or without chemotherapy. The enhancing mass volume was manually segmented on three-dimensional T1-weighted images. The rCBV maps were created by using T2-weighted dynamic susceptibility contrast perfusion MR imaging and registered to T1-weighted images, and the fraction of enhancing mass with rCBV above a range of thresholds was calculated. A receiver operating characteristic (ROC) curve was created by calculating sensitivity-specificity pairs at each threshold for rCBV fraction (< or = 20% or > 20%) by using percentage of malignant features at histologic evaluation as the reference criterion. Relationships between rCBV and probability of recurrence were estimated by using logistic regression analysis. ROC analysis showed excellent discriminating accuracy of rCBV fraction (area under the ROC curve, 0.97 +/- 0.03 [standard error]) and high efficiency (93%) with an rCBV threshold of 1.8 times that of normal-appearing white matter. Logistic regression analysis showed that a unit increase of rCBV is associated with a 254-fold increase (95% confidence interval: 43, 1504, P < .001) of the odds that enhanced tissue is recurrence, adjusting for age, treatment, volume of enhancing tissue, and time to suspected recurrence. The fraction of malignant histologic features in enhancing masses recurring after treatment for brain neoplasms can be predicted by using the rCBV fraction, with improved differentiation between recurrent neoplasm and TRN.

Optic nerve head changes in early glaucoma: a comparison between stereophotography and Heidelberg retina tomography

To evaluate and compare the diagnostic accuracy of the Heidelberg retina tomograph (HRT) version 3 with that of glaucoma specialists using stereophotography in discriminating between normal eyes and patients with early glaucomatous visual field loss. A total of 105 eyes of 105 individuals were prospectively and consecutively recruited. The sample comprised 51 normal and 54 early glaucomatous eyes, as defined by intraocular pressure and standard automated perimetry results, regardless of optic disc appearance. Receiver operating characteristic (ROC) curves were plotted for the HRT3 parameters and a linear discriminant function (LDF) developed in our hospital. Best sensitivity-specificity pairs were compared between the HRT3 parameters, with the highest areas under the ROC curve (AUCs) and evaluation of optic disc stereophotographs. Agreement between methods for measuring vertical cup-to-disc ratio was evaluated with the Bland-Altman plot. The average visual field mean deviation was -2.90 dB. The HRT3 parameters with the largest AUCs were our LDF (0.900), rim volume (0.883), and vertical cup/disk ratio (0.880), with no significant differences between these parameters. Sensitivity-specificity pairs were 79.6-100% (clinical evaluation), 83.3-86.3% (our LDF), 64.8-96.1% (final glaucoma probability score), and 68.5-90.2% (global Moorfields regression analysis). The diagnostic accuracy for differentiating normal eyes from those with early visual field defects was similar between clinical evaluation of the optic disc and evaluation with the HRT3. The use of our LDF increased the sensitivity-specificity balance with respect to the HRT-provided parameters. The diagnostic accuracy of the HRT classifications was comparable to that of an experienced glaucoma specialist.

Diagnostic Usefulness of Carbohydrate Antigen-125 in Cancerous and Noncancerous Peritoneal Effusions

Carbohydrate antigen-125 (CA-125) is a tumor marker that has been used for differential diagnosis of peritoneal malignancies. The aim of the present study was to evaluate the diagnostic usefulness of simultaneous quantification of CA-125 in peritoneal fluid and serum for abdominal cancer cases and noncancer diseases. Noncancer disease group included cirrhotic patients (n=28) and spontaneous bacterial peritonitis (SBP) patients (n=11). Abdominal cancer group was composed of histologically diagnosed various malignancies (n=10), such as gastric cancer. CA-125 levels were quantified by chemiluminescent enzyme immuno-assay. Diagnostic usefulness tests and receiver operating characteristics (ROC) curve analysis were performed for the levels of peritoneal fluid CA-125 (pCA-125) and serum CA-125 (sCA-125), and the ratio of pCA-125 to sCA-125 (p/sCA-125). The sCA-125 levels were significantly higher in noncancer patients than those in the cancer patients, while the pCA-125 levels showed no significant difference between the two groups. Notably, the p/sCA-125 ratio was significantly lower in the noncancer patients than that in the cancer patients. Area under the ROC curve was 0.267 for sCA-125, 0.542 for pCA-125 and 0.831 for p/sCA-125. The accepted cutoff values were the combination of values that gave the greatest diagnostic sensitivity plus specificity. Either sCA-125 or pCA-125 value gave lower diagnostic accuracy, whereas p/sCA-125 value demonstrated a significantly higher diagnostic accuracy (sensitivity-specificity pairs: 0.40-0.33 for sCA-125, 0.60-0.54 for pCA-125, and 0.80-0.72 for p/sCA-125, respectively). Hence, determination of p/sCA-125 improves the biochemical discrimination of abdominal cancerous cases from noncancerous diseases.

Objective method for localization of cortical asymmetries using positron emission tomography to aid surgical resection of epileptic foci

We designed a semiautomated method for the objective detection of abnormal regions of tracer accumulation in the brain. The purpose of the present study was to examine the diagnostic performance of this method by applying it to patients with clinically intractable epilepsy of unilateral origin; they underwent [F-18] deoxyglucose positron emission tomography (PET) prior to surgical resection of epileptic foci. A semiautomated method for assessment of asymmetries in the brain cortex was developed that compares activity concentrations in homotopic cortical areas. When these differences exceeded a predefined threshold, the areas with lower activity were marked and 3-dimensional surface rendered images were created to guide placement of intracranial electrodes (ECoG) followed by surgical resection. The normal amount of asymmetry between small (0.5-0.7 cm2) homotopic cortical regions was determined as 5.9 +/- 4.0% (mean +/- SD). The false-positive fraction was determined for cutoff thresholds of 1 SD (10%), 1.5 SD (12%), and 2 SD (15%) outside the mean and was found to be 89, 44, and 0%, respectively. The obtained sensitivity-specificity pairs for correct localization of epileptogenic lobes based on the ECoG results were best for the 15% threshold (80/94%, accuracy 0.90). This objective PET method allows the accurate determination of cortical asymmetries, and it proved to be highly efficient in guiding epilepsy surgery.

Objective Method for Localization of Cortical Asymmetries Using Positron Emission Tomography to Aid Surgical Resection of Epileptic Foci

We designed a semiautomated method for the objective detection of abnormal regions of tracer accumulation in the brain. The purpose of the present study was to examine die diagnostic performance of this method by applying it to patients with clinically intractable epilepsy of unilateral origin; they underwent [F-18] deoxyglucose positron emission tomography (PET) prior to surgical resection of epileptic foci. A semiautomated method for assessment of asymmetries in the brain cortex was developed that compares activity concentrations in homotopic cortical areas. When these differences exceeded a predefined threshold, the areas with lower activity were marked and 3-dimensional surface rendered images were created to guide placement of intracranial electrodes (ECoG) followed by surgical resection. The normal amount of asymmetry between small (0.5–0.7 cm2) homotopic cortical regions was determined as 5.9 ± 4.0% (mean ± SD). The false-positive fraction was determined for cutoff thresholds of 1 SD (10%), 1.5 SD (12%), and 2 SD (15%) outside the mean and was found to be 89, 44, and 0%, respectively. The obtained sensitivity-specificity pairs for correct localization of epileptogenic lobes based on the ECoG results were best for the 15% threshold (80/94%, accuracy 0.90). This objective PET method allows the accurate determination of cortical asymmetries, and it proved to be highly efficient in guiding epilepsy surgery.

Serum catalase as marker of graft-vs-host disease in allogeneic bone marrow transplant recipients: pilot study

We evaluated the efficacies of serum catalase (CAT), 5'-nucleotidase (5'NT), and tumor necrosis factor-alpha (TNF) as diagnostic markers of acute graft-vs-host disease (GVHD) in 28 allogeneic bone marrow transplant recipients by comparing their abilities to discriminate between GVHD-related and non-GVHD-related complications. Mean peak serum CAT concentrations for patients with GVHD-related complications (n = 17) were about fivefold higher than concentrations in patients with non-GVHD-related complications (n = 25; P = 0.003), whereas the mean peak concentrations of serum 5'NT and TNF were not substantially different. Similarly, the sensitivity and specificity of serum CAT (100% and 88%, respectively) for use as a diagnostic marker of GVHD were much better than those of serum 5'NT (88% and 24%, respectively) or serum TNF (65% and 4%, respectively). Receiver-operating characteristic plots of all possible sensitivity-specificity pairs obtained over the whole range of results also showed that serum CAT has the best diagnostic accuracy. Low specificities of serum TNF and 5'NT were caused mainly by their increase in septicemia, fungal infection, and veno-occlusive disease and after the use of granulocyte-macrophage colony-stimulating factor to stimulate donor cell engraftment. Serum CAT may prove to be a rapid and relatively noninvasive test for the diagnosis of acute GVHD.

Combined serum amylase and lipase determinations for diagnosis of suspected acute pancreatitis

Serum amylase and lipase measurements are often used to diagnose acute pancreatitis. This study addresses the question of whether it is advantageous to order serum amylase and lipase tests simultaneously. We evaluated performance of the two tests separately and in combination through a retrospective study of patients for whom both amylase and lipase determinations were ordered. Initial analysis of test performance was conducted with a uniformly applied criterion based on determination of optimal sensitivity-specificity pairs. Individual tests and combinations of tests, including the "AND" and "OR" rules and discriminant functions, were examined. Only the discriminant approach demonstrated better performance than the lipase test alone. This finding was subsequently confirmed by logistic regression analysis. We conclude that ordering both tests simultaneously can be advantageous in diagnosing acute pancreatitis when a bivariate approach is used; however, this must be weighed against the difficulties associated with clinical implementation of such approaches.

Screening for HTLV-III Antibodies: The Relation Between Prevalence and Positive Predictive Value and Its Social Consequences

To the Editor.— We are concerned that in the recent article by Weiss et al,1some of the test operating characteristics reported may have been misleading. Specifically, the authors conclude that their screening test for human T-cell leukemia (lymphotropic) retrovirus type III (HTLV-III) antibodies (as a marker for the ability of a unit of blood to transmit acquired immunodeficiency syndrome [AIDS]) was highly sensitive and specific based on values calculated after excluding results. Neglecting these borderline results is inappropriate, as test results in the overlap zone between distributions for and nondisease actuallydefinea test's sensitivity and specificity.2 Sensitivity-specificity pairs for different cutoffs can be determined from data in the article's Table 1 (combining all patients with AIDS to define disease and Burlington blood donors to define nondisease). However, it may be more informative in examining a screening test to scrutinize the ratio of false-positives to