Performance Evaluation Of Diagnostic Tests Research Articles

Deep learning for prediction of IDH mutation in gliomas-a meta-analysis of evaluation of diagnostic test performance with a Bayesian approach (P1-1.Virtual)

Deep learning for prediction of IDH mutation in gliomas-a meta-analysis of evaluation of diagnostic test performance with a Bayesian approach (P1-1.Virtual)

Erratum. The T2-FLAIR-mismatch sign as an imaging biomarker for IDH and 1p/19q status in diffuse low-grade gliomas: a systematic review with a Bayesian approach to evaluation of diagnostic test performance.

"Erratum. The T2-FLAIR–mismatch sign as an imaging biomarker for IDH and 1p/19q status in diffuse low-grade gliomas: a systematic review with a Bayesian approach to evaluation of diagnostic test performance" published on May 2020 by American Association of Neurological Surgeons.

The T2-FLAIR-mismatch sign as an imaging biomarker for IDH and 1p/19q status in diffuse low-grade gliomas: a systematic review with a Bayesian approach to evaluation of diagnostic test performance.

With the revised WHO 2016 classification of brain tumors, there has been increasing interest in imaging biomarkers to predict molecular status and improve the yield of genetic testing for diffuse low-grade gliomas (LGGs). The T2-FLAIR-mismatch sign has been suggested to be a highly specific radiographic marker of isocitrate dehydrogenase (IDH) gene mutation and 1p/19q codeletion status in diffuse LGGs. The presence of T2-FLAIR mismatch indicates a T2-hyperintense lesion that is hypointense on FLAIR with the exception of a hyperintense rim. In accordance with PRISMA guidelines, we performed a systematic review of the Ovid Medline, Embase, Scopus, and Cochrane databases for reports of studies evaluating the diagnostic performance of T2-FLAIR mismatch in predicting the IDH and 1p/19q codeletion status in diffuse LGGs. Results were combined into a 2 × 2 format, and the following diagnostic performance parameters were calculated: sensitivity, specificity, positive predictive value, negative predictive value, and positive (LR+) and negative (LR-) likelihood ratios. In addition, we utilized Bayes theorem to calculate posttest probabilities as a function of known pretest probabilities from previous genome-wide association studies and the calculated LRs. Calculations were performed for 1) IDH mutation with 1p/19q codeletion (IDHmut-Codel), 2) IDH mutation without 1p/19q codeletion (IDHmut-Noncodel), 3) IDH mutation overall, and 4) 1p/19q codeletion overall. The QUADAS-2 (revised Quality Assessment of Diagnostic Accuracy Studies) tool was utilized for critical appraisal of included studies. A total of 4 studies were included, with inclusion of 2 separate cohorts from a study reporting testing and validation (n = 746). From pooled analysis of all cohorts, the following values were obtained for each molecular profile-IDHmut-Codel: sensitivity 30%, specificity 73%, LR+ 1.1, LR- 1.0; IDHmut-Noncodel: sensitivity 33.7%, specificity 98.5%, LR+ 22.5, LR- 0.7; IDH: sensitivity 32%, specificity 100%, LR+ 32.1, LR- 0.7; 1p/19q codeletion: sensitivity 0%, specificity 54%, LR+ 0.01, LR- 1.9. Bayes theorem was used to calculate the following posttest probabilities after a positive and negative result, respectively-IDHmut-Codel: 32.2% and 29.4%; IDHmut-Noncodel: 95% and 40%; IDH: 99.2% and 73.5%; 1p/19q codeletion: 0.4% and 35.1%. The T2-FLAIR-mismatch sign is an insensitive but highly specific marker of IDH mutation but not 1p/19q codeletion in diffuse LGGs, although there may be significant exceptions. These findings support the utility of T2-FLAIR mismatch as an imaging-based biomarker for positive selection of patients with IDH-mutant gliomas.

Field Evaluation of Diagnostic Test Sensitivity and Specificity for Salmonid Alphavirus (SAV) Infection and Pancreas Disease (PD) in Farmed Atlantic salmon (Salmo salar L.) in Norway Using Bayesian Latent Class Analysis.

Salmonid alphavirus (SAV) is the OIE-listed, viral cause of pancreas disease (PD) in farmed Atlantic salmon. SAV is routinely detected by PCR–methods while typical histopathological lesions are additionally used to confirm the diagnosis. Field evaluation of diagnostic test performance is essential to ensure confidence in a test's ability to predict the infection or disease status of a target animal. For most tests used in aquaculture, characteristics like sensitivity (Se) and specificity (Sp) at the analytical level may be known. Few tests are, however, evaluated at the diagnostic level according to the OIE standard. In the present work, we estimated diagnostic test sensitivity (DSe) and diagnostic test specificity (DSp) for five laboratory tests used for SAV detection. As there is no gold standard, the study was designed using Bayesian latent class analysis. Real-time RT-PCR, cell culture, histopathology, virus neutralization test, and immunohistochemistry were compared using samples taken from three different farmed Atlantic salmon populations with different infection status; one population regarded negative, one in an early stage of infection, and one in a later stage of infection. The average fish weight in the three populations was 2.0, 1.6, and 1.5 kg, respectively. The DSe and DSp of real-time RT-PCR is of particular interest due to its common use as a screening tool. The method showed high DSe (≥0.977) and moderate DSp (0.831) in all 3-populations models. The results further suggest that a follow-up test of serum samples in real-time RT-PCR negative populations may be prudent in cases where epidemiological information suggest a high risk of infection and where a false negative result is of high consequence. This study underlines the need to choose a test appropriate for the purpose of the testing. In the case of a weak positive PCR-result, a follow-up test should be conducted to verify the presence of SAV. Cell culture showed high DSe and DSp and may be used to verify viral presence.

Two-way partial AUC and its properties.

Simultaneous control on true positive rate and false positive rate is of significant importance in the performance evaluation of diagnostic tests. Most of the established literature utilizes partial area under receiver operating characteristic (ROC) curve with restrictions only on false positive rate (FPR), called FPR pAUC, as a performance measure. However, its indirect control on true positive rate (TPR) is conceptually and practically misleading. In this paper, a novel and intuitive performance measure, named as two-way pAUC, is proposed, which directly quantifies partial area under ROC curve with explicit restrictions on both TPR and FPR. To estimate two-way pAUC, we devise a nonparametric estimator. Based on the estimator, a bootstrap-assisted testing method for two-way pAUC comparison is established. Moreover, to evaluate possible covariate effects on two-way pAUC, a regression analysis framework is constructed. Asymptotic normalities of the methods are provided. Advantages of the proposed methods are illustrated by simulation and Wisconsin Breast Cancer Data. We encode the methods as a publicly available R package tpAUC.

Diagnostic Availability (DA) based on Pythagorean Theorem as a Novel Index for Integrative Evaluation of Diagnostic Tests

Accurate diagnosis is the prerequisite for effective treatment of diseases. Various biostatistics indices have been used to evaluate the performance of diagnostic tests. The distribution of test results commonly shows a relatively fixed overlap between the diseased and non-diseased populations, and the two main properties of laboratory tests sensitivity (Se) and specificity (Sp) usually demonstrate an inverse relationship of each other. An inverse relationship is also seen in the two sides of a right triangle where the hypotenuse serves as an integrated functional index of the two sides, as reflected in the geometric equation Pythagorean Theorem (PT). We have developed a novel test performance index named diagnostic availability (DA) based on the theory of PT, which integrates and reflects the combined efficiency of Se and Sp. When Se and Sp were changing inversely as seen in the vast majority of clinical settings, DA was superior to the other indices diagnostic efficiency (DE), Youden index, and Kappa coefficient (К) in reflecting the overall test performance of a test. It was also the only one responsive to the reciprocal changes of Se and Sp. Therefore, we propose the additional consideration of DA besides Se and Sp during the determination of the optimal test result cut-off point. In the rare scenarios that Se and Sp are changing in the same direction, DA, J and К outperforms DE in reflecting test performance. Furthermore, DA can also help identify the better test when two tests exhibit similar areas under the curve (AUC) in the receiver operating characteristic curve (ROC) analysis. In conclusion, DA is a novel index for overall performance evaluation of diagnostic tests reflecting the combined efficiency of Se and Sp. It shows superiority over other commonly used indices in the assessment of overall test performance and serves as a valuable addition to the existing indices.

Application of Receiver Operating Characteristic (ROC) Curve for Evaluation of Diagnostic Test Performance

In the field of clinical medicine, diagnostic accuracy studies refer to the degree of agreement between the index test and the reference standard for the discriminatory ability to identify a target disorder of interest in a patient. The receiver operating characteristic (ROC) curve offers a graphical display the trade-off between sensitivity and specificity at each cutoff for a diagnostic test and is useful in assigning the best cutoff for clinical use. In this end, the ROC curve analysis is a useful tool for estimating and comparing the accuracy of competing diagnostic tests. This paper reviews briefly the measures of diagnostic accuracy such as sensitivity, specificity, and area under the ROC curve (AUC) that is a summary measure for diagnostic accuracy across the spectrum of test results. In addition, the methods of creating an ROC curve in single diagnostic test with five-category discrete scale for disease classification from healthy individuals, meaningful interpretation of the AUC, and the applications of ROC methodology in clinical medicine to determine the optimal cutoff values have been discussed using a hypothetical example as an illustration.

Investigation of Diagnostic Test Performance Using Receiver Operating Characteristic and Fundamental Concepts of Information Theory

Aims: Receive Roperating Characteristic (ROC) curveanda fundamental concept of information theory is directly applicable to evaluation of diagnostic test performance. In this study,the performance of the two diagnosis tests on the field of rheumatic disorder is analyzed byusing receiver operating characteristic and fundamental concepts of information theory. The aims of this study to investigate which diagnosis tests has better performance and to demonstrate which test can be an alternative to gold standard test by carrying out ROC and fundamental concepts of information theory. Study Design: ROC analysis and application of fundamentals concept of information theory

Comparison of the performance of two spontaneous sedimentation techniques for the diagnosis of human intestinal parasites in the absence of a gold standard

Performance evaluation of diagnostic tests is critical in the search for accurate diagnoses. A gold standard test is usually absent in parasitology, thus rendering satisfactory assessment of diagnostic accuracy difficult. Moreover, reliability (assessed by the study of repeatability) is a rarely studied characteristic of diagnostic tests. This study compared and evaluated the performance (repeatability, concordance and accuracy) of the spontaneous sedimentation technique (SST) and the Paratest for the diagnosis of Giardia lamblia, Entamoeba histolytica complex, Blastocystis spp., Ascaris lumbricoides, hookworm, Trichuris trichiura and Calodium hepaticum. Fecal samples of 143 individuals were separated into three replicates for each test. Concordance and homogeneity of the results between replicates of each test and between tests were evaluated. Proportions of positives, sensitivity and specificity were estimated using a Bayesian Latent Class Model. High repeatability of both tests was found for the detection of intestinal parasites, except for Blastocystis spp. and hookworm. Concordance between tests was generally high (concordance correlation coefficient, 0.72–0.88), except for Blastocystis spp., hookworm and T. trichiura. The Paratest detected more cases of Blastocystis spp. and fewer of hookworm than the SST. The tests were quite discordant in the detection of T. trichiura. A low sensitivity (39.4–49.2% for SST, 35.8–53.8% for Paratest) and a high specificity (93.2–97.2%) were found for both tests. The Paratest presented a slightly higher sensitivity for the diagnosis of Blastocystis spp. (53.8%), and SST did so for hookworm (49.2%). This is the first study on repeatability and accuracy (using a Bayesian approach) of two spontaneous sedimentation techniques. These results suggest underdiagnosis of little dense parasitic forms due to technical limitations in both tests. We conclude that the combined study of repeatability, concordance and accuracy is a key strategy for better evaluation of the performance of tests and is also useful for the identification of technical limitations.

Evaluation and Comparison of Diagnostic Test Performance Based on Information Theory

A fundamental concept of information theory, relative entropy and mutual information, is directly applicable to evaluation of diagnostic test performance. The aim of this study is to demonstrate how basic concepts in information theory apply to the problem of quantifying major depressive disorder diagnostic test performance. In this study, the per- formances of the Dexamethasone Suppression Test-DST and the Thyroid-Stimulating Hormone Test-TSH, two of the di- agnosis tests of Major Depressive Disorder, are evaluated with the method of Information Theory. The amount of informa- tion gained by performing a diagnostic test can be quantified by calculating the relative entropy between the posttest and pretest probability distributions. And also demonstrates that diagnostic test performance can be quantified as the average amount of information the test result provides about the disease state. It is aimed that this study will hopefully give various points of view to the researchers who want to make research on this subject by explaining how the tests used for the diag- nosis of various diseases are evaluated with this way.

A General Latent Class Model for Performance Evaluation of Diagnostic Tests in the Absence of a Gold Standard: An Application to Chagas Disease

We propose a new general Bayesian latent class model for evaluation of the performance of multiple diagnostic tests in situations in which no gold standard test exists based on a computationally intensive approach. The modeling represents an interesting and suitable alternative to models with complex structures that involve the general case of several conditionally independent diagnostic tests, covariates, and strata with different disease prevalences. The technique of stratifying the population according to different disease prevalence rates does not add further marked complexity to the modeling, but it makes the model more flexible and interpretable. To illustrate the general model proposed, we evaluate the performance of six diagnostic screening tests for Chagas disease considering some epidemiological variables. Serology at the time of donation (negative, positive, inconclusive) was considered as a factor of stratification in the model. The general model with stratification of the population performed better in comparison with its concurrents without stratification. The group formed by the testing laboratory Biomanguinhos FIOCRUZ-kit (c-ELISA and rec-ELISA) is the best option in the confirmation process by presenting false-negative rate of 0.0002% from the serial scheme. We are 100% sure that the donor is healthy when these two tests have negative results and he is chagasic when they have positive results.

Quantitative synthesis in systematic reviews.

The final common pathway for most systematic reviews is a statistical summary of the data, or meta-analysis. The complex methods used in meta-analyses should always be complemented by clinical acumen and common sense in designing the protocol of a systematic review, deciding which data can be combined, and determining whether data should be combined. Both continuous and binary data can be pooled. Most meta-analyses summarize data from randomized trials, but other applications, such as the evaluation of diagnostic test performance and observational studies, have also been developed. The statistical methods of meta-analysis aim at evaluating the diversity (heterogeneity) among the results of different studies, exploring and explaining observed heterogeneity, and estimating a common pooled effect with increased precision. Fixed-effects models assume that an intervention has a single true effect, whereas random-effects models assume that an effect may vary across studies. Meta-regression analyses, by using each study rather than each patient as a unit of observation, can help to evaluate the effect of individual variables on the magnitude of an observed effect and thus may sometimes explain why study results differ. It is also important to assess the robustness of conclusions through sensitivity analyses and a formal evaluation of potential sources of bias, including publication bias and the effect of the quality of the studies on the observed effect.

DECIDE: a software for computer-assisted evaluation of diagnostic test performance

The evaluation of the performance of clinical tests is a complex problem involving different steps and many statistical tools, not always structured in an organic and rational system. This paper presents a software which provides an organic system of statistical tools helping evaluation of clinical test performance. The program allows (a) the building and the organization of a working database, (b) the selection of the minimal set of tests with the maximum information content, (c) the search of the model best fitting the distribution of the test values, (d) the selection of optimal diagnostic cut-off value of the test for every positive/negative situation, (e) the evaluation of performance of the combinations of correlated and uncrorrelated tests. The uncertainty associated with all the variables involved is evaluated. The program works in a MS-DOS environment with EGA or higher performing graphic card.