Data-source effects on the sensitivities and specificities of clinical features in the diagnosis of rheumatoid arthritis: the relevance of multiple sources of knowledge for a decision-support system.

Abstract

An experimental computer system was developed to support diagnosis of rheumatic disorders by computing diagnostic probabilities using modified likelihood ratios. The authors examined whether the performance of the model was affected by the settings in which the data used to derive the likelihood ratios were collected. The sensitivities and specificities of various clinical features for diagnosing rheumatoid arthritis (RA) were obtained from: 1) a study of 1,570 consecutive outpatients at a rheumatology clinic; 2) a review of the literature; 3) estimates by rheumatologists; and 4) a population study. Considerable variations in sensitivity and specificity but satisfactory agreement in likelihood ratios were found across the four data sets. The likelihood ratios were then used to compute the probabilities of RA in a test series of 570 of the rheumatology clinic outpatients. The model's diagnoses with likelihood ratios from the other sources were adequate. When the likelihood ratios from these sources were combined, discrimination came close to what could be achieved by using the likelihood ratios based on the data from the clinic. The method applied in the study, which makes use of variation of input data instead of variation of test series, and the results are relevant to assessing the external validity and transferability of Bayesian decision-support systems.