Measuring agreement between healthcare survey instruments using mutual information.

Yuncheol Kang,Ann M Kolanowski,Donna Fick,Melinda R Steis,Vittaldas V Prabhu

doi:10.1186/s12911-016-0335-y

Abstract

BackgroundHealthcare researchers often use multiple healthcare survey instruments to examine a particular patient symptom. The use of multiple instruments can pose some interesting research questions, such as whether the outcomes produced by the different instruments are in agreement. We tackle this problem using information theory, focusing on mutual information to compare outcomes from multiple healthcare survey instruments.MethodsWe review existing methods of measuring agreement/disagreement between the instruments and suggest a procedure that utilizes mutual information to quantitatively measure the amount of information shared by outcomes from multiple healthcare survey instruments. We also include worked examples to explain the approach.ResultsAs a case study, we employ the suggested procedure to analyze multiple healthcare survey instruments used for detecting delirium superimposed on dementia (DSD) in community-dwelling older adults. In addition, several examples are used to assess the mutual information technique in comparison with other measures, such as odds ratio and Cohen’s kappa.ConclusionsAnalysis of mutual information can be useful in explaining agreement/disagreement between multiple instruments. The suggested approach provides new insights into and potential improvements for the application of healthcare survey instruments.

Highlights

Healthcare researchers often use multiple healthcare survey instruments to examine a particular patient symptom
In the comparison of Confusion Assessment Method (CAM) and a family version of CAM (FAM-CAM) in terms of the ‘Inattention’ feature, the amount of local mutual information from the disagreement sections is measured at −0.148, while that of agreement sections is measured at only 0.196, resulting in 0.048 as mutual information, which represents a low level of agreement compared to pair 1
We conjecture that FAM-CAM shows better performance in terms of explaining agreement compared to Delirium Rating Scale (DRS)

Summary

Introduction

Healthcare researchers often use multiple healthcare survey instruments to examine a particular patient symptom. The use of multiple instruments can pose some interesting research questions, such as whether the outcomes produced by the different instruments are in agreement. We tackle this problem using information theory, focusing on mutual information to compare outcomes from multiple healthcare survey instruments. Many are used to diagnose a particular symptom, serving as a diagnostic survey instrument. Since such diagnostic survey instruments are noninvasive, several instruments can be used on the same patient. By utilizing the area below a receiver operating characteristic (ROC) curve, one can combine multiple results to increase diagnostic

Objectives

Methods

Results

Discussion

Conclusion