How “consistent” is “consistent”? A clinician-based assessment of the reliability of expressions used by radiologists to communicate diagnostic confidence

Abstract

To evaluate the degree of variability in clinicians' interpretation of expressions used by radiologists to communicate their level of diagnostic confidence within radiological reports. Clinicians were solicited to complete a prospective survey asking them to select the approximate perceived level of certainty, expressed as a percentage, associated with 20 expressions used by radiologists to communicate their level of diagnostic confidence within radiological reports. The median and inter-decile range (IDR) were computed for each expression, with a smaller IDR indicating greater reproducibility. Clinicians were also asked questions regarding their attitudes about radiologists' communication of diagnostic confidence. Forty-nine surveys were completed. Median confidence associated with the expressions ranged from 10-90%. Reproducibility of the expressions was variable, as IDR ranged from 15-53%, although a median IDR of 40% indicated overall poor reproducibility. Expressions with relatively higher reproducibility included "most likely", "likely", and "unlikely" (IDR 15-20%), whereas expressions with relatively lower reproducibility included "compatible with", "suspicious for", "possibly," and "can be seen in the setting of" (IDR ≥45%). Only 20% of clinicians agreed or strongly agreed that radiologists consistently use such expressions within their reports. Fifty-five percent of clinicians preferred that diagnostic confidence be communicated as a percentage rather than as a textual expression. There was poor reproducibility in clinicians' interpretations of many expressions used by radiologists to communicate their level of diagnostic confidence. Use of percentages to convey diagnostic confidence within reports may mitigate this source of ambiguity in radiologists' communication with clinicians.