Designing comprehensible healthcare public reports: An investigation of the use of narratives and tests of quality metrics to support healthcare public report sensemaking

Abstract

A key challenge for designers of healthcare public reports is the development of a presentation format that accurately communicates the variability in the quality of care among healthcare systems. This study conducted in the United States explored whether presenting public report information within narratives and with tests of healthcare quality metrics supported the public report sensemaking process. The study involved 200 participants and employed a 2 (public report information presented in the standard format, presented within a narrative) * 2 (no tests of quality metrics added to standard report, metrics tests added) between-subjects experimental design. The participants viewed the scenario of a patient looking for dialysis facility-related information. They were then asked which dialysis facility they would choose for their care and their level of confidence in their choice. Subsequently, a knowledge quiz evaluating how the participants interpreted the information presented to them, the NASA-TLX workload survey, and a usability questionnaire were administered. The results showed that the probability of choosing the better facility from the perspective of the quality measures included in the report changed from 0.69 to 0.89 when information was presented within a narrative rather than with the standard public report format. The results also found a significantly higher comprehension score (M = 54.58, SD = 18.51) when information was presented within the narrative than when presented in the standard public report format (M = 44.57, SD = 25.13). When information was presented to the participants within a narrative, the narrative may have enabled the participants to visualize themselves as the person depicted in the narrative and this may have increased the perceived relevance of the quality measures. Total workload, mental demand and perceived usability were higher when information was presented within the narrative than when presented in the standard format. The high workload and mental demand may be due to the stress placed on the information processing channels while reading a narrative and the effort expended to relate it to the quality measures. They may also be markers of more deliberative decision making facilitated by the narratives. No significant effect of tests of quality metrics was found on the dependent variables of choice of the better healthcare facility, comprehension, and usability. There was also no significant effect of quality metrics tests on overall workload. However, the effect of quality metrics tests on the mental demand subscale of NASA-TLX was significant. Mental demand was higher without quality metrics tests than with quality metrics tests. No significant interaction was found between the two independent variables on the dependent variables of choice of the better healthcare facility, comprehension, workload, and usability. It is recommended that narratives be used to present public report information to support informed healthcare decisions.