The Impact of Data Quality Tags on Decision-Making Outcomes and Process

Abstract

It has been proposed that metadata describing data quality (DQ), termed DQ tags, be made available in situations where decision makers are unfamiliar with the data context, for example, in data warehouses. However, there have been conflicting reports as to the impact of such DQ tags on decision-making outcomes. Early studies did not explicitly consider the usability and semantics of the DQ tag designs used experimentally or the impact of such tags on decision process, except in suggestions for future research. This study addresses these issues, focusing on the design of usable DQ tags whose semantics are explicitly specified and exploring the impact of such DQ tags on decision outcomes and process. We use the information quality framework InfoQual, the interaction design technique of contextual inquiry, and cognitive process tracing to address DQ tag semantics, usability, and impact on decision process, respectively. In distinct contrast to earlier laboratory experiments, there was no evidence that the preferred decision choice changed with DQ tags, but decision time was significantly increased and there were indications of reduced consensus. These results can be explained by understanding the impact of DQ tags on decision process using concurrent protocol analysis, which involves participants verbalizing thoughts while making a decision. The protocol analysis study shows that DQ tags are associated with increased cognitive processing in the earlier phases of decision making, which delays generation of decision alternatives.