Defining a Learning Metric for DSS Success Monitoring

Abstract

The decision-making process is a knowledge-intensive activity, supported by DSS, that warrants close monitoring in most enterprises to ensure its success. Numerous frameworks for the evaluation of DSS effectiveness were proposed in the literature. However, many use metrics that are survey-based to reflect users’ perception of the system’s value. Based on the premise that metrics should be as objective as possible, this paper proposes a learning metric that assess the cognitive effects of DSS and their impact on decision performance. Drawing from the current tendency of using DSS in e-learning platforms, we define a learning metric that includes factors such as time spent on tasks, decision-aids use versus cumulated personal experience from previous usage, regret avoidance, decision outcome, and decision rejection/acceptance from higher management. Based on a criteria application process, we validate the proposed metric by first specifying its intent of use to determine the appropriate validation criteria, then demonstrating its viability against these criteria. An experimental case study is conducted to further attest to the validity of the proposed learning metric.