The added value of signal detection theory as a method in evidence-informed decision-making in higher education: A demonstration

Abstract

Signal Detection Theory (SDT) is rarely used in higher education, yet has much potential in informing decision-making. In this methodological paper, we describe the potential of SDT for different higher education contexts and demonstrate its practical application. Both the commonly used regression analyses and SDT analyses provide information on the accuracy of a predictor, and thus which instrument(s) to use. SDT analyses, in addition, provide information on the effects of setting specific cut-off scores on outcomes of interest. SDT provides the sensitivity and specificity information for the chosen instrument(s) at specific cut-off scores (criteria in SDT). This allows for evidence-informed, deliberate choice of cut-off scores to steer toward desired outcomes. Depending on how undesirable false positives and false negatives are considered in a specific situation, a lower or higher cut-off score can be deemed adequate. Using SDT analyses in our example, we demonstrate how to use the results to optimize “real-life” student selection. However, selection is only one of many decision-making practices where SDT is applicable and valuable. We outline some of the areas within higher education decision-making and quality assurance, where SDT can be applied to answer specific questions and optimize decision-making.