Simultaneous optimization of quota‐restricted selection decisions with mastery scores

Abstract

Combinations of elementary test‐based decisions in psychology and education arise when one decision problem leads to another, which, in turn, may lead to a third one, and so on. An example is test‐based decision making in computer‐aided instruction (CAI) in education. Such systems can be described as instructional networks with individual routes for the students. The purpose of this paper is to formulate optimal rules for deciding on such routes using data from achievement tests and data gathered at earlier decisions in the network. As an example, a common decision problem in education and psychology, consisting of a quota‐restricted selection decision for a treatment followed by a mastery decision, is analysed in the case of several relevant subpopulations. The quota‐restricted selection and mastery decisions are optimized simultaneously using the framework of Bayesian decision theory. Conditions for optimal simultaneous rules to be monotone are presented. Results from an empirical example of instructional decision making in CAI will be presented to illustrate the differences between simultaneous optimization of the quota‐restricted selection and mastery decisions and optimizing each of the two decisions separately.