Data-driven preference learning in multiple criteria decision making in the evidential reasoning context

Abstract

In some situations, such as the diagnosis of thyroid nodules, a decision maker considers observations on multiple criteria to provide the overall assessments and advice on what will be done in the next step. To guarantee the quality of the assessments and advice and their consistency with observations, this paper proposes a method of learning the preferences of the decision maker from the observations on multiple criteria and the overall assessments provided. The constraints on preferences are learned first to avoid extreme and irrational preferences. Within the feasible region formed by the constraints, the preferences are learned. When gold standards, which can be used to judge the correctness of the overall assessments, are available, the issue of how to learn the constraints and the preferences that satisfy the constraints is presented. With and without the consideration of gold standards, the way in which solutions can be generated using the learned preferences is introduced. To demonstrate the process of preference learning based on observations and overall assessments, a case study is conducted using the examination reports generated by three radiologists from 2013 to 2017 in a hospital located in Hefei, Anhui, China.