Stepwise benchmarking for multiple criteria sorting

Abstract

We propose a stepwise benchmarking framework for multiple criteria sorting, where alternatives are assigned to pre-defined and ordered decision classes. Our focus is on the alternatives currently attaining unfavorable classifications and aiming to reach a more preferred category through gradual performance modifications distributed over a few steps. We introduce strategies for the generation of development paths that incorporate either existing alternatives or fictive benchmarks. The latter ones are constructed using a suitably adapted framework for Post Factum Analysis. Its role is to highlight how the alternatives’ performances need to be modified minimally so that the desired sorting recommendation is attained. The proposed method is applicable in the context of classifications arrived with precise values of preference model parameters or multiple feasible parameter sets within the framework of robustness analysis. A Decision Maker is allowed to specify the constraints on the feasible performance improvements and define whether to build the development path based on all criteria or a subset of criteria. We propose the measures that capture the balance in modifications of performances on different criteria and various steps. They can evaluate all generated improvement plans and indicate the path providing the smoothest development toward more preferred classes by following the intermediate benchmarks. We also offer some supportive measures that quantify the contribution of different criteria in attaining the target assignment. The use of the proposed framework is illustrated in a real-world problem of parametric evaluation of research units. We analyze the outcomes derived with a dedicated outranking-based approach employed by the Polish Ministry of Science and Higher Education and discuss the development plans for some example units assigned to the least preferred class.