Methodological support of the working group in predicting the results of the classification expertise

Abstract

The paper considers the specifics of the working group activity in preparation of expertise classification and the features of approaches for the choice of expert assessments and expert selection methods. The analysis focuses on potentially weak (illegibly) formalized customer requirements, which are quite typical for expertise classification. The used methods suffer from many drawbacks, which make them practically inapplicable in terms of planning, preparing and predicting the possible expertise classification reliability. The authors developed a new approach for predicting the reliability of the expertise classification at the stage of its preparation that depends on the reliability of the proposed expert group. The approach involves a probabilistic representation of the possible results of the work (namely, classification) of particular experts. The authors propose a number of probabilistic models (probabilistic matrices), which reflect the reliability (correctness) of the classification of certain objects both at the level of particular experts and at the level of entire expertise results. The set of procedures developed for a random group of experts allows obtaining probabilistic characteristics of the objects classification correctness when the group works as a part of an expert commission. The proposed approach can be used as a tool for working groups, which not only simplifies the process of an expert group selection, but also allows predicting the reliability of possible results and thereby makes it possible to take measures in advance in order to meet customer requirements. This approach can serve as a methodological basis for automating the problem solution in the expert selection process for the expertise classification at the level of its preparation, depending on customer requirements (restrictions). The proposed models and procedures will improve the efficiency of expertise classification, as well as save time for its preparation.