Formation of a spectrum of system-information criteria for the consistency of expert opinions

Abstract

The article is devoted to the development of a system-information methodology for establishing the consistency of generalized expert opinions. After all, in the process of conducting any research in which expert information is revealed, a paradoxical situation may arise when the indicator of a generalized group opinion, usually obtained based on the “group normalization effect”, is actually determined, but there is no individual opinion that would coincide with it. Moreover, in the context of this publication, we are talking about the consistency of group systems of specialists’ advantages. Under the system of benefits, we mean an ordered series of indicators, characteristics of professional activity or objects of expertise: from more acceptable (important, significant, etc.) to less acceptable. And it is clear that the inconsistency of opinions arises precisely when individual preference systems are aggregated into a group one. It is determined that the consistency of group opinions should be checked in the following areas: 1) for each individual alternative, which is ordered. It is advisable to implement the direction after the introduction into practice of expert research — the methodology and the differential method for establishing a part of the total significance of the compared alternatives; 2) coincidence/non-coincidence of individual systems of preferences of group members; 3) integrally — using the Kendall concordance coefficient; 4) complex. Based on the experience of using a multi-step technology for identifying and screening out marginal thoughts, eliminating the “systematic error of the survivor”, as well as building a “reference” group system of advantages, a set of known and new system-information criteria for the consistency of expert opinions is formulated. We are talking about the requirements for: the probability of the Kendall concordance coefficient, tested using the statistical criterion for testing hypotheses “хі-square”, and its minimum acceptable value, which should be equal to W≥0,7; the need for statistically probable matching of all individual preference systems with group thought in the final group preference system; the need to match the individual system of preferences with the majority opinions of group members etc. Moreover, the last two criteria are implemented based on the Spearman rank correlation coefficient, and their reliability is checked using Student’s t-test. Information about the effectiveness of these criteria is provided.