Qualimetric assessment of research and development performance of a higher education institution

Abstract

The study presents an original methodology for assessing the quality of scientific activity of the units of the Military Medical Academy named after S.M. Kirov. As a tool for its solution, we used qualimetric evaluation of all possible areas of scientific activity of departments and criteria that characterize these areas. Qualimetric methods of expert survey and weighting of performance indicators and descriptive statistics were used. Twenty-five experts took part in structuring the scientific performance of the academy. The resulting structure included six main sections, each described by a specific set of attributes. The attributes were both quantitative and qualitative with certain graduations and units of measurement, each of which was assigned a weighting coefficient. The values of the coefficients were expressed in fractions of 1, and the sum was 1. The consistency of the experts proved to be quite high, and the coefficient of concordance was 0.82 (p 0.001). Based on the comprehensive assessment of each division of the academy, a scale score was calculated, which helped in determining the rating of a division. A sigma (using the mean square deviation) estimate was chosen as a scaling tool, which shows that 68% of all complex estimates of the units being evaluated are expected in the range from the arithmetic mean minus sigma to the arithmetic mean plus sigma. That is, units with the value of a complex indicator that falls within this interval are recognized as units with an average assessment of scientific activity. Approximately 16% of units whose value of the complex indicator will be greater than the calculated interval will make up a group of units with scientific activity above the average, and the same number of units whose value of the complex indicator will be less than the calculated interval will make up a group of units with scientific activity as below the average. In general, following the structuring and weighting coefficients, a calculation complex was developed to determine the scientific status of a unit, which was performed using an Excel spreadsheet and can be used, along with the evaluation of educational and methodological activities, in the general evaluation of units.