Research fund evaluation based on academic publication output analysis: the case of Chinese research fund evaluation

Abstract

Government funding is a key scientific research resource, and it has made a concrete contribution to the world’s science and technology development. But these funds come from common taxpayers, so we need to evaluate the effectiveness of these funds. Generally speaking, policymakers adopt the method of peer review to make assessments. Compared to kinds of shortcomings of peer review, the paper here proposed the benchmarking evaluation method based on the academic publication outputs of supporting funds, mainly guiding indicators from scientometrics. At first, with the academic publication output extracted from the concluding report project manager submitted to the government after the fund finished, we designed the analysis framework to search and define the research field the fund belonged to. And then from the following three perspectives, including quantity, quality and relative influence, we compared the research fund output to the field output. Later, we took one fund program from national program on key basic research project of China (973 Program) in the field of quantum physics as an example to make an empirical analysis to demonstrate its effectiveness. At last, we found that the funded program performance was superior to the field, and even about 11.65% of the research achievement reaches the top 1/1000 of the world, but the research was lacking in remarkable papers, so it needs further improvement.