Abstract
Past performance is a key consideration when rationalising the allocation of grants and other opportunities to individual researchers. The National Research Foundation of South Africa (NRF) has long used a highly structured system of ‘rating’ the past performance of individual researchers. This system relies heavily on peer review, and has seldom been benchmarked against bibliometric measures of research performance such as Hirsch’s h-index. Here I use data for about 600 rated researchers in the biological sciences to evaluate the extent to which outcomes of peer review correspond to bibliometric measures of research performance. The analysis revealed that values of the h-index based on the Scopus database are typically 5–20 for researchers placed in the NRF’s C rating category (‘established’), 20–40 for those in the B rating category (‘considerable international recognition’) and >40 for those in the A rating category (‘leading international scholars’). Despite concerns that citation patterns differ among disciplines, the mean h-index per rating category was remarkably consistent across five different disciplines in the biological sciences, namely animal sciences, plant sciences, ecology, microbiology and biochemistry/genetics. This observation suggests that the NRF rating system is equitable in the sense that the outcomes of peer review are generally consistent with bibliometric measures of research performance across different disciplines in the biological sciences. However, the study did reveal some notable discrepancies which could reflect either bias in the peer-review process or shortcomings in the bibliometric measures, or both.
 Significance:
 
 NRF rating categorisations (estimates of standing in a research field based on peer review) are a reasonably good predictor of the h-index of individual researchers in the biological sciences.
 The relationships between rating categorisations and the h-index are remarkably consistent across five sub-disciplines in the biological sciences.
 Peer review and the h-index were compared in terms of their relative advantages and disadvantages and the combined use of both approaches is advocated for measurement of research performance.
Highlights
Peer review is one of the pillars of the global research enterprise.[1]
Whether of grants or submitted manuscripts, is usually performed without added remuneration and there are limits to the time that researchers are willing to allocate to this process, if it involves an assessment of the entire track record of each individual researcher
The aim of the present study was to establish, across different disciplines of biology, the relations between estimates of the standing of researchers in their field based on peer review and those based on bibliometric analysis of their h-index
Summary
Peer review is one of the pillars of the global research enterprise.[1]. At a political level, national governments often steer overall research activities in particular directions through funding allocations to programmes, but they almost always devolve the final allocation of resources to a system of peer review. One possible way of reducing the burden on peer reviewers (and the institutions that manage the process) is to separate periodic evaluation of the track record of the applicant from the evaluation of proposed research, such that a single measure of the applicant’s track record can be used for multiple decision-making processes This approach has been adopted in South Africa for several decades and takes the form of peer evaluation of the research performance of researchers in the higher education and research sector. This evaluation is usually based on the opinions of about six peers (some suggested by the candidate and some suggested by a panel of disciplinespecific experts) and is codified as a particular ‘rating’ which lasts for a period of 6 years. This information can be used by administrative panels when making grant allocations or for other purposes such as informing a university selection committee about the past performance and general standing of an applicant in a research field
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
