Bibliometric indicators versus expert opinion in assessing research performance

Abstract

AbstractThis study compares bibliometric indicators versus expert judgment as indicators of the research performance of major pharmaceutical companies, a context which may be uniquely capable of permitting such a comparison. For each company, a refined composite research output score was calculated based on that company's drug output (1965–1976). These research production scores, normalized by research budget as an indicator of research organizational size, produced an indicator of research productivity, an output/input ratio. The best and most consistent predictors of drug research success in general were the number of clinical articles, and in particular highly cited clinical articles—both their absolute value and their proportionate occurrence among publications. In general, there was a slight but consistent tendency for bibliometric indicators to perform better than expert judgments in predicting research performance as measured by a third and independent indicator of research output. This consistent trend was composed of three aspects. First, the subject‐specific bibliometric indicators correlated more highly with the drug output performance measures than did general bibliometric measures or the expert judgments. Second, when the drug output measures were subjected to a regression analysis, the expert judgment variables were conspicuous by their absence, even in secondary or tertiary positions. Third, the expert judgment variables appeared to be very predictable from the bibliometric measures, while no such converse relationship existed.