Abstract
A system called QUALIS was implemented in Brazil in 2009, intended to rank graduate programs from different subject areas and promote selected national journals. Since this system uses a complicated suit of criteria (differing among subject areas) to group journals into discrete categories, it could potentially create incentives to publish in low-impact journals ranked highly by QUALIS. Here I assess the influence of the QUALIS journal ranking system on the global impact of Brazilian science. Brazil shows a steeper decrease in the number of citations per document since the implementation of this QUALIS system, compared to the top Latin American countries publishing more scientific articles. All subject areas showed some degree of bias, with social sciences being usually more biased than natural sciences. Lastly, the decrease in the number of citations over time proved steeper in a more biased subject area, suggesting a faster shift towards low-impact journals. Overall, the findings documented here suggest that the QUALIS system has undermined the global impact of Brazilian science, and reinforce a recent recommendation from an official committee evaluating graduate programs to eliminate QUALIS. A system based on impact metrics could avoid introducing distorted incentives, and thereby boost the global impact of Brazilian science.
Highlights
In 1998 the Brazilian agency responsible for establishing criteria for evaluating the performance of higher education institutions (CAPES) launched a journal ranking system called “QUALIS”, which classified journals according to their distribution and their quality within subject areas (A, B and C) (Andrade & Galembeck 2009)
I was able to retrieve CiteScore for 9,985 of the journals comprised in QUALIS (36%), and only 3% of the QUALIS journals indexed by Scopus did not contain a CiteScore (Fig.2, see absolute values in Supplementary Material - Table SI)
Results reveal that the QUALIS system, originally intended to rank graduate programs from different subject areas and promote selected national journals, has been unable to increase the relative impact of Brazilian science, compared to other Latin-American countries, since its implementation in 2009
Summary
In 1998 the Brazilian agency responsible for establishing criteria for evaluating the performance of higher education institutions (CAPES) launched a journal ranking system called “QUALIS”, which classified journals according to their distribution (local, national or international) and their quality within subject areas (A, B and C) (Andrade & Galembeck 2009). The QUALIS system has been subject to substantial criticism (da Silva 2009, Andriolo et al 2010, Fernandes & Manchini 2019, Ferreira et al 2013, Rocha-e-Silva 2009b), no systematic cross-subject area assessment has been yet performed to quantify its influence on the global impact of Brazilian science. This is surprising considering the system could create incentives to publish in low-impact journals ranked highly by QUALIS, thereby resulting in a decreased global impact.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
