Abstract
The aim of this paper is to understand whether the probability of receiving positive peer reviews is influenced by having published in an independently assessed, high-ranking journal: we eventually interpret a positive relationship among peer evaluation and journal ranking as evidence that journal ratings are good predictors of article quality. The analysis is based on a large dataset of over 11,500 research articles published in Italy in the period 2004-2010 in the areas of architecture, arts and humanities, history and philosophy, law, sociology and political sciences. These articles received a score by a large number of externally appointed referees in the context of the Italian research assessment exercise (VQR); similarly, journal scores were assigned in a panel-based independent assessment, which involved all academic journals in which Italian scholars have published, carried out under a different procedure. The score of an article is compared with that of the journal it is published in: more specifically, we first estimate an ordered probit model, assessing the probability for a paper of receiving a higher score, the higher the score of the journal; in a second step, we concentrate on the top papers, evaluating the probability of a paper receiving an excellent score having been published in a top-rated journal. In doing so, we control for a number of characteristics of the paper and its author, including the language of publication, the scientific field and its size, the age of the author and the academic status. We add to the literature on journal classification by providing for the first time a large scale test of the robustness of expert-based classification.
Highlights
There is a large degree of agreement on the notion that research assessment in humanities and social sciences (HSS) is made more complex by a variety of factors
The influence of journal classification on the article score We assume that the probability for an article i, published in the journal j, of receiving a score equal to x ∈ {-2; 1} is influenced by the class assigned to the journal, once controlling for a number of characteristics of the article: P (Score = x) = F(Journal class, Paper characteristics ) (1)
Using a very large dataset of journal articles published in HSS, the paper proves that independent classifications of journals may be considered as good predictors of the score assigned to individual articles
Summary
There is a large degree of agreement on the notion that research assessment in humanities and social sciences (HSS) is made more complex by a variety of factors In these fields the structure of academic publication is largely different, with a large weight assigned to books and monographs, and to production in national language (Finkenstaedt, 1990). It is agreed that peer review is still the most important evaluation methodology, so large efforts are made in making it more sophisticated, methodologically controlled, based on sound principles of evaluation methodology in social sciences, and free from unwanted biases, distortions and unexpected side effects. An additional line of work refers to the classification of books and publishers
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
