Is it all bafflegab? – Linguistic and meta characteristics of research articles in prestigious economics journals

Abstract

• This study investigates whether linguistic and meta characteristics of journal articles in economics are associated with scientific prestige. • Prestige is operationalized as the normalized SCImago Journal Rank (SJR) of the journal that the article was published in. • The data set of over 200,000 articles is segmented into topically homogeneous clusters using an LDA topic model. • Beta regression models are estimated using componentwise gradient boosting to determine the most important correlates of scientific prestige. • In summary, several linguistic and meta features are found to be associated with scientific prestige consistently across clusters. In competitive research environments, scholars have a natural interest to maximize the prestige associated with their scientific work. In order to identify factors that might help them address this goal more effectively, the scientometric literature has tried to link linguistic and meta characteristics of academic papers to the associated degree of scientific prestige, conceptualized as cumulative citation counts. In this paper, we take an alternative approach that instead understands scientific prestige in terms of the rankings of the journals that the articles appeared in, as such rankings are routinely used as surrogate research quality indicators. For the purpose of determining the most important drivers of suchlike prestige, we use state-of-the-art text mining tools to extract 344 interpretable features from a large corpus of over 200,000 journal articles in economics. We then estimate beta regression models to investigate the relationship between these predictors and a cross-sectionally standardized version of SCImago Journal Rank (SJR) in multiple topically homogeneous clusters. In so doing, we also reinvestigate the bafflegab theory , according to which more prestigious research papers tend to be less readable, in a methodologically novel way. Our results show the consistently most informative predictors to be associated with the length of the paper, the span of coreference chains in its full text, the deployment of a personal and moderately informal writing style, the “density” of the article in terms of sentences per page, international and institutional collaboration in research teams and the references cited in the paper. Moreover, we identify various linguistic intricacies that matter in the association between readability and scientific prestige, which suggest this relationship to be more complicated than previously assumed.