The Citation Impact of German Sociology Journals: Some Problems with the Use of Scientometric Indicators in Journal and Research Evaluations

Abstract

We analyze the citation impacts (‘cited’) and citation practices (‘citing’) of sociological journals which publish in German, and discuss drawbacks of using the journal impact factor (IF) to assess the quality of these journals. Whereas journal impact factors measure short-term (2-year) impacts at research fronts, sociological literature moves slowly in terms of citations: the median age of citations to sociology journals is mostly longer than ten years. Citation in a field without a research front may indicate codification with a different dynamics. For example, Merton’s (1968) Matthew Effect is frequently cited by network analysts as an example of preferential attachment. Secondly, circulation can be impeded due to a language barrier. Thirdly, citation distributions are heavily skewed because of the positive loops (preferential attachment). Given this skewness one should not use quasi-averages (such as the IF). Using non-parametric statistics (e.g., percentiles), however, requires delineation of field-specific reference sets. What would be a relevant reference set in the case we examine: other sociological journals or other social-science journals in German? The German sociology journals are cited by the latter, but are citing the former as the knowledge bases. Fourth, the attribution of journal measures to individual paper implies an ecological fallacy: inferences about the nature of single records (here: papers) are deduced from inferences for the groups to which these records belong (here: the journal where the papers were published). Finally, algorithmic constructs (e.g., the various rankings) cannot be used for policy or management purposes without validation or specification of statistical error.