Transforming Academic Journal Assessment from “Quality” to “Impact”: A Case Study of the SDG Impact Intensity Academic Journal Rating Artificial Intelligence System

Abstract

This chapter details how artificial intelligence (AI) can be used—and is very much needed—to reinvent academic research evaluation in terms of quality and impact. First, we review current literature on academic research assessment examining two overarching questions: What normative standards are employed to determine the quality of academic research? What explicit methodological rationales, unstated assumptions, and academic cultural dynamics determine the definition of quality? Second, we offer a literature review and alternative model for assessing the quality of academic research in terms of impact—that is, does academic research make a positive difference in the material conditions of humanity and sustain the natural environment, and how does academic research praxis transform “words into worlds”? Third, we examine how impact is being adopted by the publishing industry in a paradigm shift from limited notions of “quality” to the broader consequences of societal and sustainable “impact.” We critique how the Journal Impact Factor, as a gold standard of academic publication value, has eclipsed other possibilities for assessing real-world impacts of academic outputs: quantification without qualification. Fourth, we present a case study applying an AI rating system to ascertain degrees of impact for academic journals—SDG Impact Intensity. The AI system utilizes the United Nations 17 Sustainable Development Goals (SDGs) criteria as normative standards to rate the impact of academic journals. We discuss the implications of such a system and SDG-focused research on business schools and scholarly outputs for academia. We critically consider the challenges of employing our algorithmic technique as a digital transformation technology for responsible management education, and ultimately to serve the Common Good and environmental sustainability.