Don't judge a journal by its cover?: Appearance of a Journal's website as predictor of blacklisted Open‐Access status

Abstract

AbstractThe nature of scientific research has motived an open‐access model of publication supported by article processing fees. Under this rapidly evolving environment and financial incentives, some dubious venues would publish almost anything—for a fee. Many entities keep track of the standards of these new journals, “blacklisting” those deemed problematic. Anecdotal evidence suggests that blacklisted journals tend to have websites with subpar appearance (e.g., old web technologies, unprofessional design). In this work, we systematically explore whether this anecdotal evidence is true. In particular, we evaluate the websites of journals whitelisted and un‐whitelisted by the Directory of Open Access Journals (DOAJ). We use a convolutional neural network to predict whether a journal is whitelisted based on a screenshot of its website and analyze the factors that predict one output vs. the other. Our results show that appearance is indeed a predictive factor, achieving a medium performance (AUC of 0.736). Further, our interpretation suggests that the network considers whitelisting those websites with a table of content, social media links and packed content. Conversely, our model mistakenly whitelists blacklisted journals hosted by Elsevier and blacklists whitelisted websites with serif fonts and non‐Latin characters.