From alpha to omega and beyond! A look at the past, present, and (possible) future of psychometric soundness in the Journal of Applied Psychology.

Abstract

The psychometric soundness of measures has been a central concern of articles published in the Journal of Applied Psychology (JAP) since the inception of the journal. At the same time, it isn't clear that investigators and reviewers prioritize psychometric soundness to a degree that would allow one to have sufficient confidence in conclusions regarding constructs. The purposes of the present article are to (a) examine current scale development and evaluation practices in JAP; (b) compare these practices to recommended practices, previous practices, and practices in other journals; and (c) use these comparisons to make recommendations for reviewers, editors, and investigators regarding the creation and evaluation of measures including Excel-based calculators for various indices. Finally, given that model complexity appears to have increased the need for short scales, we offer a user-friendly R Shiny app (https://orgscience.uncc.edu/about-us/resources) that identifies the subset of items that maximize a variety of psychometric criteria rather than merely maximizing alpha. (PsycInfo Database Record (c) 2020 APA, all rights reserved).