A nuanced view of the extent to which samples from narrow populations are scientifically problematic.

Abstract

Psychologists have a traditional concern with participant samples from narrow populations and deleterious effects on researchers' ability to generalize findings. Recently, both individuals and authoritative organizations, such as the American Psychological Association, have merged this external validity concern with diversity and inclusion concerns. The American Psychological Association directive for researchers to include diverse samples seems obviously well-taken as it purports to mitigate these problems at once; it simultaneously increases external validity and promotes diversity and inclusion. However, we show that there are complications. These include problems with internal and external validity conceptualizations; that sometimes generalization failures can support, rather than detract from, external validity; the crucial role auxiliary assumptions play in impacting internal and external validity; Lakatosian degenerative science and its problematic application; and distinguishing between merely including diverse groups in research samples versus analyzing for group differences. These complications imply a nuanced perspective of whether samples from narrow populations are undesirable. That a sample is from a narrow population might, or might not, preclude strong support or disconfirmation for the theory, including its ability to generalize. Our nuanced perspective militates against the current trend of journal directives to require diverse samples. Sample suitability for particular researcher goals should be judged on a case-by-case basis that takes into account that sometimes samples from narrow populations can nevertheless engender impressive scientific progress and sometimes not. (PsycInfo Database Record (c) 2024 APA, all rights reserved).