Traditional and Modern Convenience Samples: An Investigation of College Student, Mechanical Turk, and Mechanical Turk College Student Samples

Abstract

Two of the most popular populations for convenience sampling used in the psychological sciences are college students and Mechanical Turk (MTurk) workers. College students represent a traditional type of convenience sample, whereas MTurk workers provide a more modern source of data. However, little research has examined how these populations differ from each other in salient characteristics. Additionally, no research to date has investigated how MTurk college students (a traditional sample collected using modern methods) compare to either population. The current study examined 1,248 participants comprising three samples: MTurk noncollege workers ( n = 533), MTurk college students ( n = 385), and traditional college students ( n = 330). We compared the samples on demographic characteristics, study completion time, attention, and individual difference variables (i.e., personality, social desirability, need for cognition, personal values, and social attitudes). We examined the individual difference variables in terms of mean responses, internal consistency estimates, and subscale intercorrelations. Results indicated the samples were distinct from each other in terms of all variables assessed; in addition, adding demographic characteristics as covariates to the analyses of individual difference variables did not effectively account for sample differences. We conclude that research using convenience samples should take these differences into account.