The Use of Mobile Devices in High‐stakes Remotely Delivered Assessments and Testing

Abstract

With Internet access no longer restricted to desktop and laptop computers, job applicants now have the opportunity to complete remotely delivered assessments on mobile, handheld small screen devices such as smartphones, and personal digital assistants. In this study, a large dataset is used to investigate demographic and score differences between job applicants who completed a remotely delivered high‐stakes assessment on a mobile device and those who completed it on a nonmobile device. Based on a sample of 3,575,207 job applicants who completed an unproctored Internet‐based assessment between January 2011 and April 2012, the percentage of applicants completing the assessment on a mobile device was small, 1.93%, but nevertheless represented more than 69,000 people. Overall, there were small test‐taker demographic differences in the use of mobile devices versus nonmobile devices in that mobile devices were slightly more likely to be used by women, African‐Americans and Hispanics, and younger applicants. Scores on a personality measure were similar for mobile and nonmobile devices but scores on a general mental ability test were substantially lower for mobile devices. Tests of measurement invariance also indicated equivalence across the mobile and nonmobile samples. Test taker and organizational implications for completing remotely delivered high‐stakes noncognitive and cognitive assessments on mobile versus nonmobile devices are discussed.