Examining the effect of mid-test warnings on factor structure of personality scores: A field experiment

Abstract

Warning messages delivered in the middle of a personality test (i.e., mid-test warnings) have emerged as a promising method to mitigate applicant faking in terms of lowering personality mean scores. However, little is known about whether mid-test warnings might also affect personality factor structure. The present study investigated this prominent issue through a field experiment. Participants were applicants for graduate programs at a large public university in China, who were instructed to complete an online Big-Five personality test as part of the selection procedure. During the personality test, three messages (control, mid-test warning with accusation, and mid-test warning without accusation) were randomly delivered to applicants. Our analysis focused on the ten personality subscales included in both the initial block (before the mid-test messages) and the main block (after the mid-test messages). Results indicated that (a) the control message did not lead to the violation of configural, metric, and scalar invariance, and (b) the two mid-test warnings did not result in the violation of configural and metric invariance but led to scalar non-invariance. These findings suggest that mid-test warnings reduce faking by lowering intercepts, but do not adversely affect the configural and metric aspects of personality factor structure.