Handling the Inconsistency between Self-Report and the Actual Behavior: Validity of Excluding Survey Participants with Insufficient Effort Responding

Abstract

In this study, we aimed to understand and reduce the difference between self-report in a survey and the actual behavior. Thus, we investigated whether such a difference was caused by participants who engaged in insufficient effort responding (IER), which has been receiving increasing research attention. We collected and analyzed data of actual and self-reported smartphone game usage from behavior logs and survey responses including the items associated with the IER scale, respectively. The results confirmed a strong tendency of overreporting and low correlations between the behavior log and survey responses for IER participants. Although the distributions of survey responses differed between IER participants and others, those of the behavior log did not. In conclusion, when IER participants are excluded, the difference between the behavior log and survey responses reduces, but the distribution of the actual behavior remains the same without selection bias.