Abstract
Each year, crowdsourcing organizational research grows increasingly popular. However, this source of sampling receives much scrutiny focused on data quality and related research methods. Specific to the present research, survey attentiveness poses a unique dilemma. Research on updated conceptualizations of attentiveness--insufficient effort responding (IER)--shows that it carries substantial concerns for data quality beyond random noise, which further warrants deleting inattentive participants. However, personal characteristics predict IER, so deleting data may cause sampling bias. Therefore, preventing IER becomes paramount, but research seems to ignore whether IER prevention itself may create systematic error. This study examines the detection and prevention of IER in Amazon's Mechanical Turk (Mturk) by evaluating three IER detection methods pertinent to concerns of attentiveness on the platform and using two, promising, IER prevention approaches--Mturk screening features and IER preventive warning messages. I further consider how these issues relate to organizational research and answer the call for a more nuanced understanding of the Mturk population by focusing on psychological phenomena often studied/measured in organizational literature--the congruency effect and approach-avoidance motivational theories, Big Five personality, positive and negative affectivity, and core self-evaluations. I collected survey data from screened and non-screened samples and manipulated warning messages using four conditions--no warning, gain-framed, loss-framed, and combined-framed messages. I used logistic regression to compare the prevalence of IER across conditions and the effectiveness of warning messages given positively or negatively valenced motivational tendencies. I also used 4x2 factorial ANCOVAs to test for differences in personal characteristics across conditions. The sample consisted of 1071 Mturk workers (turkers). Results revealed differences in IER prevalence among detection methods and between prevention conditions, counter-intuitive results for congruency effects and motivational theories, and differences across conditions for agreeableness, conscientiousness, and positive and negative affectivity. Implications, future research, and recommendations are discussed.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.