MTurk Participants Research Articles

Dementia-related anxiety and coping styles associated with suicidal ideation

Objective: Dementia-related anxiety is associated with general anxiety and depression, which are risk factors for suicidal ideation. Consequently, dementia-related anxiety may be associated with suicidal ideation. When faced with a negative event (e.g. developing dementia), individuals primarily employ monitoring or blunting styles of coping. The present study investigated whether dementia-related anxiety and coping styles were associated with suicidal ideation in adults and whether coping styles moderated the relation between dementia-related anxiety and suicidal ideation.Methods: Online Mturk participants (n = 330) completed the Miller Behavioral Style Scale, Dementia Worry Scale, and Patient Health Questionnaire-9. Suicidal ideation was assessed with the Patient Health Questionnaire.Results: Dementia-related anxiety and monitor/blunter coping styles were independently associated with suicidal ideation in a multiple logistic regression; coping styles did not moderate the relation between dementia-related anxiety and suicidal ideation.Discussion: Greater dementia-related anxiety and less use of monitoring coping style were associated with suicidal ideation. Interventions to decrease suicide risk may benefit from taking into account individual differences in dementia-related anxiety and coping styles.

Threat, trust, and Trump: identity and voting in the 2016 presidential election

ABSTRACTUsing longitudinal data from 421 MTurk participants, collected over 6 months surrounding the 2016 election between Hillary Clinton and Donald Trump, we investigated the relationships among identity centrality, institutional trust, and vote. Specifically, we were interested in whether trust in science and the media mediated the relationship between the centrality of racial, gender, and class identities, and how the importance of these identities was related to participants’ votes. Results indicated that trust in these institutions did mediate the relationship between centrality and vote in different ways depending on level of privilege, identity domain, and type of trust. For instance, class centrality among working-class people predicted greater trust in the media, which predicted voting for Hillary Clinton, while race centrality among White people predicted less trust in science, which predicted voting for Trump.

Profiling public and private self-consciousness on self-presentation tactic use

It seems generally accepted that public self-consciousness (PuSC) relates to increased use of self-presentation and private self-consciousness (PrSC) does not, but some researchers believe that PuSC and PrSC relate to different approaches to self-presentation. The present study aimed to profile PuSC and PrSC on self-presentation tactic use but advance previous investigations by differentiating PrSC dimensions: internal state awareness (ISA) and self-reflection (SR). We hypothesized that ISA and SR would manifest divergent self-presentation profiles and that neither profile would correspond closely with PuSC. MTurk participants (N = 298; Mage = 37.66; 61% female) first completed indices of PuSC, ISA, and SR, and then reported their past use of various self-presentation tactics. ISA and SR manifested divergent self-presentation profiles, and neither profile converged with PuSC. Specifically, ISA revealed a self-presentation profile that generally conveyed a principled and socially-potent identity; SR revealed a profile that conveyed an exploitative and opportunistic approach to identity construction; PuSC related specifically to ingratiation. In sum, PrSC is associated with self-presentation, but its dimensions reflect divergent self-presentational approaches. PuSC largely relates to tactical ingratiation and does not appear to represent a broad self-presentational orientation.

Psychometric Properties of the Multidimensional Schizotypy Scale and Multidimensional Schizotypy Scale–Brief: Item and Scale Test–Retest Reliability and Concordance of Original and Brief Forms

We conducted 2 studies examining the psychometric properties of the Multidimensional Schizotypy Scale (MSS) and the Multidimensional Schizotypy Scale–Brief (MSS–B). These studies offered the first examination of the scales’ test–retest reliability (dependability), impact of testing interval on test–retest reliability, and concordance of the 2 versions. The MSS and MSS–B contain positive, negative, and disorganized schizotypy subscales that reflect current models of schizotypy. In Study 1, MTurk participants (n = 245) completed the scales at 2 assessments across a 7-week time interval. Test–retest reliabilities (intraclass correlations) ranged from .84 to .90 for the MSS subscales and from .77 to .85 for the MSS–B subscales, and were unaffected by the interval length between administrations. The association between the same subscales of the MSS and MSS–B across the 2 assessments ranged from .79 to .87, indicating good correspondence between the MSS and MSS–B. Scores on the MSS–B subscales in Study 1 were derived from the full-length MSS. Therefore, we replicated the findings using the actual MSS–B subscales in Study 2 in 3 samples (total n = 339). The results indicated that the MSS and MSS–B have good internal consistency and test–retest reliabilities (dependability), and that there is high concordance between the full-length and brief versions.

Why Do People Experience the Fear of Missing Out (FoMO)? Exposing the Link Between the Self and the FoMO Through Self-Construal

With the advent of social media, it is getting easier to be informed about different aspects of individuals’ lives. Therefore, experiencing fear of missing out (FoMO), which is characterized by the desire to stay continually connected with others, is becoming increasingly widespread among people. The concept of FoMO has drawn considerable attention from researchers; however, past research has yet to expose the link between the self-concept and FoMO. Drawing on the self-construal theory, we argue that the construal of the interdependent self underlies FoMO. In other words, the current article proposes that people with interdependent self-construal are more vulnerable to FoMO. In Study 1, we conducted online survey research with 566 MTurk participants, 287 U.S. participants and 279 Indian participants. In Study 2, we conducted an experiment with two conditions with 188 MTurk participants residing in the United States. The findings from the two studies provide consistent and convergent findings that FoMO is positively associated with interdependent self-construal. Implications are discussed and further research suggestions are provided.

MTurk participants have substantially lower evaluative subjective well-being than other survey participants

Amazon's MTurk platform has become a popular site for obtaining relatively inexpensive and convenient adult samples for use in behavioral research. Concerns have been raised about selection issues, because MTurk workers chose to participate in the platform and select the tasks they perform (of many offered to them). Prior studies have documented demographic and psychological differences with national samples. In this paper we studied evaluative subjective well-being (the Cantril Ladder) in an MTurk sample, a national Internet panel sample, and a national telephone survey conducted by Gallup-Sharecare. A surprising finding was that MTurk participants' Ladder scores were substantial lower than the other two samples. Analyses controlling for six demographic differences among the samples only slightly reduced the mean differences. However, patterns of demographic—well-being associations were similar within the samples. To corroborate these results, we conducted a secondary analysis on another three samples, one MTurk sample and two Internet panel samples. The same group differences in Ladder scores were observed. These findings add to the growing literature documenting the characteristics of MTurk samples and we discuss the implications for future research with such samples.

Intentional memory and online data collection: A test of the effects of animacy and threat on episodic memory

ABSTRACTAs research in the behavioural sciences increasingly relies on Amazon’s Mechanical Turk (MTurk) for recruiting participants, it is an empirical question whether an intentional memory experiment could be conducted online and whether performance of MTurk participants and college students would be comparable. College students and MTurk participants viewed threatening and nonthreatening items that were animate or inanimate and completed a recall test. Recent research indicates that human memory systems evolved to prioritise survival-relevant information, including animacy of items. The results of both studies indicated that participants recalled more animate than inanimate items and recalled more threatening items than nonthreatening items. The results indicate that it is possible to conduct memory experiments online and that results match patterns found in laboratory studies of the animacy effect (Nairne, VanArsdall, Pandeirada, Cogdill, & LeBreton, 2013), but that researchers should include safeguards to ensure the online participants are attending to and following directions.

Crowdsourcing Intelligent Research Participants: A Student versus MTurk Comparison

ABSTRACTThe use of online workers as research participants has grown in recent years, prompting interest in how online workers compare to traditional accounting research participants. To date, no study has compared the relative intelligence of online workers to student subjects. Such a comparison may be important to behavioral accounting researchers given the homogeneity of accounting students relative to online subject pools and given prior research suggesting accounting students have relatively high analytic ability. While graduate students outperform both undergraduate and MTurk participants on common intelligence tests and across two reasonably complex tasks, we find that MTurk participants perform similarly to undergraduate accounting students. Overall, our results provide incremental assurance that MTurk participants are suitable subjects when accounting expertise is not explicitly required. We also provide evidence that screening MTurk workers on intelligence scores may benefit researchers who require participants with relatively high cognitive ability.

Prevalence of and Factors Associated With Patient Nondisclosure of Medically Relevant Information to Clinicians

Patient failure to disclose medically relevant information to clinicians can undermine patient care or even lead to patient harm. To examine the frequency of patients failing to disclose to their clinicians information that is relevant to their care and their reasons for doing so. Two national nonprobability samples were recruited to participate in an online survey, one using Amazon's Mechanical Turk (MTurk) from March 16 to 30, 2015 (2096 respondents), followed by one using Survey Sampling International (SSI) from November 6 to 17, 2015 (3011 respondents). Data analysis was conducted from September 28 to October 8, 2018. After dropping respondents meeting the exclusion criteria, the final sample sizes were 2011 (MTurk) and 2499 (SSI). The primary outcome measures were self-reported nondisclosure of 7 types of information to their clinician (eg, did not understand instructions, medication use) and reasons for nondisclosure (eg, embarrassment, not wanting to be judged). There was a total of 4510 overall respondents. Of 2096 respondents, 2013 completed the MTurk survey (96.0% completion rate) and 2011 were included in the analysis. Of 3011 respondents, 2685 completed the SSI survey (89.2% completion rate) and 2499 were included in the analysis. The mean (SD) age of the participants was 36 (12.4) years for MTurk and 61 (7.59) years for SSI. Both samples were predominantly white (MTurk: 1696 [84.3%]; SSI: 1968 [78.8%]). A total of 1630 MTurk participants (81.1%) and 1535 SSI participants (61.4%) avoided disclosing at least 1 type of information. Disagreeing with the clinician's recommendation (MTurk: 918 of 2010 respondents [45.7%]; SSI: 785 of 2497 respondents [31.4%]) and not understanding the clinician's instructions (MTurk: 638 of 2009 respondents [31.8%]; SSI: 607 of 2497 respondents [24.3%]) were the most common occurrences. The most commonly reported reasons for nondisclosure included not wanting to be judged or lectured (MTurk: 81.8% [95% CI, 79.8%-83.9%]; SSI: 64.1% [95% CI, 61.5%-66.7%]), not wanting to hear how harmful the behavior is (MTurk: 75.7% [95% CI, 73.5%-78.0%]; SSI: 61.1% [95% CI, 58.5%-63.8%]), and being embarrassed (MTurk: 60.9% [95% CI, 58.9%-62.9%]; SSI: 49.9% [95% CI, 47.8%-52.1%]). In both samples, participants who were women (MTurk: odds ratio [OR], 1.88 [95% CI, 1.49-2.37]; SSI: OR, 1.38 [95% CI, 1.17-1.64]), younger (MTurk: OR, 0.98 [95% CI, 0.97-0.99]; SSI: OR, 0.98 [95% CI, 0.97-0.99]), and with worse self-rated health (MTurk: OR, 0.87 [95% CI, 0.76-0.99]; SSI: OR, 0.80 [95% CI, 0.72-0.88]) were more likely to report withholding information. Many respondents in these surveys intentionally withhold important information from their clinicians and were most likely to do so when they disagreed with or misunderstood their clinician's instructions. A better understanding of how to increase patients' comfort with reporting this information may improve the clinician-patient relationship and patient care.

Investors' Responses to Reported Earnings when Management Issues Goal versus Expectation Earnings Guidance: An Experimental Investigation

ABSTRACTWe investigate whether nonprofessional investors' responses to a company's reported earnings differ when management earnings guidance is presented as a goal or an expectation. We present 64 M.B.A. students and 262 MTurk participants with earnings guidance, manipulating between subjects whether management provides the guidance as a “goal” or an “expectation” and whether the company's reported earnings fall short or exceed investors' expectations as derived from management's earnings guidance. Our experimental results suggest that if earnings guidance is issued as a goal rather than as an expectation, investors respond less negatively when earnings fall short of investors' expectations, but not less positively when earnings exceed investors' expectations. Mediation analysis supports the interpretation that earnings falling short of investors' expectations leads investors to perceive managers as less competent and to be more disappointed when managers issue expectation rather than goal guidance, which in turn influences investors' attractiveness judgments of the company.

Stigmatizing attitudes toward persons with psoriasis among laypersons and medical students

Perceived stigma among patients with psoriasis contributes to poor quality of life. To determine the prevalence and predictors of stigmatizing attitudes toward persons with psoriasis among laypersons and medical trainees. Laypersons were recruited from Amazon.com's MechanicalTurk (MTurk) (n=198). Additionally, 187 medical students were recruited via e-mail. Participants completed an online survey in which they viewed images of persons with visible psoriasis. Participants reported their desire to socially avoid the persons in the images, their emotional responses to the persons in the images, and their endorsement of psoriasis-related stereotypes and myths. MTurk participants endorsed social avoidance items such as not wanting to shake hands with (39.4%) or have the persons in the images in their home (32.3%). Participants stereotyped persons with psoriasis as contagious (27.3%) and endorsed the myth that psoriasis is not a serious disease (26.8%). Linear regression analyses showed that having heard of or knowing someone with psoriasis predicted fewer stigmatizing attitudes (P<.05). The medical students reported less stigmatizing attitudes than the MTurk participants (P<.01). Self-report, single-institution study. Stigmatizing views of persons with psoriasis are prevalent among people in the United States. Educational campaigns for the public and medical trainees may reduce stigma toward persons with psoriasis.

How many hours does it take to make a friend?

The question of this investigation is, how many hours does it take to make a new friend? Drawing from Dunbar’s social brain hypothesis and Communicate Bond Belong theory, friendship status was examined as a function of hours together, shared activities, and everyday talk. In Study 1, MTurk participants ( N = 355) who had recently relocated estimated time spent with a new acquaintance. Hours together was associated with closer friendships. Time spent engaging in leisure activities also predicted closeness. In Study 2, first-year students ( N = 112) reported the number of hours spent with two new acquaintances three times over 9 weeks. Hours together was associated changes in closeness between waves. Two types of everyday talk predicted changes in closeness.

Values of Participants in Behavioral Accounting Research: A Comparison of the M-Turk Population to a Nationally Representative Sample

ABSTRACTThe external validity of conclusions from behavioral accounting experiments is in part dependent upon the representativeness of the sample compared to the population of interest. Researchers are beginning to leverage the availability of workers via online labor markets, such as Amazon's Mechanical Turk (M-Turk), as proxies for the general population (e.g., investors, jurors, and taxpayers). Using over 200 values-based items from the World Values Survey (WVS), the purpose of the current study is to explore whether U.S. M-Turk workers' values are similar to those of the U.S. population. Results show for the majority of items collected, M-Turk participants' values are significantly different from the WVS participants (e.g., values related to trust, ethics, religious beliefs, and politics). We present select items and themes representing values shown to influence judgments in prior research and discuss how those values may affect inferences of behavioral accounting researchers.Data Availability: Data are available from the authors upon request.

Using Online Labor Market Participants for Nonprofessional Investor Research: A Comparison of MTurk and Qualtrics Samples

ABSTRACTRecently, researchers have begun using online labor markets to recruit participants for experimental studies examining the judgments and decisions of nonprofessional investors. This study investigates the quality and generalizability of data collected from these sources by replicating an experimental task from Elliott, Hodge, Kennedy, and Pronk (2007) using nonprofessional investor participants from two popular online labor markets—Amazon's Mechanical Turk (MTurk) and Qualtrics Online Sample (Qualtrics). Compared to Qualtrics participants, we find that MTurk participants pay greater attention to the experimental materials and better acquire and recall information. Further, the MTurk sample more closely replicates EHKP's investment club member results on measures of information integration than does the Qualtrics sample. These results provide some evidence that many interesting research questions can be satisfactorily answered using nonprofessional investor participants from MTurk. We believe further investigation is needed before Qualtrics can be endorsed as a high-quality source of nonprofessional investor participants.

A self-report measure for the ICD-11 dimensional trait model proposal: The personality inventory for ICD-11.

Proposed for the 11th edition of the World Health Organization's International Classification of Diseases (ICD-11) is a dimensional trait model for the classification of personality disorder (Tyrer, Reed, & Crawford, 2015). The ICD-11 proposal consists of 5 broad domains: negative affective, detachment, dissocial, disinhibition, and anankastic (Mulder, Horwood, Tyrer, Carter, & Joyce, 2016). Several field trials have examined this proposal, yet none has included a direct measure of the trait model. The purpose of the current study was to develop and provide initial validation for the Personality Inventory for ICD-11 (PiCD), a self-report measure of this proposed 5-domain maladaptive trait model. Item selection and scale construction proceeded through 3 initial data collections assessing potential item performance. Two subsequent studies were conducted for scale validation. In Study 1, the PiCD was evaluated in a sample of 259 MTurk participants (who were or had been receiving mental health treatment) with respect to 2 measures of general personality structure: The Eysenck Personality Questionnaire-Revised and the 5-Dimensional Personality Test. In Study 2, the PiCD was evaluated in an additional sample of 285 participants with respect to 2 measures of maladaptive personality traits: The Personality Inventory for DSM-5 and the Computerized Adaptive Test for Personality Disorders. Study 3 provides an item-level exploratory structural equation model with the combined samples from Studies 1 and 2. The results are discussed with respect to the validity of the measure and the potential benefits for future research in having a direct, self-report measure of the ICD-11 trait proposal. (PsycINFO Database Record

A Technical Guide to Using Amazon's Mechanical Turk in Behavioral Accounting Research

ABSTRACT Multiple social science researchers claim that online data collection, mainly via Amazon's Mechanical Turk (MTurk), has revolutionized the behavioral sciences (Gureckis et al. 2016; Litman, Robinson, and Abberbock 2017). While MTurk-based research has grown exponentially in recent years (Chandler and Shapiro 2016), reasonable concerns have been raised about online research participants' ability to proxy for traditional research participants (Chandler, Mueller, and Paolacci 2014). This paper reviews recent MTurk research and provides further guidance for recruiting samples of MTurk participants from populations of interest to behavioral accounting researchers. First, we provide guidance on the logistics of using MTurk and discuss the potential benefits offered by TurkPrime, a third-party service provider. Second, we discuss ways to overcome challenges related to targeted participant recruiting in an online environment. Finally, we offer suggestions for disclosures that authors may provide about their efforts to attract participants and analyze responses.

When Talking Differences Makes Us Closer: Communicative Dynamics of Group Salience in Intergroup Contact

Group salience, although a key variable in generalizing contact effects toward the target out-group, often negatively affects the quality of intergroup contact. Countering this pessimistic view, we investigated whether or not communicatively-induced group salience could yield positive contact effects by inducing prosocial intergroup emotions and by increasing relational closeness with the out-group interactant in the context of cross-group friendships. A survey conducted with MTurk participants (N = 331) revealed that communicating group-related topics within intergroup friendships increased prosocial intergroup emotions, which was further predictive of relational closeness and intergroup prejudice. However, relational closeness showed no evidence of mediating the effect of prosocial emotions on intergroup prejudice. The results contribute to intergroup scholarship in expanding the discussion of the influence of communicative dimensions of salience on intergroup relations.

Assessing Diabetes-Relevant Data Provided by Undergraduate and Crowdsourced Web-Based Survey Participants for Honesty and Accuracy.

BackgroundTo eliminate health disparities, research will depend on our ability to reach select groups of people (eg, samples of a particular racial or ethnic group with a particular disease); unfortunately, researchers often experience difficulty obtaining high-quality data from samples of sufficient size. ObjectivePast studies utilizing MTurk applaud its diversity, so our initial objective was to capitalize on MTurk’s diversity to investigate psychosocial factors related to diabetes self-care.MethodsIn Study 1, a “Health Survey” was posted on MTurk to examine diabetes-relevant psychosocial factors. The survey was restricted to individuals who were 18 years of age or older with diabetes. Detection of irregularities in the data, however, prompted an evaluation of the quality of MTurk health-relevant data. This ultimately led to Study 2, which utilized an alert statement to improve conscientious behavior, or the likelihood that participants would be thorough and diligent in their responses. Trap questions were also embedded to assess conscientious behavior.ResultsIn Study 1, of 4165 responses, 1246 were generated from 533 unique IP addresses completing the survey multiple times within close temporal proximity. Ultimately, only 252 responses were found to be acceptable. Further analyses indicated additional quality concerns with this subsample. In Study 2, as compared with the MTurk sample (N=316), the undergraduate sample (N=300) included more females, and fewer individuals who were married. The samples did not differ with respect to race. Although the presence of an alert resulted in fewer trap failures (mean=0.07) than when no alert was present (mean=0.11), this difference failed to reach significance: F1,604=2.5, P=.11, ƞ²=.004, power=.35. The modal trap failure response was zero, while the mean was 0.092 (SD=0.32). There were a total of 60 trap failures in a context where the potential could have exceeded 16,000.ConclusionsPublished studies that utilize MTurk participants are rapidly appearing in the health domain. While MTurk may have the potential to be more diverse than an undergraduate sample, our efforts did not meet the criteria for what would constitute a diverse sample in and of itself. Because some researchers have experienced successful data collection on MTurk, while others report disastrous results, Kees et al recently identified that one essential area of research is of the types and magnitude of cheating behavior occurring on Web-based platforms. The present studies can contribute to this dialogue, and alternately provide evidence of disaster and success. Moving forward, it is recommended that researchers employ best practices in survey design and deliberately embed trap questions to assess participant behavior. We would strongly suggest that standards be in place for publishing the results of Web-based surveys—standards that protect against publication unless there are suitable quality assurance tests built into the survey design, distribution, and analysis.

Investorss Reactions to Earnings Surprises When Management Issues Goal versus Expectation Earnings Guidance

We investigate whether nonprofessional investors’ responses to a company’s reported earnings differ when management earnings guidance is presented as a goal or an expectation. We present 64 MBA students and 262 Mturk participants with management earnings guidance, manipulating between-subjects whether management provides the guidance in the form of a “goal” or an “expectation” and whether the company’s reported earnings fall short or exceed investors’ expectations that are derived from management’s earnings guidance. Our experimental results suggest that investors respond less negatively when reported earnings fall short of investors’ expectations, but not less positively when earnings exceed investors’ expectations, when earnings guidance is issued as a goal rather than as an expectation. Mediation analysis of post-experiment questions supports the interpretation that earnings falling short of investors’ expectations leads investors to perceive managers as less competent and be more disappointed when managers issue expectation guidance rather than goal guidance, which in turn influences investors’ attractiveness judgments of the company.

Exploring beyond simple demographic variables: Differences between traditional laboratory samples and crowdsourced online samples on the Big Five personality traits

Amazon's Mechanical Turk (MTurk), a popular crowdsourcing website, is increasingly being utilized by researchers to obtain psychological data. This transition has prompted evaluation of sourcing costs, psychometric properties, and motivations of participants. However, research is limited comparing traditional and crowdsourced participants on personality measures. Therefore, in the current study laboratory participants (drawn from three universities) and MTurk workers completed the Big Five inventory and provided demographic information using web-based surveys. Controlling for age and gender, laboratory participants were significantly lower in Openness (d̂=0.26), and higher in Extraversion (d̂=0.37), Agreeableness (d̂=0.15), and Neuroticism (d̂=0.05) than MTurk participants. However, pairwise comparisons among individual sites revealed there were means above and below that for MTurk participants for Openness and Conscientiousness. Given these differences, researchers are encouraged to consider how such personality characteristics may influence the outcomes of their research when designing and conducting psychological studies that use crowdsourcing techniques to recruit participants.