Amazon Mechanical Turk Sample Research Articles

Prescription for Love: An Experimental Investigation of Laypeople’s Relative Moral Disapproval of Love Drugs

New technologies regularly bring about profound changes in our daily lives. Romantic relationships are no exception to these transformations. Some philosophers expect the emergence in the near future of love drugs: a theoretically achievable biotechnological intervention that could be designed to strengthen and maintain love in romantic relationships. We investigated laypeople’s resistance to the use of such technologies and its sources. Across two studies (Study 1, French and Peruvian university students, N after exclusion = 186; Study 2, Amazon Mechanical Turk sample, N after exclusion = 693, pre-registered), we found that the use of love drugs designed to strengthen and maintain love in romantic relationships are considered as more morally problematic than psychological therapy with the same aim. In Study 2, we show that this last effect is partially due to the fact that the love resulting from the use of love drugs is perceived as less authentic, intense, and durable. We discuss the specific role of authenticity in the relative moral disapproval of love drugs.

Measurement invariance of a Posttraumatic Stress Disorder symptoms measure (PCL-5) in college student and Amazon's mechanical TURK samples.

College student and Amazon's Mechanical TURK (MTURK) samples are regularly utilized in trauma research. Recent literature, however, has criticized these samples for not being generalizable to the general U.S. The purpose of this study was to determine whether college student (n = 255) and MTURK (n = 316) samples are invariant on the Posttraumatic Stress Disorder Checklist for DSM-5. Measurement invariance using confirmatory factor analyses was used to determine whether groups are invariant across factor structure, factor loadings, item intercepts, and residual error variances on a given measure of Post-traumatic Stress Disorder (PTSD) symptom severity. Model fit indices indicated the seven-factor Hybrid model was the best-fitting model, but the six-factor Anhedonia model was the most parsimonious model. Both models demonstrated equivalence in factor at the strictest level, indicating MTURK and college student samples are similar in regard to PTSD symptom severity. These findings provide evidence that these groups can be combined in future studies to increase sample size for trauma research. Only the Anhedonia factor exhibited mean differences between groups, which may be related to true differences between college students and MTURK survey-takers. This study provides further evidence that the findings from trauma studies using these populations are generalizable to each other. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Outcomes Affect Evaluations of Decision Quality: Replication and Extensions of Baron and Hershey’s (1988) Outcome Bias Experiment 1

Outcome bias is the phenomenon whereby decisions which resulted in successful outcomes were rated more favorably than when the same decisions resulted in failures. We conducted a pre-registered replication and extension of Experiment 1 (original’s: N = 20) from the classic Baron and Hershey (1988) with an online Amazon Mechanical Turk sample using CloudResearch (N = 692), switching from a within-participants design in the original experiment to a between-participants design. We tested outcome bias by measuring participants’ ratings of the quality of decisions in medical scenarios. For the replication (pre-registered) part of the study, we successfully replicated signal and direction of the outcome bias (original: dpaired = 0.21 – 0.53; replication: dindependent = 0.77 [0.62, 0.93] to 1.1 [0.94, 1.26]), and even for participants who stated that outcomes should not be taken into consideration when evaluating decisions (d = 0.64 [0.21, 1.08]). For the extension part of the study, we found differences, dependent on outcome types, in evaluations of the perceived importance of considering the outcome, the perceived responsibility of decision-makers, and the perception that others would act similarly given the choice by outcome type. Materials, data, and code are available on Open Science Framework (OSF): https://osf.io/knjhu/.

Revisiting the Links Between Numeracy and Decision Making: Replication Registered Report of Peters et al. (2006) With an Extension Examining Confidence

Numeracy is individuals’ capacity to understand and process basic probability and numerical information required to make decisions. We conducted a Replication Registered Report of Peters et al. (2006) examining numeracy as a predictor of positive-negative framing effect (Study 1), frequency-percentage effect (Study 2), ratio effect (Study 3), and bets effect (Study 4). With an online US American Amazon Mechanical Turk sample (N = 860), our replication using the target’s dichotomizing of the numeracy measure found support for the original findings regarding interactions between numeracy and three decision-making effects. Numeracy was associated with weaker framing effect (η2p = 0.01, 90% CI [0.00, 0.02]), weaker ratio bias (Cramer’s V = 0.17, 95% CI [0.10, 0.24]), and stronger bets effect (η2p = 0.02, 90% CI [0.01, 0.04]), yet we found no support for the frequency-percentage effect (η2p = 0.00, 90% CI [0.00, 0.01]). However, we found support for associations with all four studies when treating numeracy as a continuous variable. We extended the replication to examine confidence, yet the results were mixed with support found for only three conditions (Study 1 positive framing condition: r = -0.11, 95% CI [-0.20, -0.02]; Study 3: r = 0.15, 95% CI [0.08, 0.21]; Study 4 no-loss bet condition: r = 0.10, 95% CI [0.01, 0.20]), suggesting a much weaker and more complex relationship than anticipated. Materials, data, and code are available on: https://osf.io/4hjck/.

Revisiting the Differential Centrality of Experiential and Material Purchases to the Self: Replication and Extension of Carter and Gilovich (2012)

Carter and Gilovich (2012) investigated the centrality of experiential and material purchases to the self and concluded people have stronger associations with their experiential than their material purchases. In a pre-registered experiment with a US American Amazon Mechanical Turk sample, (N=743), we successfully replicated their Studies 3A, 3B, 3C, and 5. Experiential purchases were perceived as more reflective of true-self than material purchases for both self (d=0.65[0.57,0.73]) and for strangers (d=0.88[0.80,0.96]), and that when meeting a new person, information about experiential purchases was considered to be more insightful (d=1.13[1.04,1.22]), useful (d=1.14[1.05,1.23]), and fun to talk about (d=1.96[1.83,2.08]) than material purchases. Self-concept was more strongly associated with experiential purchases than with material purchases (d=0.39[0.25,0.54]), and that there was a negative association between experiential purchase satisfaction and the willingness to exchange memory (r=-.34[-.43,-.24]) (all effects above were p<.001). We added an extension examining change in evaluations of material and experiential purchases over time and found that current evaluations were more negative than past evaluations, yet to a lesser extent for experiential compared to material purchases. Materials, data, and code are available on: https://osf.io/v2w5h/

The destination marketing and media profile – travel motives nexus amid tourism crisis: the mediating effect of the nation brand

PurposeThe ongoing COVID-19 pandemic highlights the importance of destination marketing and media profiling to re-engage international tourists. However, potential crisis-induced nation brand (NB) deficits must be addressed to re-ignite tourism demand. The study examines the possible intervening effect of the contemporary NB in the international destination marketing and media-travel motives nexus.Design/methodology/approachA deductive quantitative study was undertaken with an online Amazon Mechanical Turk sample of n = 454 respondents. Hypotheses were tested using PROCESS Macro, Model 4.FindingsThe results show that the NB [people and negative events] had a practically significant partial mediating effect in the destination marketing – nature-cultural oriented travel motivation nexus.Practical implicationsNew insights are provided via a practical model which facilitates the measurement of potential nuances in the influence of destination marketing and media profiling on leisure tourists' travel motives amid crises. The intervening effect implies that a better understanding of the NB as an indirect antecedent to travel motivation may result in more effective crisis communications and tourism recovery-oriented marketing.Originality/valueThe study is amongst the first to extend marketing and behavioural theory to explore the interplay between the marketing and media profile, a nation's brand and tourists' travel behaviour amid a crisis. The study addresses a discernible dearth of knowledge related to the influence of the NB on tourist behaviour from an emerging market perspective.

The Roles of Gender and Employment Status in Abdication Decisions.

Abdication-the act of giving up a choice to another person-has been shown to prompt reciprocal generosity (Kardas et al., 2018), but it is not clear whether men or women are more likely to engage in abdication. The present research utilized a gift card scenario adapted from Kardas et al. (2018) in two Amazon Mechanical Turk samples to explore whether women abdicated more frequently than men, whether men abdicated to women less frequently than to other men, and whether employment status predicts abdication. In experiment one, participants (N = 520) were assigned to one of three conditions in which they had to either choose who received a hypothetical gift card themselves or abdicate the decision to a friend (sex not specified, male friend, or female friend). Chi-square analyses revealed no significant differences between participants who abdicated and those who allocated; men and women had similar abdication patterns; there was no significant difference in abdication as a function of friend gender for male or female participants. In experiment two, participants (N = 707) were again assigned to one of the three conditions but were also asked their employment status. Chi-square analyses indicated that participants generally preferred to be allocators rather than abdicators; women showed a similar abdication pattern to men; there was no significant difference in abdication as a function of friend gender for male or female participants. Lastly, unemployed participants abdicated more frequently (70%) than their employed counterparts (42%), and this was especially likely for women. These results have implications for potential factors that influence abdication decisions.

Extreme party animals: Effects of political identification and ideological extremity

AbstractMany Americans’ political identity is more relevant, salient, and central than their American identity. Political parties provide party members with an identity and belief system. Political identities influence identity‐based ideologies which increase affective polarization and potential anti‐social intergroup conflict between political parties. The purpose of the current research is to determine how political party identification and ideological extremity influences affective polarization for Republicans and Democrats across two studies. Participants in both studies completed a survey which measured political identification, ideological extremity, and uniquely operationalized measures of affective polarization, including support for co‐partisans, attitudes about political rivals, emotions towards political rivals, action tendencies towards political rivals, and perceived threat from political rivals. Study 1 (N = 530) used responses to the survey from a convenience sample crowdsourced by Amazon Mechanical Turk (AMT) in 2018. Study 1 showed that higher political party identification is associated with significantly higher affective polarization for members of both parties and that ideological extremity partially mediates this effect. Study 2 (N = 260) used a 2020 AMT sample to replicate Study 1’s results. Findings extend our understanding of identity‐based ideological extremity and suggest that affective polarization could be driven by political party identification and partisans’ ideological extremity.

Characteristics and clinical implications of the relationship between veganism and pathological eating behaviours.

Veganism may serve as a socially acceptable means to restrict food intake and disguise pathological eating behaviours. Studies that include vegan participants typically group them with other meat avoiders (e.g., vegetarians), potentially masking risk factors unique to veganism. We addressed this issue by recruiting two Amazon Mechanical Turk samples of 110 vegan and 118 omnivore participants, with comparable gender composition. We aimed to examine whether vegans showed higher disordered eating than omnivores, and if motives for pursuing a vegan diet impacted disordered eating. We assessed disordered eating using the Eating Attitudes Test, the Eating Disorder Examination-Questionnaire, and the Eating Pathology Symptom Inventory. Vegans displayed more pathological eating behaviours than omnivores, which was significantly predicted by cognitive restraint. However, body dissatisfaction levels were higher in omnivores than vegans. Diet motives did not influence vegans' disordered eating. We propose vegans have high levels of cognitive restraint, possibly due to their intention to avoid animal products. In turn, cognitive restraint subscales in eating disorder measures might be over-pathologising rates of eating disorders in vegans. Future research should monitor the progression of people's eating-related attitudes and behaviours before and after they transition to veganism to establish whether veganism increases the risk of disordered eating, or vice versa. IV, cross-sectional study.

Psychometric Properties of the AUDIT-C within an Amazon Mechanical Turk Sample

Objectives: Amazon's Mechanical Turk (MTurk) has become a popular data collection tool in the addiction sciences. We sought to examine the psychometric properties of the AUDIT-C in an MTurk sample. Methods: Data collection was facilitated via MTurk (N=309; 52.8% female), where an online survey assessed demographic data, alcohol use behaviors (AUDIT-C), and alcohol-related consequences (CAPS-r). Responses to the AUDIT-C were subjected to a principal component analysis to evaluate the structure of the 3-item measure. Alcohol-related consequences were used as a measure of convergent validity. Results: Results provided evidence for a single-factor structure. Pearson's product-moment correlation coefficients between AUDIT-C scores and CAPS-r scores produced statistically significant results (r = 0.51, p < .001). Using biological sex-based suggested cut-off scores for the AUDIT-C, hazardous drinkers (M = 19.15, SD = 8.27) demonstrated statistically significantly higher levels of alcohol-related consequences than non-hazardous drinkers (M = 12.56, SD = 5.35; t(295) = -8.34, p < .001). Reliability and stability statistics demonstrated strong internal consistency. Conclusions: Results demonstrate the sound psychometric properties of the AUDIT-C for an MTurk sample and provide evidence supporting the use of AUDIT-C as a screening tool to be employed with digitally accessed populations to identify and reach hazardous drinkers.

Evaluating the Effectiveness of NoteAid in a Community Hospital Setting: Randomized Trial of Electronic Health Record Note Comprehension Interventions With Patients.

BackgroundInterventions to define medical jargon have been shown to improve electronic health record (EHR) note comprehension among crowdsourced participants on Amazon Mechanical Turk (AMT). However, AMT participants may not be representative of the general population or patients who are most at-risk for low health literacy.ObjectiveIn this work, we assessed the efficacy of an intervention (NoteAid) for EHR note comprehension among participants in a community hospital setting.MethodsParticipants were recruited from Lowell General Hospital (LGH), a community hospital in Massachusetts, to take the ComprehENotes test, a web-based test of EHR note comprehension. Participants were randomly assigned to control (n=85) or intervention (n=89) groups to take the test without or with NoteAid, respectively. For comparison, we used a sample of 200 participants recruited from AMT to take the ComprehENotes test (100 in the control group and 100 in the intervention group).ResultsA total of 174 participants were recruited from LGH, and 200 participants were recruited from AMT. Participants in both intervention groups (community hospital and AMT) scored significantly higher than participants in the control groups (P<.001). The average score for the community hospital participants was significantly lower than the average score for the AMT participants (P<.001), consistent with the lower education levels in the community hospital sample. Education level had a significant effect on scores for the community hospital participants (P<.001).ConclusionsUse of NoteAid was associated with significantly improved EHR note comprehension in both community hospital and AMT samples. Our results demonstrate the generalizability of ComprehENotes as a test of EHR note comprehension and the effectiveness of NoteAid for improving EHR note comprehension.

Correlates of obsessive-compulsive and related disorders symptom severity during the COVID-19 pandemic

We investigated changes in the severity of obsessive-compulsive and related disorders (OCRDs) symptoms as a result of the COVID-19 pandemic. An Amazon Mechanical Turk sample of 829 individuals was evaluated with a series of instruments assessing the severity of the OCRDs before and during the pandemic. Additional questionnaires about sociodemographic factors, personal and family histories of OCRD, COVID-19 related events, compulsivity and impulsivity traits, schizotypal symptoms, and the severity of depression, anxiety and stress levels, were also used. Participants reported that OCD, hoarding disorder (HD) and skin picking disorder (SPD) symptoms significantly worsened during the pandemic along with increased disability, more affective symptoms and reduced quality of life. Female gender, a higher number of COVID-19 related stressful events, and higher pre-COVID-19 fear of harm and symmetry symptoms predicted more severe OCD symptoms during the pandemic, whereas lack of a HD diagnosis by a mental health professional and more severe schizotypal symptoms predicted worsened hoarding symptoms. Greater compulsivity traits were associated with more severe COVID-19 pandemic obsessive-compulsive and hoarding symptoms. These data indicate that the immense distress resulting from the COVID-19 included significant deterioration of OCRDs' symptoms, particularly of OCD, HD and SPD. It was also possible to identify a pre-pandemic profile of people most at risk of pandemic-related deterioration in OCRDs' symptoms, which may prove valuable for preventative initiatives in relation to the likely future waves of COVID-19 or of other communicable diseases. Future studies should follow up these findings longitudinally.

Mechanical Turk or volunteer participant? Comparing the two samples in the study of intelligent personal assistants

AbstractA challenge in academic and practitioner research is recruiting study participants that match target demographics, possess a desired skillset, and will participate for little to no compensation. An alternative to traditional participant recruitment struggles is crowdsourcing participants through online labor markets, such as Amazon Mechanical Turk (AMT). AMT is a platform that provides the tool for finding and recruiting participants with diverse demographics, skills, and experiences. This paper aims to demystify the use of crowdsourcing, and particularly AMT, by comparing the performance of traditionally recruited volunteers and AMT participants on tasks related to the evaluation of intelligent personal assistants (IPAs such as Amazon Alexa, Google Assistant, Apple Siri, and Microsoft Cortana). The comparison of AMT and non‐AMT samples indicated that while the two samples differed on demographics, their task performance was not significantly different. The paper discusses the costs and benefits of using AMT samples and would be of particular relevance to researchers who employ questionnaires and/or task‐specific data collection methods in their work.

Materialism and purchase-evoked happiness: A moderated mediation model of purchase type and purchase’s impact on self

ABSTRACTAlthough it is suggested that materialism value leads to more purchase-evoked happiness, limited research has investigated the underlying mechanism of the relationship. The purpose of the paper is to fill the research gap, demonstrating how materialism increases purchase-evoked happiness. Two surveys with a student sample (n = 112) and an Amazon Mechanical Turk sample (n = 207) were conducted. Bootstrapping analysis was used to examine the data. The findings identify the mediating role of purchase’s impact on self plays between materialism and purchase-evoked happiness, and reveal that purchase type (material vs. experiential purchase) moderates the mediation. For material purchases, higher-materialism value leads to more perceived impact of the purchases on self, which in turn, results in more purchase-evoked happiness. For experiential purchases, there is no such mediation effect of the purchases’ impact on self between materialism and purchase-evoked happiness. This paper is the first research providing empirical support for materialists’ belief that material possessions influence them personally and identifying the mechanism under which materialism increases purchase-evoked happiness. It expands the research on materialism and purchase type by showing that materialism as a personal value interplays with purchase type in influencing how happiness is obtained from purchases.

Comparing Internet Experiences and Prosociality in Amazon Mechanical Turk and Population-Based Survey Samples

Given the high cost of traditional survey administration (postal mail, phone) and the limits of convenience samples such as university students, online samples offer a much welcomed alternative. Amazon Mechanical Turk (AMT) has been especially popular among academics for conducting surveys and experiments. Prior research has shown that AMT samples are not representative of the general population along some dimensions, but evidence suggests that these differences may not undermine the validity of AMT research. The authors revisit this comparison by analyzing responses to identical survey questions administered to both a U.S. national sample and AMT participants at the same time. The authors compare the two samples on sociodemographic factors, online experiences, and prosociality. The authors show that the two samples are different not just demographically but also regarding their online behaviors and standard survey measures of prosocial behaviors and attitudes. The authors discuss the implications of these findings for data collected on AMT.

Precursors of the emotional cascade model of borderline personality disorder: The role of neuroticism, childhood emotional vulnerability, and parental invalidation.

The emotional cascade model proposes that the emotional instability and engagement in maladaptive behaviors within borderline personality disorder (BPD) may be linked to rumination. Previous research has established that BPD is highly related to neuroticism, childhood emotional vulnerability, and parental invalidation. Therefore, the purpose of the current study was to assess whether the constructs of the emotional cascade model relate to other constructs within the BPD nomological network. More specifically, neuroticism, childhood emotional vulnerability, and parental invalidation should relate to the ruminative process and engagement in maladaptive behaviors as described by the emotional cascade model. The current study investigated the relation between these factors using a cross-lagged panel design with data collected online at 3 time points in a student sample and an Amazon Mechanical Turk sample. Neuroticism predicted rumination 1 month later across both samples and for 3 measures of rumination. Childhood emotional vulnerability and parental invalidation predicted rumination 2 months later for 2 measures of rumination in the student sample only. Future studies should continue to investigate the potential role of personality traits and BPD vulnerability factors within the emotional cascade model. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Predictors of Attitudes Toward Autonomous Vehicles: The Roles of Age, Gender, Prior Knowledge, and Personality.

Autonomous vehicles (AVs) hold considerable promise for maintaining aging adults’ mobility as they develop impairments in driving skill. Nonetheless, attitudes can be a significant barrier to adoption as has been shown for other technologies. We investigated how different introductions to AV, video with a driver in the front seat, the rear seat, and a written description, affected attitudes, as well as how individual difference variables such as age, gender, prior knowledge, and personality traits predict attitudes within a middle-aged (Median age = 34, IQR = 20, n = 441) Amazon Mechanical Turk sample. The 16-item attitude survey uncovered three factors: Concern with AV, Eagerness to Adopt AV technology, and Willingness to Relinquish Driving Control. ANOVAs showed that only age (younger less concerned) and gender, (females more concerned) were significant factors in Concern with AV. Only gender affected Willingness to Relinquish Driving Control, with males more willing. Multiple regressions that included previous knowledge level and personality traits showed a different pattern. Female gender and greater conscientiousness were associated with greater Concern about AV. Prior knowledge of AV was associated with less concern. Emotional stability and openness to experience were positive predictors of Eagerness to Adopt AV, whereas conscientiousness was a negative predictor. Prior knowledge and openness to experience, positively, and extraversion, negatively, were associated with being willing to relinquish driving control. These results suggest that different information dissemination campaigns are needed to persuade consumers to adopt AV technology. We discuss potential approaches.

Sexual Passion in Committed Relationships: Measurement and Conceptual Issues.

The purpose of this study is to discuss the need for a new triadic model of sexual passion in relationships and to present the preliminary psychometric properties of a scale designed to measure these three approaches to passion (harmonious, obsessive, and inhibited) for use in clinical and scholarly work. Existing theory and measures of general passion are based on a dualistic model of passion that includes the harmonious and obsessive approaches to passion. We added the inhibited approach to passion from the sexuality research and develop measures for assessing sexual passion in relationships. We utilized an Amazon Mechanical Turk sample of 1,421 individuals in committed relationships to test this measure. Reliability analyses and confirmatory factory analyses evinced that these three approaches to sexual passion were unique constructs and distinct from sexual satisfaction. Sexual passion showed predictive validity above and beyond relationship length, sexual desire toward a partner, and a broader variable of sexual drive. Harmonious sexual passion robustly predicted higher sexual satisfaction and relationship satisfaction, and inhibited sexual passion moderately predicted lower sexual satisfaction and relationship satisfaction. Obsessive sexual passion had minimal associations with either outcome. These new constructs, especially harmonious and inhibited sexual passion, may help scholars and practitioners improve their understanding of sexual satisfaction and overall relationship satisfaction.

The Message Matters: The Influence of Fact-Checking on Evaluations of Political Messages

Fact-checks have become prolific in U.S. campaigns over the last ten years. As a result, fact-checks have become one of the easiest ways for individuals to analyze the truthfulness of politicians’ statements. The increase in both fact-checking and its accessibility to voters led us to ask whether fact-checks influence individuals’ attitudes and evaluations of political candidates and campaign messages. To examine the impact of fact-checking, we conduct two original experiments using the 2012 Ohio Senate race between Republican challenger Josh Mandel and Democratic incumbent Sherrod Brown. The first experiment utilized a sample of over 300 students during the fall of 2013. The second experiment is a crowdsourced Amazon Mechanical Turk Sample in the fall of 2014. We find the content of fact-check messages are influential in altering assessments of candidates’ advertisements. We also find the source of the fact-check only modestly impacts assessments. The findings illustrate the potential power of fact-checks to influence the effectiveness of candidates’ messages and reaffirm the important role the news media plays in validating candidate claims and arguments during political campaigns.

Temporal framing and the hidden-zero effect: rate-dependent outcomes on delay discounting.

Recent research suggests that presenting time intervals as units (e.g., days) or as specific dates, can modulate the degree to which humans discount delayed outcomes. Another framing effect involves explicitly stating that choosing a smaller-sooner reward is mutually exclusive to receiving a larger-later reward, thus presenting choices as an extended sequence. In Experiment 1, participants (N = 201) recruited from Amazon Mechanical Turk completed the Monetary Choice Questionnaire in a 2 (delay framing) by 2 (zero framing) design. Regression suggested a main effect of delay, but not zero, framing after accounting for other demographic variables and manipulations. We observed a rate-dependent effect for the date-framing group, such that those with initially steep discounting exhibited greater sensitivity to the manipulation than those with initially shallow discounting. Subsequent analyses suggest these effects cannot be explained by regression to the mean. Experiment 2 addressed the possibility that the null effect of zero framing was due to within-subject exposure to the hidden- and explicit-zero conditions. A new Amazon Mechanical Turk sample completed the Monetary Choice Questionnaire in either hidden- or explicit-zero formats. Analyses revealed a main effect of reward magnitude, but not zero framing, suggesting potential limitations to the generality of the hidden-zero effect.