MTurk Participants Research Articles

Lament: Measuring a Christian Meaning-Making Coping Practice

Research demonstrates that people draw on their religions in coping with difficult life circumstances, and that more religious people tend to cope better than less religious people. Lament is a religious prayer practice that constitutes a form of meaning-making coping. Here, we explicate this form of coping and, in a series of three studies with Christian participants, report on the development of a lament scale. In the first study, we develop items and test the items for clarity and for generalizability to diverse Christian groups using expert review and cognitive interviewing with participants representing five Christian groups. In the second study, we report results of exploratory factor analysis using data from primarily White and Protestant MTurk participants ( N = 303). In the third study, we report on factor stability and construct validity using data from largely White and African American Protestant and Catholic Prolific participants ( N = 346). The studies demonstrated generalizability to diverse Christian groups, a three-factor structure consistent with our theoretical formulation, good construct validity and relationships with well-being. Moderation analyses also indicated that when both Complaint and Praise are either high or low, the severity of the event is inversely related to flourishing. However, when either Complaint or Praise is high, the negative relationship between severity of the event and flourishing is ameliorated. The Lament Scale is a reliable and valid instrument for assessing lament in ways that are consistent with the conceptualization of lament as a meaning-making coping practice.

Examining the effect of impossible problem solving on state anger: Comparing in-person and MTurk participants

This research comprises of two studies examining the effect of impossible problem solving on state anger. For the first study, using 155 participants on Amazon’s MTurk, a significant difference was observed on state anger from baseline to post-intervention (p = .04). Due to the well documented concerns about using MTurk participants, a second study involving 111 participants from a small liberal arts college was conducted. Using the same design, no significant effect of time on state anger was observed (p > .05). Post-hoc analyses indicated a significant difference between MTurk and in-person participants on trait anger (p < .05), pre-test state anger (p < .05), and post-test state anger (p < .05). Implications are discussed.

Investigating the transdiagnostic potential of rumination in relation to Cluster B personality disorder symptoms.

This study examined several forms of repetitive negative thinking in relation to measures of Cluster B disorders to identify unique and co-occurring cognitive mechanisms contributing to these symptom presentations. MTurk participants (N = 725; 53% men; 76% White) completed categorical and dimensional trait-based measures of Cluster B disorders alongside six measures of rumination and worry. Oversampling strategies were used during participant recruitment to obtain a sample with clinically relevant personality features. Path analyses examined each form of repetitive negative thinking in relation to Cluster B measures while controlling for shared variance between each construct. Anger rumination demonstrated notable relations across all four Cluster B personality disorders. Similarly, sadness rumination was positively associated with all four Cluster B personality disorders. Self-critical rumination and depressive rumination were uniquely associated with borderline symptoms. Worry demonstrated negative associations with measures of narcissistic, histrionic, and antisocial/psychopathy symptoms. Findings expand previous research by highlighting probable transdiagnostic and distinguishing cognitive process contributing to the presentation of Cluster B personality pathology. However, partialling the influence of overlapping constructs presents notable challenges in identifying and interpreting unique relations between various forms of repetitive negative thinking and Cluster B symptoms. Future research investigating these relations within outpatient and inpatient samples may provide avenues for the development of effective cognitive-based interventions for treating these symptoms. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

First impressions: Do faces with scars and palsies influence warmth, competence and humanization?

A glance is enough to assign psychological attributes to others. Attractiveness is associated with positive attributes ('beauty-is-good' stereotype). Here, we raise the question of a similar but negative bias. Are people with facial anomalies associated with negative personal characteristics? We hypothesized that biases against faces with anomalies arise because of negative stereotypes (less warmth and competence) and forms of dehumanization (animalistic and mechanistic). We enrolled 1493 mTurk participants (N = 1306 after exclusion) to assess 31 traits of photographed people using 60 pairs of photographs of the same person before and after plastic surgery. Half anomalous faces had a scar and the other half had a palsy. To calculate warmth and competence, we conducted a principal components analysis of the 31 attributes. Animalistic dehumanization was assessed by averaging reverse-scored ratings corresponding to moral sensibility and rationality/logic, and mechanistic dehumanization by averaging across reverse-scored ratings corresponding to emotional responsiveness and interpersonal warmth. We found that both kinds of anomalous faces were seen as less warm, competent and were dehumanized. Our findings suggest that an 'anomalous-is-bad' stereotype generalizes regardless of the aetiology of the anomaly. This effect may be related to a reverse halo effect, that is, the horn effect.

The impact of death priming on climate change denial: a preliminary investigation.

Climate change denial (CCD) has been found to be more pronounced among individuals with a conservative political orientation. Terror Management Theory posits that an individual's worldview serves as a primary defense mechanism in coping with existential threats, such as the reality of climate change. Drawing on these premises, we conducted preliminary research investigating the impact of death priming on CCD from the Terror management theory perspective. We administered a death priming task and measured CCD in its immediate wake or following a delay task. We confirmed among 219 Amazon's MTurk participants that immediately following death priming, CCD was reduced among all participants. In the delayed death priming condition, we acquired preliminary evidence that CCD was heightened among Republican participants. These findings demonstrate the relevance of death priming to CCD tendencies and potentially spawn future research regarding CCD as a particular form of coping with existential threats.

Does the Paraphilia Scale Work for Everyone? Confirmatory Factor Analysis and Measurement Invariance Across Gender and Sexual Orientation Groups

ABSTRACT We conducted three studies to examine the factor structure and measurement invariance of the Paraphilia Scale, a measure of paraphilic interests used in multiple studies. In the first study, we conducted a confirmatory factor analysis (CFA) testing different a priori models with a community sample of 1,040 adults previously reported by Seto et al. (2021), and found support for a hierarchical four-factor model: An agonistic continuum involving coercion or physical pain (biastophilia, sexual sadism, masochism), chronophilias (pedophilia, hebephilia), courtship disorders (voyeurism, exhibitionism, and frotteurism), and fetishism (object fetishism, transvestic fetishism, urophilia-coprophilia). This factor structure was replicated in a second study comprising a combined sample of 400 mTurk participants and 870 university students. The third study analyzed the community sample and found evidence of configural invariance but not scalar or metric invariance across gender (man or woman) and sexual orientation for gender (heterosexual or other sexual orientation). This indicates that the factor structure of the Paraphilia Scale is robust for gender and sexual orientation for gender, but factor loadings differ across these groups, as do the loadings of individual items on the four factors. Implications for research on gender and sexual orientation differences in paraphilic interests are discussed.

It's not you (well, it is a bit you), it's me: Self- versus social image in warm-glow giving.

Attempts by charities to motivate giving tend to focus on potential donors' altruistic tendencies. However, prior research suggests that approximately 50% of individuals are to some extent motivated by warm glow, the satisfaction received from the act of giving. The satisfaction derives from looking good to themselves (self-image) and/or to others (social image). We conduct an online experiment on MTurk participants (n = 960) with a more realistic simulation of being watched to determine the importance of self- and social image to warm-glow giving. We find evidence that suggests that social image concerns do not increase the likelihood that someone will give but they do increase the amount given; average giving is significantly higher in the treatments when feelings of being watched are stimulated. Our results suggest that charities looking to increase their donor bases might effectively do so by focusing on self-image concerns. Charities wishing to increase the amount donated might effectively do so by focusing on the social image concerns of the donor.

Mothers of transgender youth experience stigma‐by‐association

AbstractThe current research investigated whether mothers of transgender youth experience stigma‐by‐association. Mturk participants (N = 489) were randomly assigned to read a vignette about a family in which the social identity (transgender, gay/lesbian, cisgender/heterosexual control) and gender (girl, boy) of a child was manipulated, while all other information was held constant. Results revealed stigma targeting mothers as a function of children's social identity (but not gender), such that mothers of transgender girls and boys were viewed substantially more negatively than identical mothers of cisgender/heterosexual youth. Moreover, this stigma was particularly robust among politically conservative participants. In contrast, mothers of gay/lesbian youth did not encounter systematic stigma, though they were sometimes perceived more negatively than mothers of cisgender/heterosexual youth. Results provide novel experimental evidence of stigma‐by‐association targeting mothers of transgender youth and raise serious concerns about the treatment of parents who seek to affirm their transgender children.

Does improved upward social mobility foster frustration and conflict? A large-scale online experiment testing Boudon’s model

The rise of populism has reignited scholarly interest in the paradox of societal advancement leading to frustration and social tension. Globalization and digitalization have increased social opportunities for parts of the population, but a substantial portion of society feels disadvantaged, resulting in discontent. This study, rooted in Boudon’s model of relative deprivation, examines the mechanisms that fuel this frustration. We conducted an online experiment involving 2114 US-based MTurk participants, in which we manipulated the availability of status positions to create varying degrees of upward social mobility. We also varied group sizes to ensure robustness. We assessed relative deprivation with structural, subjective, and behavioral measures. For example, frustration was measured using the “joy-of-destruction game,” in which subjects had to make the costly decision to destroy part of another player’s winnings. Contrary to the model’s prediction, we found that the proportion of individuals who were worse off, the losers, decreased consistently as mobility increased. This outcome can be attributed to overentry in conditions of low mobility and underentry in conditions of intermediate or high mobility. The losers displayed increased frustration and hostility towards noncompetitors and winners. Intriguingly, winners also exhibited heightened hostility. However, at the aggregate level, hostile behavior did not surge as conditions improved. In our exploratory analyses at the individual level, we identified several distinct patterns. Risk-tolerant individuals and women were more likely to enter competition. Conversely, those with advanced education levels showed a decreased inclination to competitiveness. Risk-tolerant individuals reported greater feelings of frustration and displayed increased hostility. This effect was also observed particularly among politically right-leaning individuals.

White Americans' blame attributions and empathy towards Black victims of police violence: How pejorative stereotypes 'engulf the field'.

We examined the dynamics of minority-directed police violence by considering how our White participants' empathy for Black victims may be influenced by critical intragroup differences related to racial stereotyping. Although the role of stereotyping in reactions to Black Americans accused of crime is well-established, we explore the influence of pejorative Black stereotypes on reactions to Black victims of police violence. Specifically, we investigated the roles of individual differences in the endorsement of the Black criminal stereotype among White observers and manipulated the crime-unrelated stereotypicality (i.e. stereotypical, counterstereotypical) of Black victims of police violence. White US MTurk participants read about a White policeman shooting a Black man (Study 1, n = 140) or sexually assaulting a Black woman (Study 2, n = 166). Across both studies, strong stereotype endorsers reported relatively low empathy for stereotypical victims, mediated by greater blame towards those victims. This finding demonstrates the relevance of heretofore untested motivated reasoning processes in the outgroup empathy deficits literature. Weak stereotype endorsers showed relatively high empathy and low victim blame regardless of Black victim stereotypicality, indicating limited sensitivity to outgroup member suffering is not inevitable. We consider the practical implications of the findings for policing and for citizenship education.

The Effects of an Incarcerated Person’s Gender Identity and Crime Type on Mock Parole Board Members’ Decisions

Parole members are tasked with deciding which incarcerated person is granted parole; however, factors such as the incarcerated person’s gender identity in combination with the crime committed could negatively affect this decision. This study investigated the effects of the incarcerated person’s gender identity (male, female, nonbinary) and crime committed (sexual assault, theft/larceny) on mock parole members’ decisions and perceptions of the incarcerated person. MTurk participants (N = 467) acted as mock parole board members. Participants read vignettes describing an incarcerated person eligible for parole, and participants rated their likelihood to release the incarcerated person and perception of the incarcerated person. We found that the incarcerated person’s gender identity affected participants’ parole decisions and perceptions of the incarcerated person, such that, in general, male incarcerated people were perceived more negatively than nonbinary and female incarcerated people, and female incarcerated people were perceived the most positively. Crime severity also affected parole decisions, such that incarcerated people who commit a crime of sexual assault were less likely to receive parole and were perceived the most negatively compared to incarcerated people who commit a crime of theft/larceny. There was no interaction between the incarcerated person’s gender identity and crime severity. This study can provide insight on perceptions of nonbinary incarcerated people, and how these perceptions compare to binary incarcerated people.

Conducting Rehabilitation Research Through Amazon Mechanical Turk: A User-Friendly Guide

Amazon Mechanical Turk (MTurk) is a crowdsourcing website that has become a popular participant recruitment tool in social science research. MTurk provides access to a large, diverse, and inexpensive pool of willing participants, and the unique clinical characteristics of MTurk participants may be of particular interest to rehabilitation researchers. In this article, the authors provide a step-by-step guide to set up and conduct a Qualtrics online survey study via MTurk. Several considerations including quality control of data, cost reduction, and ethical considerations are discussed, and strategies to improve data quality and promote research ethics are provided.

The Theory of Planned Behaviour doesn't reveal 'attitude-behaviour' gap? Contrasting the effects of moral norms vs. idealism and relativism in predicting pro-environmental behaviours.

The inconsistency between pro-environmental attitudes and behaviours called the 'attitude-behaviour' gap, has been reported in many scenarios relating to sustainable actions. However, the reasons for it are not entirely clear. It has been proposed that the 'attitude-behaviour' gap is driven by the moral dimension whereby moral attitudes fail to translate into subsequent moral behaviours. If so, the lacking integration of moral dimension into the environmental domain serves as a generalisable factor to improve the accuracy of predicting pro-environmental behaviours. Hence, we aimed to explore (i) whether the addition of a moral element to the Theory of Planned Behaviour (TPB) shows the 'attitude-behaviour' gap and (ii) whether the ethics position questionnaire (EPQ) is a more suitable measure of morality within the TPB framework, as compared to that of moral norms. The results from 181 US MTurk participants disclosed that the addition of the moral element to the TPB framework did not reveal the presence of the 'attitude-behaviour' gap, despite both moral norms and idealism significantly predicting pro-environmental attitudes. The findings do not indicate whether moral norms or idealism should be used as a more accurate measure of morality within the TPB framework, although relativism was found to have no significant effects. Further investigation of why the moral element does not reveal the 'attitude-behaviour' gap within the TPB framework predicting pro-environmental behaviours would help understand the reasons why rational choice models tend to overestimate theoretical vs. real-life engagement with sustainability.

Comparing attentional disengagement between Prolific and MTurk samples

Attention often disengages from primary tasks in favor of secondary tasks (i.e., multitasking) and task-unrelated thoughts (i.e., mind wandering). We assessed whether attentional disengagement, in the context of a cognitive task, can substantially differ between samples from commonly used online participant recruitment platforms, Prolific and Mechanical Turk (MTurk). Initially, eighty participants were recruited through Prolific to perform an attention task in which the risk of losing points for errors was varied (high risk = 80% chance of loss, low risk = 20% chance of loss). Attentional disengagement was measured via task performance along with self-reported mind wandering and multitasking. On Prolific, we observed surprisingly low levels of disengagement. We then conducted the same experiment on MTurk. Strikingly, MTurk participants exhibited more disengagement than Prolific participants. There was also an interaction between risk and platform, with the high-risk group exhibiting less disengagement, in terms of better task performance, than the low-risk group, but only on MTurk. Platform differences in individual traits related to disengagement and relations among study variables were also observed. Platform differences persisted, but were smaller, after increasing MTurk reputation criteria and remuneration in a second experiment. Therefore, recruitment platform and recruitment criteria could impact results related to attentional disengagement.

The Development of the Five-Factor Schizoid Inventory.

The current study sought to provide evidence for a measure of schizoid personality disorder (SZD PD) traits using the Five-Factor Model framework of personality. In the first study, undergraduate participants (n = 496) completed the Five-Factor Schizoid Inventory (FFZI) and other self-report measures. The first half of the sample was used to develop the FFZI, while the second half was used to validate it. The FFZI demonstrated excellent internal consistency, convergent validity with measures of SZD PD and hypothesized IPIP-NEO facets, and discriminant validity with other PDs and non-hypothesized IPIP-NEO facets. The second study recruited MTurk participants (n = 181) and demonstrated preliminary support for the reliability and validity of the FFZI in an online, community sample. Ultimately, these data suggest that the FFZI is a useful measure of SZD PD and provide further evidence that SZD PD can be conceptualized as a maladaptive extension of introversion traits.

Decision-making in terrorism sting operations: is entrapment driven by the sunk-cost effect and intergroup biases?

ABSTRACT Previous research has found that entrapment – inducing suspects to commit crimes they were not already predisposed to commit – is widespread in US terrorism sting operations, particularly those involving Muslims and racial minorities. To test whether the sunk cost effect and/or intergroup biases could explain decisions to authorise these operations, we conducted an experiment in which college student and MTurk participants (n = 1009) from the US and India decided whether to approve an operation featuring apparent entrapment. Sunk costs and suspect characteristics were manipulated across experimental conditions. Results for US participants (n = 792) show that high sunk costs increased approval of the operation only for relatively young participants, and only in scenarios with non-Muslim suspects. Scenarios with Muslim suspects yielded higher approval rates for participants who were confident in their decision. Indian participants (n = 217) were more likely than US participants to approve the operation and were unaffected by suspect religion and sunk costs. Results suggest that the sunk cost bias may play some role in entrapment, although other factors, such as outgroup bias, may also contribute to these decisions. The observed cross-cultural differences could result from international variations in susceptibility to the sunk cost effect, among other factors.

Effects of pay rate and instructions on attrition in crowdsourcing research.

Researchers in social sciences increasingly rely on crowdsourcing marketplaces such as Amazon Mechanical Turk (MTurk) and Prolific to facilitate rapid, low-cost data collection from large samples. However, crowdsourcing suffers from high attrition, threatening the validity of crowdsourced studies. Separate studies have demonstrated that (1) higher pay rates and (2) additional instructions-i.e., informing participants about task requirements, asking for personal information, and describing the negative impact of attrition on research quality-can reduce attrition rates with MTurk participants. The present study extended research on these possible remedies for attrition to Prolific, another crowdsourcing marketplace with strict requirements for participant pay. We randomly assigned 225 participants to one of four groups. Across groups, we evaluated effects of pay rates commensurate with or double the US minimum wage, expanding the upper range of this independent variable; two groups also received additional instructions. Higher pay reduced attrition and correlated with more accurate performance on experimental tasks but we observed no effect of additional instructions. Overall, our findings suggest that effects of increased pay on attrition generalize to higher minimum pay rates and across crowdsourcing platforms. In contrast, effects of additional instructions might not generalize across task durations, task types, or crowdsourcing platforms.

The Cognitive Control Model of Work-related Flow.

Although several models of flow have been proposed that include environmental and trait-based antecedents of the state, elements of cognitive control that enable workers to experience flow and its subsequent outcomes at work have largely been overlooked. This research proposes and provides empirical support for the "Cognitive Control Model of Work-related Flow," which integrates antecedents of flow at work related to the ability to focus concentration of cognitive resources toward experiencing flow at work. Along with flow at work, the model includes the antecedents of grit, flow metacognition, and mindfulness at work and the outcomes of work performance, engagement, and burnout. Findings across three studies (a cross-sectional, a time-lagged, and a one-day experience sampling method study) utilizing MTurk participants provided support for the model, as grit, mindfulness, and flow metacognition predicted flow, and flow predicted subjective performance, engagement, and burnout. Theoretical implications and the potential for developing flow interventions at work are discussed.

Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages.

To understand human behavior, social scientists need people and data. In the last decade, Amazon's Mechanical Turk (MTurk) emerged as a flexible, affordable, and reliable source of human participants and was widely adopted by academics. Yet despite MTurk's utility, some have questioned whether researchers should continue using the platform on ethical grounds. The brunt of their concern is that people on MTurk are financially insecure, subject to abuse, and earn inhumane wages. We investigated these issues with two representative probability surveys of the U.S. MTurk population (N = 4094). The surveys revealed: (1) the financial situation of people on MTurk mirrors the general population, (2) most participants do not find MTurk stressful or requesters abusive, and (3) MTurk offers flexibility and benefits that most people value above other options for work. People reported it is possible to earn more than $10 per hour and said they would not trade the flexibility of MTurk for less than $25 per hour. Altogether, our data are important for assessing whether MTurk is an ethical place for research.

The Brief Survey of Interpersonal Exploitative Behavior: Preliminary Development and Validation

The development and preliminary validation of a new measure of interpersonal exploitativeness, i.e., Brief Survey of Interpersonal Exploitative Behavior (BSIEB), is described in two studies. Interpersonal exploitativeness is a transdiagnostic feature of Narcissistic Personality Disorder and Antisocial Personality Disorder. The BSIEB focuses on instantiating interpersonal exploitative behavior by surveying specific, "real world” exploitative behaviors. In Study 1, an item pool was generated by 50 Amazon Mechanical Turk (MTurk) participants. Inter-rater reliability was assessed via independent ratings from a second group of MTurk participants. The BSIEB was administered to a new sample of 508 adults (64.5% males; M age = 37.1). The BSIEB demonstrated excellent internal consistency reliability (Cronbach’s α = .94) and convergent validity (r = .64) with a measure of exploitativeness based on the Five-Factor Model of personality. A 1-factor measurement model fit well and was invariant for gender. In Study 2, the BSIEB demonstrated convergent validity with other measures of exploitativeness and incremental validity, registering the most significant share of unique variance in predicting the outcome variables of social aggression and alcohol and substance use, which are associated with narcissism and antisocial personality disorder.  The results support using the BSIEB as a valid research measure that expands construct coverage of interpersonal exploitative behavior.