MTurk Participants Research Articles

Is Amazon's Mechanical Turk (MTurk) a comparable recruitment source for trauma studies?

Trauma researchers have recently begun using Amazon's Mechanical Turk (MTurk) as a data collection platform that is both time- and cost-efficient. Research is needed to determine the utility, generalizability, and validity of MTurk as a recruitment source for trauma-exposed samples. Data were collected from 266 trauma-exposed MTurk participants on several clinical and psychological constructs relevant to trauma research. The mean scores, prevalence rates, and correlation strengths of the MTurk sample were compared to those reported in previously published studies of undergraduate, community, and treatment-seeking samples. Findings indicated that prevalence rates of posttraumatic stress disorder (PTSD) and depression were not significantly different from comparison samples, but prevalence rates of generalized anxiety were significantly higher than that of a community sample. The MTurk sample showed significantly lower mean scores of PTSD, depression, and generalized anxiety symptoms than all comparison samples. Correlations were examined to determine whether established relationships between common trauma-related constructs were correlated for MTurk participants as they were in other samples. Correlations between PTSD symptom severity, posttraumatic cognitions (PTCs), and trauma coping self-efficacy (CSE-T) in the MTurk sample were not significantly different from the correlations observed in all comparison samples. Finally, MTurk participants who met criteria for probable PTSD scored significantly higher on measures of depression, generalized anxiety, and PTCs, and lower on CSE-T, than those without probable PTSD. Future trauma researchers utilizing MTurk should consider potential similarities and differences between MTurk samples and community, clinical, and undergraduate samples when interpreting the generalizability of findings. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Is a Reasonable Woman Different from a Reasonable Person? Gender Differences in Perceived Sexual Harassment

Although the problem of sexual harassment can be approached from a variety of perspectives, the present research focused on the role played by individuals’ perceptions, specifically those that may differ between men and women. We examined whether perceived sexual harassment would vary depending on observers’ gender, on gender of the harasser and of the victim, and on whether and what type of sexual harassment definition was provided to observers. In doing so, we attempted to update and clarify inconsistent results from prior studies. Results from 413 young adult U.S. MTurk participants responding to an online survey revealed fairly large effects for participants’ gender, such that women perceived a wider range of situations as sexual harassment than did men. In addition, the dyad of a man harassing a woman was construed as more definitely involving sexual harassment than other dyads. Surprisingly, these gender differences were smaller for situations judged as comprising sexual harassment to a lesser extent (i.e., those involving derogatory attitudes and nonsexual physical contact). Results are discussed in relation to prior findings and their legal implications, especially as they relate to a psychological assumption of the so-called “reasonable woman” standard used in U.S. courts, that women perceive sexual harassment to a greater degree than men. The results are also relevant for those wishing to curtail harassment within organizations and for those counseling victims of sexual harassment.

Race, Gender and Beauty: The Effect of Information Provision on Online Hiring Biases

We conduct a study of hiring bias on a simulation platform where we ask Amazon MTurk participants to make hiring decisions for a mathematically intensive task. Our findings suggest hiring biases against Black workers and less attractive workers and preferences towards Asian workers female workers and more attractive workers. We also show that certain UI designs including provision of candidates information at the individual level and reducing the number of choices can significantly reduce discrimination. However provision of candidates information at the subgroup level can increase discrimination. The results have practical implications for designing better online freelance marketplaces.

Factor Structure and Multi-Group Measurement Invariance of Posttraumatic Stress Disorder Symptoms Assessed by the PCL-5

The Posttraumatic Stress Disorder Checklist (PCL) is widely used in research and clinical settings to assess posttraumatic stress disorder (PTSD) symptomatology, including to examine PTSD symptoms/severity across diverse groups. However, the validity of these studies depends on the degree to which the PCL captures a conceptually equivalent construct of PTSD across the compared groups. This study examined the factor structure and invariance of PTSD as assessed by the PCL for DSM-5 (PCL-5) across a sample of trauma-exposed students (n = 412) and trauma-exposed participants recruited online through Amazon’s Mechanical Turk (MTurk; n = 346) platform. Participants from both samples completed the Stressful Life Events Screening Questionnaire (exposure to traumatic events) and the PCL-5. Using confirmatory factor analysis (CFA), we examined the factor structure of the DSM-5 and five alterative models of PTSD separately in both samples. We then tested measurement invariance of the DSM-5 and the optimal model of PTSD across these samples. All PTSD models fit the data well; however, the Hybrid model provided a significantly better fit for both samples. Both the Hybrid and DSM-5 models of PTSD demonstrated invariance for item mappings on factors (configural), factor loadings (metric), intercepts (scalar), and error variances (residual). Our findings support the stability, applicability, and meaningful comparisons of the PCL-5 assessed DSM-5 and Hybrid model factors across samples of trauma-exposed students and MTurk participants.

Revisiting narcissism and contingent self-esteem: A test of the psychodynamic mask model

The “psychodynamic mask model” of narcissism posits that narcissists' self-esteem is inherently fragile. Available research provides inconsistent and ambiguous support for this model. In light of weaknesses associated with some previous tests, we provided novel evidence on whether narcissism dimensions relate to an indicator of “contingent self-esteem;” here, contingent self-esteem was operationally defined as the literal relation between self-esteem and self-reported satisfaction in various life domains. Using a sample of 450 MTurk participants, we examined whether several narcissism constructs moderate the relation between self-esteem and self-reported satisfaction in various life domains. In general, relations between self-esteem and satisfaction in various life domains decreased as a function of two different indicators of grandiose narcissism; by contrast, relations between self-esteem and satisfaction in various life domains were generally enhanced (albeit weakly and non-significantly) as a function of a vulnerable narcissism indicator. In sum, grandiose narcissism appeared related to reduced contingent self-esteem and vulnerable narcissism appeared mostly unrelated to contingent self-esteem. We discuss these findings in relation to previous evidence on narcissism and contingent self-esteem and evaluating the mask model.

Facts only the perpetrator could have known? A study of contamination in mock crime interrogations.

This paper examines contamination in interrogations: the process by which an interrogator divulges privileged information to a suspect. In Experiment 1, we predicted that mock investigators would communicate critical crime details when they interview mock suspects about a crime-and that innocent and guilty suspects alike would later produce confessions that contained these details. In Experiment 2, we hypothesized that observers who listened only to the confessions would exhibit a greater guilt bias than those who also had exposure to the eliciting interview. Experiment 1 (N = 59) used student participants in a mock crime scenario to test whether contamination is natural to communication even in the absence of external incentives. In Experiment 2, MTurk participants (N = 499) listened to audio-clips from Experiment 1 to test whether presenting observers with the full interview decreases guilt ratings for false confessors. Investigators divulged crime information to both innocent and guilty suspects, and even false confessions later included accurate details. Although Experiment 2 observers exhibited a guilt bias, exposure to the interview (not just the confession) attenuated this effect for innocent confessors. The information disclosure associated with contamination is a normal cognitive process that occurs even without external incentives to secure a confession. Experiment 2 showed that seeing contamination in action may decrease judgments of guilt for innocent suspects. Interrogations should be recorded in their entirety to provide fact finders with an objective record of the source of crime details contained within narrative confessions. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Why Are People Honest? Internal and External Motivations to Report Honestly

ABSTRACTWe create and validate measures capturing internal and external motivations to report honestly as trait‐like characteristics. Both measures have high levels of reliability, as well as convergent and divergent validity. To test their predictive validity, we conduct two experiments. In the first experiment, MTurk participants have the opportunity and incentive to misreport with no immediate consequences, and in the second experiment, participants with management experience report how they would make a hypothetical accounting allocation decision. In both experiments, we find that participants who are higher in internal motivations to report honestly are more likely to report honestly than those lower in internal motivations, confirming this measure's predictive validity. Both experiments also provide limited support for the predictive validity of our external measure, finding that those who are higher in external motivation do not report differently than those who are lower in external motivations in the absence of controls. Our study also reveals that individuals who are higher in internal motivations have a diminished reaction to different management controls: MTurk participants to a control that punishes misreporting, and manager participants to a control that rewards honest reporting. Results suggest that management and those charged with governance should consider that some employees can react negatively to controls that are perceived as constraining. Our measures are useful to researchers who investigate honest reporting by allowing them to identify, ex ante, individuals who want to be honest versus wanting to appear honest.

Self-reported willingness to share political news articles in online surveys correlates with actual sharing on Twitter.

There is an increasing imperative for psychologists and other behavioral scientists to understand how people behave on social media. However, it is often very difficult to execute experimental research on actual social media platforms, or to link survey responses to online behavior in order to perform correlational analyses. Thus, there is a natural desire to use self-reported behavioral intentions in standard survey studies to gain insight into online behavior. But are such hypothetical responses hopelessly disconnected from actual sharing decisions? Or are online survey samples via sources such as Amazon Mechanical Turk (MTurk) so different from the average social media user that the survey responses of one group give little insight into the on-platform behavior of the other? Here we investigate these issues by examining 67 pieces of political news content. We evaluate whether there is a meaningful relationship between (i) the level of sharing (tweets and retweets) of a given piece of content on Twitter, and (ii) the extent to which individuals (total N = 993) in online surveys on MTurk reported being willing to share that same piece of content. We found that the same news headlines that were more likely to be hypothetically shared on MTurk were also shared more frequently by Twitter users, r = .44. For example, across the observed range of MTurk sharing fractions, a 20 percentage point increase in the fraction of MTurk participants who reported being willing to share a news headline on social media was associated with 10x as many actual shares on Twitter. We also found that the correlation between sharing and various features of the headline was similar using both MTurk and Twitter data. These findings suggest that self-reported sharing intentions collected in online surveys are likely to provide some meaningful insight into what content would actually be shared on social media.

How to kiss and make-up (or not!): Postconflict behavior and affective recovery from conflict.

We designed 3 studies to identify postconflict behaviors that aid or hinder couple partners' emotional recoveries from their conflicts. For Study 1, we created a codebook of 18 postconflict behaviors, derived from 230 participants' daily descriptions of reconciliation efforts over a 3-week period. In Study 2, 340 MTurk participants used a checklist to report which of the 18 behaviors they engaged in following their most recent conflict with their partner. An orthogonal factor analysis revealed four dimensions of postconflict behavior: avoidance (e.g., sulk/withdraw), active repair (e.g., apologize), gain a new perspective (e.g., seek help from friends), and let go (e.g., drop the conflict). In Study 3, 226 cohabiting couples completed a 2-week diary for which they reported on their postconflict reconciliation strategies. Results revealed that postconflict behavior dimensions active repair and gain a new perspective predicted better postconflict residual affective recovery on days with conflict. In contrast, avoidance predicted poorer affective recovery on days with conflict. These results suggest that couples' behavior after conflict can facilitate or inhibit them from reconnecting intimately. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Women’s and Men’s Evaluations of Lesbians and Gay Men and their Levels of Outness

The coming out process is an important component of gay identity development (Troiden 1988) because it allows for an opportunity to overcome feelings of shame and increase self-acceptance (Eliason and Schope 2007). Given the importance of being openly gay (compared to not-openly gay), two experiments were performed. Experiment 1 examined whether heterosexual people’s evaluations of gay individuals are influenced by their outness levels. Experiment 1 recruited 295 U.S. MTurk participants who read a vignette describing either an openly gay or not-openly gay individual. Results illustrate that openly gay targets were evaluated more favorably than not-openly gay targets. Experiment 2 examined whether the reasons justifying the level of outness influence such evaluations. Experiment 2 recruited 492 U.S. MTurk participants who read a vignette describing either an openly gay or not-openly gay individual who provided one of three reasons for their outness level: being true to oneself, honesty in a relationship, and societal prejudice. Results illustrate that the more strongly participants endorsed homonegativity, the less favorably they evaluated openly gay targets. Moreover, female participants perceived societal prejudice and honesty in a relationship as better reasons for outness than being true to oneself. Understanding how levels of outness and reasons for outness can influence evaluations by heterosexual people may help gay people navigate social situations and, perhaps, make informed decisions regarding when it is a safe or possibly unsafe to disclose a gay identity.

Renewal and resurgence phenomena generalize to Amazon's Mechanical Turk.

Amazon's Mechanical Turk (MTurk) is a crowdsourcing platform that provides researchers with the potential for obtaining behavioral data for very little cost. However, the extent to which the results of common behavioral phenomena found in basic, translational, and applied laboratories may be reproduced (as a first step towards prospective research) via MTurk remains relatively unexplored. We evaluated renewal and resurgence arrangements using MTurk as the subject recruitment platform as a first step to determining the generality of the obtained data. Results suggested that MTurk participants produced renewal and resurgence data similar to those reported in basic, translational, and applied studies.

How caring is “nullified”: Strong racial identity eliminates White participant empathy effects when police shoot an unarmed Black male.

Objective: Despite growing national and international attention given to White police officers’ shooting unarmed Black men, there has been minimal empirical focus on the factors that might influence how Whites respond to such violence. Among a sample of White Americans, we assessed whether the influence of trait empathy on responses to a White police officer shooting an unarmed Black male would be moderated by White racial identity. Method: 349 MTurk participants completed measures of racial identity and trait empathy, then read a passage describing a White police officer shooting an unarmed stereotypical or counterstereotypical Black male. The victim's conformity to racial stereotypes was manipulated as a potential moderator. Participants subsequently indicated whether the Black victim was perceived as a threat, whether the White police officer was viewed as racist, and the extent to which the victim should be blamed and compensated. Results: At lower levels of White racial identity, there was less perceived victim threat, greater perceptions of officer racism, and more favorable victim responding for high empathic relative to low empathic participants. Conversely, at higher levels of racial identity, the impact of empathy was significantly diminished. This moderation effect only occurred when the victim conformed to common racial stereotypes. Conclusions: The findings indicate that the empathy and racial identification of White observers, and Black victim characteristics drive reactions to both the victim and White police officer. Thus, these variables have implications for jury selection, investigations of police misconduct, and even how the media portrays victims of interracial violence.

Plea decision-making: the influence of attorney expertise, trustworthiness, and recommendation

ABSTRACT Attorney recommendations influence defendant plea decisions; and the degree of influence likely rests on the perceived trustworthiness and level of expertise of the attorney (factors of source credibility). We explored attorney source credibility factors and how these characteristics influence defendants’ plea decision-making. MTurk participants read a hypothetical plea scenario and were asked to imagine themselves as the defendant in a DWI/DUI case making a plea decision; in the scenario, we manipulated the defense attorney’s level of trustworthiness, expertise, and plea recommendation. There was a significant interaction between attorney recommendation and trustworthiness on defendants’ plea decisions; participants who were advised to accept the guilty plea were more likely to plead guilty when the attorney was high in trustworthiness compared to low in trustworthiness. Attorney trustworthiness did not affect plea decisions for defendants advised to reject the guilty plea. Importantly, attorney trustworthiness affected defendants’ decision to follow the attorney’s recommendation and ultimate plea decision (regardless of expertise), and attorney expertise affected defendants’ confidence in their decision (regardless of trustworthiness). Results suggest individual-level characteristics of defense attorneys affect the influence of the attorney and their recommendation, and ultimately defendants’ plea decision-making.

Product performance implications of framing white space in advertising

Purpose The purpose of this paper is to investigate the impact of framing white space around a product image (e.g. silver polish) and its intended effect image (e.g. tarnish-free, shiny silverware) in an ad on the performance perceptions of the advertised product. Design/methodology/approach Four experimentally designed studies demonstrate a positive impact of framing the white space around the product and effect images. A total of 270 MTurk participants provided data for these studies through online surveys. Findings Findings across four different product categories reveal that when the white space surrounding the product and effect images in an ad is enclosed by a frame, consumers perceive product performance to be significantly higher than when there is no white space framing. This effect is mediated by enhanced perceived fit between the two images based on the white space framing. Research limitations/implications The current findings contribute to multiple marketing research streams including the use of white space and frames in visual promotions, as well as the literature on product performance, perceived fit, stylistic manipulations (such as the framing of white space in this case) and advertising communication. Practical implications The use of product and effect visuals in advertising to convey product performance is one of the most widely used tactics in the consumer packaged goods industry. Managers and creative directors can leverage the current findings to improve consumer product performance perceptions by using a simple, stylistic manipulation (which does not alter the core ad elements but only the production aspects) of framing the white space around the product and effect images in their product ads. Originality/value To date, there is only a small body of research examining the role of white space and visual frames on marketing-related outcomes. However, none of that looks at white space framing. The current work is the first to examine how localization of white space around the product and effect images improves consumer perceptions about the advertised product’s performance.

Profiling HEXACO factors on self-presentation tactic use

The idea that personality is closely connected to the strategic presentation of the self is occasionally echoed across social-personality and clinical psychology; although self-presentation may be a feature of personality, investigations into this possibility are rather rare and possess various shortcomings that make them somewhat inconclusive. To advance understanding, MTurk participants (N = 298) first completed indices of HEAXCO factors and then reported their past use of various self-presentation tactics. Data revealed that each HEXACO factor generally positively related to self-presentation tactics that, together, portray the factor's social orientation and characteristics. In sum, our data show that the apparent basic elements of personality relate to strategic portrayals of identities consistent with those elements; we discuss the implications of these findings for understanding self-presentation and personality.

The Interaction of Perceived Subjectivity and Pay Transparency on Professional Judgment in a Profit Pool Setting: The Case of Large Law Firms

ABSTRACTThis paper examines how the interaction of perceived subjectivity and pay transparency in profit allocation is associated with an important aspect of law partners' professional judgment, namely their tendency to accede to the wishes of their client and fellow partner (labeled hereafter as partner accedence). Based on interviews with 56 corporate law partners working in large Canadian law firms, we find higher partner accedence in a less subjective system than in a more subjective system, but only under no pay transparency. We find that pay transparency (versus no transparency) is associated with increased accedence in a more subjective system, but it is marginally associated with decreased accedence in a less subjective system. In an experiment where we randomly assign MTurk participants to conditions, we replicate the finding that pay transparency (versus no transparency) has a more positive effect on partner accedence as subjectivity level increases.Data Availability: Lawyers participated in the study upon which this paper is based only after signing agreements that strict confidentiality of all data would be maintained by the researchers. As such, we are bound by these confidentiality agreements with individual lawyers interviewed for the study. Experiment data from Amazon Mechanical Turk are available from the authors.JEL Classifications: M12; M40; M52.

The Effects of Perspective Taking Implementing Intentions on Employee Evaluations and Hostile Sexism

The current research examined whether gender bias in the workplace could be reduced through perspective taking implementation intentions, which are if–then statements that specify how to accomplish goals (Gollwitzer, 1999). Amazon MTurk participants (N = 180, 53% male) learned they would complete a two-step performance review for a consulting company. Prior to receiving a male or female employee’s record, all participants were given a goal strategy to be fair in their review, with half also receiving an if–then strategy that encouraged perspective taking. Participants rated the employee on three work related dimensions (skillset, performance, and traits), provided an overall promotion recommendation, and completed the Ambivalent Sexism Inventory (Glick & Fiske, 1996). Although we did not find evidence of gender bias on the work measures, we found that the implementation intention strategy resulted in more positive employee evaluations overall and less hostile sexism than a simple goal strategy. We discuss the potential organizational benefits of employing perspective taking implementation intentions.

Assessment of Patient Nondisclosures to Clinicians of Experiencing Imminent Threats

Patient disclosure to their clinician about experiencing an imminent threat is a critical step toward receiving support or assistance. To examine the frequency of patients not disclosing their experience of imminent threats to their clinician and their reasons for doing so. Survey study incorporating results from 2 national, nonprobability samples of 2011 US adults recruited from Amazon's Mechanical Turk (MTurk) from March 16 to 30, 2015, and 2499 recruited from Survey Sampling International (SSI) from November 6 to 17, 2015. Data analysis was conducted from December 20 to 28, 2018. Self-reported nondisclosure of 4 types of imminent threats (depression, suicidality, abuse, and sexual assault) to their clinician and reasons for nondisclosure. There were 2011 participants in the MTurk sample (1210 [60.3%] female; 1696 [60.2%] white; mean [SD] age, 35.7 [12.4] years; age range, 18-79 years) and 2499 participants (1273 [51.0%] female; 1968 [78.8%] white; mean [SD] age, 61.0 [7.6] years; age range, 50-91 years) in the SSI sample. Among those who reported experiencing at least 1 of the 4 imminent threats, 613 of 1292 MTurk participants (47.5%) and 581 of 1453 SSI participants (40.0%) withheld information from their clinician. The most commonly endorsed reasons for withholding this information included being embarrassed (MTurk: 72.7%; SSI: 70.9%), not wanting to be judged or lectured (MTurk: 66.4%; SSI: 53.4%), and not wanting to engage in a difficult follow-up behavior (MTurk: 62.4%; SSI: 51.1%). Respondents who experienced at least 1 of the 4 imminent threats had significantly higher odds of nondisclosure in both samples if they were female (MTurk: odds ratio [OR], 1.66 [95% CI, 1.30-2.11]; and SSI: OR, 1.33 [95% CI, 1.07-1.67]) or younger (MTurk: OR, 0.99 [95% CI, 0.98-1.00]; and SSI: OR, 0.98 [95% CI, 0.97-1.00]). Worse self-rated health was also associated with nondisclosure, but only in the SSI sample (OR, 0.85 [95% CI, 0.74-0.96]). This study suggests that many people withhold information from their clinicians about imminent health threats that they face. A better understanding of how to increase patients' comfort with reporting this information is critical to allowing clinicians to help patients mitigate these potentially life-threatening risks.

Moral Content in Workplace Conflict

Using moral foundations theory and text analysis tools, two studies were conducted to explore the moral content in workplace conflict accounts. In Study 1, conflict narratives provided by MTurk participants were text-analyzed to uncover moral content differences across the two main types of conflicts–relationship conflict and task conflict. This analysis revealed that relationship conflicts had greater purity language than task conflicts and task conflicts had greater authority language than relationship conflicts. In Study 2, the high purity language in relationship conflict was replicated in a text-analysis of moral foundations words in a sample of 100 passive aggressive notes from the website PassiveAggressiveNotes.com. These studies advance moral content as a key differentiator between task and relationship conflict.

Black Intragroup Empathic Responding to Police Interracial Violence: Effects of Victim Stereotypicality and Blacks’ Racial Identification

Despite the public outrage in response to police violence against unarmed Black men, work on the psychological dynamics of reactions to these incidents is relatively rare. The present research examined whether empathy for a Black male victim of White police interracial violence would vary as a function of victim stereotypicality (stereotypic/counterstereotypic) and Black participant racial identity. In Study 1, 140 Black participants were recruited from Amazon Mechanical Turk (MTurk). As hypothesized, Black participants low in racial identification reported less empathy for the stereotypical relative to the counterstereotypical victim. Those high in racial identification showed relatively high levels of empathy regardless of the characteristics of the Black victim. Study 2 replicated these effects with 263 Black MTurk participants. This research highlights the value of considering individual differences in the Black observers (racial identification) and the characteristics of Black victims to better understand the psychological processes involved in intragroup responses to police violence.