MTurk Respondents Research Articles

White Knighting: How Help Reinforces Gender Differences Between Men and Women

Individuals often prize helping behaviors in the workplace, but these behaviors can potentially be harmful to those who receive help. Two online experiments were conducted to determine whether dependency-oriented help, which consists of solving a problem for the beneficiary (Nadler, 2002), has negative consequences for female beneficiaries and positive consequences for male helpers (Studies 1 and 2: 100 and 203 U.S. MTurk respondents respectively). I found that observers perceived women who received dependency-oriented help from men as being lower in status (Study 2), competence (Studies 1 and 2), and promotability (Studies 1 and 2) than women who received autonomy-oriented help from men. Contrary to expectations, observers’ endorsement of benevolent sexism moderated these effects such that their evaluations of female beneficiaries of dependency-oriented help was more negative the less they endorsed benevolent sexism (Studies 1 and 2). Finally, observers evaluated helpers who gave dependency-oriented help as lower in status and competence than helpers who give autonomy-oriented help (Study 2). Overall, the present findings suggest ways in which helping behaviors can have harmful effects on both helpers and beneficiaries of help in the workplace.

How serious is the ‘carelessness’ problem on Mechanical Turk?

ABSTRACTThis study compared the quality of survey data collected from Mechanical Turk (MTurk) workers and college students. Three groups of participants completed the same survey. MTurk respondents completed the survey as paid workers using the Mechanical Turk crowdsourcing platform. Student Online respondents also completed the survey online after having been recruited in class. Finally, Student Paper-and-Pencil respondents completed the survey on paper in a classroom setting. Validity checks embedded in the survey were designed to gauge participants’ haste and carelessness in survey completion. MTurk respondents were significantly more likely to fail validity checks by contradicting their own answers or simply completing the survey too quickly. Student groups showed fewer careless mistakes and longer completion times. The MTurk sample tended to be older, more educated, and more ethnically diverse than student samples. Results suggest that researchers should pay special attention to the use of validity checks when recruiting MTurk samples.

A new twist on the gig economy: conducting surveys on Amazon Mechanical Turk

There is growing interest in using online outsourcing platforms that are part of the “gig economy” to conduct surveys for academic research. This interest has been driven in part by the belief that compared to traditional survey data collection methods, internet-based marketplaces such as Amazon Mechanical Turk (MTurk) enable one to collect survey data cheaper and faster from a larger, more diverse participant pool. However, many have questioned whether models based on survey data from these online marketplaces are similar to models based on survey data from more traditional platforms. To investigate this research question, we used MTurk and Qualtrics (a traditional market research firm) to survey air travelers. Our results showed that MTurk and Qualtrics respondents had distinct socio-demographic characteristics, but we found no statistical evidence for different air trip characteristics. In our data, proportionately more MTurk respondents were in the younger, single, male, and lower-income categories than for Qualtrics respondents. We found that airline itinerary choice models estimated from the MTurk and Qualtrics survey data were similar, with the key difference related to price sensitivities. Although our results provide evidence that MTurk can be used for travel demand modeling applications, we offer words of caution for others planning to conduct surveys in online marketplaces, particularly for those seeking to recruit more than 1000 participants or for those targeting specific geographic areas.

Unemployed and Turking: Is Amazon Mechanical Turk a Viable Source for Unemployment Research?

Organizational research conducted using data obtained from Amazon’s Mechanical Turk (MTurk) has proliferated in recent years. However, there are critical concerns over the efficacy of these data for employment research. The purpose of this study was, alternatively, to assess the efficacy of MTurk data for unemployment research. We did so through four distinct analyses in which we sought to replicate existing findings that have been established using unemployment field data. Findings indicate unemployed MTurk sample demographics closely reflect US national unemployed population rates, with the exception of the MTurk sample being substantially younger, on average. Moreover, analyses aimed at replicating the effects of latent resources (self- esteem, social support, and meaning in life) on well-being among the unemployed MTurk sample were highly successful. However, analyses aimed at replicating effects requiring a comparison of unemployed to employed MTurk respondents were less successful. Coupled with evidence of differences between the employed MTurk sample and US national employment population rates on age and marital status, study implications may say as much about the potential (lack of) generalizability of findings from MTurk data for employment research as they do about the potential generalizability of findings from MTurk data for unemployment research.

How Low Can You(r Power) Go? It Depends on Whether You are Male or Female

Three online experiments were conducted to determine whether gender differences in feelings of power are most evident in objectively lower or higher power situations (total n = 1360; Studies 1 and 2: 238 and 771 U.S. MTurk respondents respectively; Study 3: 351 Canadian university students). We focused on evaluating whether men’s and women’s responses were in line with a cushioning account, whereby the higher power generally accorded to men as a group essentially serves as a back-up power source for men in lower power positions. We also evaluated support for a ceiling account, whereby women’s feelings of power are limited in higher power positions. Results were consistent with the cushioning account: Men reported feeling more powerful than women did when imagining or recalling occupying a lower power position and in a control baseline, but no gender difference was evident under higher power conditions. Results further revealed that women’s feelings of power were more variable across lower versus higher power positions than were men’s and indicated that women’s feelings of power are quite responsive to situationally afforded high power when it is available. Overall our findings suggest that occupying a higher power role eradicates gender differences in feelings of power that are otherwise evident and thus has an equalizing effect.

More Than Meets the Eye: The Effect of Intercanthal Distance on Perception of Beauty and Personality.

In judging normalcy, surgeons rely on established facial anthropometric measures and proportions. However, there exists a range of "normal," and a degree of disproportion may be considered more attractive. The authors set out to determine how changes in only intercanthal distance affect the layperson's perception of beauty and personality traits of a face. The authors used Amazon.com's Mechanical Turk (MTurk), a crowdsourcing tool, to determine how changes in intercanthal distance affect overall perception of beauty and personality. MTurk respondents provided demographic information and were asked to survey 16 female subjects, each digitally edited to be hypoteloric or hyperteloric. Data were collected from 490 MTurk crowd raters. Paired t test analysis found that respondents perceived subjects to be more submissive, friendly, and attractive with increased intercanthal distance (P < 0.05). Women respondents were less likely to perceive change in regards to how unthreatening and how intelligent the subject appeared upon intercanthal widening (P < 0.05). Compared with Caucasian respondents, minorities (Asian- and African-American) were more likely to perceive difference in submissiveness, threat, intelligence, and attractiveness with increased intercanthal distance (P < 0.05). All respondents >46 years of age were less likely to perceive a change in any of the 7 traits upon intercanthal widening, compared with respondents between 18 and 25 years of age (P < 0.05). The layperson perceives significant increases in a female subject's submissivness, friendliness, and attractiveness with an intercanthal distance increase of 10% from normal. Surgeons should be aware of this when correcting hypertelorism, given the potential positive impact of a slightly increased intercanthal distance on perceived beauty and personality.

Home alone: Why people believe others' social lives are richer than their own.

Although decades of research show that people tend to see themselves in the best possible light, we present evidence that people have a surprisingly grim outlook on their social lives. In 11 studies (N = 3,293; including 3 preregistered), we find that most people think that others lead richer and more active social lives than they do themselves. We show that this bias holds across multiple populations (college students, MTurk respondents, shoppers at a local mall, and participants from a large, income-stratified online panel), correlates strongly with well-being, and is particularly acute for social activities (e.g., the number of parties one attends or proximity to the "inner circle" of one's social sphere). We argue that this pessimistic bias stems from the fact that trendsetters and socialites come most easily to mind as a standard of comparison and show that reducing the availability of extremely social people eliminates this bias. We conclude by discussing implications for research on social comparison and self-enhancement. (PsycINFO Database Record

A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples

With the exploding use of Internet surveys, research efforts and data quality are increasingly subject to the effects of respondents who do not give the required attention to survey questions and who speed through the survey, or who intentionally cheat with their answers. We investigate respondent integrity and data quality for samples drawn from a “Regular” online panel and from Amazon's MTurk. New metrics for assessing sample integrity and online data quality are introduced. Overall, MTurk respondents in both respondent groups took less time to answer questions. The non-USA MTurk group deviated most from correct answers in attention filter questions and had more duplicate IP addresses. In addition, the results from the three Internet sample sources are substantively different. The choice of an Internet survey sample vendor is critical, as it can impact sample composition, respondent integrity, data quality, data structure and substantive results.

The Demographic and Political Composition of Mechanical Turk Samples

One of the most notable recent developments in survey research is the increased usage of online convenience samples drawn from Amazon’s Mechanical Turk (MTurk). While scholars have noted various social and political differences (e.g., age, partisanship) between MTurk and population-based samples, the breadth and depth of these variations remain unclear. We investigate the extent to which MTurk samples differ from population samples, and the underlying nature of these differences. We do so by replicating items from the population-based American National Election Studies (ANES) 2012 Time Series Study in a survey administered to a sample of MTurk respondents. With few exceptions, we not only find that MTurk respondents differ significantly from respondents completing the 2012 ANES via the Web but also that most differences are reduced considerably when controlling for easily measurable sample features. Thus, MTurk respondents do not appear to differ fundamentally from population-based respondents in unmeasurable ways. This suggests that MTurk data can be used to advance research programs, particularly if researchers measure and account for a range of political and demographic variables as needed.

What is Workplace Incivility? An Investigation of Relational Schemas (WITHDRAWN)

Stemming from the recognition that we know little about how employees make sense of incivility at work, this qualitative study investigated relational schemas of workplace incivility to refine our theoretical understanding of the incivility construct and begin collecting data to support development of applied interventions. To do so, 205 currently employed American MTurk respondents completed online surveys asking open-ended questions about various schematic features of workplace incivility and normative questions about how they believe incivility should be properly addressed in the workplace. Phenomenological methodology was utilized. Workplace incivility was defined in relational schemas similar to current scholarly conceptualizations, but respondents’ examples of prototypical incivility spanned all manner of minor to severe forms of mistreatment. Perpetrators, targets, bystanders, and interveners were identified as the primary roles engaged with incivility. Beyond these schemas, respondents also overwhelmingly called for active prevention and intervention against workplace incivility, spearheaded primarily by their organizational leadership. Researchers should take note of results indicating construct proliferation concerns in the workplace mistreatment literature, while practitioners should consider the role of organizational leadership in preventing and addressing incivility in the workplace.

“Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents

As Amazon’s Mechanical Turk (MTurk) has surged in popularity throughout political science, scholars have increasingly challenged the external validity of inferences made drawing upon MTurk samples. At workshops and conferences experimental and survey-based researchers hear questions about the demographic characteristics, political preferences, occupation, and geographic location of MTurk respondents. In this paper we answer these questions and present a number of novel results. By introducing a new benchmark comparison for MTurk surveys, the Cooperative Congressional Election Survey, we compare the joint distributions of age, gender, and race among MTurk respondents within the United States. In addition, we compare political, occupational, and geographical information about respondents from MTurk and CCES. Throughout the paper we show several ways that political scientists can use the strengths of MTurk to attract respondents with specific characteristics of interest to best answer their substantive research questions.