Mechanical Turk Workers Research Articles

College students’ and Mechanical Turk workers’ environmental factors while completing online surveys

College students’ and Mechanical Turk workers’ environmental factors while completing online surveys

Substance Use Frequency Relates to Suicidal Ideation Through Perceived Burdensomeness and to Suicide Attempts Through Capability for Suicide

Background Although substance use has been linked to both suicidal ideation and suicide attempts, the factors underlying these relations remain unclear. The Interpersonal-Psychological Theory of Suicide (ITS) provides a framework for understanding how substance use may increase suicide risk. The purpose of the current study was to examine if frequency of substance use is indirectly related to suicidal ideation and suicide attempts through core ITS variables (i.e., burdensomeness, thwarted belongingness, and suicide capability). Methods An online sample of Mechanical Turk workers (N = 365) completed measures assessing substance use frequency, burdensomeness, thwarted belongingness, suicide capability, suicidal ideation, and suicide attempts. Results After controlling for relevant clinical and demographic covariates, substance use frequency was indirectly related to suicidal ideation through burdensomeness but not thwarted belongingness. Substance use frequency was indirectly related to suicide attempts through suicide capability only. Limitations The cross-sectional design precludes conclusions about the precise nature and direction of the relations examined. The use of a community sample limits generalizability to more severe substance using samples. Conclusions Results highlight the relevance of distinct ITS factors in the relation between substance use frequency and both suicidal ideation and suicide attempts. Results may inform specific targets for novel interventions aimed at reducing suicide risk among substance-using individuals. Highlights Substance use frequency was indirectly related to SI through burdensomeness. Substance use frequency was not indirectly related to SI through thwarted belongingness. Substance use frequency was indirectly related to SA only through suicide capability.

Financial scarcity is indirectly related to multiple aspects of executive function through stress and the strength of association depends on childhood poverty

Financial scarcity, or having insufficient financial resources to meet needs, directly impairs multiple aspects of executive function (EF). Financial scarcity is also stressful and stress impairs EF, however, whether stress mediates the effect of scarcity on EF, and whether these associations vary by either the aspect of EF being measured, or self-reported developmental history of poverty, is not known. Data were collected from a cross-sectional sample of 249 female Amazon Mechanical Turk workers who self-reported their stress using Cohen's perceived stress scale and their EF using the 75-item Behavior Rating Inventory of Executive Function. They also reported their history of poverty during childhood at ages 0–6, 6–12, and 12–18 years. We measured current financial scarcity using questions that assess five aspects of perceived inability to meet one's basic needs. We used path analysis to test our primary hypothesis that higher financial scarcity is associated with higher stress and more challenges with components of EF overall. Specifically, we tested whether there are indirect effects from scarcity to three specific aspects of EF (behavioral regulation, emotion regulation, and meta-cognition) through stress. Second, we tested whether the model fit and magnitude of direct effects varied by childhood poverty history. Results supported hypotheses that stress would be negatively associated with multiple EF aspects (behavioral regulation and emotional regulation); and associations were stronger for women reporting early poverty-related experiences. Interventions designed to alleviate financial scarcity, and address stress management, may be considered as potential ways to support EF for people with scarce financial resources.

Psychosocial determinants of anxiety about the COVID-19 pandemic.

Pandemic health threats can cause considerable anxiety, but not all individuals react similarly. To understand the sources of this variability, we applied a theoretical model developed during the H1N1 pandemic of 2009 to quantify relationships among intolerance of uncertainty, stress appraisals, and coping style that predict anxiety about the COVID-19 pandemic. We surveyed 1579 U.S. Amazon Mechanical Turk workers in April 2020. Using structural equation modeling, we found that individuals who were more intolerant of uncertainty reported higher appraisals of threat, stress, and other-control, which predicted higher anxiety when emotion-focused coping was engaged, and lower anxiety when problem-focused coping was engaged. Political affiliation moderated these effects, such that conservatives relied more on self-control and other-control appraisals to mitigate anxiety than independents or liberals. These results show that how people appraise and cope with their stress interacts with political ideology to shape anxiety in the face of a global health threat.

Who creates strong passwords when nudging fails

The use of strong passwords is viewed as a recommended cybersecurity practice, as the hacking of weak passwords led to major cybersecurity breaches. The present research investigated whether nudging with messages based on participants’ self-schemas could lead them to create stronger passwords. We modeled our study on prior health-related research demonstrating positive results using messages based on self-schema categories (i.e., True Colors categories -compassionate, loyal, intellectual, and adventurous). We carried out an online study, one with 256 (185 women, 66 men, 5 other) undergraduates and one with 424 (240 men, 179 women, 5 other) Amazon Mechanical Turk (MTurk) workers, in which we randomly assigned participants to receive messages that matched or mismatched their self-schema. We also investigated whether differences across the Big Five personality traits, secure password knowledge, attitudes and behavior, need for cognition, and general risk-taking predicted the strength of passwords that participants created during the study. Multiple individual difference variables predicted password strength (i.e., conscientiousness, emotional stability, need for cognition, self-reported secure password knowledge, attitude, and behavior, and general risk-taking). MTurk workers had higher levels of cybersecurity knowledge and created stronger passwords than college students. The nudging messages did not lead to stronger passwords. Implications for strategies to increase the use of secure passwords are discussed.

Orthorexia nervosa vs. healthy orthorexia: relationships with disordered eating, eating behavior, and healthy lifestyle choices

BackgroundOrthorexia nervosa (OrNe) is a pattern of disordered eating behavior characterized by excessive preoccupation with overvalued ideas about healthy eating. Healthy orthorexia (HeOr) refers to a non-pathological interest in healthy eating and nutrition. Despite converging evidence that OrNe is correlated with measures of psychopathology and personality traits, there is relatively little empirical data relating OrNe and HeOr to eating and health-promoting behavior.MethodsAim 1: exploratory structural equation modeling was used to evaluate the factor structure of an English-language version of the Teruel Orthorexia Scale, the only measure of OrNe that also assesses HeOr in two samples (Yoga practitioners, N = 469, and Mechanical Turk workers, N = 453). Aim 2: conducted in the Mechanical Turk sample, partial correlation and linear regression analyses were used to evaluate the relationships of OrNe and HeOr with other symptoms of, and clinical impairment from, disordered eating, food choices, diet quality, body mass index (BMI), nutrition knowledge, alcohol/tobacco/vaping use, and sedentary behavior.ResultsThe two-factor structure of the TOS was replicated. HeOr and OrNe had opposing relationships with measures of disordered eating behavior and distress, food choices, diet quality, and nutrition knowledge. HeOr was negatively related to BMI, whereas OrNe was positively related to substance use and sedentary behavior.ConclusionsOrNe and HeOr are distinct latent constructs, with the latter reflecting non-pathological behavior. Only OrNe is related to elevated disordered eating and clinical impairment. Despite involving obsessions and compulsions related to healthy eating, OrNe was associated with relatively unhealthy eating and other lifestyle behaviors.Level of evidenceLevel V, descriptive cross-sectional study.

Announcement of formal controls as phase-shifting perceptions: their determinants and moderating role in the context of mobile loafing

PurposeMobile loafing, or non-work-related mobile computing, is deviant workplace behavior that can reduce productivity and increase cybersecurity risks. To thwart mobile loafing, organizations often adopt formal controls that encompass rules and policies. These formal controls can serve as a phase-shifting event. Phase shifting is a process where individuals reevaluate and revise their perceptions of the regulation of deviant behaviors. Despite the importance of understanding this process, little research has examined the announcement of formal controls as an impetus for phase shifting. The primary objectives of this study were to induce a phase-shifting perception in an organizational setting and explore its determinants and moderating role in the context of mobile loafing.Design/methodology/approachThe authors proposed and tested a model using two-wave data collected from 231 Amazon Mechanical Turk workers. To test the research hypotheses, they used covariance-based structural equation modeling and logistic regression.FindingsThe authors found that peer's mobile loafing and neutralization positively influence mobile-loafing intention before and after the announcement of formal controls. This research also shows that the higher an employee's neutralization, the likelier they perceive the announcement of formal controls as phase shifting. Also, the authors found that the moderating effect of phase-shifting perceptions functions in such a way that the relationship between T1 and T2 mobile-loafing intention is weaker when employees perceive the announcement of formal controls as a phase-shifting event.Practical implicationsThe authors’ results provide managers with useful insights into effectively using formal controls to mitigate employees' deviant behavior. To effectively use formal controls, managers should articulate formal controls that can trigger employees to revise their perceptions of counterproductive workplace behavior policies.Originality/valueThis study is one of the first in information systems research to empirically examine the announcement of formal controls as a phase-shifting event and explore its antecedents and moderating role in the context of deviant workplace behavior in general and mobile loafing in particular.

Faith and science mindsets as predictors of COVID-19 concern: A three-wave longitudinal study

The COVID-19 pandemic allowed for a naturalistic, longitudinal investigation of the relationship between faith and science mindsets and concern about COVID-19. Our goal was to examine two possible directional relationships: (Model 1) COVID-19 concern ➔ disease avoidance and self-protection motivations ➔ science and faith mindsets versus (Model 2) science and faith mindsets ➔ COVID-19 concern. We surveyed 858 Mechanical Turk workers in three waves of a study conducted in March, April, and June 2020. We found that science mindsets increased whereas faith mindsets decreased (regardless of religious type) during the early months of the pandemic. Further, bivariate correlations and autoregressive cross-lagged analyses indicated that science mindset was positive predictor of COVID-19 concern, in support of Model 2. Faith mindset was not associated with COVID-19 concern. However, faith mindset was a negative predictor of science mindset. We discuss the need for more research regarding the influence of science and faith mindsets as well as the societal consequences of the pandemic.

Designing a Gardening Intervention to Improve Health: Recommendations From New and Experienced Gardeners

Abstract Objectives Gardeners consume more fruits and vegetables (F&V) and have lower body mass indexes (BMI) than non-gardeners, making gardening a potential public health intervention. Little is known about information needed to teach gardening skills to novice gardeners. This study investigated what never gardeners (NG) and experienced gardeners (EG) perceived as necessary information to teach gardening. Methods Amazon Mechanical Turk (MTurk) workers responded to demographic and gardening questions via an online survey. After reviewing responses for completeness, 415 participants were included in the analysis. Participants were characterized by gardening experience level based on the total years they had gardened (0 years = NG, 1 + years = EG). In open ended questions, NG were asked: “What, if anything, would encourage you to start vegetable gardening?” and “What information do you think you would need to start a vegetable garden?” Experienced gardeners were asked “What information would be most important to teach someone who is new to vegetable gardening?” Qualitative responses were coded and the most frequent responses are presented. Results The overall sample had a mean age of 40 years, was 47% female, 80% white, and 68% with a college degree. There were 203 NG (48.9%) and 241 EG (51.1%). For reasons that would motivate NG to start a garden, the two most frequently indicated were, 1) having the space needed to start a garden (n = 33, 16%) or 2) if it would be a way to save money (20, 9.7%). For NGs the most frequently mentioned response to what information they would need to start vegetable gardening on their own was information regarding maintenance, such as sunlight, water, weeding, and spacing (n = 30, 14.8%). For EGs the most frequent response to what information would be most important to teach someone new to vegetable gardening, was also information related to maintenance such as, such as sunlight, water, weeding, and spacing (n = 41, 17%). Conclusions Gardening is a potential public health intervention that can influence health. Understanding space needs, costs of starting a garden, and knowing when to plant are key learning topics needed when developing these interventions for new gardeners. Future analyses will investigate the preferences and characteristics of those who would like to participate in an online versus an in-person gardening intervention. Funding Sources None.

CurmElo: The theory and practice of a forced-choice approach to producing preference rankings.

We introduce CurmElo, a forced-choice approach to producing a preference ranking of an arbitrary set of objects that combines the Elo algorithm with novel techniques for detecting and correcting for (1) preference heterogeneity induced polarization in preferences among raters, and (2) intransitivity in preference rankings. We detail the application of CurmElo to the problem of generating approximately preference-neutral identifiers, in this case four-letter and five-letter nonsense words patterned on the phonological conventions of the English language, using a population of Amazon Mechanical Turk workers. We find evidence that human raters have significant non-uniform preferences over these nonsense words, and we detail the consequences of this finding for social science work that utilizes identifiers without accounting for the bias this can induce. In addition, we describe how CurmElo can be used to produce rankings of arbitrary features or dimensions of preference of a set of objects relative to a population of raters.

How Candidates’ Age and Gender Predict Voter Preference in a Hypothetical Election

Are preferences for political candidates influenced by how old they appear to be? Amazon Mechanical Turk workers and undergraduate students were shown photos of 93 state legislators as candidates in hypothetical elections. Other information about the candidates (e.g., party affiliation) was held constant, randomized, or not presented. For very young candidates (< 35 years old), participants favored women over men. However, participants’ intention to vote for male candidates increased with age until candidates were about 45 years old and then slightly decreased. In contrast, participants’ intention to vote for female candidates consistently decreased with candidates’ age. Perceived attractiveness and warmth accounted for some of the gender differences in the effect of candidates’ perceived age.

The Influence of Stigmatizing Messages on Danger Appraisal: Examining the Model of Stigma Communication for Opioid-Related Stigma, Policy Support, and Related Outcomes

ABSTRACT Drug overdose is a leading cause of injury and death in the United States, and opioids are among the most significant of causes. For people with opioid use disorders (OUDs), opioid stigma can lead to devastating consequences, including anxiety and depression. Still, mass media may stigmatize people with OUDs by ascribing stigmatizing labels (e.g., “opioid addict”) and other stigma features to those individuals. However, it is unclear how these stigmatizing messages influence public perceptions of people with OUDs and public support for rehabilitation and Naloxone administration policies. The model of stigma communication (MSC) provides a framework for understanding these relationships. This study used the MSC in two online factorial experiments, the first among college undergraduates (N = 231) and the second among Amazon Mechanical Turk workers (N = 245), to examine how stigmatizing messages about people with OUDs influence stigma-related outcomes. Results reveal that opioid stigma messages influence different outcomes depending on the content of those messages. Classification messages with a stigmatizing mark (e.g., “Alex appears unkempt”) and label (e.g., “opioid addict”) led to greater perceptions of dangerousness and threat in both studies. High stigma classification messages also led to an increased desire for behavioral regulation and social distance in Study 2. Structural equation modeling in Study 1 also supported the applicability of the MSC in the opioid context. Implications for health communication theory development and practice are discussed.

Measuring confidence engaging in computer activities at different skill levels: Development and validation of the Brief Inventory of Technology Self-Efficacy (BITS)

People in educational settings are expected to effectively use increasingly complex technology. One of the most important factors in technology use is computer self-efficacy (CSE). Although frequently assessed, available measures of CSE have various issues. Additionally, there is no current measure that delineates CSE at different skill levels and specifically targets usage in applied fields. Consequently, this set of seven studies describes the development and psychometric properties of two CSE scales that address these issues: the 18-item Brief Inventory of Technology Self-Efficacy (BITS) and the six-item Brief Inventory of Technology Self-Efficacy – Short Form (BITS-SF). The first study encompassed conceptualization and item development. The second and third studies explored and confirmed the three-factor structure of the BITS (Novice, Advanced, and Expert) in samples of Mechanical Turk workers and college students, respectively. The fourth study examined the multigroup invariance of this factor structure across men and women. The fifth study showed evidence for the temporal stability of the BITS. The sixth study assessed for validity evidence for both scales. Finally, the seventh study used latent class analysis to determine the underlying classes of the BITS-SF. The uses of the BITS and BITS-SF are discussed. Both measures are available at www.bitssurvey.com.

Traditional and Modern Convenience Samples: An Investigation of College Student, Mechanical Turk, and Mechanical Turk College Student Samples

Two of the most popular populations for convenience sampling used in the psychological sciences are college students and Mechanical Turk (MTurk) workers. College students represent a traditional type of convenience sample, whereas MTurk workers provide a more modern source of data. However, little research has examined how these populations differ from each other in salient characteristics. Additionally, no research to date has investigated how MTurk college students (a traditional sample collected using modern methods) compare to either population. The current study examined 1,248 participants comprising three samples: MTurk noncollege workers ( n = 533), MTurk college students ( n = 385), and traditional college students ( n = 330). We compared the samples on demographic characteristics, study completion time, attention, and individual difference variables (i.e., personality, social desirability, need for cognition, personal values, and social attitudes). We examined the individual difference variables in terms of mean responses, internal consistency estimates, and subscale intercorrelations. Results indicated the samples were distinct from each other in terms of all variables assessed; in addition, adding demographic characteristics as covariates to the analyses of individual difference variables did not effectively account for sample differences. We conclude that research using convenience samples should take these differences into account.

Better Crowdcoding: Strategies for Promoting Accuracy in Crowdsourced Content Analysis

ABSTRACT In this work, we evaluate different instruction strategies to improve the quality of crowdcoding for the concept of civility. We test the effectiveness of training, codebooks, and their combination through 2 × 2 experiments conducted on two different populations – students and Amazon Mechanical Turk workers. In addition, we perform simulations to evaluate the trade-off between cost and performance associated with different instructional strategies and the number of human coders. We find that training improves crowdcoding quality, while codebooks do not. We further show that relying on several human coders and applying majority rule to their assessments significantly improves performance.

The emotion storyboard: A method to examine social judgments of emotion.

As perceivers, we need to understand context to make social judgments about emotion, such as judging whether emotion is appropriate. We propose a graphic novel-like method, the emotion storyboard, for use in research on social judgments of emotion. Across two studies, participants were randomly assigned to read emotion storyboards or written vignettes to compare the efficacy of the emotion storyboard to that of vignettes in studies on social judgments of emotion. In Study 1, undergraduates (N = 194) answered comprehension questions and rated story clarity and immersion. Participants also made social judgments of emotion by rating main character emotion control and appropriateness of intensity. To further compare the efficacy of the methods, in Study 2, Amazon Mechanical Turk workers (N = 213) answered comprehension questions while response times were recorded, rated clarity, answered a race manipulation check, and rated main character emotion type appropriateness. Overall, emotion storyboards resulted in greater clarity ratings, greater race manipulation check accuracy, and in some instances, enhanced comprehension and comprehension response times relative to vignettes. In emotion storyboards, main character emotion was rated more controlled and more appropriate in intensity, but not different in emotion type appropriateness, than in vignettes. Overall, the method offers a new method of examining social elements of emotion that enhances comprehension and maximizes experimental efficiency.

An Exploration of Stress: Leveraging Online Data from Crowdsourcing Platforms

Background: Early detection of community health risk factors such as stress is of great interest to health policymakers, but representative data collection is often expensive and time-consuming. It is important to investigate the use of alternative means of data collection such as crowdsourcing platforms. Methods: An online sample of Amazon Mechanical Turk (MTurk) workers (N = 500) filled out, for themselves and their child, demographic information and the 10-item Perceived Stress Scale (PSS-10), designed to measure the degree to which situations in one’s life are appraised as stressful. Internal consistency reliability of the PSS-10 was examined via Cronbach’s alpha. Analysis of variance (ANOVA) was utilized to explore trends in the average perceived stress of both adults and their children. Last, Rasch trees were utilized to detect differential item functioning (DIF) in the set of PSS-10 items. Results: The PSS-10 showed adequate internal consistency reliability (Cronbach’s alpha = 0.73). ANOVA results suggested that stress scores significantly differed by education (p = 0.024), employment status (p = 0.0004), and social media usage (p = 0.015). Rasch trees, a recursive partitioning technique based on the Rasch model, indicated that items on the PSS-10 displayed DIF attributable to physical health for adults and social media usage for children. Conclusion: The key conclusion is that this data collection scheme shows promise, allowing public health officials to examine health risk factors such as perceived stress quickly and cost effectively.

At whose expense? System justification and the appreciation of stereotypical humor targeting high- versus low-status groups

Abstract Many popular comedians tell complicated jokes that involve multiple levels of interpretation. The same joke may be perceived as criticizing or reinforcing the societal status quo, depending on perceivers’ assumptions about the target of the punchline (e.g., whether the joke is at the expense of high- or low-status groups). We focused on how such jokes are experienced by listeners who are psychologically prone to justifying (vs. challenging) the status quo. In a sample of Mechanical Turk workers (N = 179), we explored whether individual differences in system justification would be associated with the appreciation of group-based (stereotypical) humor, depending on the perceived target of the joke. As hypothesized, high system-justifiers found jokes targeting low-status groups (e.g., women, poor people, racial/ethnic minorities) to be funnier than low system-justifiers did. In some cases, low system-justifiers found jokes targeting high-status groups (rich people, European Americans) to be funnier than high system-justifiers did. These results expand upon previous demonstrations that humor appreciation is linked to relatively stable ideological dispositions and suggest that different individuals may perceive complex group-based humor in divergent ways.

Providing witnesses with an option to say "I'm not sure" to a showup neither improves classification performance nor the reliability of suspect identifications.

Objective: Past research with one-person showup identification procedures suggests that providing witnesses with an explicit option to opt-out reduces innocent-suspect identifications without reducing culprit identifications (Weber & Perfect, 2012). This finding suggests that improving performance from identification procedures might be as simple as providing witnesses with the option to opt-out from deciding. We examined whether providing witnesses with an option to say "not sure" improved performance from showup procedures. Hypotheses: We predicted that participants would opt-out more when given a poor view. We also predicted that classification performance would be better for those who had option to opt-out, and this improvement would be more pronounced for those who had a poor view. Finally, we predicted that the opt-out option would reduce more low-confidence than high-confidence decisions. Method: We randomly assigned Amazon Mechanical Turk Workers (Experiment 1A: N = 2,003, average age = 36.90 [SD = 11.67], 57.86% female) and university students (Experiment 1B: N = 721, average age = 19.91 (SD = 3.99), 69.72% female) to a 2 (culprit: present, absent) × 2 (memory strength: strong, weak) × 2 (not sure option: yes, no) between-participants design. We manipulated memory strength by giving witnesses either a clear or degraded view of the encoding video. After watching the encoding video, participants completed a 10-min filler task and were then presented with a showup procedure. Results: Participants who were given a poor view were more likely to opt-out than were participants who were given a clear view. Participants who were given the option to respond not sure reported higher confidence in their decisions. However, the not sure option did not improve classification performance. Conclusion: Contrary to our prediction, we found no evidence that an opt-out option improves performance from a showup procedure, which is consistent with past research examining opt-out options with lineup procedures. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Eliciting Human Judgment for Prediction Algorithms

Even when human point forecasts are less accurate than data-based algorithm predictions, they can still help boost performance by being used as algorithm inputs. Assuming one uses human judgment indirectly in this manner, we propose changing the elicitation question from the traditional direct forecast (DF) to what we call the private information adjustment (PIA): how much the human thinks the algorithm should adjust its forecast to account for information the human has that is unused by the algorithm. Using stylized models with and without random error, we theoretically prove that human random error makes eliciting the PIA lead to more accurate predictions than eliciting the DF. However, this DF-PIA gap does not exist for perfectly consistent forecasters. The DF-PIA gap is increasing in the random error that people make while incorporating public information (data that the algorithm uses) but is decreasing in the random error that people make while incorporating private information (data that only the human can use). In controlled experiments with students and Amazon Mechanical Turk workers, we find support for these hypotheses.This paper was accepted by Charles Corbett, operations management.