Crowdsourcing Research Articles

Latrine Utilization and Associated Factors Among Rural Households in Southwest Ethiopia: Risk, Attitude, Norms, Ability, and Self-Regulation Behavioral Model.

Consistent latrine utilization remains a challenge in most rural areas of poor and middle-income countries including Ethiopia. Therefore; the objective of this study was to determine the magnitude of latrine utilization and its associated factors among rural households in open defecation-free declared (ODF) and non-ODF kebeles, Southwest Ethiopia, 2022. A community-based comparative cross sectional study design was employed from May 16 to June 14, 2022 among 682 (341 ODF and 341 non-ODF) rural households which were selected by multistage systematic random sampling technique. Data were collected by face to face using a pretested structured questionnaire and an observational checklist through Open data kit. SPSS version 26 was used to analysis the data and separate logistic regression models; ODF and non-ODF were run to identify the associated factors. P-values <.05 with 95% CI were set as statistical significant. The overall magnitude of latrine utilization in the study area was 67.1% (95% CI: 63.71, 70.49). Latrine utilization was higher in ODF declared households (71.6%) than in non-ODF households (62.5%). In ODF; Graduated as model households [AOR = 5.1, 95% CI: 2.14, 12.84], visited by health extension works (HEWs) [AOR = 3.5, 95% CI: 1.67, 7.25], and latrine with privacy [AOR = 3.0, 95% CI: 1.37, 6.65] and also in non-ODF; households visited by HEWs [AOR = 2.9, 95% CI: 1.34, 6.06], latrine with privacy [AOR = 2.5, 95% CI: 1.24, 5.07], positive attitude [AOR = 3.2, 95% CI: 1.52, 6.59], and descriptive norm [AOR = 2.7, 95% CI: 1.27, 5.53] were significantly associated with latrine utilization. This study found that rural households declared as ODF utilize their latrine higher than non-ODF. Constructing latrine without privacy, lack of follow up, Attitude and societal norms were factors that limited the utilization of latrine and the sustainability of ODF. Therefore, health extension follow-up, latrine construction with privacy, normative and persuasive behavior change approach should be encouraged.

Data-Driven Many-Objective Crowd Worker Selection for Mobile Crowdsourcing in Industrial IoT

With the development of mobile networks and intelligent equipment, as a new intelligent data sensing paradigm in large-scale sensor applications such as the industrial Internet of Things, mobile crowd sensing (MCS) assigns industrial sensing tasks to workers for data collection and sharing, which has created a bright future for building a strong industrial system and improving industrial services. How to design an effective worker selection mechanism to maximize the utility of crowdsourcing is the research hotspot of mobile sensing technologies. This article studies the problem of least workers selection to make large MCS system perform sensing tasks more effective and achieve certain coverage with certain constraints being meeting. A many-objective worker selection method is proposed to achieve the desired tradeoff and an optimization mechanism is designed based on the enhanced differential evolution algorithm to ensure data integrity and search solution optimality. The effectiveness of the proposed method is verified through a large scale of experimental evaluation datasets collected from real world.

EduQG: A Multi-Format Multiple-Choice Dataset for the Educational Domain

Natural language processing technology has made significant progress in recent years, fuelled by increasingly powerful general language models. This has also inspired a sizeable body of work targeted specifically towards the educational domain, where the creation of questions (both for assessment and practice) is a laborious/expensive effort. Thus, automatic Question-Generation (QG) solutions have been proposed and studied. Yet, according to a recent survey of the educational QG community’s progress, a common baseline dataset unifying multiple domains and question forms (e.g., multiple choice vs. fill-the-gap), including readily available baseline models to compare against, is largely missing. This is the gap we aim to fill with this paper. In particular, we introduce a high-quality dataset in the educational domain, containing over 3,000 entries, comprising (i) multiple-choice questions, (ii) the corresponding answers (including distractors), and (iii) associated passages from the course material used as sources for the questions. Each question is phrased in two forms, normal and cloze (i.e., fill-the-gap), and correct answers are linked to source documents with sentence-level annotations. Thus, our versatile dataset can be used for both question and distractor generation, as well as to explore new challenges such as question format conversion. Furthermore, 903 questions are accompanied by their cognitive complexity level as per Bloom’s taxonomy. All questions have been generated by educational experts rather than crowd workers to ensure they are maintaining educational and learning standards. Our analysis and experiments suggest distinguishable differences between our dataset and commonly used ones for question generation for educational purposes. We believe this new dataset can serve as a valuable resource for research and evaluation in the educational domain. The dataset and baselines are made available to support further research in question generation for education (https://github.com/hadif ar/question-generation).

A Task-Interdependency Model of Complex Collaborative Work for Advancing Human-Centered Crowd Work

A Task-Interdependency Model of Complex Collaborative Work for Advancing Human-Centered Crowd Work

Teenage Pregnancy Patterns and Experience in SidoonTown, River Nile state, Sudan, 2022

Background: Teenage is a stage of physical and mental human development that occurs between childhood and adulthood. Teenage pregnancy is a global public health problem that affects the physical and emotional health.Teenage pregnancy is linked with many risk factors that affect the health of the mother and the outcome, as spontaneous abortions, as well as neonatal and maternal mortality and morbidity.Objective: to study issues related to teenage pregnancy patterns and experience. Methodology: This study adopted a descriptive cross-sectional community-based design. The study was conducted among a sample of 205 women using the simple random sampling method of the teenagers living inSidoon Townof the River Nile state. The data required for the purposes of the study were collected using structured face-to-face interviews. The data were analyzed using SPSS, version 23 and suitable descriptive and inferential statistical methodswere used, and P &lt; .05 was accepted for statistical significance. Results: The majority of the participants (78%) were aged between 17-19 years, and those got married and had their first pregnancy between 14-16 years were 79% and 84.4% respectively. About half of the participants (53.2%) didn’t complete their primary level school education. The results also indicated that 40.5% of the participantswere subjected to forced marriage experience, whereas 59.5% had the right to choose. Pregnancy was experienced by 176 (85.9%) of the women, out of whom 159 (86.9%) delivered their babies vaginally, while C/S birth was found among 24(13.1%) of them; abortion was experienced by 25.4% of them, most of which happened without a determined cause. Planning of pregnancy and using of family planning (FP) was found among 78(38.5%) of the participants. The majority of them used hormonal FP without medical advice and 62 (78.8%) used combined pills. Teenage pregnancy associated morbidity wasfound among 59 women (gestational diabetes, pregnancy induced hypertension, preterm labor, post-partum depression and infection: 19,13,24,3 and1 respectively). Significant correlation was found between passive smoking and history of the abortion among women,P = 0.05.Conclusion: Teenage pregnancy is linked with the participants’ morbidity. Early pregnancy interferes with the continuation of the woman’s education, and lack of awareness related to use of FP methods used was seen. Recommendation:Health education for the study community is needed to enhance their health awareness.

Designing culturally situated playful environments for early STEM learning with a Latine community

Children from underserved, minoritized, and immigrant families have less access to early out-of-school STEM learning opportunities. Playful Learning Landscapes increase the accessibility of early STEM learning in everyday public spaces (e.g., bus stops, grocery stores) by merging principles of guided play and STEM learning goals with local community's values. We used community-based design research to (1) identify Latine families’ funds of knowledge related to play, science, and math learning, and (2) create designs for playful environments merging families’ values and practices with guided play and STEM learning principles. Our design partners were 32 parents, primarily Spanish-speaking immigrant mothers from Mexico, and two directors of a local community organization. The design process consisted of co-design sessions, interviews, inductive thematic analysis, elaboration, playtest and feedback, and iteration. Our findings showed that familismo, heritage representation, and the meaning of community spaces influenced the ways families engaged in STEM learning and the learning environments they desired in their community. Moreover, families’ STEM practices were rooted in everyday experiences of playing cultural games, family food routines, and outdoor activities. Incorporating Latine parents in the design process and leveraging their funds of knowledge resulted in culturally situated designs aligned with playful and STEM learning principles. This study contributes to knowledge of Latine families’ values and practices that can help create home-community connections to strengthen children's learning.

국어사전 용례의 윤리성 자동 분류 문제와 제안

The purpose of this study is to extract unethical use cases from the Korean dictionary to determine whether they are ethical and to find the right way to present ethical use cases. In this study, the problem of misclassification and undetected ethical automatic classifier of the Korean dictionary example was analyzed through the crowd worker and the ethical automatic classifier ‘Safety Check(Korv0.5). This study argued that cases with long-established unethical bias should be reviewed again through unethical cases in the Korean dictionary. Ethical information should be structured according to the purpose of the dictionary, but unethical use cases should be given additional attachment information, or in the case of new words, ethical regulations are needed in the guidelines for preparing examples. The unethical case analysis data reviewed in this study were disclosed to help improve the model through balanced data learning in subsequent studies.

Combining Human and Machine Confidence in Truthfulness Assessment

Automatically detecting online misinformation at scale is a challenging and interdisciplinary problem. Deciding what is to be considered truthful information is sometimes controversial and also difficult for educated experts. As the scale of the problem increases, human-in-the-loop approaches to truthfulness that combine both the scalability of machine learning (ML) and the accuracy of human contributions have been considered. In this work, we look at the potential to automatically combine machine-based systems with human-based systems. The former exploit superviseds ML approaches; the latter involve either crowd workers (i.e., human non-experts) or human experts. Since both ML and crowdsourcing approaches can produce a score indicating the level of confidence on their truthfulness judgments (either algorithmic or self-reported, respectively), we address the question of whether it is feasible to make use of such confidence scores to effectively and efficiently combine three approaches: (i) machine-based methods, (ii) crowd workers, and (iii) human experts. The three approaches differ significantly, as they range from available, cheap, fast, scalable, but less accurate to scarce, expensive, slow, not scalable, but highly accurate.

The Ladder of Power: Science Communication and Citizen Science

On March 28, 2022, the Journal of Science Communication published a special issue on participatory science communication featuring 15 papers and essays. The Journal of Science Communication special issue sparked a debate among the four editors over the wording of the call for papers. What is the difference between “participatory science communication” and “citizen science”? Are they different points along a continuum stretching from “simple” to “more involved”? Does “citizen science” incorporate “participatory science communication”? And is all citizen science participatory? A key consideration is the level of involvement by “citizens” in these endeavours, and that consideration translates to questions of power. This essay explores definitions of participatory science communication and citizen science. It examines each of these concepts through the framework of shifting relationships and the implicit power imbalance between scientists and various publics. In doing this, we revisited Sherry Arnstein’s (1969) paper, “A Ladder of Citizen Participation”, and constructed complementary ladders for science communication and citizen science.

The role of system‐use practices for sustaining motivation in crowdsourcing: A technology‐in‐practice perspective

AbstractThe success of crowdsourcing (CS) systems depends on sustained participation, which is an ongoing challenge for the majority of CS providers. Unfortunately, participants are frequently demotivated by technical difficulties and the incorrect use of CS systems, which can result in CS failure. Although the literature generally assumes that sustained participation in CS is determined by a shift between intrinsic and extrinsic motivation, the role of system‐use practices in facilitating such a shift remains unknown. We explore how CS system‐use practices influence participants' sustained motivation, evolving from initiation to progression to sustention. Using the notion of technology‐in‐practice as a lens, we develop and examine a process model using an in‐depth case study of a large‐scale ongoing CS project, the Australian Newspaper Digitisation Program. The findings suggest that CS participants' motivation is shaped by an evolving combination of three basic components (i.e., contextual condition, outcome and action intensity) and mediated by two types of system‐use practice (i.e., passive and active). Passive‐use practices facilitate sustaining motivation from initiation to progression, whereas active‐use practices have a key role in sustention. Our study contributes to the emerging literature on the substantial role of system‐use practices in sustaining motivation, resulting in sustained participation. The findings also offer actionable insights into improving the viability of CS systems in retaining and motivating continuous and increased contributions from participants.

The social consequences of Machine Allocation Behavior: Fairness, interpersonal perceptions and performance

Machines increasingly decide over the allocation of resources or tasks among people resulting in what we call Machine Allocation Behavior. People respond strongly to how other people or machines allocate resources. However, the implications for human relationships of algorithmic allocations of, for example, tasks among crowd workers, annual bonuses among employees, or a robot’s gaze among members of a group entering a store remains unclear. We leverage a novel research paradigm to study the impact of machine allocation behavior on fairness perceptions, interpersonal perceptions, and individual performance. In a 2 × 3 between-subject design that manipulates how the allocation agent is presented (human vs. artificial intelligent [AI] system) and the allocation type (receiving less vs. equal vs. more resources), we find that group members who receive more resources perceive their counterpart as less dominant when the allocation originates from an AI as opposed to a human. Our findings have implications on our understanding of the impact of machine allocation behavior on interpersonal dynamics and on the way in which we understand human responses towards this type of machine behavior.

ALZHEIMER'S DISEASE AND RELATED DEMENTIAS RISK AND PSYCHOSOCIAL HEALTH NEEDS OF OLDER ADULT IMMIGRANTS AND REFUGEES

Abstract To establish priorities for an intervention addressing Alzheimer’s Disease and Related Dementias (ADRD) risk among older adult immigrants and refugees from Afghanistan and the Middle-East and North Africa (MENA). A multi-methods Community-Based Collaborative Action Research design was employed to gather qualitative and quantitative data. Twenty community leaders and stakeholders participated in interviews and community needs survey responses of 124 ethnically diverse adults over age 55 were examined. We analyzed community needs survey data using descriptive statistics. A step-wise Thematic Analysis was used to analyze interview transcripts. A major theme and standout priority was identified: needing culturally-sensitive services and resources addressing social isolation among the elderly. Other psychosocial priorities included: accessibility needs, caregiving support, safety and end-of-life resources, digital literacy, and English language services. This study exemplifies a priority-setting partnership between clinician-researchers and refugee service providers. The priorities highlighted ADRD risk factors and guide the collaborative development of a community-based intervention.

Teacher Use of School Connectedness Strategies With Underrepresented Youth in a Low-Income Middle School

Adolescents in low-income and rural communities are at a higher risk for mental health concerns. This study used a mixed-methods, community-based participatory research design to examine teachers’ experiences managing students with emotional and mental health needs in a low-income middle school serving underrepresented youth. Barriers to connecting positively with students included disruptive student behaviors, a lack of student interest, and an emphasis on accountability measures for academic achievement. Three themes emerged: (a) disruptive student behavior resulted in a whack-a-mole approach to managing emotional health needs; (b) learning was often put on the back burner as teachers attempted Maslow before Bloom; and (c) a lack of training and limited time to connect positively with students contributed to missed connections. Findings from this study indicate that additional resources and staff training are needed to help rural, low-income schools support students who have emotional and mental health needs.

Crowd work in STEM‐related fields: A window of opportunity from a gender perspective?

AbstractAfter the Global Financial Crisis (2008) many people found new job opportunities on crowd platforms. The COVID‐19 crisis reinforced this trend and virtual work is expected to increase. Although the working conditions of individuals engaged on these platforms is an emerging topic, of research, the existing literature tends to overlook the gendered dimension of the gig economy. Following a quantitative approach, based on the statistical analysis of 444 profiles (platform Freelancer.com in Spain and Argentina), we examine the extent to which the gig economy reproduces gender inequalities such as the underrepresentation of women in STEM‐related tasks and the gender pay gap. While the findings reveal lower participation of women than men, this gap is not higher in Argentina than in Spain. Moreover, gender variations in hourly wages are not as marked as expected, and such differences disappear once STEM skill levels are controlled for. Asymmetry in individuals' STEM skill level provides a better explanation than gender of the hourly wage differences. This finding opens a window of opportunity to mitigate the classical gender discrimination that women face in technological fields in traditional labor markets. Finally, the paper identifies some issues concerning the methodological bias entailed by the use of an application programming interface in cyber‐research, when analyzing gender inequalities.

The Duality of Empowerment and Marginalization in Microtask Crowdsourcing: A Replication

This paper describes an exact replication of a study by Deng, Joshi, &amp; Galliers (2016) of crowd worker values on Amazon’s Mechanical Turk (MTurk) crowdsourcing platform. The original study analyzed 210 MTurk crowd workers’ narratives using value sensitive design (VSD). The results uncovered nine shared values: access, autonomy, fairness, transparency, communication, security, accountability, making an impact, and dignity. Further analysis in the original study revealed four crowdsourcing structures: compensation, governance, technology, and microtask, and duality of crowd worker perceptions, empowerment, and marginalization. This replication study also asked Amazon Mechanical Turk crowd workers questions about their work and used the original study’s findings for a priori codes. However, new values and findings emerged in our results, which offers additional implications for further research regarding microtask crowdsourcing.

On the role of human and machine metadata in relevance judgment tasks

In order to evaluate the effectiveness of Information Retrieval (IR) systems it is key to collect relevance judgments from human assessors. Crowdsourcing has successfully been used as a method to scale-up the collection of manual relevance judgments, and previous research has investigated the impact of different judgment task design elements (e.g., highlighting query keywords in the document) on judgment quality and efficiency. In this work we investigate the positive and negative impacts of presenting crowd human assessors with more than just the topic and the document to be judged. We deploy different variants of crowdsourced relevance judgment tasks following a between-subjects design in which we present different types of metadata to the human assessor. Specifically, we investigate the effect of human metadata (e.g., what other human assessors think of the current document, as in which relevance level has already been selected by the majority crowd workers), machine metadata (e.g., how IR systems scored this document such as its average position in ranked lists, statistics about the document such as term frequencies). We look at the impact of metadata on judgment quality (i.e., the level of agreement with trained assessors) and cost (i.e., the time it takes for workers to complete the judgments) as well as at how metadata quality positively or negatively impact the collected judgments.

Label confidence-based noise correction for crowdsourcing

In crowdsourcing scenarios, each instance obtains multiple noisy labels from different crowd workers and then gets its integrated label via a label aggregation method. In spite of the effectiveness of label aggregation methods, a certain level of noise still remains in the integrated labels. To solve this problem, several noise correction methods have been proposed in recent years. However, to our best knowledge, these methods seldom take the label confidence of each instance into account. Therefore, we propose a label confidence-based noise correction (LCNC) method. At first, LCNC calculates the label confidence of each instance using its multiple noisy labels to filter all instances and obtains an original clean set and noise set. Second, LCNC builds multiple random trees on the original clean set to recalculate the label confidence of each instance using its predicted labels generated by these random trees and then refilters all instances to obtain a new clean set and noise set. Finally, LCNC builds two heterogeneous classifiers on the new clean set to correct the noise instances in the new noise set according to a consensus voting strategy. The experimental results on 34 simulated and two real-world crowdsourced datasets show that LCNC significantly outperforms all the other state-of-the-art noise correction methods used for comparison.

Immunization Coverage among Children Aged Between 12-23 Months in West Pokot County, Kenya

Vaccination has been shown to be one of the most cost-effective health interventions worldwide. Despite the efforts made towards global immunization coverage, the number of unvaccinated and under-vaccinated infants is still high. The situation is no different in Kenya. Basic vaccination coverage reduced from 77% in 2008 to 70% in 2017. National basic vaccination coverage of 70% is significantly lower than the global target of 90% by the year 2020. There are gaping regional differences in immunization coverage. The lowest national immunization coverage is documented at 31% in West Pokot County. This study sought to identify the determinants of vaccination coverage among children between 12-23 months of age in West Pokot County. The study used a community-based cross-sectional design where mothers/guardians of children between 12-23 months old were randomly sampled. The objectives of the study were to estimate the immunization coverage, investigate factors influencing immunization coverage and determine the influence of health service utilization on immunization coverage. Binary logistic regression was conducted to determine the influence of the independent variables on the dependent variable. The findings showed that, majority (62.4%) of the mothers/guardians had partially immunized their children, followed by those who had fully immunized their children (36.6%) and lastly 1.1% had not immunized their children. Age of the mother/guardian is significantly associated with immunization coverage (OR = 1.128, p =0.000&lt;0.05). This suggests that the older the mother/guardian, the higher the chances of the child being fully immunized. The distance to nearest health facility was found to have no significant relationship with immunization coverage (p&gt;0.05). Respondents who paid for the services were more likely to have their children fully immunized than those who did not pay for the services (OR = 3.546, p =0.019&lt;0.05). It was also observed that Paternal occupation influences coverage OR = 2.006, P =0.035&lt;0.05), visiting the health facility in last year also influenced immunization (OR = 3.147, P =0.01&lt;0.05). The study concluded that the immunization coverage among children between 12-23 months in West Pokot County is below the set target. It is therefore recommended that the county government should intensify vaccination campaigns and develop policies aimed at economically empowering the residents of West Pokot County. Keywords: Immunization coverage, children aged between 12-23 months, West Pokot County

What scholars and IRBs talk when they talk about the Belmont principles in crowd work‐based research

AbstractHow scholars and IRBs perceive and apply the Belmont principles in crowd work‐based research was an open and largely neglected question. As crowd work becomes increasingly popular for scholars to implement research and collect data, such negligence, signaling a lack of attention to the ethical issues in crowd work‐based research more broadly, seemed alarming. To fill this gap, we conducted a qualitative study with 32 scholars and IRB directors/analysts in the United States to inquire into their perceptions and applications of the Belmont principles in crowd work‐based research. We found two dilemmas in applying the Belmont principles in crowd work‐based research, namely the dilemma between the dehumanization and expected autonomy of crowd workers, and the dilemma between the monetary incentive/reputationall risks and the conventional notion of research benefits/risks. We also compared the scholars' and IRBs' ethical perspectives and proposed our research implications for future work.

Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative View

Microtask crowdsourcing holds great potential as an employment opportunity with the flexibility and anonymity that individuals with disability may require. Though prior research has explored the accessibility of crowd work, the lived crowd work experiences of the broader community of workers with disability are still largely under-explored, especially when it comes to how their experiences are similar to or different from the experiences of workers without disability. In this work, we aim to obtain a deeper understanding of the microtask crowdsourcing experience for people with disabilities, especially regarding their financial and social experiences of participating in crowd work, along with the benefits and challenges that they encounter through this work. Specifically, we first surveyed 1,200 crowd workers both with and without disability about their experiences using the Amazon Mechanical Turk platform, and the differences we found inspired the design of a follow-up survey to gain greater understanding of the crowd work experience for workers with disability. Our findings reveal that workers with disability receive unique benefits from performing crowd work, such as a greater sense of purpose, but also encounter many challenges, such as completing tasks on time and earning a livable wage, causing them to turn to online communities for assistance. Although many of the challenges they face are not unique to crowd workers with disability, workers with disability may be disproportionately impacted by these challenges. From our findings, we provide implications for crowd platforms, as well as the gig economy as a whole, that seek to promote greater consideration of workers with a diverse range of conditions to create a more valuable work experience for them.