Interactive Crowdsourcing Research Articles

Interactive crowdsourcing to fact-check politicians.

The discourse of political leaders often contains false information that can misguide the public. Fact-checking agencies around the world try to reduce the negative influence of politicians by verifying their words. However, these agencies face a problem of scalability and require innovative solutions to deal with their growing amount of work. While the previous studies have shown that crowdsourcing is a promising approach to fact-check news in a scalable manner, it remains unclear whether crowdsourced judgements are useful to verify the speech of politicians. This article fills that gap by studying the effect of social influence on the accuracy of collective judgements about the veracity of political speech. In this work, we performed two experiments (Study 1: N = 180; Study 2: N = 240) where participants judged the veracity of 20 politically balanced phrases. Then, they were exposed to social information from politically homogeneous or heterogeneous participants. Finally, they provided revised individual judgements. We found that only heterogeneous social influence increased the accuracy of participants compared to a control condition. Overall, our results uncover the effect of social influence on the accuracy of collective judgements about the veracity of political speech and show how interactive crowdsourcing strategies can help fact-checking agencies. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Supplemental Material for Interactive Crowdsourcing to Fact-Check Politicians

Supplemental Material for Interactive Crowdsourcing to Fact-Check Politicians

A Novel Crowdsourcing Task Recommendation Method Considering Workers’ Fuzzy Expectations: A Case of ZBJ.COM

Crowdsourcing recommendation is necessary to surmount information overload and assist workers to identify tasks of interest efficiently. However, existing studies mainly concentrate on investigating workers’ qualifications or requirements, neglecting their fuzzy expectations and psychological behavior. Besides, workers can only passively accept the results and do not have any right to change them even if they are not satisfactory. As workers will pay different attention to task aspects, we regard the crowdsourcing recommendation as a multi-criteria decision-making (MCDM) problem. In this paper, we propose an MCDM-based method for automated and interactive crowdsourcing task recommendations while comprehensively considering workers’ fuzzy expectations and psychological behavior. Given a large volume of available tasks on a crowdsourcing platform, the requirement of massive human efforts in traditional MCDM is not applicable in such situation. Therefore, we formulate a set of quantitative task evaluation criteria for automated task assessments based on the attributes that have the most impact on workers’ participation behavior. Further, to capture workers’ fuzziness in expectations, an intuitionistic fuzzy 2-tuple linguistic (IF2L) term is introduced, which are further leveraged as reference points to evaluate their “gains” and “losses” based on the prospect theory. Finally, the D-S evidence theory is adopted to gather workers’ uncertain prospect values. A case of ZBJ.COM, a popular crowdsourcing platform in China, shows that the proposed method is efficient and effective in recommending suitable and personalized tasks to workers.

SeCrowd: Efficient secure interactive crowdsourcing via permission-based signatures

Crowdsourcing is a powerful paradigm to provide services by soliciting help from a pool of contributors. However, the security threats in crowdsourcing, especially in interactive crowdsourcing where contributors interact with each other for collaboration, are not well studied. External adversaries who may inject malicious results are the major concerns of all involved parties in crowdsourcing. In this paper, an efficient secure interactive crowdsourcing framework, called SeCrowd, is proposed to protect the authenticity in the crowdsourcing platform against external adversaries. The framework is characterized by a recently proposed cryptographic primitive, called permission-based signature, which reveals the signer’s identity to permissive verifiers only, while safeguards it against others. In this way, contributors can identify each other based on their signed contributions, but their identities remain unknown to other users. Moreover, users can verify whether the signed contributions are made by legal contributors without being aware of the actual signer’s identities, which guarantees that the contributions cannot be tampered or forged. We prove that SeCrowd achieves unforgeability in the random oracle model. Comprehensive analysis and experiments showed that our framework is secure and highly efficient in terms of computation and communication cost.

Best Paper Selection

Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017 Apr 10;17(1):36 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5387195/ Blijleven V, Koelemeijer K, Wetzels M, Jaspers M. Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care, and efficiency of care. JMIR Hum Factors 2017 Oct 5;4(4):e27 https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/28982645/ Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A. Safety risks associated with the lack of integration and interfacing of hospital health information technologies: a qualitative study of hospital electronic prescribing systems in England. BMJ Qual Saf 2017 Jul;26(7):530-41 http://qhc.bmj.com/cgi/pmidlookup?view=long&pmid=27037303 Dufendach KR, Koch S, Unertl KM, Lehmann CU. A randomized trial comparing classical participatory design to VandAID, an interactive crowdsourcing platform to facilitate user-centered design. Methods Inf Med 2017 Oct 26;56(5):344-9 http://www.thieme-connect.com/DOI/DOI?10.3414/ME16-01-0098 Luna DR, Rizzato Lede DA, Otero CM, Risk MR, González Bernaldo de Quirós F. User-centered design improves the usability of drug-drug interaction alerts: experimental comparison of interfaces. J Biomed Inform 2017 Feb;66:204-13 https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(17)30009-6

Innovation through interactive crowdsourcing: The role of boundary objects

The aim of this study is to explore company–consumer collaboration in the innovation process through the practice of interactive crowdsourcing. This type of crowdsourcing lies within the co-creation process and is characterized by close interactions between the two categories of actors. To pursue our aim, we draw on the concept of “boundary object,” developed within sociology. This concept can help understand the dynamics of interactions and collective action among disparate actors. The study’s findings show that boundary objects and boundary negotiating artifacts facilitate the exchange of knowledge and expertise between the world of companies and the world of consumers. They also highlight the role of these objects in revealing influence and power games liable to generate disappointment and tensions between actors. The study sheds new light on the advantages, limitations, and challenges of a co-creation process.

Post Disaster Situation Awareness and Decision Support Through Interactive Crowdsourcing

In today's world, online social media has established itself as one of the most important information carriers, and that applies in the field of disaster management as well. However, posts in online social media like Twitter and Facebook are mostly unstructured, and it is extremely difficult to extract meaningful information from such unstructured posts. Therefore, disaster management authorities face problems in using these posts for proper situation analysis and decision-making. To alleviate this problem, we propose a decision support framework that collects situational information through interactive crowd-sourcing using SMS from the “crowd” present at the disaster site, and summarizes such responses to have situational awareness and appropriate decision-making regarding damage or need assessment. We evaluated our proposed system using 3000 real-time interactive responses from a disaster-prone coastal area of India (named, the Sunderbans). Since the use of smart-phone in rural India is significantly low, SMS from the “crowd” were used to form the core information repository for post-disaster situation analysis.

Crowdsourcing Public Opinion Using Urban Pervasive Technologies: Lessons From Real‐Life Experiments in Oulu

The civic potential of pervasive computing technologies has been increasingly explored within academic and urban planning communities. One vision is of communities and cities enriched with pervasive computing, where citizens can leverage the new technology and interfaces for their own purposes, and at the same time be empowered to be heard through the use of such technology. This article reports lessons learned from a project that created public human interfaces for interacting with city officials in Oulu, Finland. We focus on civic engagement and reflect on a deployment of public interactive crowdsourcing technologies that are openly available to everyone, right in the heart of the city. The series of field trials reported here involve public displays, mobile phones, and Internet technologies. We find differences in the performance of different kinds of input mechanisms, and present lessons learned from cooperating with officials in a joint civic engagement effort.

Crowdsourcing in Public Policy: Technologies, Subjects and Its Socio-Political Role

The article describes the potential of the new interactive crowdsourcing technologies in the public policy. Theconceptual basis of the research implies the network theory of policy, the theory of deliberate democracy and theconcept “governance”. The empirical study was based on the methodology of double reflexivity and conductedusing such tools as case study, focus-group interview and monitoring of the online platforms of Russian andforeign crowdsourcing resources. The authors emphasize and characterize the main technologies of politicalcrowdsourcing: prediction market crowdsourcing, network brainstorming, project crowdsourcing, andcrowdfunding. The authors think that the socio-political role of crowdsourcing lies in the extension of space inwhich the authorities and citizens interact, the initiation and introduction of political innovations, and thedevelopment of the new forms of civil and political participation.

What the disability community can teach us about interactive crowdsourcing

In this forum we will celebrate research that helps to successfully bring the benefits of computing technologies to children, older adults, people with disabilities, and other populations that are often ignored in the design of mass-marketed products. Juan Pablo Hourcade, Editor