Crowdsourcing System Research Articles

Verification of a labor market domain using an academic crowdsourcing system

Students desiring to become a valuable good in the labor market are willing to pay a considerable monetary cost to obtain knowledge about their prospective job opportunities, nowadays with a diminishing interest in the obtainment of a diploma. Considering the behavior of the labor market as a domain theory under uncertainty, it is straightforward to expect the presence of contradictions, in the form of salaries unable to be classified due to high inconsistency and variation. We provide an algorithm to verify a labor market domain theory based on a crowdsourcing academic system, in which feedback about possible contradictions is generated as a result of consultations with experts inside of the market and clustered into different contexts. We found that the verification process can be repeated iteratively as long as the students overall tuition is equal or greater than a quantity partially defined by the number of different profiles of the students.

Financial Advisor Recruitment: A Smart Crowdsourcing-Assisted Approach

Successful portfolio management requires, in addition to advanced optimization strategies, effective recruitment of specialized financial advisors. Hiring the wrong ones can be detrimental to investors' financial goals. In this article, we propose an automated crowdsourcing system to organize cooperation between financial advisors and investors. Without interfering with their private portfolio optimization techniques, we design a recruitment framework that matches financial advisors to investors based on their profiles and features, as well as the previous activities of their peers. Using the database of the crowdsourcing platform, we employ an unsupervised technique to regroup advisors with a high degree of similarities into clusters and, hence, shrink the search space. Afterward, we train a machine learning regression model to predict the matching score that can be achieved if an investor hires a particular advisor. These scores are converted to weights of bipartite graphs to which we apply a double-phased many-to-many maximum weight matching algorithm to determine a suitable investor-advisor combination. In the simulations, we investigate the performance of the proposed recruitment approach and show that, compared with other traditional approaches, higher returns can be reached for both investors and financial advisors.

Verification of a labor market domain using an academic crowdsourcing system

Students desiring to become a valuable good in the labor market are willing to pay a considerable monetary cost to obtain knowledge about their prospective job opportunities, nowadays with a diminishing interest in the obtainment of a diploma. Considering the behavior of the labor market as a domain theory under uncertainty, it is straightforward to expect the presence of contradictions, in the form of salaries unable to be classified due to high inconsistency and variation. We provide an algorithm to verify a labor market domain theory based on a crowdsourcing academic system, in which feedback about possible contradictions is generated as a result of consultations with experts inside of the market and clustered into different contexts. We found that the verification process can be repeated iteratively as long as the students’ overall tuition is equal or greater than a quantity partially defined by the number of different profiles of the students.

Task assignment for social-oriented crowdsourcing

Crowdsourcing has become an efficient measure to solve machine-hard problems by embracing group wisdom, in which tasks are disseminated and assigned to a group of workers in the way of open competition. The social relationships formed during this process may in turn contribute to the completion of future tasks. In this sense, it is necessary to take social factors into consideration in the research of crowdsourcing. However, there is little work on the interactions between social relationships and crowdsourcing currently. In this paper, we propose to study such interactions in those social-oriented crowdsourcing systems from the perspective of task assignment. A prototype system is built to help users publish, assign, accept, and accomplish location-based crowdsourcing tasks as well as promoting the development and utilization of social relationships during the crowdsourcing. Especially, in order to exploit the potential relationships between crowdsourcing workers and tasks, we propose a “worker-task” accuracy estimation algorithm based on a graph model that joints the factorized matrixes of both the user social networks and the history “worker-task” matrix. With the worker-task accuracy estimation matrix, a group of optimal worker candidates is efficiently chosen for a task, and a greedy task assignment algorithm is proposed to further the matching of worker-task pairs among multiple crowdsourcing tasks so as to maximize the overall accuracy. Compared with the similarity based task assignment algorithm, experimental results show that the average recommendation success rate increased by 3.67%; the average task completion rate increased by 6.17%; the number of new friends added per week increased from 7.4 to 10.5; and the average task acceptance time decreased by 8.5 seconds.

ITrust-A Trustworthy and Efficient Mapping Scheme in Elliptic Curve Cryptography.

Recently, many platforms have outsourced tasks to numerous smartphone devices known as Mobile Crowd-sourcing System (MCS). The data is collected and transferred to the platform for further analysis and processing. These data needs to maintain confidentiality while moving from smartphones to the platform. Moreover, the limitations of computation resources in smartphones need to be addressed to balance the confidentiality of the data and the capabilities of the devices. For this reason, elliptic curve cryptography (ECC) is accepted, widespread, and suitable for use in limited resources environments such as smartphone devices. ECC reduces energy consumption and maximizes devices’ efficiency by using small crypto keys with the same strength of the required cryptography of other cryptosystems. Thus, ECC is the preferred approach for many environments, including the MCS, Internet of Things (IoT) and wireless sensor networks (WSNs). Many implementations of ECC increase the process of encryption and/or increase the space overhead by, for instance, incorrectly mapping points to EC with extra padding bits. Moreover, the wrong mapping method used in ECC results in increasing the computation efforts. This study provides comprehensive details about the mapping techniques used in the ECC mapping phase, and presents performance results about widely used elliptic curves. In addition, it suggests an optimal enhanced mapping method and size of padding bit to secure communications that guarantee the successful mapping of points to EC and reduce the size of padding bits.

Online Graphic Designers and the Ambiguity of Capital Democratization

The development of information and communication technology has brought a lot of progress and helped encourage the emergence of the digital economy - one of which is in the creative industry in the field of graphic design. This development raises an opportunity, namely crowdsourcing system. Crowdsourcing system is the process of getting services, ideas, or content by collecting contributions from a group of people, especially the online community for various activities. According to Howe, this crowdsourcing system results in the of or the sort of system we call today the sharing economy of the digital economy. But instead Christian Fuchs (2014) expressed his criticism by saying that the practice of crowdsourcing is only a pretext for producers to exploit to increase profits by using unpaid workers. One of the attractions of the crowdsourcing system is the meritocracy offer which provides opportunities for those who do not have an academic background in graphic design education to be equal and have the same opportunities as academic graphic designers or even professionals. The subject of this study was the self-taught graphic designer community in Salaman village, Magelang, which underwent an online crowdsourcing system. This study aims to gain a critical understanding of the existence of ambiguity in the practice of 'capital democratization' offered in the crowdsourcing system. This research tries to open up new insights about the link between capital democratization offered by the crowdsourcing system and the attitude of online graphic designers through Fuchs (2014) approach of digital workers.

Towards fair and efficient task allocation in blockchain-based crowdsourcing

Crowdsourcing has been a popular paradigm leveraging the power of the crowd to accomplish a common goal. Traditional crowdsourcing systems rely on a centralized platform to allocate tasks and rewards to users, facing severe problems of single node failure and malicious behaviors of the platform. Recently, some works take advantage of blockchain to solve the drawback of centralization. However, the key issue, fair and efficient task allocation, has not been explored in existing blockchain-based crowdsourcing systems. In this paper, we design a novel blockchain-based framework for crowdsourcing, in which a distributed reverse and blind auction-based task allocation mechanism (RbatAlloc) is proposed utilizing user profile and bidding price to realize fair and efficient task allocation in transparent blockchain environment. Finally, we implement a prototype of our system and deploy it to a locally developed network. The experimental results demonstrate the effectiveness of the proposed framework and mechanism.

A Crowdsourcing Framework for Collecting Tabular Data

In crowdsourcing, human workers are employed to tackle problems that are traditionally difficult for computers (e.g., data cleaning, missing value filling, and sentiment analysis). In this paper, we study the effective use of crowdsourcing in filling missing values in a given relation (e.g., a table containing different attributes of celebrity stars, such as nationality and age). A task given to a worker typically consists of questions about the missing attribute values (e.g., What is the age of Jet Li?). Although this problem has been studied before, existing work often treats related attributes independently, leading to suboptimal performance. In this paper, we present T-Crowd, which is a crowdsourcing system that considers attribute relationships. Particularly, T-Crowd integrates each worker's answers on different attributes to effectively learn his/her trustworthiness and the true data values. The attribute relationship information is used to guide task allocation to workers. Our solution seamlessly supports categorical and continuous attributes. Our extensive experiments on real and synthetic datasets show that T-Crowd outperforms state-of-the-art methods, improving the quality of truth inference and reducing the monetary cost of crowdsourcing.

Origins of Algorithmic Instabilities in Crowdsourced Ranking

Crowdsourcing systems aggregate decisions of many people to help users quickly identify high-quality options, such as the best answers to questions or interesting news stories. A long-standing issue in crowdsourcing is how option quality and human judgement heuristics interact to affect collective outcomes, such as the perceived popularity of options. We address this limitation by conducting a controlled experiment where subjects choose between two ranked options whose quality can be independently varied. We use this data to construct a model that quantifies how judgement heuristics and option quality combine when deciding between two options. The model reveals popularity-ranking can be unstable: unless the quality difference between the two options is sufficiently high, the higher quality option is not guaranteed to be eventually ranked on top. To rectify this instability, we create an algorithm that accounts for judgement heuristics to infer the best option and rank it first. This algorithm is guaranteed to be optimal if data matches the model. When the data does not match the model, however, simulations show that in practice this algorithm performs better or at least as well as popularity-based and recency-based ranking for any two-choice question. Our work suggests that algorithms relying on inference of mathematical models of user behavior can substantially improve outcomes in crowdsourcing systems.

A Dynamic Privacy Protection Mechanism for Spatiotemporal Crowdsourcing

In spatiotemporal crowdsourcing applications, sensing data uploaded by participants usually contain spatiotemporal sensitive data. If application servers publish the unprocessed sensing data directly, it is easy to expose the privacy of participants. In addition, application servers usually adopt the static publishing mechanism, which is easy to produce problems such as poor timeliness and large information loss for spatiotemporal crowdsourcing applications. Therefore, this paper proposes a spatiotemporal privacy protection (STPP) method based on dynamic clustering methods to solve the privacy protection problem for crowd participants in spatiotemporal crowdsourcing systems. Firstly, the working principles of a dynamic privacy protection mechanism are introduced. Then, based on k-anonymity and l-diversity, the spatiotemporal sensitive data are anonymized. In addition, this paper designs the dynamic k-anonymity algorithm based on the previous anonymous results. Through extensive performance evaluation on real-world data, compared with existing methods, the proposed STPP algorithm could effectively solve the problem of poor timeliness and improve the privacy protection level while reducing the information loss of sensing data.

Crowdsourced Photographs as an Effective Method for Large-Scale Passive Tick Surveillance.

As tick vector ranges expand and the number of tickborne disease cases rise, physicians, veterinarians, and the public are faced with diagnostic, treatment, and prevention challenges. Traditional methods of active surveillance (e.g., flagging) can be time-consuming, spatially limited, and costly, while passive surveillance can broadly monitor tick distributions and infection rates. However, laboratory testing can require service fees in addition to mailing and processing time, which can put a tick-bite victim outside the window of potential prophylactic options or under unnecessary antibiotic administration. We performed a retrospective analysis of a national photograph-based crowdsourced tick surveillance system to determine the accuracy of identifying ticks by photograph when compared to those same ticks identified by microscopy and molecular methods at a tick testing laboratory. Ticks identified by photograph were correct to species with an overall accuracy of 96.7% (CI: 0.9522, 0.9781; P < 0.001), while identification accuracy for Ixodes scapularis Say (Ixodida: Ixodidae), Amblyomma americanum Linnaeus (Ixodida: Ixodidae), and Dermacentor variabilis Say (Ixodida: Ixodidae), three ticks of medical importance, was 98.2% (Cohen's kappa [κ] = 0.9575; 95% CI: 0.9698, 0.9897), 98.8% (κ = 0.9466, 95% CI: 0.9776, 0.9941), and 98.8% (κ = 0.9515, 95% CI: 0.9776, 0.9941), respectively. Fitted generalized linear models revealed that tick species and stage were the most significant predictive factors that contributed to correct photograph-based tick identifications. Neither engorgement, season, nor location of submission affected identification ability. These results provide strong support for the utility of photograph-based tick surveillance as a tool for risk assessment and monitoring among commonly encountered ticks of medical concern.

Task recommendation in crowdsourcing systems: A bibliometric analysis

Existing studies on task recommendation in crowdsourcing systems provide additional insights into the field from their perspectives, methodologies, frameworks, and disciplines, resulting in a highly productive but unorganized knowledge domain. This paper is motivated to exploit bibliometric techniques to derive insights that exceed the boundaries of individual systems and identify the potentially transformative changes from 268 published articles. The explicit features (i.e., affiliation, author, citation, and keywords) and implicit information (i.e., topic distribution, potential structure, hidden insights, and evolutionary trend) of domain literature are discovered by network analysis, cluster analysis, and timeline analysis. We summarize the generic framework based on knowledge domain structure and highlight the position of knowledge source, especially textual information, in task recommendation models. Drawing on the Shneider four-stage model, the temporal evolution trend is graphically illustrated to emphasize avenues for future research. Our study conveys accumulated and synthesized specialty knowledge to researchers or newcomers to help them design, initiate, implement, manage, and evaluate recommender systems in crowdsourcing.

Abnormal Crowd Traffic Detection for Crowdsourced Indoor Positioning in Heterogeneous Communications Networks

WiFi fingerprint-based indoor positioning system emerges to provide fundamental location-related service in heterogeneous communications networks. It relies on crowdsourcing technology in the collection of received signal strength (RSS) to dynamically update fingerprint database. However, this crowdsourced indoor positioning system is vulnerable to the intrusion of dishonest users (i.e., attackers). Attackers may manipulate the number of users who submit RSS fingerprints, and finally mislead the evaluation of crowd traffic evaluation. In this paper, we propose an abnormal crowd traffic detection (ACTD) scheme to identify attackers according to their abnormal RSS sensing behaviors. Specifically, a fog server is explored to serve as the crowdsourcing platform to perform data storage and detection. We first categorize attackers into three different levels according to their real geographical locations and collusion. Then, through the analysis of pseudonym changing behavior in RSS submission, we propose a rarity-based outlier detection to classify attackers of level-1. Furthermore, we propose a variable-length Markov model, i.e., probabilistic suffix tree (PST), to detect the colluded users who are not at the target point of interest (POI). In addition, a metric learning algorithm is developed to detect the collusion of AP organizer based on RSS fingerprint distance difference. The extensive simulation results show that the ACTD scheme can effectively resist attackers with high accuracy and appropriately deal with traffic evaluation from RSS fingerprint information.

Digital Hunters

&#x0D; &#x0D; &#x0D; As the infrastructure of the internet continues to expand, networked computational surveillance becomes an essential practice of territorial and biopolitical control. The feedback loop between information technologies and global structures of power creates new territorial and biopolitical regimes that sanction the mobility of people and information across Earth. These new ‘techno-territories’ lead to the emergence of new agents of power, who weave virtual and material worlds together in order to exercise control over these new spaces and the bodies that flow through them. This article discusses the emergence of ‘digital hunters’ as both subjects and objects of power through a discursive analysis of AZ: move and get shot (2011-2014) and The Virtual Watchers (2016), two artworks by Joana Moll based on research into crowdsourced surveillance systems at the US/Mexico border. Through a discussion of these projects I trace the emergence of digital hunting as a new practice of territorial control through networked images, as citizens are militarized through participatory architectures of surveillance and social media.&#x0D; &#x0D; &#x0D;

Delivery systems with crowd‐sourced drivers: A pickup and delivery problem with transfers

AbstractRapid urban growth, the increasing importance of e‐commerce and high consumer service expectations have given rise to new and innovative models for freight delivery within urban environments. Crowdsourced solutions—where drivers are not employed by a carrier but occasionally offer their services through on‐line platforms and are contracted as required by carriers—are receiving growing attention from industry. We consider a crowdsourced system where drivers express their availability to perform delivery tasks for a given period of time and the platform communicates a schedule with requests to serve. We investigate the potential benefits of introducing transfers to support driver activities. At transfer locations, drivers can drop off packages for pick up by other drivers at a later time. We frame the problem as a multidepot pickup and delivery problem with time windows and transfers, and propose an adaptive large neighborhood search algorithm that effectively identifies beneficial transfer opportunities and synchronizes driver operations. Computational experiments indicate that introducing transfer options can significantly reduce system‐wide travel distance as well as the number of drivers required to serve a given set of requests, especially when drivers have short availability and requests have high service requirements.

Communicating Benefits from Vaccines Beyond Preventing Infectious Diseases.

Despite immunisation being one of the greatest medical success stories of the twentieth century, there is a growing lack of confidence in some vaccines. Improving communication about the direct benefits of vaccination as well as its benefits beyond preventing infectious diseases may help regain this lost confidence. A conference was organised at the Fondation Merieux in France to discuss what benefits could be communicated and how innovative digital initiatives can used for communication. During this meeting, a wide range of indirect benefits of vaccination were discussed. For example, influenza vaccination can reduce hospitalisations and deaths in older persons with diabetes by 45% and 38%, respectively, but the link between influenza and complications from underlying chronic non-communicable diseases such as diabetes is frequently underestimated. Vaccination can reduce antimicrobial resistance (AMR), which is growing, by reducing the incidence of infectious disease (though direct and indirect or herd protection), by reducing the number of circulating AMR strains, and by reducing the need for antimicrobial use. Disease morbidity and treatment costs in the elderly population are likely to rise substantially, with the ageing global population. Healthy ageing and life-course vaccination approaches can reduce the burden of vaccine-preventable diseases, such as seasonal influenza and pneumococcal diseases, which place a significant burden on individuals and society, while improving quality of life. Novel disease surveillance systems based on information from Internet search engines, mobile phone apps, social media, cloud-based electronic health records, and crowd-sourced systems, contribute to improved awareness of disease burden. Examples of the role of new techniques and tools to process data generated by multiple sources, such as artificial intelligence, to support vaccination programmes, such as influenza and dengue, were discussed. The conference participants agreed that continual efforts are needed from all stakeholders to ensure effective, transparent communication of the full benefits and risks of vaccination.

A Reliable Weighting Scheme for the Aggregation of Crowd Intelligence to Detect Fake News

Social networks play an important role in today’s society and in our relationships with others. They give the Internet user the opportunity to play an active role, e.g., one can relay certain information via a blog, a comment, or even a vote. The Internet user has the possibility to share any content at any time. However, some malicious Internet users take advantage of this freedom to share fake news to manipulate or mislead an audience, to invade the privacy of others, and also to harm certain institutions. Fake news seeks to resemble traditional media to establish its credibility with the public. Its seriousness pushes the public to share them. As a result, fake news can spread quickly. This fake news can cause enormous difficulties for users and institutions. Several authors have proposed systems to detect fake news in social networks using crowd signals through the process of crowdsourcing. Unfortunately, these authors do not use the expertise of the crowd and the expertise of a third party in an associative way to make decisions. Crowds are useful in indicating whether or not a story should be fact-checked. This work proposes a new method of binary aggregation of opinions of the crowd and the knowledge of a third-party expert. The aggregator is based on majority voting on the crowd side and weighted averaging on the third-party side. An experimentation has been conducted on 25 posts and 50 voters. A quantitative comparison with the majority vote model reveals that our aggregation model provides slightly better results due to weights assigned to accredited users. A qualitative investigation against existing aggregation models shows that the proposed approach meets the requirements or properties expected of a crowdsourcing system and a voting system.

BTR: A Feature-Based Bayesian Task Recommendation Scheme for Crowdsourcing System

The crowdsourcing system is a distributed problem-solving platform, in which tasks are delivered to the crowd (i.e., crowdworkers) in the form of an open call. Usually, large-scale crowdsourcing systems contain abundant microtasks, and the overhead of a crowdworker spending on searching the appropriate task may be comparable to the cost of completing the task. Therefore, task recommendation is necessary. However, existing work ignores the dynamics in crowdsourcing system, i.e., new tasks continually arrive, which leads to the issues of task cold-start. To overcome the challenge of the new coming task recommendation, this article proposes a feature-based Bayesian task recommendation (BTR) scheme. The key idea to deal with the dynamics of the crowdsourcing system lies in that the BTR learns the latent factor of the task through the task features instead of task ID and then learns the user’s preference according to their historical behaviors. Specifically, based on task features and the user’s historical behavior records, BTR can not only timely provide crowdworkers with personalized task recommendations but also solve the task cold-start problem. The simulations based on the real crowdsourced data set demonstrate that BTR performs better than other typical schemes that target at recommending the newly arrived tasks to crowdworkers.

Walrasian Equilibrium-Based Multiobjective Optimization for Task Allocation in Mobile Crowdsourcing

With the rapid development of Industry 5.0 and mobile devices, the research of mobile crowdsensing networks has become an important research focus. Task allocation is an important research content that can inspire crowd workers to participate in crowd tasks and provide truthful sensed data in mobile crowdsourcing systems. However, how to inspire crowd workers to participate in crowd tasks and provide truthful sensed data still has many challenges. In this article, based on the Markov model and collaborative filtering model, the similarities, trajectory prediction, dwell time, and trust degree are considered to propose the Markov and Collaborative filtering-based Task Recommendation (MCTR) model. Then, based on the Walrasian equilibrium, the optimum solution is researched to maximize the social welfare of mobile crowdsourcing systems. Finally, the comparison experiments are carried out to evaluate the performance of the proposed multiobjective optimization and the Markov-based task allocation with other methods. Through comparison experiments, the efficiency and adaptation of mobile crowdsourcing systems could be improved by the proposed task allocation.

Content to cash: Understanding and improving crowdsourced live video broadcasting services with monetary donations

Crowdsourced live video broadcasting (livecast) services such as Twitch and Douyu have become increasingly popular in recent years. In such a service, how to allocate limited service capacities, including video transcoding and delivery capacities among numerous channels, is a critical problem. Previous studies allocate capacities based on popularity. In this paper, we analyze Douyu, a leading crowdsourced livecast website in China, with a measurement approach. We find that Douyu is deployed upon a video delivery network (VDN), and it prioritizes popular channels when allocating service capacities; we also find that viewers’ willingness to donate monetary gifts in a channel is closely related to their streaming experiences, which are decided by service capacities allocated in the channel. On the other hand, a livecast channel’s profitability is only moderately correlated to its popularity. In other words, there exists a mismatch between the popularity-based service strategies and Douyu’s business model. Motivated by our analysis, we propose that channels’ profitability as well as popularity should be considered in capacity allocating. We present proactive and reactive algorithms, which balance viewers’ satisfaction with system’s monetary profit, for allocating transcoding capacity among livecast channels. We also propose a practical strategy for VDN edge nodes to select channels to replicate, by taking channels’ popularity, profitability, and bandwidth consumptions into consideration. Experiments driven by real-world measurement data show that our proposed solutions can effectively improve the overall benefits for a crowdsourced livecast system and individual VDN edge nodes, and avoid adjusting channels’ transcoding schemes too often during livecast sessions.