Crowdsourced Data Research Articles

Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms

This perspective paper explores challenges associated with online crowdsourced data collection, particularly focusing on longitudinal tasks with time-sensitive outcomes like response latencies. Based on our research, we identify two significant sources of bias: technical shortcomings such as low, variable frame rates, and human factors, contributing to high attrition rates. We explored potential solutions to these problems, such as enforcing hardware acceleration and defining study-specific frame rate thresholds, as well as pre-screening participants and monitoring hardware performance and task engagement over each experimental session. With this discussion, we intend to provide recommendations on how to improve the quality and reliability of data collected via online crowdsourced platforms and emphasize the need for researchers to be cognizant of potential pitfalls in online research.

Leto: Crowdsourced Radio Map Construction With Learned Topology and a Few Landmarks

Existing crowdsourced indoor positioning systems (CIPSs) usually require prior knowledge about the site and a tedious calibration process. Moreover, they may require a large number of landmarks while ignoring the topology information that may be contained in the crowdsourced data. In this paper, we present Leto, a system that uses <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">le</b> arned <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">to</b> pology information from combined user traces to construct a radio map. Leto relies on crowdsourced WiFi and accelerometer signals only without requiring any prior knowledge about the site. Our key idea is that learned topology information can reduce the required number of landmarks, while available landmarks can transform the topology into a map. We propose a novel framework that efficiently learns the map topology by a hybrid multidimensional scaling (HMDS) algorithm and accurately rectifies the map using only a few anchors by an adaptive force-directed (AFD) algorithm. We also provide a theoretical convergence analysis of the HMDS algorithm. Experimental results on real-world datasets show that Leto can capture useful topology information and achieve significant improvements in radio map construction compared to existing systems.

Reel reflections: the role of entertainment media narratives in coping among young adult cancer survivors

Introduction Young adult cancer survivors (YACS; ages 18–39) report a significant psychological burden. Entertainment media narratives (e.g., books, movies, shows that are produced for mass consumption) might be an effective tool for reducing this distress, although little is known about present use among YACS. Method YACS completed a survey about their use of entertainment media narratives to cope with cancer using an adapted version of the Brief COPE. Additionally, YACS reported their use of entertainment media narratives to start conversations about their experience with others, and they described features of entertainment media narratives that they found helpful in coping. Results We recruited 108 YACS from three recruitment sites. Most participants were White (n = 65), female (n = 54), and recruited from Prolific (n = 56), an Internet-based, crowdsourced data collection platform. Participants were, on average, 30 years old and 45 months from the completion of their primary cancer treatment. YACS who used entertainment media to cope with cancer (n = 32; 29.6%) were significantly younger and significantly closer to the end of their primary treatment. Compared to nonusers, users of entertainment media narratives to cope were also more likely to identify as Black; identifying as Black was associated with a 2.05-factor increase in using narrative entertainment media to cope with cancer even when controlling for other demographic differences. Additionally, compared to their peers, Black YACS reported greater use of narratives to start cancer-related conversations. Emotional and inspirational storylines were the most helpful story features. Discussion Some YACS, especially Black YACS, use stories to cope with their cancer experience. YACS patients could find stories useful in exploring their cancer-related emotions, although the exact benefits are still unknown.

Fusing crowdsourced data to an adaptive wireless traffic signal control system architecture

In this study, a novel wireless adaptive traffic signal control System architecture is proposed, in which crowdsourced data is integrated to detect the traffic state on each approach of an intersection. Under the wireless architecture, the green signal timing is calculated using a travel time-based Max Pressure control algorithm, which is performed on a single-board computer (SBC, RPi 4). The resulting green timings are sent from SBC to the master controller (ESP32) over serial UART and then to the slave (ESP32) wirelessly using ESP-NOW; slave ESP32 is placed on each approach of the intersection. It integrates a fail-safe mechanism to avoid any vehicular conflicts. The proposed system architecture also embeds a crowdsourced data-based pre-emption of intersections for efficient movement of emergency vehicles. The efficacy of the proposed system is compared using a simulation tool with real-world data. With respect to actuated and Max Pressure based on traffic flow, the proposed algorithm reduces the total delay by 54.8% and 22.96%, respectively. Further, the total queue dissipation time decreases by 8.32% and 5.45%, respectively. It outperforms the actuated signal control as well as Max Pressure based on traffic flows. The capability of transforming existing traffic signal controllers into wireless adaptive traffic signal control systems makes it a very affordable and scalable solution.

Efficient Online Crowdsourcing with Complex Annotations

Crowdsourcing platforms use various truth discovery algorithms to aggregate annotations from multiple labelers. In an online setting, however, the main challenge is to decide whether to ask for more annotations for each item to efficiently trade off cost (i.e., the number of annotations) for quality of the aggregated annotations. In this paper, we propose a novel approach for general complex annotation (such as bounding boxes and taxonomy paths), that works in an online crowdsourcing setting. We prove that the expected average similarity of a labeler is linear in their accuracy conditional on the reported label. This enables us to infer reported label accuracy in a broad range of scenarios. We conduct extensive evaluations on real-world crowdsourcing data from Meta and show the effectiveness of our proposed online algorithms in improving the cost-quality trade-off.

A Review of Crowdsourcing Update Methods for High-Definition Maps

High-definition (HD) maps serve as crucial infrastructure for autonomous driving technology, facilitating vehicles in positioning, environmental perception, and motion planning without being affected by weather changes or sensor-visibility limitations. Maintaining precision and freshness in HD maps is paramount, as delayed or inaccurate information can significantly impact the safety of autonomous vehicles. Utilizing crowdsourced data for HD map updating is widely recognized as a superior method for preserving map accuracy and freshness. Although it has garnered considerable attention from researchers, there remains a lack of comprehensive exploration into the entire process of updating HD maps through crowdsourcing. For this reason, it is imperative to review and discuss crowdsourcing techniques. This paper aims to provide an overview of the overall process of crowdsourced updates, followed by a detailed examination and comparison of existing methodologies concerning the key techniques of data collection, information extraction, and change detection. Finally, this paper addresses the challenges encountered in crowdsourced updates for HD maps.

Interpretable machine learning for predicting urban flash flood hotspots using intertwined land and built-environment features

Pluvial flash floods are fast-moving hazards and causes significant disruptions in urban areas. With the increase in heavy precipitations, the ability to proactively identify flash floods hotspots in cities is critical for flood nowcasting and predictive monitoring of risks. While rainfall runoff models and hydrologic models are useful models for flash flood prediction, these models are computationally expensive and effort intensive to be used for flood nowcasting. To address this challenge, this study presents interpretable machine learning models for predicting urban flash flood hotspots based on intertwined land and built environment features. The task of predicting flash flood hotspots is formulated as a binary classification problem, and three recent flash flood events in U.S. cities are selected for data collection and model validation. Various features related to land and built environment characteristics are constructed using diverse datasets, and the occurrences of flash floods are captured using crowdsource data from the events. Using these features and datasets, the flash flood hotspots of cities are predicted with two ensemble models based on decision trees. The results demonstrate that the models can achieve good accuracy (0.8) in identifying flooded/non-flooded locations. Especially, the models can achieve high true positive rate (0.83–0.89) and low missing rate, demonstrating the methods' practicability for accurately predicting flooded hotspots. The model interpretation results indicate that land features related to hydrological and topological features have greater impacts on flash flood risk, than built environment features. Further analysis reveals that the feature importance, model performance, and model transferability performance vary among cities and localized specifications of the models are needed for accurate prediction of flash flood for a particular city. The data-driven machine learning models presented in this study provide a useful tool for predicting flash flood hotspots based on the intertwined features of land and the built environment in cities to enable nowcasting and proactive monitoring of flash flood hotspots for emergency response and also inform integrated urban design and development towards flash flood risk reduction.

Designing a Human-centered AI Tool for Proactive Incident Detection Using Crowdsourced Data Sources to Support Emergency Response

Time of incident reporting is a critical aspect of emergency response. However, the conventional approaches to receiving incident reports have time delays. Non-traditional sources such as crowdsourced data present an opportunity to detect incidents proactively. However, detecting incidents from such data streams is challenging due to inherent noise and data uncertainty. Naively maximizing detection accuracy can compromise spatial-temporal localization of inferred incidents, hindering response efforts. This article presents a novel human-centered AI tool to address the above challenges. We demonstrate how crowdsourced data can aid incident detection while acknowledging associated challenges. We use an existing CROME framework to facilitate training and selection of best incident detection models, based on parameters suited for deployment. The human-centered AI tool provides a visual interface for exploring various measures to analyze the models for the practitioner’s needs, which could help the practitioners select the best model for their situation. Moreover, in this study, we illustrate the tool usage by comparing different models for incident detection. The experiments demonstrate that the CNN-based incident detection method can detect incidents significantly better than various alternative modeling approaches. In summary, this research demonstrates a promising application of human-centered AI tools for incident detection to support emergency response agencies.

EDIR: Efficient Distributed Image Retrieval of Novel Objects in Mobile Networks

Crowdsourcing data collection from a network of mobile devices is useful in various applications. Mobile devices store a large amount of visual data that can aid in different application scenarios. Trained Convolutional Neural Networks (CNNs) can be deployed on mobile devices to be used in searching for objects of interest. Querying for novel objects, for which models have not been trained yet, presents some unique challenges. When novel objects are queried, new models must be trained and distributed to all edge devices. In this paper, we propose an efficient method and a system, called EDIR, which enables answering these queries while taking into account the bandwidth limitations encountered in wireless networks, as well as the limited energy and computational power on mobile devices. Through extensive experimentation, we show that using distance-based classifiers, specifically those relying on the Cosine distance, leads to more efficient utilization of network resources by reducing the number of false positives. We perform analysis that enables the requester to tune the parameters of interest before issuing the query, and validate our theoretical results. EDIR reduces the amount of transferred data by more than 45% compared to other approaches while simultaneously achieving a good F1 score.

REVIEW OF GLOBAL PRACTICE OF USING CELL PHONE DATA TO MEASURE TRAVELLER DATA

JSC “Ukrzaliznytsia” is a national integrated railway company and the largest employer in the country, playing a crucial role in the Ukrainian economy and the labour market. The rail companies faces significant risks, mainly due to prolonged disruptions due to the ongoing armed conflict on Ukrainian territory [1]. In such conditions, the question of the need for a comprehensive solution to the problems of the functioning of passenger railway transport, where the number of travellers serves as a pivotal indicator of the demand for transportation services, offering essential information for planning and evaluations. Advancements in technology have enabled the collection of crowd-sourced traffic data [2]. Crowdsourced smartphone data can collect information about user locations, travel paths, route choices, travel times, and speeds. This analysis explores how the number of passengers in trains is measured, examining technologies and methods that use cell phone data globally to assess their potential application in Ukraine.

A crowdsensing-based framework for sound and vibration data analysis in smart urban environments

In this paper, we introduce a framework tailored to the precise analysis of sound and vibration data in the dynamic context of smart cities. By harnessing the power of crowdsensing data, our system offers a robust and highly adaptable solution for the real-time monitoring and comprehensive assessment of acoustic and vibrational parameters within urban environments. The rise of smart cities, fueled by sensor tech and data analysis, demands advanced tools to tackle urban environmental issues and improve residents’ quality of life. Our framework empowers city stakeholders, offering insights for informed decisions in urban planning, transportation, infrastructure maintenance, and public health. Key features of our system include data acquisition through mobile crowdsensing application, advanced signal processing algorithms for noise and vibration identification, and the integration of geospatial information to provide location-specific context. Furthermore, our framework supports scalable and adaptable data analytics, ensuring the efficient utilization of resources and the effective management of urban environments. The presented framework addresses the challenges of noise pollution, structural health monitoring, transportation optimization, and public safety in smart cities. By exploiting crowdsourced data, it promotes a collaborative approach to data collection, analysis, and decision-making, fostering an environment where cities can continuously evolve and adapt to the evolving needs of their residents. This paper offers a detailed exploration of the system’s architecture, and showcases its practical implementation, affirming its potential as a powerful starting point for advancing the science of smart cities.

Climate and biodiversity change constrain the flow of cultural ecosystem services to people: A case study modeling birding across Africa under future climate scenarios

Global change is currently impacting ecosystems and their contributions to people (i.e. ecosystem services). These impacts have consequences for societies and human well-being, especially in Africa. Historically, efforts have focused on assessing global change from a social or biophysical perspective, treating them as separate entities. Yet, our understanding of impacts to social-ecological systems remains limited, particularly in the Global South, due to a lack of data, tools, and approaches accounting for social and ecological aspects of ecosystem services. This is especially relevant for cultural ecosystem services as they are less tangible. We use a simple indicator and important provider of a multitude of cultural ecosystem services, birding, to understand how climate, biodiversity, and land use change will impact cultural ecosystem services across Africa. We explore how emerging tools and data can overcome limitations in mapping and modeling cultural ecosystem services, particularly in analyzing human preferences and behavior at large spatiotemporal scales and in data-poor regions. Leveraging crowdsourced data from eBird and using machine learning techniques we map and model recreational birding to assess the underlying social-ecological relationships and the impact of future climate and environmental change. We show that bird species richness, protected areas, accessibility, and max temperature contribute most to birding suitability across the continent. Further, we show spatial shifts in the suitability of birding under three future climate scenarios (SSP126, 370, and 585). Models suggest climate and biodiversity change will increasingly constrain the flow of birding related cultural ecosystem services across Africa. This has implications for human-nature interactions, development of countries, management of protected areas, and overall human well-being in the future. More generally, we highlight opportunities for crowdsourced datasets and machine learning to integrate non-material ecosystem services in models and thus, enhance the understanding of future impacts to ecosystem services and human well-being.

Applications of using connected vehicle data for pavement quality analysis

Current quantitative methods to evaluate pavement conditions in the United States are most commonly focused on construction acceptance using the International Roughness Index (IRI). However, from an asset management perspective, qualitative visual inspection techniques are the most prevalent. Modern vehicles with factory-equipped sensors drive these roadways daily and can passively assess the condition of infrastructure at an accuracy level somewhere between qualitative assessment and rigorous construction acceptance techniques. This paper compares crowdsourced ride quality data with an industry standard inertial profiler on a 7-mile bi-directional construction zone. A linear correlation was performed on 14 miles of I-65 that resulted in an R2 of 0.7 and a p-value of &amp;lt;0.001, but with a modest fixed offset bias. The scalability of these techniques is illustrated with graphics characterizing IRI values obtained from 730,000 crowdsourced data segments over 5,800 miles of I-80 in April of 2022 and October 2022. This paper looks at the use of standard original equipment manufacturer (OEM) on-board sensor data from production vehicles to assess approximately 100 miles of roadway pavements before, during, and after construction. The completed construction projects observed IRI improvements of 10 in/mi to 100 in/mi. These results suggest that it is now possible to monitor pavement ride quality at a system level, even with a small proportion of connected vehicles (CV) providing roughness data.

Patterns of Competitiveness in Russian Gubernatorial Elections

AbstractCompetitiveness in electoral authoritarian regimes is compromised by management of the electoral arena through various kinds of manipulation. This study links two dimensions of competitiveness, namely fairness and contestation. We analyze the regional heterogeneity of both dimensions for Russian gubernatorial elections held between 2012 and 2019. To assess competitiveness, we use crowd-sourced electoral observation reports and data from the Central Election Commission. Our analysis reveals significant regional variation in both dimensions. Nevertheless, opposition victories are rare and only occur in exceptional cases. Low levels of electoral malpractices seem to be a necessary but not sufficient condition for close election results. Windows of opportunity for the opposition arise only if the regime misperceives the closeness of the race and if the opposition nominates prominent candidates. Overall, these findings shed light on the complex factors influencing the competitiveness of elections in electoral authoritarian regimes like Russia.

The use of crowdsourced mobile data in estimating pedestrian and bicycle traffic: A systematic review

To address the need for better non-motorized traffic data, policymakers and researchers are collaborating to develop new approaches and methods for estimating pedestrian and bicyclist traffic volumes. Crowdsourced mobile data, which has higher spatial and temporal coverage and lower collection costs than data collected through traditional approaches, may help improve pedestrian and bicyclist traffic estimation despite their limitations or biases. This systemic literature review documents how researchers have used crowdsourced mobile data to estimate pedestrian and bicyclist traffic volumes. We find that one source of commercial fitness application data (i.e., Strava) has been used much more frequently than other crowdsourced mobile data, and that most studies have used crowdsourced mobile data to estimate bicyclist volumes. Comparatively few studies have estimated pedestrian volumes. The most common approach to the use of crowdsourced counts is as independent variables in direct demand models. Variables constructed from crowdsourced mobile data not only have significant correlations with observed counts in statistical models but also have larger relative importance than other factors in machine learning models. Studies also show that including crowdsourced mobile data can significantly improve estimation performance. Future research directions include application of crowdsourced mobile data in more pedestrian traffic estimations, comparison of the performance of different crowdsourced mobile data, incorporation of multiple data sources, and expansion of the methods using crowdsourced mobile data for non-motorized traffic estimation.

Wildfires exacerbate inequalities in indoor pollution exposure

Wildfires lead to dramatic increases in fine particulate matter pollution concentrations. Based on the premise that higher-income households purchase more defensive investments to reduce the degree to which outdoor pollution infiltrates indoors, in this study, we investigate how income contributes to outdoor–indoor pollution infiltration rates during wildfire events. Using crowd-sourced data from the PurpleAir Real-Time Air Quality Monitoring Network and econometric models that explore variations in monitor readings over time, we find increases in outdoor pollution lead to significant increases in indoor pollution, but disproportionately so in lower-income areas. The results highlight a new inequality in pollution exposure: not only are outdoor pollution levels higher for lower-income individuals, but indoor pollution levels are higher even for similar outdoor pollution levels.

Examining the validity of the addictions neuroclinical assessment domains in a crowdsourced sample of adults with current alcohol use.

Several dimensional frameworks for characterizing heterogeneity in alcohol use disorder (AUD) have been proposed, including the Addictions Neuroclinical Assessment (ANA). The ANA is a framework for assessing individual variability within AUD across three domains corresponding to the proposed stages of the addiction cycle: reward (binge-intoxication stage), negative emotionality (withdrawal-negative affect stage), and cognitive control (preoccupation-anticipation stage). Recent work has evaluated the ANA's three-factor structure and construct validity, primarily in treatment-seekers with AUD. We extended this research by examining the factor structure, bias across alcohol use severity, longitudinal invariance, and concurrent and predictive validity of a novel assessment of the ANA domains in adults with past 12-month regular (10 + alcohol units/week) alcohol use. Participants recruited from Prolific (N = 732), a crowdsourced data collection platform, completed various self-report measures. A test-retest subsample (n = 234) completed these measures 30 days later. Split-half exploratory factor analysis and confirmatory factor analysis supported the three-factor structure of the ANA. The overall factor structure was invariant across 30 days. Concurrently and prospectively, ANA domains demonstrated convergent validity concerning theoretically aligned alcohol-related, psychological, and personality measures. However, there was evidence of poor discriminant validity, and several cognitive control and reward items demonstrated bias across alcohol use severity. Future research is needed to improve the measurement of ANA domains using multimodal indicators, examine longitudinal changes in domains and their relationship with alcohol use severity, characterize phenotypic subgroups based on relative levels of domains, and compare the utility of the ANA with other proposed frameworks for measuring AUD heterogeneity. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Crowd-sourced symptom data in pollen allergy: testing a novel study approach for assessing the efficacy of food supplements

Summary Purpose Pollen allergy can have a significant impact on a person’s quality of life. Recently, food supplements have gained in importance for persons suffering from pollen allergy. Still, there is not much research data on the efficacy of food supplements and the experienced relief, especially when choosing study designs that require less effort than randomized controlled trials (RCT). Methods A food supplement containing an Astragalus membranaceus root extract was administered to 328 voluntary participants during the 2018 birch, grass and ragweed pollen season in Austria. Participants documented their symptoms and medication intake in the online Patients Hay-fever Diary (PHD). All participants were asked to answer a quality-of-life questionnaire at the end of the study. Alongside with the adherence rate the overall symptom load index (SLI) and the nasal symptoms of the participants were monitored and compared to a filtered group of users from the PHD. Results The adherence rates range from 32.8% to 77.3%. Additional data like a quality-of-life questionnaire and the mean overall/nasal SLI showed differences between participants and the baseline. However, they were only statistically significant for the ragweed pollen season and the nasal symptoms. Conclusion The study design relied on the evaluation of crowd-source symptom data alone by comparing the participants with a baseline. The participant adherence turned out lower than expected; however a trend for reduced symptoms was found and is also supported by the quality-of-life questionnaire.

Can infrastructure, built environment, and geographic factor negate weather impact on Strava cyclists?

Cycling participation is context-sensitive and weather condition is reportedly a significant factor. How weather affects cyclists with different demographics, trip purposes, and in the context of cycling infrastructure, built environment and geographic factors is less well understood by existing literature. This paper applies autoregressive models to explain difference in Strava cycling volume from the same hour of the previous day as a function of change in weather conditions, and day of the week; the contextual effect of cycling infrastructure, built environment and geographic factors is accounted for using interaction terms. We use Strava crowdsourced cycling data in Sydney, Australia, as a case study; commute and leisure cyclists, male and female, young and older cyclists are modeled separately. We find weather conditions have a statistically significant effect on cycling participation; rain, rainfall in the last 2 hours and wind are general deterrents to cycling. Physically separated cycling lanes reduce the adverse effect of precipitation on leisure cyclists and male cyclists but have little effect in retaining commute cyclists and female cyclists. The adverse effect of precipitation and wind on commute cycling is amplified in areas with good access to jobs, possibly due to the availability of better alternative modes of transport. Inland locations generally attenuate effects of windy conditions, except for young adults. This paper sheds light on factors attenuating adverse weather effects on cycling participation and provides useful guidance for future cycling infrastructure.

RollBack: A New Time-Agnostic Replay Attack Against the Automotive Remote Keyless Entry Systems

Automotive Keyless Entry (RKE) systems provide car owners with a degree of convenience, allowing them to lock and unlock their car without using a mechanical key. Today’s RKE systems implement disposable rolling codes, making every key fob button press unique, effectively preventing simple replay attacks. However, a prior attack called RollJam was proven to break all rolling code–based systems in general. By a careful sequence of signal jamming, capturing, and replaying, an attacker can become aware of the subsequent valid unlock signal that has not been used yet. RollJam, however, requires continuous deployment indefinitely until it is exploited. Otherwise, the captured signals become invalid if the key fob is used again without RollJam in place. We introduce RollBack, a new replay-and-resynchronize attack against most of today’s RKE systems. In particular, we show that even though the one-time code becomes invalid in rolling code systems, replaying a few previously captured signals consecutively can trigger a rollback-like mechanism in the RKE system. Put differently, the rolling codes become resynchronized back to a previous code used in the past from where all subsequent yet already used signals work again. Moreover, the victim can still use the key fob without noticing any difference before and after the attack. Unlike RollJam, RollBack does not necessitate jamming at all. In fact, it requires signal capturing only once and can be exploited at any time in the future as many times as desired. This time-agnostic property is particularly attractive to attackers, especially in car-sharing/renting scenarios in which accessing the key fob is straightforward. However, while RollJam defeats virtually any rolling code–based system, vehicles might have additional anti-theft measures against malfunctioning key fobs, hence against RollBack. Our ongoing analysis (with crowd-sourced data) against different vehicle makes and models has revealed that ∼ 50% of the examined vehicles in the Asian region are vulnerable to RollBack, whereas the impact tends to be smaller in other regions, such as Europe and North America.