Wisdom Of Crowds Research Articles

What if the Forecaster Knew: Assessing Forecast Reliability Via Simulation

The FIFA Men’s World Cup (FWC) is the most important football (soccer) competition, attracting worldwide attention. A popular practice among football fans in Brazil is to organize contests in which each participant informs guesses on the final score of each match. The participants are then ranked according to some scoring rule. Inspired by these contests, we created a website to hold an online contest, in which participants were asked for their probabilities on the outcomes of upcoming matches of the FWC. After each round of the tournament, the ranking of all participants based on a proper scoring rule was published. In this article we estimate, by means of simulations, the ability of the best forecasters of our contest, considering that their good performances could be due to randomness. We also study the performance of some methods to aggregate individual forecasts, in order to study if some sort of wisdom of crowds (WOC) phenomenon was verified in the contest.

Crowdsmelling: A preliminary study on using collective knowledge in code smells detection

Code smells are seen as major source of technical debt and, as such, should be detected and removed. However, researchers argue that the subjectiveness of the code smells detection process is a major hindrance to mitigate the problem of smells-infected code. We proposed the crowdsmelling approach based on supervised machine learning techniques, where the wisdom of the crowd (of software developers) is used to collectively calibrate code smells detection algorithms, thereby lessening the subjectivity issue. This paper presents the results of a validation experiment for the crowdsmelling approach. In the context of three consecutive years of a Software Engineering course, a total "crowd" of around a hundred teams, with an average of three members each, classified the presence of 3 code smells (Long Method, God Class, and Feature Envy) in Java source code. These classifications were the basis of the oracles used for training six machine learning algorithms. Over one hundred models were generated and evaluated to determine which machine learning algorithms had the best performance in detecting each of the aforementioned code smells. Good performances were obtained for God Class detection (ROC=0.896 for Naive Bayes) and Long Method detection (ROC=0.870 for AdaBoostM1), but much lower for Feature Envy (ROC=0.570 for Random Forrest). Obtained results suggest that crowdsmelling is a feasible approach for the detection of code smells, but further validation experiments are required to cover more code smells and to increase external validity.

Forecasting foreign exchange rates as group experiment: actuality bias and fact-convergence effect within wisdom of crowds

PurposeThis article examines the accuracy and bias inherent in the wisdom of crowd effect. The purpose is to clarify what kind of bias crowds have when they make predictions. In the theoretical inquiry, the effect of the accumulated absolute deviation was simulated. In the empirical study, the observed biases were examined using data from forecasting foreign exchange rates.Design/methodology/approachIn the theoretical inquiry, the effect of the accumulated absolute deviation was simulated based on mathematical propositions. In the empirical study, the data from 2004 to 2011 were provided by Nikkei, which holds the “Nikkei Yen Derby” competition. In total, 3,657 groups forecasted the foreign exchange rate, and the first prediction was done in early May to forecast the rate at the end of May. The second round took place in June in a similar manner.FindingsThe average absolute deviation in May was smaller than that in June. The first round of prediction was more accurate than the second round one. Predictors were affected by the observable real exchange rate, such that they modified their forecasts by referring to the actual data in early June. An actuality bias existed when the participants lost their diverse prospects. Since the standard deviations of the June forecasts were smaller than those of May, the fact-convergence effect was supported.Originality/valueThis article reports novel findings that affect the wisdom of crowd effect—referred to as actuality bias and fact-convergence effect. The former refers to a forecasting bias toward the observable rate near the forecasting date. The latter implies that predictors, as a whole, indicate smaller forecast deviations by observing the realized foreign exchange rate.

A Secure Blockchain Framework for Storing Historical Text: A Case Study of the Holy Hadith

Historical texts are one of the main pillars for understanding current civilization and are used to reference different aspects. Hadiths are an example of one of the historical texts that should be securely preserved. Due to the expansion of the online resources, fabrications and alterations of fake Hadiths are easily feasible. Therefore, it has become more challenging to authenticate the online available Hadith contents and much harder to keep these authenticated results secure and unmanipulated. In this research, we are using the capabilities of the distributed blockchain technology to securely archive the Hadith and its level of authenticity in a blockchain. We selected a permissioned blockchain customized model in which the main entities approving the level of authenticity of the Hadith are well-established and specialized institutions in the main Islamic countries that can apply their own Hadith validation model. The proposed solution guarantees its integrity using the crowd wisdom represented in the selected nodes in the blockchain, which uses voting algorithms to decide the insertion of any new Hadiths into the database. This technique secures data integrity at any given time. If any organization’s credentials are compromised and used to update the data maliciously, 50% + 1 approval from the whole network nodes will be required. In case of any malicious or misguided information during the state of reaching consensus, the system will self-heal using practical Byzantine Fault Tolerance (pBFT). We evaluated the proposed framework’s read/write performance and found it adequate for the operational requirements.

The Effects of COVID-19 on the Adoption of “On-the-Shelf Technologies”: Virtual Dressing Room Software and the Expected Rise of Third-Party Reverse-Logistics

The fashion industry is adapting to the new situation caused by the COVID-19 pandemic by changing the structure of its supply chain, much like other industries that experience exogenous shocks. The pandemic affected conditions of apparel retailers: downstream among consumers and upstream among suppliers. It induced retailers to accelerate the adoption of technologies to make their supply chains more flexible and resilient. Before COVID-19, the apparel industry had gradually adopted virtual dressing room (VDR) technologies and crowd wisdom software, both of which reduce the risk associated with online purchases. Apparel retailers also altered the structure of their supply chains by outsourcing product turns via third-party logistics providers. This article analyzes how changes in market demand and supply conditions following the outbreak of COVID-19 accelerated the adoption of VDR technologies. The analysis is based on a conceptual microeconomic model of adopting technologies and changes in the supply chain. We support our theoretical findings with business cases. Understanding the nonlinear relationships among changes in demand, supply chains, and retail technology adoption triggered by exogenous shocks is essential for managers and affects the quality of service.

Collective Intelligence in Design Crowdsourcing

This study investigates how collective intelligence emerges in crowdsourcing for architectural design. Previous studies have revealed that collective intelligence emerges from collaboration and can outperform individual intelligence. As design is a highly collaborative practice, collective intelligence plays a vital role in the design process. In this study, we compare the structure of two architectural design crowdsourcing systems using several methods. The results of the analysis suggest that design crowdsourcing systems can give rise to the following three types of collective intelligence: (1) discussive, which emerges from a conversation between designers and clients; (2) synthetic, which emerges from a parallel and sequential design development; and (3) evaluative, which is based on the wisdom of the crowd in evaluating and selecting designs. The article concludes with recommendations for collaborative design method.

Enhancing Mobility as a Service (MaaS) concept through Social Interaction and Crowdsourcing applications

The present paper considers the rapid growth of private transport usage in urban environments, that is linked with increased traffic congestion and imbalance between parking supply and demand, to promote novel Mobility as a Service (MaaS) concepts towards overcoming both issues. Based on the findings of two research projects, SocialCar and SocialPARK, focusing on supporting social interaction of travelers and the proper use of crowd-coursed information and data, sustainable solutionsfor relieving urban congestion and vital parking management are being presented. Projects findings, proved to have a significant impact on increasing interest for carpooling and PT combined usage, while the parking availability sharing information seems attractive for the majority of citizens. Combining both concepts, and exploiting the “wisdom of the crowd” for providing mobility and parking solutions in real time conditions, under a monitored and accurate frame of operation, adds on the creation of a novel value-chain for mobility and parking towards sustainability.

Mapping and population size estimates of people who inject drugs in Afghanistan in 2019: Synthesis of multiple methods.

IntroductionMapping and population size estimates of people who inject drugs (PWID) provide information needed for monitoring coverage of programs and planning interventions. The objectives of this study were to provide the locations and numbers of PWID in eight cities in Afghanistan and extrapolate estimates for the country as a whole.MethodsMultiple population size estimation methods were used, including key informant interviews for mapping and enumeration with reverse tracking, unique object and service multipliers, capture-recapture, and wisdom of the crowds. The results of the several methods were synthesized using the Anchored Multiplier–a Bayesian approach to produce point estimates and 95% credible intervals (CI). Using the prevalence of PWID in the eight cities and their correlation with proxy indicators, we extrapolated the PWID population size for all of Afghanistan.ResultsKey informants and field mapping identified 374 hotspots across the eight cities from December 29, 2018 to March 20, 2019. Synthesizing results of the multiple methods, the number of male PWID in the eight study cities was estimated to be 11,506 (95% CI 8,449–15,093), corresponding to 0.69% (95% CI 0.50–0.90) of the adult male population age 15–64 years. The total number of women who injected drugs was estimated at 484 (95% CI 356–633), corresponding to 0.03% (95% CI 0.02–0.04) of the adult female population. Extrapolating by proxy indicators, the total number of PWID in Afghanistan was estimated to be 54,782 (95% CI 40,250–71,837), men and 2,457 (95% CI 1,823–3,210) women. The total number of PWID in Afghanistan was estimated to be 57,207 (95% CI 42,049–75,005), which corresponds to 0.37% (95% CI 0.27–0.48) of the adult population age 15 to 64 years.DiscussionThis study provided estimates for the number of PWID in Afghanistan. These estimates can be used for advocating and planning services for this vulnerable at-risk population.

When Is Society Susceptible to Manipulation?

We consider a social learning model where agents learn about an underlying state of the world from individual observations as well as from exchanging information with each other. A principal (e.g., a firm or a government) interferes with the learning process in order to manipulate the beliefs of the agents. By utilizing the same forces that give rise to the “wisdom of the crowd” phenomenon, the principal can get the agents to take an action that is not necessarily optimal for them but is in the principal’s best interest. We characterize the social norms and network structures that are susceptible to this kind of manipulation and derive conditions under which a social network is impervious and cannot be manipulated. In the process, we develop a new centrality measure and describe how our model offers insights into designing networks that are resistant to manipulation. This paper was accepted by Gabriel Weintraub, revenue management and market analytics.

Safety in numbers: how social choice theory can inform avalanche risk management

ABSTRACT Avalanche studies have undergone a transition in recent years. Early research focused mainly on environmental factors. More recently, attention has turned to human factors in decision making, such as behavioural and cognitive biases. This article adds a social component to this human turn in avalanche studies. It identifies lessons for decision making by groups of skiers from the perspective of social choice theory, a sub-field of economics, decision theory, philosophy and political science that investigates voting methods and other forms of collective decision making. In the first part, we outline the phenomenon of wisdom of crowds, where groups make better decisions than their individual members. Drawing on the conceptual apparatus of social choice theory and using idealised scenarios, we identify conditions under which wisdom of crowds arises and also explain how and when deciding together can instead result in worse decisions than may be expected from individual group members. In the second part, we use this theoretical understanding to offer practical suggestions for decision making in avalanche terrain. Finally, we make several suggestions for risk management in other outdoor and adventure sports and for outdoor sports education.

AI: Coming of age?

AbstractAI has had many summers and winters. Proponents have overpromised, and there has been hype and disappointment. In recent years, however, we have watched with awe, surprise, and hope at the successes: Better than human capabilities of image-recognition; winning at Go; useful chatbots that seem to understand your needs; recommendation algorithms harvesting the wisdom of crowds. And with this success comes the spectre of danger. Machine behaviours that embed the worst of human prejudice and biases; techniques trying to exploit human weaknesses to skew elections or prompt self-harming behaviours. Are we seeing a perfect storm of social media, sensor technologies, new algorithms and edge computing? With this backdrop: is AI coming of age?

Hint: harnessing the wisdom of crowds for handling multi-phase tasks

Hint: harnessing the wisdom of crowds for handling multi-phase tasks

Prognostication in Acute Neurological Emergencies

BackgroundFor patients with acute, serious neurological conditions presenting to the emergency department (ED), prognostication is typically based on clinical experience, scoring systems and patient co-morbidities. Because estimating a poor prognosis influences caregiver decisions to withdraw life-sustaining therapy, we investigated the consistency of prognostication across a spectrum of neurology physicians. MethodsFive acute neurological presentations (2 with large hemispheric infarction; 1 with brainstem infarction, 1 with lobar hemorrhage, and 1 with hypoxic-ischemic encephalopathy) were selected for a department-wide prognostication simulation exercise. All had presented to our tertiary care hospital's ED, where a poor outcome was predicted by the ED neurology team within 24 hours of onset. Relevant clinical, laboratory and imaging data available before ED prognostication were presented on a web-based platform to 120 providers blinded to the actual outcome. The provider was requested to rank-order, from most to least likely, the predicted 90-day modified Rankin Scale (mRS) score. To determine the accuracy of individual outcome predictions we compared the patient's the actual 90-day mRS score to highest ranked predicted mRS score. Additionally, the group's “weighted” outcomes, accounting for the entire spectrum of mRS scores ranked by all respondents, were compared to the actual outcome for each case. Consistency was compared between pre-specified provider roles: neurology trainees versus faculty; non-vascular versus vascular faculty. ResultsResponses ranged from 106-110 per case. Individual predictions were highly variable, with predictions matching the actual mRS scores in as low as 2% of respondents in one case and 95% in another case. However, as a group, the weighted outcome matched the actual mRS score in 3 of 5 cases (60%). There was no significant difference between subgroups based on expertise (stroke/neurocritical care versus other) or experience (faculty versus trainee) in 4 of 5 cases. ConclusionAcute neuro-prognostication is highly variable and often inaccurate among neurology providers. Significant differences are not attributable to experience or subspecialty expertise. The mean outcome prediction from group of providers (“the wisdom of the crowd”) may be superior to that of individual providers.

Anticipated innovations for the blue economy: Crowdsourced predictions for the North Sea Region

The mission policy approach to the sustainable blue economy has identified as critical the ability to anticipate the emergence of a wide range of feasible innovations as they enter the transactional environment of organizations in the marine and maritime sector. This article contributes to that growing effort by harnessing the wisdom of the crowd and presents more than 60 crowdsourced, time-specific innovation forecasts expected to impact maritime, shipbuilding, ports, offshore wind, and ocean infrastructure. Data were collected in 2020 by the EU-funded Interreg VB PERISCOPE Project, a North Sea Region initiative to catalyze transregional innovation. The results can be used strategically to develop collaborative, transregional planning and policy for innovation based on data reflecting public expectations for the future. Years from now, this article can also act as a snapshot of public expectations at the onset of the decade.

Normalized Attraction Travel Personality Representation for Improving Travel Recommender Systems

Travel recommender systems (TRSs) aim to reduce travel-related search overload. A significant part of a TRS is representing attractions in a way that reflect the explicit and implicit features of attractions. However, traditional attraction representation methods may not provide a complete image of attractions. Building on the notions of user travel styles (UTSs) and the wisdom of crowds, we propose a method derived from topic-model-based models to represent travel attractions, called the Normalized Attraction Travel Personality (NATP) representation. This approach attempts to leverage the semantics of attraction reviews to model user travel personalities (UTPs), which collectively can construct the attraction travel personality (ATP) representation. Furthermore, we regularize and normalize the ATP representation to obtain our proposed representation. This NATP-based attraction representation could capture implicit characteristics of attractions revealed by the wisdom of crowds. Our experiments show that our representation method gained better results when evaluated against comparative approaches in terms of rating prediction and recommendation ranking quality, indicating the effectiveness of the proposed attraction representation. Lastly, we qualitatively investigate how our attraction representation surpasses the state-of-the-art representation methods.

Deliberation and the Wisdom of Crowds

Deliberation and the Wisdom of Crowds

Deliberation and the Wisdom of Crowds

Deliberation and the Wisdom of Crowds

Betting on a buzz, mispricing and inefficiency in online sportsbooks

Bookmakers sell claims to bettors that depend on the outcomes of professional sports events. Like other financial assets, the wisdom of crowds could help sellers to price these claims more efficiently. We use the Wikipedia profile page views of professional tennis players involved in over ten thousand singles matches to construct a buzz factor. This measures the difference between players in their pre-match page views relative to the usual number of views they received over the previous year. The buzz factor significantly predicts mispricing by bookmakers. Using this fact to forecast match outcomes, we demonstrate that a strategy of betting on players who received more pre-match buzz than their opponents can generate substantial profits. These results imply that sportsbooks could price outcomes more efficiently by listening to the buzz.

Policy Choice and the Wisdom of Crowds

Policy Choice and the Wisdom of Crowds

Wisdom of Crowds and Commodity Pricing

Wisdom of Crowds and Commodity Pricing