Wisdom Of Crowds Research Articles

Collaboration to improve cross-race face identification: Wisdom of the multi-racial crowd?

Face identification is particularly prone to error when individuals identify people of a race other than their own - a phenomenon known as the other-race effect (ORE). Here, we show that collaborative "wisdom-of-crowds" decision-making substantially improves face identification accuracy for own- and other-race faces over individuals working alone. In two online experiments, East Asian and White individuals recognized own- and other-race faces as individuals and as part of a collaborative dyad. Collaboration never proved more beneficial in a social setting than when individual identification decisions were combined computationally. The reliable benefit of non-social collaboration may stem from its ability to avoid the potential negative outcomes of group diversity such as conflict. Consistent with this benefit, the racial diversity of collaborators did not influence either general or race-specific face identification accuracy. Our findings suggest that collaboration between two individuals is a promising strategy for improving cross-race face identification that may translate effectively into forensic and eyewitness settings.

The Wisdom of the Crowd is not a Forgone Conclusion. Effects of Self-Selection on (Collaborative) Knowledge Construction.

Web 2.0 has elevated the possibilities of collaboration to unprecedented levels. Therein lies great potential, as the aptly coined phenomenon "Wisdom of the Crowd" implies. When it comes to controversial topics, however, there is no safety in numbers alone. On the contrary, collaboration among only like-minded people may even exacerbate biases (e.g., Echo Chambers). Yet, it is human nature to seek out like-minded others. Consequently, the process of self-selection is crucial if the heterogeneity of opinions serves as a safeguard against undesirable effects of group processes (e.g., attitude polarization). Accordingly, online environments that invite more heterogeneous (vs. homogeneous) users should produce less biased content. We tested this hypothesis in a field study, comparing articles on the same 20 controversial topics from the online encyclopedias Conservapedia and RationalWiki with Wikipedia (and Britannica serving as a gold standard) and exploring the opinions of discussants in the three online encyclopedias. As expected, articles from Conservapedia and RationalWiki were significantly less balanced than articles from Wikipedia and Britannica. We replicated this finding in a lab study with 257 participants who self-selected to one of three online wikis (Vegan Love, Nutrition, Meat & Fish) and individually as well as collaboratively wrote an encyclopedia-like article about "Diets." As expected, Wikis with a specific focus (Vegan Love, Meat & Fish) predominantly attracted authors with a positive attitude toward this focus and, as a consequence, resulted in more biased content than in the Nutrition Wiki. Overall, our results suggest that crowds alone do not guarantee wisdom-self-selection is a crucial process that needs to be taken into account.

Tracking Home-Owners’ Sentiments: Subjective Indices and Convergent Validity

Abstract Surveys regularly ask home-owners to guess what their property might be worth in the current housing market. We develop suitable statistical techniques to construct hedonic and repeat-sales style house price indices from these owner-estimated values (OEVs). The resulting series are then linked to a large set of quality-adjusted residential property price indices estimated from transaction data allowing us to perform a variety of convergent validity tests. Based on results for 20 countries, several decades and different OEV elicitation techniques, we conclude that the “wisdom of the home-owner crowd” is sufficient to study objective house price dynamics. Yet, surveys fail to accurately measure house price levels.

Wisdom of Two Crowds: Misinformation Moderation on Reddit and How to Improve this Process---A Case Study of COVID-19

Past work has explored various ways for online platforms to leverage crowd wisdom for misinformation detection and moderation. Yet, platforms often relegate governance to their communities, and limited research has been done from the perspective of these communities and their moderators. How is misinformation currently moderated in online communities that are heavily self-governed? What role does the crowd play in this process, and how can this process be improved? In this study, we answer these questions through semi-structured interviews with Reddit moderators. We focus on a case study of COVID-19 misinformation. First, our analysis identifies a general moderation workflow model encompassing various processes participants use for handling COVID-19 misinformation. Further, we show that the moderation workflow revolves around three elements: content facticity, user intent, and perceived harm. Next, our interviews reveal that Reddit moderators rely on two types of crowd wisdom for misinformation detection. Almost all participants are heavily reliant on reports from crowds of ordinary users to identify potential misinformation. A second crowd--participants' own moderation teams and expert moderators of other communities--provide support when participants encounter difficult, ambiguous cases. Finally, we use design probes to better understand how different types of crowd signals---from ordinary users and moderators---readily available on Reddit can assist moderators with identifying misinformation. We observe that nearly half of all participants preferred these cues over labels from expert fact-checkers because these cues can help them discern user intent. Additionally, a quarter of the participants distrust professional fact-checkers, raising important concerns about misinformation moderation.

Searching for the Metaverse: Neuroscience of Physical and Digital Communities.

What distinguishes real-world communities from their online counterparts? Social and cognitive neuroscience research on social networks and collective intentionality will be used in the article to answer this question. Physical communities are born in places. And places engage "we-mode" neurobiological and cognitive processes as behavioral synchrony, shared attention, deliberate attunement, interbrain synchronization, and so on, which create coherent social networks of very different individuals who are supported by a "wisdom of crowd." Digital technologies remove physical boundaries, giving people more freedom to choose their activities and groups. At the same time, however, the lack of physical co-presence of community members significantly reduces their possibility of activating "we-mode" cognitive processes and social motivation. Because of this, unlike physical communities that allow interaction between people from varied origins and stories, digital communities are always made up of people who have the same interests and knowledge (communities of practice). This new situation disrupts the "wisdom of crowd," making the community more radical and less accurate (polarization effect), allowing influential users to wield disproportionate influence over the group's beliefs, and producing inequalities in the distribution of social capital. However, a new emergent technology-the Metaverse-has the potential to reverse this trend. Several studies have revealed that virtual and augmented reality-the major technologies underlying the Metaverse-can engage the same neurobiological and cognitive "we-mode" processes as real-world environments. If the many flaws in this technology are fixed, it might encourage people to engage in more meaningful and constructive interactions in online communities.

Wisdom of the Crowd: insights gained from comparing predicted and observed effects of blood pressure lowering strategies

In a first of its kind assessment in cardiovascular research, we assessed whether pooled cardiovascular expertise could accurately predict efficacy and tolerability for both a novel and an established treatment option. A survey was administered prior to the publication of the QUARTET (A Quadruple UltrA-low-dose tReatment for hypErTension) trial. QUARTET was a multicentre, double-blind, parallel-group, trial that randomised participants to initial treatment with either monotherapy or an ultra-low dose quadruple single pill combination for 12 weeks. Survey participants were asked to predict blood pressure (BP) at 12 weeks and 52 weeks for both groups.

Achieving Ecological Reflexivity: The Limits of Deliberation and the Alternative of Free-Market-Environmentalism

Environmental problems are often highly complex and demand a great amount of knowledge of the people tasked to solve them. Therefore, a dynamic polit-economic institutional framework is necessary in which people can adapt and learn from changing environmental and social circumstances and in light of their own performance. The environmentalist literature refers to this knowledge producing and self-correcting capacity as ecological reflexivity. Large parts of the literature agree that deliberative democracy is the right institutional arrangement to achieve ecological reflexivity. Our paper sheds doubt on this consensus. While we agree with the critique of centralized, technocratic planning within the literature on deliberative democracy and agree that ecologically reflexive institutions must take advantage of the environmental ‘wisdom of the crowd’, we doubt that deliberative democracy is the right institutional arrangement to achieve this. Ecological deliberation fails to address its own epistemic shortcomings in using crowd wisdom: Rational ignorance, rational irrationality and radical ignorance weaken the performance of deliberative institutions as an alternative and reflexive form of ecological governance. Instead, we propose an institutional order based on market-based approaches as the best alternative for using the environmental wisdom of the crowd.

Supplemental Material for Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-Making

Supplemental Material for Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-Making

Quantifying the complexity and similarity of chess openings using online chess community data

Chess is a centuries-old game that continues to be widely played worldwide. Opening Theory is one of the pillars of chess and requires years of study to be mastered. In this paper, we use the games played in an online chess platform to exploit the “wisdom of the crowd” and answer questions traditionally tackled only by chess experts. We first define a relatedness network of chess openings that quantifies how similar two openings are to play. Using this network, we identify communities of nodes corresponding to the most common opening choices and their mutual relationships. Furthermore, we demonstrate how the relatedness network can be used to forecast future openings players will start to play, with back-tested predictions outperforming a random predictor. We then apply the Economic Fitness and Complexity algorithm to measure the difficulty of openings and players’ skill levels. Our study not only provides a new perspective on chess analysis but also opens the possibility of suggesting personalized opening recommendations using complex network theory.

Optimising collective accuracy among rational individuals in sequential decision-making with competition

Theoretical results underpinning the wisdom of the crowd, such as the Condorcet Jury Theorem, point to substantial accuracy gains through aggregation of decisions or opinions, but the foundations of this theorem are routinely undermined in circumstances where individuals are able to adapt their own choices based after observing what other agents have chosen. In sequential decision-making, rational agents use the choices of others as a source of information about the correct decision, creating powerful correlations between different agents’ choices that violate the assumptions of independence on which the Condorcet Jury Theorem depends. In this paper, I show how such correlations emerge when agents are rewarded solely based on their individual accuracy, and the impact of this on collective accuracy. I then demonstrate how a simple competitive reward scheme, where agents’ rewards are greater if they correctly choose options that few have already chosen, can induce rational agents to make independent choices, returning the group to optimal levels of collective accuracy. I further show that this reward scheme is robust, offering improvements to collective accuracy across wide range of competition strengths, suggesting that such schemes could be effectively implemented in real-world contexts to improve collective wisdom.

The wisdom of crowds for improved disaster resilience: a near-real-time analysis of crowdsourced social media data on the 2021 flood in Germany

Transformative disaster resilience in times of climate change underscores the importance of reflexive governance, facilitation of socio-technical advancement, co-creation of knowledge, and innovative and bottom-up approaches. However, implementing these capacity-building processes by relying on census-based datasets and nomothetic (or top-down) approaches remains challenging for many jurisdictions. Web 2.0 knowledge sharing via online social networks, whereas, provides a unique opportunity and valuable data sources to complement existing approaches, understand dynamics within large communities of individuals, and incorporate collective intelligence into disaster resilience studies. Using Twitter data (passive crowdsourcing) and an online survey, this study draws on the wisdom of crowds and public judgment in near-real-time disaster phases when the flood disaster hit Germany in July 2021. Latent Dirichlet Allocation, an unsupervised machine learning technique for Topic Modeling, was applied to the corpora of two data sources to identify topics associated with different disaster phases. In addition to semantic (textual) analysis, spatiotemporal patterns of online disaster communication were analyzed to determine the contribution patterns associated with the affected areas. Finally, the extracted topics discussed online were compiled into five themes related to disaster resilience capacities (preventive, anticipative, absorptive, adaptive, and transformative). The near-real-time collective sensing approach reflected optimized diversity and a spectrum of people’s experiences and knowledge regarding flooding disasters and highlighted communities’ sociocultural characteristics. This bottom-up approach could be an innovative alternative to traditional participatory techniques of organizing meetings and workshops for situational analysis and timely unfolding of such events at a fraction of the cost to inform disaster resilience initiatives.

Identifying dynamical persistent biomarker structures for rare events using modern integrative machine learning approach.

The evolution of omics and computational competency has accelerated discoveries of the underlying biological processes in an unprecedented way. High throughput methodologies, such as flow cytometry, can reveal deeper insights into cell processes, thereby allowing opportunities for scientific discoveries related to health and diseases. However, working with cytometry data often imposes complex computational challenges due to high-dimensionality, large size, and nonlinearity of the data structure. In addition, cytometry data frequently exhibit diverse patterns across biomarkers and suffer from substantial class imbalances which can further complicate the problem. The existing methods of cytometry data analysis either predict cell population or perform feature selection. Through this study, we propose a "wisdom of the crowd" approach to simultaneously predict rare cell populations and perform feature selection by integrating a pool of modern machine learning (ML) algorithms. Given that our approach integrates superior performing ML models across different normalization techniques based on entropy and rank, our method can detect diverse patterns existing across the model features. Furthermore, the method identifies a dynamic biomarker structure that divides the features into persistently selected, unselected, and fluctuating assemblies indicating the role of each biomarker in rare cell prediction, which can subsequently aid in studies of disease progression.

Practical Applications of Questioning the Wisdom of Crowds to Design Portfolio Diversification Strategies

In <b><i>Questioning the Wisdom of Crowds to Design Portfolio Diversification Strategies</i></b>, from the Winter 2023 issue of <b><i>The Journal of Alternative Investments</i></b>, <b>Vadim Zlotnikov</b>, <b>Igor Halperin</b>, <b>Lisa Huang</b>, <b>Cathy Pena</b> (all of <b>Fidelity</b>), and <b>Mikhail Stukalo</b> (of <b>Gainy</b>, a fintech startup), develop a diversification strategy based on measures of crowdedness. They design a portfolio with the positive characteristics of uncrowded stocks that also harvests the skill of active managers. They create a measure of crowdedness using holdings data on active managers. Stocks can become crowded based on conviction, reflecting manager skill. But they also can become crowded due to a consensus view, which leads to problematic characteristics; crowded stocks can exhibit negatively skewed returns with high forward-looking volatility. The authors also construct a long–short portfolio based on two different measures of crowdedness. The portfolio is beta-neutral and can be used to improve the Sharpe ratio of a 60/40 portfolio by dampening volatility. Maintaining smart money investors’ insights, the equity component does not sacrifice returns, as other defensive strategies do.

Can individual subjective confidence in training questions predict group performance in test questions?

When people have to solve many tasks, they can aggregate diverse individuals' judgments using the majority rule, which often improves the accuracy of judgments (wisdom of crowds). When aggregating judgments, individuals' subjective confidence is a useful cue for deciding which judgments to accept. However, can confidence in one task set predict performance not only in the same task set, but also in another? We examined this issue through computer simulations using behavioral data obtained from binary-choice experimental tasks. In our simulations, we developed a "training-test" approach: We split the questions used in the behavioral experiments into "training questions" (as questions to identify individuals' confidence levels) and "test questions" (as questions to be solved), similar to the cross-validation method in machine learning. We found that (i) through analyses of behavioral data, confidence in a certain question could predict accuracy in the same question, but not always well in another question. (ii) Through a computer simulation for the accordance of two individuals' judgments, individuals with high confidence in one training question tended to make less diverse judgments in other test questions. (iii) Through a computer simulation of group judgments, the groups constructed from individuals with high confidence in the training question(s) generally performed well; however, their performance sometimes largely decreased in the test questions especially when only one training question was available. These results suggest that when situations are highly uncertain, an effective strategy is to aggregate various individuals regardless of confidence levels in the training questions to avoid decreasing the group accuracy in test questions. We believe that our simulations, which follow a "training-test" approach, provide practical implications in terms of retaining groups' ability to solve many tasks.

Hidden population size estimation and diagnostics using two respondent-driven samples with applications in Armenia.

Estimating the size of hidden populations is essential to understand the magnitude of social and healthcare needs, risk behaviors, and disease burden. However, due to the hidden nature of these populations, they are difficult to survey, and there are no gold standard size estimation methods. Many different methods and variations exist, and diagnostic tools are needed to help researchers assess method-specific assumptions as well as compare between methods. Further, because many necessary mathematical assumptions are unrealistic for real survey implementation, assessment of how robust methods are to deviations from the stated assumptions is essential. We describe diagnostics and assess the performance of a new population size estimation method, capture-recapture with successive sampling population size estimation (CR-SS-PSE), which we apply to data from 3 years of studies from three cities and three hidden populations in Armenia. CR-SS-PSE relies on data from two sequential respondent-driven sampling surveys and extends the successive sampling population size estimation (SS-PSE) framework by using the number of individuals in the overlap between the two surveys and a model for the successive sampling process to estimate population size. We demonstrate that CR-SS-PSE is more robust to violations of successive sampling assumptions than SS-PSE. Further, we compare the CR-SS-PSE estimates to population size estimations using other common methods, including unique object and service multipliers, wisdom of the crowd, and two-source capture-recapture to illustrate volatility across estimationmethods.

Computing word meanings by aggregating individualized distributional models: Wisdom of the crowds in lexical semantic memory

Linguistic experience varies across individuals and is impacted by both demography and personal preferences, leading to differences in word meanings across languages (Thompson et al., 2020) and people (Johns, 2022). An active area of study in the cognitive sciences that examines the impact of varied knowledge across individuals is the wisdom of the crowd effect, where it is found that the aggregate judgement of a group of individuals is often better than the judgement of the best individual in the group (Surowiecki, 2004). The goal of this article was to determine if there is a wisdom of the crowd effect in lexical semantic memory, such that the aggregated word similarity values from many individual language users exceeds the fit of the best fitting individual. This was accomplished by training 500 different distributional models from 500 high-level commenters on the internet forum Reddit. By deriving aggregated word similarity values from these individuals, a strong wisdom of the crowd effect was found where the aggregated similarity values far exceeded the performance of the best fitting individual for each dataset tested. Additionally, it was found that even aggregating only a small number of users provided a large increase in fit relative to the individual corpora, but with the best fitting measure including word similarity values from all possible users. The results of this article provide an avenue for future distributional model development by demonstrating that the best pathway towards better distributional models may lie in the aggregation of multiple representations attained from individual users of a language.

On an effective and efficient method for exploiting the wisdom of the inner crowd

Researchers have shown that even an individual can produce the wisdom of the crowds, called “the wisdom of the inner crowd.” However, the previous methods leave room for improvements in terms of efficacy and response time. This paper proposes a more efficient method, which required a short time, based on findings from cognitive and social psychology. The procedure is to ask participants to give two answers to the same question: first, their own estimate and, second, their estimate of public opinion. Experiments using this method showed that the averages of the two estimates were more accurate than the participants’ first estimates. That is, the wisdom of the inner crowd elicited. In addition, we found that the method could be superior to other methods in terms of efficacy and convenience. Moreover, we identified the conditions where our method worked better. We further clarify the availability and limitations of using the wisdom of the inner crowd. Overall, this paper proposes an effective and short-time method for harvesting the wisdom of the inner crowd.

Wisdom of Crowds in Quantum Machine Learning

A collective outcome or decision from a crowd often prevails over that of a single expert. Here we study this phenomenon through the lens of quantum machine learning. We compare the performance of an expert, a highly trained quantum network, to a crowd, several poorly trained ones, for quantum information processing tasks such as quantum tomography and entanglement recognition. We also show quantum-enhanced performance from a temporal crowd by using time multiplexing on a single poorly trained quantum network. Given the same resources for training, we show that the crowd outperforms the expert by a definitive margin.

Religious Belief and the Wisdom of Crowds

In their simplest form, consensus gentium arguments for theism argue that theism is true on the basis that everyone believes that theism is true. While such arguments may have been popular in history, they have all but fallen from grace in the philosophy of religion. In this short paper, we reconsider the neglected topic of consensus gentium arguments, paying particular attention to the value of such arguments when deployed in the defence of theistic belief. We argue that while consensus gentium arguments are unlikely to offer anything close to overwhelming support for theism, their probative value is nevertheless underappreciated, and that they have been unfairly maligned as a consequence.

Collective intelligence improves probabilistic diagnostic assessments.

Collective intelligence, the "wisdom of the crowd," seeks to improve the quality of judgments by aggregating multiple individual inputs. Here, we evaluate the success of collective intelligence strategies applied to probabilistic diagnostic judgments. We compared the performance of individual and collective intelligence judgments on two series of clinical cases requiring probabilistic diagnostic assessments, or "forecasts". We assessed the quality of forecasts using Brier scores, which compare forecasts to observed outcomes. On both sets of cases, the collective intelligence answers outperformed nearly every individual forecaster or team. The improved performance by collective intelligence was mediated by both improved resolution and calibration of probabilistic assessments. In a secondary analysis looking at the effect of varying number of individual inputs in collective intelligence answers from two different data sources, nearly identical curves were found in the two data sets showing 11-12% improvement when averaging two independent inputs, 15% improvement averaging four independent inputs, and small incremental improvements with further increases in number of individual inputs. Our results suggest that the application of collective intelligence strategies to probabilistic diagnostic forecasts is a promising approach to improve diagnostic accuracy and reduce diagnostic error.