Wisdom Of Crowds Research Articles

Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy.

Human forecasting accuracy improves through the "wisdom of the crowd" effect, in which aggregated predictions tend to outperform individual ones. Past research suggests that individual large language models (LLMs) tend to underperform compared to human crowd aggregates. We simulate a wisdom of the crowd effect with LLMs. Specifically, we use an ensemble of 12 LLMs to make probabilistic predictions about 31 binary questions, comparing them with those made by 925 human forecasters in a 3-month tournament. We show that the LLM crowd outperforms a no-information benchmark and is statistically indistinguishable from the human crowd. We also observe human-like biases, such as the acquiescence bias. In another study, we find that LLM predictions (of GPT-4 and Claude 2) improve when exposed to the median human prediction, increasing accuracy by 17 to 28%. However, simply averaging human and machine forecasts yields more accurate results. Our findings suggest that LLM predictions can rival the human crowd's forecasting accuracy through simple aggregation.

Identifying Wisdom (of the Crowd): A Regression Approach

Identifying Wisdom (of the Crowd): A Regression Approach

Predicting user behavior on s-commerce platforms: a novel model

PurposeThis study investigates the influence of the wisdom of the crowd (WSC), trust and perceived value on consumer purchase intentions within social commerce (s-commerce) platforms. By constructing a novel theoretical model, the research aims to delineate the interrelationships among these variables, addressing an emerging area in social interactions and information technology.Design/methodology/approachTo empirically test and validate the proposed model, the study collected data from 442 Facebook users familiar with online shopping. The analysis employed Structural Equation Modeling – Partial Least Squares (SEM-PLS) to test the hypotheses and examine the relationships between WSC, trust, perceived value and purchase intentions.FindingsThe findings confirm that WSC significantly influences trust, perceived value and the intention to purchase on s-commerce platforms. Both perceived value and trust are substantial determinants of purchase intentions. Notably, the PLS analysis revealed that WSC accounts for 36.8% of the variance in trust and 33.1% of the variance in perceived value related to purchasing decisions on s-commerce platforms.Originality/valueThis research contributes uniquely to the ongoing discourse on s-commerce adoption by integrating WSC as a pivotal factor in understanding perceived value and consumer adoption intentions. It provides a fresh perspective on how collective intelligence affects consumer behavior in digital marketplaces, enriching the theoretical and practical understanding of s-commerce dynamics.

The price evolution in financial markets under influence of published opinions

Opinions by media experts and pundits in both traditional media outlets and online venues influence investors. While a published opinion could be more accurate than that of a typical investor, its influence displaces many independent views and has the potential to introduce a shared error. My probabilistic trading model characterizes such influence and demonstrates how the “wisdom of the crowds” effect leads convergence of prices to the fundamental value. I use two corporate events to test the model, and find evidence consistent with the theoretical predictions.

Collaboration, crowdsourcing, and misinformation.

One of humanity's greatest strengths lies in our ability to collaborate to achieve more than we can alone. Just as collaboration can be an important strength, humankind's inability to detect deception is one of our greatest weaknesses. Recently, our struggles with deception detection have been the subject of scholarly and public attention with the rise and spread of misinformation online, which threatens public health and civic society. Fortunately, prior work indicates that going beyond the individual can ameliorate weaknesses in deception detection by promoting active discussion or by harnessing the "wisdom of crowds." Can group collaboration similarly enhance our ability to recognize online misinformation? We conducted a lab experiment where participants assessed the veracity of credible news and misinformation on social media either as an actively collaborating group or while working alone. Our results suggest that collaborative groups were more accurate than individuals at detecting false posts, but not more accurate than a majority-based simulated group, suggesting that "wisdom of crowds" is the more efficient method for identifying misinformation. Our findings reorient research and policy from focusing on the individual to approaches that rely on crowdsourcing or potentially on collaboration in addressing the problem of misinformation.

How large language models can reshape collective intelligence.

Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can achieve outcomes that exceed the capabilities of individuals-even experts-resulting in improved accuracy and novel capabilities. Often, collective intelligence is supported by information technology, such as online prediction markets that elicit the 'wisdom of crowds', online forums that structure collective deliberation or digital platforms that crowdsource knowledge from the public. Large language models, however, are transforming how information is aggregated, accessed and transmitted online. Here we focus on the unique opportunities and challenges this transformation poses for collective intelligence. We bring together interdisciplinary perspectives from industry and academia to identify potential benefits, risks, policy-relevant considerations and open research questions, culminating in a call for a closer examination of how large language models affect humans' ability to collectively tackle complex problems.

Evaluating the wisdom of scholar crowds from the perspective of knowledge diffusion

Evaluating the wisdom of scholar crowds from the perspective of knowledge diffusion

Using Cognitive Models to Improve the Wisdom of the Crowd

The wisdom of the crowd is the finding that aggregating the judgments of many people can lead to surprisingly accurate group judgments. Usually statistical methods are used to aggregate people’s judgments, but there are advantages to using cognitive models instead. Crowd judgments based on cognitive modeling can (a) identify experts and amplify their judgments, (b) provide a representational structure for aggregating complicated multidimensional judgments, (c) debias judgments that are affected by heuristic cognitive processes or competitive social situations, and (d) diversify the crowd by incorporating predictions about judgments that have not been observed. Demonstrations of these advantages are provided in case studies involving ranking, probability estimation, and categorization problems.

P244. Crowd wisdom among experts and reduction of fusion failure for spondylolisthesis

P244. Crowd wisdom among experts and reduction of fusion failure for spondylolisthesis

The wisdom of the crowd with partial rankings: A Bayesian approach implementing the Thurstone model in JAGS.

We develop a Bayesian method for aggregating partial ranking data using the Thurstone model. Our implementation is a JAGS graphical model that allows each individual to rank any subset of items, and provides an inference about the latent true ranking of the items and the relative expertise of each individual. We demonstrate the method by analyzing data from new experiments that collected partial ranking data. In one experiment, participants were assigned subsets of items to rank; in the other experiment, participants could choose how many and which items they ranked. We show that our method works effectively for both sorts of partial ranking in applications to US city populations and the chronology of US presidents. We discuss the potential of the method for studying the wisdom of the crowd and other research problems that require aggregating incomplete or partial rankings.

Wisdom of crowds or awkward squad? Social interaction and the information efficiency of the Chinese capital market

Existing studies on the impact of social media on the information efficiency of capital markets have opposing views. One often-overlooked aspect in these studies is the role of investor interaction, a core feature of social media. To address this gap, we employ social network analysis (SNA) to investigate how investor interactions on Eastmoney, China's largest online stock forum, influence capital market efficiency. Our findings indicate that online investor communication can effectively disseminate firm-specific information through social networks when investor interactions are at a moderate level. However, beyond a certain threshold, heightened interactions contribute to market noise, thereby distorting the stock price discovery process. Our mechanistic analysis reveals that increased investor interactions stimulate irrational emotions at both market and stock levels, ultimately impairing capital market efficiency. This study contributes to social network analysis and behavioral finance by illuminating the impact of online social interactions among retail investors on capital market information efficiency.

Individual investors and accounting information disclosure

This research aimed to understand the scientific productivity regarding accounting information within the environment of individual investors, focusing on the role of brokerage firms and social media in this informational context. The study conducted a bibliometric analysis that involved collecting international articles published over the last 33 years from the Scopus database. The methodology employed was bibliographic, documentary, descriptive, and quantitative. The final sample included 84 articles published between 1989 and 2021. It was observed that there has been an expansion in the discussion of the topic over the analyzed period, with contributions from researchers in 27 different countries. Co-authorship was primarily dominated by three researchers per article, with most authors contributing only one article in the field. The articles employ various approaches to understand the aspects and determinants affecting decision-making and the financial market. Points such as investor awareness, the role of brokerage firms and use of social media, new standards in financial reporting, wisdom of crowds, and investor attention and protection were discussed. Additionally, the research highlights the landscape, opportunities, and challenges in the field. This study becomes relevant for advancing academic knowledge and enhancing the efficiency and transparency of financial markets. It has the potential to trigger significant impacts on both accounting theory and the practical aspects of the financial industry.

The asynchronous DeGroot dynamics

AbstractWe analyze the asynchronous version of the DeGroot dynamics: In a connected graph with nodes, each node has an initial opinions in and an independent Poisson clock. When a clock at a node rings, the opinion at is replaced by the average opinion of its neighbors. It is well known that the opinions converge to a consensus. We show that the expected time to reach ‐consensus is poly in undirected graphs and in Eulerian digraphs, but for some digraphs of bounded degree it is exponential. Our main result is that in undirected graphs and Eulerian digraphs, if the degrees are uniformly bounded and the initial opinions are i.i.d., then for every fixed . We give sharp estimates for the variance of the limiting consensus opinion, which measures the ability to aggregate information (“wisdom of the crowd”). We also prove generalizations to non‐reversible Markov chains and infinite graphs. New results of independent interest on fragmentation processes and coupled random walks are crucial to our analysis.

Co-creating a global shared research agenda on violence against women in low- and middle-income countries

BackgroundDespite a large growth in evidence on violence against women (VAW) over the last 25 years, VAW persists, as do gaps in the field’s knowledge of how to prevent and respond to it. To ensure that research on VAW in low- and middle-income countries (LIMCs) is addressing the most significant gaps in knowledge, and to prioritise evidence needs to reduce VAW and better support victims/survivors, the Sexual Violence Research Initiative (SVRI) and Equality Institute (EQI) led a process of developing a global shared research agenda (GSRA) on VAW in LMICs.MethodsThe GSRA was developed through a six-stage adaptation of the Child Health and Nutrition Research Initiative (CHNRI) method, which draws on the principle of the ‘wisdom of the crowd’. These steps included: a review of the literature on VAW in LMICs and development of domains; the generation of research questions within four domains by an Advisory Group; the consolidation of research questions; scoring of research questions by a Global Expert Group and the Advisory Group according to three criteria (applicability, effectiveness and equity); consultation and validation of the findings with the Advisory Group; and wide dissemination of the findings.ResultsThe highest ranked research questions in the GSRA pertain to the domain of Intervention research, with some highly ranked questions also pertaining to the domain of Understanding VAW in its multiple forms. Questions under the other two domains, Improving existing interventions, and Methodological and measurement gaps, were not prioritised as highly by experts. There was strong consistency in top ranked research questions according to experts’ characteristics, albeit with some important differences according to experts’ gender, occupation and geographical location.ConclusionsThe GSRA findings suggest that currently the VAW field is shifting towards intervention research after several decades of building evidence on understanding VAW, including prevalence, drivers and impacts of violence. The findings also suggest a strong emphasis on under-served populations, and under-researched forms of VAW. Future priority setting exercises in LMICs that seek to decolonise knowledge should ensure that methodologies, and modalities of engagement, put diverse voices at the centre of engagement.Trial registration Not applicable

Promoting Erroneous Divergent Opinions Increases the Wisdom of Crowds

The aggregation of many lay judgments generates surprisingly accurate estimates. This phenomenon, called the “wisdom of crowds,” has been demonstrated in domains such as medical decision-making and financial forecasting. Previous research identified two factors driving this effect: the accuracy of individual assessments and the diversity of opinions. Most available strategies to enhance the wisdom of crowds have focused on improving individual accuracy while neglecting the potential of increasing opinion diversity. Here, we study a complementary approach to reduce collective error by promoting erroneous divergent opinions. This strategy proposes to anchor half of the crowd to a small value and the other half to a large value before eliciting and averaging all estimates. Consistent with our mathematical modeling, four experiments (N = 1,362 adults) demonstrated that this method is effective for estimation and forecasting tasks. Beyond the practical implications, these findings offer new theoretical insights into the epistemic value of collective decision-making.

SAILoR: Structure-Aware Inference of Logic Rules.

Boolean networks provide an effective mechanism for describing interactions and dynamics of gene regulatory networks (GRNs). Deriving accurate Boolean descriptions of GRNs is a challenging task. The number of experiments is usually much smaller than the number of genes. In addition, binarization leads to a loss of information and inconsistencies arise in binarized time-series data. The inference of Boolean networks from binarized time-series data alone often leads to complex and overfitted models. To obtain relevant Boolean models of gene regulatory networks, inference methods could incorporate data from multiple sources and prior knowledge in terms of general network structure and/or exact interactions. We propose the Boolean network inference method SAILoR (Structure-Aware Inference of Logic Rules). SAILoR incorporates time-series gene expression data in combination with provided reference networks to infer accurate Boolean models. SAILoR automatically extracts topological properties from reference networks. These can describe a more general structure of the GRN or can be more precise and describe specific interactions. SAILoR infers a Boolean network by learning from both continuous and binarized time-series data. It navigates between two main objectives, topological similarity to reference networks and correspondence with gene expression data. By incorporating the NSGA-II multi-objective genetic algorithm, SAILoR relies on the wisdom of crowds. Our results indicate that SAILoR can infer accurate and biologically relevant Boolean descriptions of GRNs from both a static and a dynamic perspective. We show that SAILoR improves the static accuracy of the inferred network compared to the network inference method dynGENIE3. Furthermore, we compared the performance of SAILoR with other Boolean network inference approaches including Best-Fit, REVEAL, MIBNI, GABNI, ATEN, and LogBTF. We have shown that by incorporating prior knowledge about the overall network structure, SAILoR can improve the structural correctness of the inferred Boolean networks while maintaining dynamic accuracy. To demonstrate the applicability of SAILoR, we inferred context-specific Boolean subnetworks of female Drosophila melanogaster before and after mating.

Could controversies in the arena of fatty liver disease be a potential gate for the democratization of science

The term “democratization of science” describes the process of more evenly allocating epistemic authority between scientists, members of dominant civilizations, and the academic community at large, or members of less dominant societies. This means that it includes initiatives aimed at democratizing the decision-making process by acknowledging the presence of diverse types of “wisdom of crowd” and so reducing the barriers between the various stakeholders. Our purpose is to separate influence from involvement that contributes to the breakdown of conventional closed-circuit authority structures and to prevent future abuses of power by academic institutions, scientific societies, and even individual opinion leaders. A conceptual framework for comprehending the idea of the democratization of science is presented in this perspective piece. Our considerations are pertinent to the politics of widespread academic engagement in scientific decision-making, even though they were spurred by the discussion surrounding the definitions of fatty liver disease.

Using the Wisdom of the Crowd to Improve the Condition Assessment of Residential Real Estate

The “wisdom of the crowd” refers to the phenomenon where by the average estimate of a group is superior to the estimate of any individual within the group, even if that individual is an expert and the others are not. When applied to real estate, it demonstrates that a group of non–real estate professionals can arrive at a more accurate physical property categorization than a real estate expert. Importantly, this more accurate assessment of the property’s physical condition can be achieved by merely viewing photos of the property. This appraisal trend makes the finding even more relevant to practical implementation.

Large Language Models and the Wisdom of Small Crowds.

Recent advances in Large Language Models (LLMs) have raised the question of replacing human subjects with LLM-generated data. While some believe that LLMs capture the "wisdom of the crowd"-due to their vast training data-empirical evidence for this hypothesis remains scarce. We present a novel methodological framework to test this: the "number needed to beat" (NNB), which measures how many humans are needed for a sample's quality to rival the quality achieved by GPT-4, a state-of-the-art LLM. In a series of pre-registered experiments, we collect novel human data and demonstrate the utility of this method for four psycholinguistic datasets for English. We find that NNB > 1 for each dataset, but also that NNB varies across tasks (and in some cases is quite small, e.g., 2). We also introduce two "centaur" methods for combining LLM and human data, which outperform both stand-alone LLMs and human samples. Finally, we analyze the trade-offs in data cost and quality for each approach. While clear limitations remain, we suggest that this framework could guide decision-making about whether and how to integrate LLM-generated data into the research pipeline.

Boosting wisdom of the crowd for medical image annotation using training performance and task features.

A crucial bottleneck in medical artificial intelligence (AI) is high-quality labeled medical datasets. In this paper, we test a large variety of wisdom of the crowd algorithms to label medical images that were initially classified by individuals recruited through an app-based platform. Individuals classified skin lesions from the International Skin Lesion Challenge 2018 into 7 different categories. There was a large dispersion in the geographical location, experience, training, and performance of the recruited individuals. We tested several wisdom of the crowd algorithms of varying complexity from a simple unweighted average to more complex Bayesian models that account for individual patterns of errors. Using a switchboard analysis, we observe that the best-performing algorithms rely on selecting top performers, weighting decisions by training accuracy, and take into account the task environment. These algorithms far exceed expert performance. We conclude by discussing the implications of these approaches for the development of medical AI.