Decision aggregation with reliability propagation

Abstract

People often make decisions differently, even when faced with the same decision-making scenario and objectives, due to their varying abilities to access, process, and comprehend information relevant to the decisions at hand. To reconcile these differing perspectives and arrive at a unified decision, various approaches such as crowd-sourced systems have been developed to tap into the collective intelligence embodied in the opinions from a group of individuals. The diversity of opinions is both cure and curse for the effective use of crowd-sourced intelligence. To unify crowd-sourced intelligence for a well-informed decision, we propose an algorithmic approach for decision aggregation that accurately quantifies the reliability of information from multiple sources. The key idea behind this approach is to model the propagation of reliability in decisions based on an ensemble of relevance graphs, where the optimization of both the reliability propagation and the graph ensemble are mutually reinforced. The propagated reliability can be used to aggregate intelligence from multiple sources and facilitate decision-making by leveraging various types of inter-correlations of information sources and the subjects of the information. Meanwhile, the optimized graph ensemble can retain the relevance structures with respect to the crowd-sourced intelligence. We evaluate our approach with large-scale data sets, and the results show that, when aggregating analysts’ recommendations in stock markets, our approach not only outperforms alternative methods, but also provides interesting insights into the reliability of the information.