Understanding and Scaffolding Constructive Collaboration

Abstract

Is There a Decision Bias For Information From Internally Consistent Sources? Shenghua Luan (herome@ufl.edu) Department of Psychology, University of Florida P.O. Box 112250, Gainesville, FL, 32611 USA Robert D. Sorkin (sorkin@ufl.edu) Departments of Psychology and Industrial & Systems Engineering, University of Florida P.O. Box 112250, Gainesville, FL, 32611 USA Jesse Itzkowitz (itz@ufl.edu) Department of Psychology, University of Florida P.O. Box 112250, Gainesville, FL, 32611 USA A person faced with a decision often obtains opinions from other sources. These information sources may be composed of several individual sub-sources. The sub-sources may be partially correlated and may differ in their level of expertise. This study asked how decision makers weigh the estimates received from different sources when those sources varied in their internal consistency and individual expertise. We paid people to perform a graphical decision task while aided by simulated information sources. Each participant observed a graphical display of a signal-plus-noise or noise-alone event and made an estimate of signal likelihood. The participant then was shown likelihood estimates generated from two simulated information sources. The participant then made a yes-no decision about the occurrence of signal on that trial. A monetary payoff was contingent on the accuracy of this yes-no decision. The estimates from each information source consisted of likelihood ratings generated by four sub-sources. Thus, on each trial the participant was shown 8 likelihood estimates to aid in her decision, four estimates from information source “A” and four estimates from information source “B”. In order to estimate the decision weight that the participant gave to each source, we constructed a multiple linear regression model that related the participant’s initial estimate and each source’s average estimate, to the participant’s final decision. In different conditions of the experiment, we manipulated the overall information value of a source and the level of expertise and pair-wise correlation among a source’s sub- sources. Source expertise was manipulated using the following formula adapted from Sorkin and Dai (1994): d ' source  m σ 2 m ( µ d ' ) 2  =  d ' +  1 − ρ 1 + ρ ( m − 1)  where d′ source is the detection index (aggregate expertise) of the source, m is the number of sub-sources in the group, ρ is the correlation among the sub-source estimates, σ 2d’ is the variance of the sub-sources' expertise and µ d’ is the average detection ability of those sub-sources. For example, one condition tested which of two equal- information sources (i.e., two sources that have the same overall detection ability, d’) would be given the higher weight: the one whose four sub-sources had partial pair- wise correlations and high sub-source d’s, or the one whose four sub-sources had zero pair-wise correlation and lower sub-source d’s. The results indicated that participants gave a significantly higher weight to the information source that had the higher consistency and higher component expertise, even though the information available from the two sources was identical. This bias was mainly evident on trials when the aggregate opinions of the two sources disagreed. Other conditions compared performance with sources that had different overall information values as well as different levels of sub-source expertise. In these conditions, the participants tended to overweigh the information from the sources having the higher information value and higher level of sub-source expertise. These biases reflect the participants’ sensitivity to across- and within-trial differences in the accuracy and internal consistency of information sources. Acknowledgments These experiments were conducted as part of the requirements for the M.S. degree for the first author. This research was partially supported by a grant from the Air Force Office of Scientific Research to the second author. Reference Sorkin, R. & Dai, H. (1994). Signal detection analysis of the ideal group. Organizational Behavior and Human Decision Processes, 60, 1-13.