Bayes's Theorem and Reliability: A Reply to Levin

Abstract

Psychological research appears to show that people are bad at inductive reasoning. In a well known experiment due to Kahneman and Tversky, I subjects are told that a cab was involved in an accident in a city in which 15% of the cabs are blue and 85% are green, and that a witness testified that the cab was blue. Kahneman and Tversky ask subjects to estimate the probability that the testimony is correct given that the witness can reliably identify blue and green cabs 80% of the time. The majority say the probability that the offending cab was blue, given the testimony, is around .8. Using Bayes's theorem, the experimenters calculate the probability to be .41. Levin is not the first to argue that this experiment does not show that people are bad at inductive reasoning. But he is the first to argue that the experiment fails because the Bayesian calculation involves a mistaken analysis of reliability. Further, he contends that given the correct analysis of reliability, the probability that the witness's testimony is correct is indeed .8. The court tests a witness's reliability by showing him a series of blue and green cabs in conditions similar to those in which the accident occurred. As Levin puts it, the test shows the witness is right 80% of the time about when a cab is a Blue and when it is a Green (63). According to the Bayesians the witness's reliability is (1) P(witness testifies blue/cab was blue) == .8. In evaluating the testimony, however, Bayesians must determine (2) P(cab was blue/witness testifies blue),which requires Bayes's theorem as well as the prior probability that the cab was blue (.15)2, the prior probability that the cab was green (.85), and the probability that the witness testified the cab was blue even though it was green (.2). By Bayes's theorem, (2) is