In a study conducted by Tversky and Kahneman, participants were asked about the probability of a witness correctly identifying the right color of a cab that was in an accident. The participants were given the following information:
A cab was involved in a hit and run accident at night.
There are two cab companies in the city, one with green cars and the other with blue cars. 85% of the cabs in the city are green and 15% are blue.
An eyewitness identified the cab as being blue. The court tested the reliability of the witness under the same circumstances that existed on the night of the accident and concluded that the witness correctly identified the colors 80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accident was blue?
Almost all participants provided a probability greater than 50%. Many gave answers of 80% or higher.
Based our decision on only the base rate of cabs in the city we would guess that there is an 85% chance that it was a green cab involved in the accident rather than a blue cab.
However, people relied on the testimony of the witness to a much greater degree than on the base rate information even with the information about the unreliability of the witness. This problem is known as the base rate fallacy. That is, people will often ignore the base rate when making decisions.
Knowing how good the witness is in identifying the different color cabs at night and the base rates of blue and green cars, we can use Bayes Theorem to calculate how likely it is that the car is in fact blue.
There is a 12% chance of the witness correctly identifying a blue cab (True Positive; the base rate of blue cabs, 15% multiplied with the probability of the witness correctly identifying the color of a cab, 80%) .
There is a 17% chance of the witness incorrectly identifying a green cab as blue (False Positive; the base rate of green cabs, 85% multiplied with the probability of the witness identifying the wrong color cab, 20%).
There is a 29% chance (17% added to 12%) that the witness will identify any cab as blue.
This results in a 41% chance (12% divided by 29%) that the cab identified as blue is actually blue (Positive Predictive Value).
See also this entry about base rates and diagnostic decisions for more information about Bayes Theorem.