file-drawer-problem2The File Drawer Problem is a publication bias in which results that are significant are published more that results that are not statistically significant. That is, non-significant findings end up in the file drawer.

The file drawer problem can come from many sources. One potential source could be a confirmation bias of the researcher. The researcher may only report significant findings that confirm his or her research hypotheses and leave nonsignificant results out of the published report. Another case might be a researcher who does not find significant results and thus, not publishing anything from that study.

Another potential reason for this bias is that journal editors may not want to publish results that are not significant. Journal editors, to some extent, want to make their journal more important and read by more people. Nonsignificant findings rarely make the headlines and don’t really add to the popularity of the journal. To take an extreme example, I cannot come up with one example of an article in the journal Science that focuses on a nonsignificant finding.

Why is the File Drawer Problem a problem? If positive results are the only results that are published, we provide only part of the picture. That is, the positive result featured in one article may have been a Type I error. It could be balanced by ten other studies which did not find a significant effect, but these ten studies were not published and thus can not be used as a comparison to this one significant effect.

Furthermore, if one wanted to compute an average effect size from the existing literature, this effect would be positively influenced by the lack of published studies without a positive effect. For example, if we wanted to compute the effect of hypnosis on smoking cessation, we would look at the existing literature. However, studies that did not find hypnosis decreasing smoking use may not be in the published literature. Thus, the effect of hypnosis on smoking cessation may be overestimated because there are few published studies which show no effect of hypnosis on smoking cessation.

A great article on this topic comes from Robert Rosenthal:
Rosenthal, R., 1979, The “File Drawer Problem” and Tolerance for Null Results, Psychological Bulletin, 86, p. 638-641.