How many studies are in the file drawer? An estimate from the family/marital psychotherapy literature

Abstract

There has been much speculation but little data about the number of unpublished studies located in “file drawers.” If the number is high, and if those studies yield systematically different effects, then literature reviews that ignore them might yield biased results. As part of an ongoing meta-analysis of the effectiveness of marital/family therapies, the authors asked 519 randomly sampled members of five relevant organizations if they had file drawer studies. A total of 375 respondents yielded three such unpublished studies that would otherwise have qualified for inclusion in the meta-analysis. Resulting population estimates suggest that there may be almost as many family/marital psychotherapy studies in the file drawer as there are published studies and dissertations. However, because so few file drawer studies were actually obtained, very large confidence intervals surrounded the estimate of the magnitude of effect sizes. Hence the degree of bias resulting from the file drawer problem is still in doubt. Available evidence in this and other studies suggests that the conservative conclusion is to assume that population effect sizes are only 70–90% as large as those computed from published studies.