Objectives: To provide a further test of possible hypothesis conformity or confirmation bias in science based on differential citations of two sets of articles published between 1979 and 2011 that concerned same-sex parenting, as part of a larger issue of similar biases in all sciences and medical research in general. Methods: Two sets of articles used here formed nearly perfect natural experiments because the articles were from the same university team of scholars, published about the same time, often concerned the same samples, with some being published in the same scholarly journal. Differences in citation rates might typically be related to article quality, but with these two sets of articles, research value was usually similar. Based on Google Scholar citations, citation rates were compared using binomial tests, including normal approximation z scores, and one-sample chi-square tests. Results: For both sets of articles, those that presented findings more favorable to same-sex parenting were cited far more often than those with significant but less favorable results. Some authors disregarded their own significant research findings in later articles. Conclusions: Confirmation bias or hypothesis conformity, considered to be a questionable research practice (QRP), have existed in at least one area of the sciences for as long as four decades. Even when noticed and reported, such QRPs have continued, almost certainly leading to errors in reviews of the literature in such areas, errors – if accepted - that could lead to incorrect policy decisions. However, scholars who dare to challenge incorrect scholarly “consensus”, report fraudulent research, or who dare to dispute widely accepted findings may have their professional careers destroyed. Without better policing of QRPs, the value of social science research for society may decline even if it continues to benefit those who engage in QRPs. However, our observations are only a narrow slice of a much larger problem of various kinds of bias in science and medical or public health research in general.
Read full abstract