Reply

Abstract

We highly appreciate the comments by Zheng et al. on the use of individual participant data (IPD) in diagnostic studies. We agree that meta-analysis of using IPD from multiple clinical studies enables detailed investigation of diagnostic studies. We also found that increasing numbers of IPD meta-analyses (IPDMA) from observational data are being conducted to enhance the statistical power and detail of epidemiological studies,1 and this approach is being increasingly applied in many research studies.2 However, there is some skepticism as to whether IPDMA is really the gold standard for diagnostic studies, considering the fact that it involves a number of challenges. On this point, we have our own view. First, in order to get a better comparison with the previous study conducted by Shaheen and Myers in 2007,3 we used aggregate data (AD) instead of IPD to perform this updated meta-analysis. Second, extra cost in effort, time, and complexity is required to obtain and manage raw data in IPDMA. Commendable examples of IPDMA are those conducted by the Emerging Risk Factors Collaboration (ERFC),4 who have remarkably collected IPD from 116 prospective studies and more than 1.2 million participants. Although meta-analysis methods using AD are well established and fairly routine, methods for IPDMA are more complex but less well known. Currently, although the number of published articles related to “meta-analysis” has risen dramatically from hundreds of articles per year in the early 1990s to thousands of articles every year since 2003, the number of articles of IPDMA only accounts for a negligible proportion of less than 1.71% (Fig. 1). Number of articles related “meta-analyses” published in PubMed from January 1991 to December 25, 2010 compared with number of distinct, applied IPDMA published from January 1991 to December 25, 2010, as identified by a systematic review of PubMed. Finally, should one embark on an IPDMA when few studies provide their IPD, making it difficult to estimate random effects? Likewise, is an IPDMA reliable when only a proportion of existing studies provide IPD? Unfortunately this is the current situation that regrettably leads to what Riley5 referred to as availability bias—a human cognitive bias that tends to overestimate probabilities of events associated with memorable or vivid occurrences, where studies that provide IPD are a kind of biased subset of all existing studies. In conclusion, we think that methods for IPDMA are prone to be affected by bias with inadequate generalizability despite their widely recognized strength. The appropriate strategy at this moment is probably to use both approaches in a complementary fashion, in which an AD meta-analysis is conducted in the first step rather than an IPDMA. As Riley pointed out, IPD is not the be-all and end-all for meta-analysis just yet.5 Yong-Ning Xin M.D.* , Zhong-Hua Lin M.D. , An-Jin Chen M.D.* , Shi-Ying Xuan M.D.* , * College of Medicine and Pharmaceutics Ocean University of China Qingdao, Shandong Province, China, Qingdao Municipal Hospital, Qingdao Shandong Province, China, School of Medicine, Qingdao University, Qingdao Shandong Province, China.