Influence of imperfect reference standard bias on the diagnostic performance of MRI in the detection of lymphomatous bone marrow involvement

Abstract

Sir — We read with interest the paper by Jiang et al., 1 Jiang X.X. Yan Z.X. Song Y.Y. et al. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. Clin Radiol. 2013; 68: e143-e153 Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar which was recently published in the Journal. Jiang et al. 1 Jiang X.X. Yan Z.X. Song Y.Y. et al. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. Clin Radiol. 2013; 68: e143-e153 Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar performed a meta-analysis of 13 studies on the diagnostic performance of magnetic resonance imaging (MRI) for the detection of bone marrow involvement in lymphoma, and reported MRI to have a pooled sensitivity of 82% [95% confidence interval (CI): 70–90%] and a pooled specificity of 79% (95% CI: 72–84%). When assessing the performance of a diagnostic test (both in original research studies and diagnostic meta-analyses), it is of crucial importance to use a reliable reference standard. 2 Obuchowski N.A. Special topics III: bias. Radiology. 2003; 229: 617-621 Crossref PubMed Scopus (35) Google Scholar In their Materials and Methods section, Jiang et al. 1 Jiang X.X. Yan Z.X. Song Y.Y. et al. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. Clin Radiol. 2013; 68: e143-e153 Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar do not describe which reference standard they used to calculate measures of diagnostic accuracy. From Table 1 in their article1 it appears that bone marrow biopsy (BMB) was used as reference standard in seven of 13 included studies. It should be realized that BMB only assesses a very small portion of the entire bone marrow and may, therefore, suffer from sampling errors (i.e., false negatives). For example, previous studies have shown that 10–60% patients with unilaterally proven bone marrow involvement have a negative contralateral BMB of the iliac crest, 3 Brunning R.D. Bloomfield C.D. McKenna R.W. et al. Bilateral trephine bone marrow biopsies in lymphoma and other neoplastic diseases. Ann Intern Med. 1975; 82: 365-366 Crossref PubMed Scopus (182) Google Scholar , 4 Coller B.S. Chabner B.A. Gralnick H.R. Frequencies and patterns of bone marrow involvement in non-Hodgkin lymphomas: observations on the value of bilateral biopsies. Am J Hematol. 1977; 3: 105-119 Crossref PubMed Scopus (101) Google Scholar , 5 Haddy T.B. Parker R.I. Magrath I.T. Bone marrow involvement in young patients with non-Hodgkin's lymphoma: the importance of multiple bone marrow samples for accurate staging. Med Pediatr Oncol. 1989; 17: 418-423 Crossref PubMed Scopus (64) Google Scholar , 6 Wang J. Weiss L.M. Chang K.L. et al. Diagnostic utility of bilateral bone marrow examination: significance of morphologic and ancillary technique study in malignancy. Cancer. 2002; 94: 1522-1531 Crossref PubMed Scopus (96) Google Scholar and that in 33% of paired ipsilateral BMBs of the iliac crest, only one specimen is positive for bone marrow infiltration. 5 Haddy T.B. Parker R.I. Magrath I.T. Bone marrow involvement in young patients with non-Hodgkin's lymphoma: the importance of multiple bone marrow samples for accurate staging. Med Pediatr Oncol. 1989; 17: 418-423 Crossref PubMed Scopus (64) Google Scholar For this reason, the pooled measures of diagnostic accuracy (particularly specificity) that were reported by Jiang et al. 1 Jiang X.X. Yan Z.X. Song Y.Y. et al. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. Clin Radiol. 2013; 68: e143-e153 Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar are obviously biased and not reliable. Of note, preliminary data from our ongoing prospective study in more than 100 patients show that the sensitivity of MRI is substantially lower than the pooled sensitivity that was reported by Jiang et al. 1 Jiang X.X. Yan Z.X. Song Y.Y. et al. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. Clin Radiol. 2013; 68: e143-e153 Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar Despite the relatively low sensitivity of MRI for the detection of bone marrow involvement in lymphoma, we do believe that MRI can play an important role in the evaluation of these patients; MRI of the entire body (i.e., whole-body MRI) is currently a clinically feasible technique for both nodal and extranodal staging of lymphoma 7 Kwee T.C. van Ufford H.M. Beek F.J. et al. Whole-body MRI, including diffusion-weighted imaging, for the initial staging of malignant lymphoma: comparison to computed tomography. Invest Radiol. 2009; 44: 683-690 Crossref PubMed Scopus (100) Google Scholar , 8 Punwani S. Taylor S.A. Bainbridge A. et al. Pediatric and adolescent lymphoma: comparison of whole-body STIR half-Fourier RARE MR imaging with an enhanced PET/CT reference for initial staging. Radiology. 2010; 255: 182-190 Crossref PubMed Scopus (109) Google Scholar and can complement BMB (i.e., improve the sensitivity of BMB alone), either as blind BMB combined with MRI or as MRI in combination with MRI-guided BMB. A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphomaClinical RadiologyVol. 68Issue 3PreviewTo explore the role of magnetic resonance imaging (MRI) in detecting bone marrow infiltration of malignant lymphoma through a systematic review and meta-analysis. Full-Text PDF Re: A pooled analysis of MRI in the detection of bone marrow infiltration in patients with malignant lymphoma. A replyClinical RadiologyVol. 68Issue 7PreviewSir — We appreciate the attention that our article1 has received and welcome this opportunity to address the thoughtful comments. In the comment, Adams et al. indicate that it is of crucial importance to use a reliable reference standard when assessing the performance of a diagnostic test (both in original research studies and diagnostic meta-analyses). They also mentioned that bone marrow biopsy (BMB) only assesses a very small portion of the entire bone marrow and may, therefore, suffer from sampling errors (i.e., false negatives). Full-Text PDF