Quality of Interventional Radiology Journals and Papers

Abstract

We read with great interest the paper recently published in Journal of Vascular and Interventional Radiology (JVIR), by Meyer et al., entitled “Quality of Interventional Radiology Literature: A Review of Articles Published in JVIR and CVIR” (over a period of one year) [1]. Although we support any initiative to improve the quality of papers published in JVIR and CVIR, we feel that this paper gives a wrong impression of the present status of the quality of current interventional literature. The authors state in their introduction, “We performed this investigation of the current interventional literature to assess the quality of the published studies in this field.” However, to keep abreast of all developments in interventional radiology, it is necessary to expand the range of literature beyond JVIR and CVIR. High-quality interventional radiology papers, including randomized control trials, are most often submitted and published elsewhere; they appear in the New England Journal of Medicine, Lancet, JAMA, Circulation, and other high-indexed journals. The enormous impact factor of these journals still play an important role at academic careers. So, drawing conclusions about the quality of interventional radiology literature based solely on JVIR and CVIR can lead to misconceptions. Undoubtedly high-quality randomized control trials in interventional radiology have been performed for vertebroplasty, UAE, iliac stenting, EVAR, renal stenting, carotid stenting, SFA stenting, and drug-eluting balloons, to mention but a few, of which almost all have been published in one of the aforementioned top journals. Not recognizing this fact leads to a damaging impression of interventional radiology, and furthermore, no tribute is paid to those whose efforts have been invested in these trials. The main issue is not the quality of papers in CVIR and JVIR but, rather, their lower impact factor. A pivotal reason for this is the Atlantic Ocean or, to be more precise, the lack of a common global interventional radiology spirit across the ocean. Cross-references between JVIR and CVIR, or, rather, the United States and Europe, are rare, and most papers in these journals mainly quote work performed on their own continent, often overlooking relevant papers overseas. A change in this practice would certainly improve the citation index of both journals. A number of European universities have become aware of the issue of impact factor and have subsequently, very wisely, switched from the citation index to the Hirsch index, which calculates the significance of the total scientific work of an author by looking at the number of citations of all papers of an author. Even a paper published in a lower-ranking journal can be very important and therefore generate many citations. The opposite is also true, however; not all highly ranked journals publish important papers only. A very interesting article by John P. A. Ionnidis, “Why Most Published Research Findings Are False,” was published in 2005 [2]. He discusses the most crucial flaws in research: bias, proxy endpoints, insufficient power, reporting of only positive data, bad study design, incorrect study design, missing data, short follow-up, and low statistical power. Some of these flaws can also be found in the paper by Meyer et al., as well as in other papers published in JVIR and CVIR. Meyer et al. report on the quality of articles published in JVIR and CVIR during the course of 1 year. The two main conclusions of their paper, besides the incorrect assessment of the general quality of interventional radiology papers, as discussed above, are that (i) papers published in JVIR are of a higher quality compared to papers published in CVIR, and (ii) articles published in both journals display substantial weaknesses. These conclusions do not seem to be scientifically correct. For the overall comparison of the journals, total scores were calculated for each article, the results of which were compared between the two journals. However, if each item were evaluated separately, no difference would be noticed. Figure 1 in the paper in JVIR [1] shows that the proportions of outcomes for most of the consort criteria were comparable in both journals. It is only the summation of scores that suggests a difference. Moreover, Meyer et al. did not take into account the types of studies. Not all items are applicable to all studies, e.g., as some items can only be addressed in randomized controlled trials, they will always have a higher level of evidence. It therefore seems appropriate that (i) only results for the same item are compared and (ii) the types of studies are reported. We wonder whether the differences would still be significant if the authors had reported the findings as stated above. Furthermore, it remains open whether their p-values were obtained from the Wilcoxon rank-sum test or Student’s t-test, as it is also not clear why the authors stated mean values when performing a nonparametric test. Another issue is that assessment of the quality of papers should always be done by two reviewers independently [3]. Although Meyer et al. state that each investigator independently reviewed papers, it seems that each observer reviewed six issues of JVIR and three issues of CVIR. Some of the CONSORT criteria could be subjectively interpreted, as authors stated that there was ambiguity or uncertainty in scoring. All scores assigned by reviewer 1 were lower than those by reviewer 2. Although the authors state that this is a minor issue, these differences were found to be significant, and it is therefore not clear whether reviewer 1 consequently assigned these findings lower scores or reviewer 1 had more studies from CVIR with lower scores (no data on the number of articles per reviewer are reported). It is more or less customary to assess all papers independently, and not a selection. Therefore it is an important matter of concern that both observers did not perform this procedure appropriately. The paper is a good illustration, although perhaps unintentionally so, of how ambiguous an interventional radiology paper can be in scientific terms. Recognizing the real problems of lower impact factor and minimal trans-Atlantic recognition with minimal cross-fertilization might have been a much better starting point for examining the quality of interventional radiology papers published in both journals. CIRSE has recognized for years that education in medical statistics, good clinical practice, and evidence-based medicine is an important and long-term investment to produce better science. At every annual CIRSE meeting small-scale basic courses on these topics are available. Rather than conveying a productive message to improve the quality of interventional radiology publications in both journals, the attitude of “them and us” has been adopted. An unbiased bilateral discussion about scientific guidelines and strategic planning between the JVIR and the CVIR editorial boards could be the first step toward bringing both journals to a higher scientific level. Maybe one day this could finally lead to a prominently ranked journal dedicated to interventional radiology exclusively serving the needs of global interventional radiology. It is now time to go farther than our own backyards and to stop looking at interventional radiology papers and journals from a watchtower.