Virtual colonoscopy: Good results and not so good results—why the difference?

Abstract

Cotton PB, Durkalski VL, Pineau BC, Palesch YY, Mauldin PD, Hoffman B, Vining DJ, Small WC, Affronti J, Rex D, Kopecky KK, Ackerman S, Burdick JS, Brewington C, Turner MA, Zfass A, Wright AR, Iyer RB, Lynch P, Sivik MV, Butler H (Medical University of South Carolina, Charleston, South Carolina). Computed tomographic colonography (virtual colonoscopy): a multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713–1719. Colorectal cancer (CRC) is the second leading cause of cancer death in the United States resulting in approximately 6000 deaths annually. Although death from colon cancer is largely preventable through routine screening of asymptomatic patients for early detection and removal of premalignant adenomatous polyp precursors, patient compliance remains poor, rarely exceeding 50% in population surveys. Of the currently recommended CRC screening tests (fecal occult blood test, flexible sigmoidoscopy, double contrast barium enema, conventional colonoscopy), conventional colonoscopy is recognized as the gold standard by virtue of its high sensitivity for detecting polyps and cancers throughout the entire length of the colon, as well as its ability to remove or biopsy detected lesions. However, conventional colonoscopy has serious drawbacks as a screening test. It is invasive, with a small but definite risk of serious complications. It is expensive, costing as much as $2000 in some locales, and it requires intravenous sedation and a bowel prep, which discourages many patients from undergoing the test. Moreover, there is a significant manpower resource limitation among endoscopists, and several-month waiting lists for elective colonoscopies are common nationwide. Thus, there is a clear need for simpler screening methods that would allow conventional colonoscopy to be used more selectively and efficiently. Computed tomographic colonography (CTC), also referred to as “virtual colonoscopy,” is rapidly evolving as a promising candidate for CRC screening. Several single center studies (N Engl J Med 1999;341:1496–1503, Radiology 2001;219:685–692, Radiology 2002;224:383–392, Am J Surg 2002;183:124–131) and a recently published multicenter study (N Engl J Med 2003;349:2191–200) have reported sensitivities of more than 90% for detection of polyps greater than 10 mm, the size threshold for so-called “advanced adenomas,” which is the agreed principle target for colorectal cancer screening (N Engl J Med 2000;343:169–174). Sensitivity for smaller lesions 6–9 mm in size has generally been lower, ranging between 50% and 80%. Although detection (and exclusion) of all lesions is a desirable goal, the key screening parameter is the ability to detect patients with at least one clinically significant lesion (10 mm or larger), which would lead logically to therapeutic colonoscopy. In this study, Cotton et al sought to assess the performance of CTC in a large multi-center trial. The study was a nonrandomized, reader blinded, prospective trial of 615 patients age 50 years or older who were referred for routine clinically indicated colonoscopy in 9 major hospital centers between April 2000 and October 2001. Patients who had undergone colonoscopy within 3 years were excluded. Imaging was performed using 2- and 4-row multidetector computerized tomographic (CT) scanners of various manufacturers. Nominal slice widths of 2.5 to 5.0 mm and reconstruction increments of 1.5 to 1 mm were used. Both prone and supine acquisitions were obtained, each in a single breath-hold. Study data was read from 2-dimensional slices and, when necessary, by focal 3-dimensional snapshot reconstructions. Radiologists recorded their interpretation in 5 sealed envelopes, 1 for each of the 5 colon segments, ie, rectum and sigmoid, descending colon and splenic flexure, transverse colon and hepatic flexure colon, ascending colon, and the cecum. Endoscopists were blinded to the CTC results during conventional colonoscopy. The colon was examined during withdrawal with the results recorded for each segment. After each segment was examined and the results recorded, the CTC results for that segment were revealed to the endoscopist allowing for immediate re-examination for any discrepancy, ie, “segmental unblinding.” Radiologists and endoscopists recorded adequacy of bowel preparation (presence of fluid or stool) for each colon segment, as well as level of confidence for each detected polyp and each colon segment. The authors chose to report findings as “lesions” rather than adenomas vs nonadenomas and established a ground truth gold standard as a combination of initial findings on conventional colonoscopy and any additional findings after segmental unblinding to the CTC reports. This meant that the positive predictive value of conventional colonoscopy was 100%, by definition. The primary goal and outcome measure of the study was to assess the sensitivity and specificity of CTC in identifying patients with and without lesions at least 6 mm in size. Secondary outcome measures include the correct detection for lesions of any size at CTC and conventional colonoscopy, the positive and negative predictive values, and patient preferences for CTC vs. conventional colonoscopy. Initially, 1500 patients were projected to achieve statistical significance, but recruitment was slower than expected and, for this reason, the study was stopped after 615 patients were enrolled. A total of 827 lesions were detected in 308 of 600 patients who underwent both procedures; 104 patients had at least 1 lesion 6 mm in size. Overall, only 308 patients (51%) actually had lesions, meaning that 49% of patients did not have any pathology detected. Of the total of 827 lesions, 654 (79%) were 1–5 mm in size, 119 (14%) were 6–9 mm, and 54 (6.5%) were at least 10 mm. Sensitivity of CTC for detecting patients with one or more lesions using a cut off size of 6 mm was 39% and for lesion size at least 10 mm it was 55%. These results were significantly lower than those for conventional colonoscopies with sensitivities of 99% and 100%, respectively. Specificity of CT and conventional colonoscopy for detecting participants without any lesion size at least 6 mm was 90% and 100%, respectively, and without lesions at least 10 mm, 96% and 100%, respectively. CTC missed 2 of 8 cancers. The accuracy of CTC varied considerably between centers. One center had far greater recruitment success (N = 184 patients) and had the best results with a primary outcome sensitivity of CTC for lesions greater than 10 mm of 82%. Sensitivity for all other centers combined was only 24%. However, there was no formal feedback to the participating radiologists during the period of the study trial and thus critical aspects of learning were missing. Colonoscopy reached the cecum in 98.5% of patients, but both CTC and conventional colonoscopy failed to detect 2 large lesions (2 cm in the transverse colon and 4.3 cm in the rectum, both found by surgery); both of these patients had other large lesions of at least 10 mm that were detected by conventional colonoscopy. Radiologists used 3-dimensional snapshots for troubleshooting in 60% of cases in which lesions were suspected. The rate of correct lesion identification was 28% when 3-dimensional snapshots were used and 13% when they were not. “Fly-through” data were analyzed at a later date by the same radiology readers without referring back to their initial 2-dimensional review. The sensitivity and specificity for detecting patients with and without at least 1 lesion of at least 6 mm were 45% and 93%, respectively. Sensitivity for detecting individual lesions was 15% for any size and 36% for lesions of at least 6 mm. Incorporating the “fly-through” data with the initial evaluation increased the sensitivity of CTC for the primary outcome (detecting patients with lesions greater than 6 mm) by 17% to 56% but reduced specificity by 5%. In patients with lesions at least 10 mm in size, the “fly-through” data increased the sensitivity by 12% to 67% and decreased the specificity by 1%. For the primary outcome of the study, the authors used 6 mm or above as a discriminator and found that CTC detected 32% of lesions in 39% of patients. For lesions 10 mm or greater, detection was 52% per lesion and 55% per patient. It was also of note that the CTC detection rate for 6–9 mm polyps was 23% per lesion and 30% per patient. The authors expressed surprise and disappointment at these results and noted contrast with many other studies, almost all of which came from single center studies and in which the lead author was a radiologist. In analyzing this poor performance, the authors pointed first to reader experience, indicating that only one of the centers had substantial prior involvement with the technique before the study. This center, in fact, contributed the most participants (N = 184) and had the best results with a primary outcome sensitivity of 82% for polyps 10 mm or greater. The authors did touch in their Discussion on a much more recently conducted study by radiologists from the Department of Defense (N Engl J Med 2003;349:2191–2200), in which CTC detected 94% of polyps greater than 10 mm in 1233 asymptomatic adults and 89% of patients with lesions larger than 6 mm. However, they did not elaborate on the specific reasons for the relatively poorer performance in their own study. Based on their own results, the authors concluded that CTC using their techniques was not ready for routine use at this time as a primary screening technique for colorectal neoplasm. Virtual colonoscopy is a rapidly involving, technique-intensive test for colorectal cancer screening that is being introduced into an environment with significant turf and economic implications. As investigators at an academic center that has achieved good results with virtual colonoscopy (N Engl J Med 1999;341:1496–1503), we believe it is important to underscore that virtual colonoscopy is a sophisticated technologically advanced imaging procedure in which excellent results tend to be multifactorial. Indeed, the present study has been widely compared and contrasted to the Department of Defense study published by radiologists in December, 2003 (N Engl J Med 2003;349:2191–2200). That study included twice as many patients (1233) who were totally asymptomatic, studied with a nearly identical technique and study design, and gave far superior results. The Cotton study did show certain commendable design attributes including multidetector CT technology, same-day single-visit back-to-back examinations, and segmental unblinding of CTC results during colonoscopy. However, there were several specific weaknesses of execution, which tend to mitigate the significance of the Cotton et al results. First, reader inexperience was the main flaw. Radiologist CTC investigators believe that a minimum of 25 and preferably 50 proctored cases be studied before competence can be assumed. In the Cotton study, only 10 training cases were requested. As noted by the authors, in the single most active center where the radiologists had considerable prior experience, the results trended towards those better results reported by radiologist investigators in 2 recent studies (N Engl J Med 1999;341:1496–1503, 2003;349:2191–2200). Second, the nature of the reader software used by Cotton et al was largely unspecified and certainly not standardized. Thus, the exact type of 2-dimensional visualization and the definition of “snapshot” 3-dimensional viewing were left undefined. CTC reading software has advanced considerably in the 3+ years since this study was conducted and thus, the results obtained by the authors in this very rapidly moving computer-dependent technology are substantially obsolete as presented. Reporting of results was also misleading in 2 fundamental ways: (1) By presenting ≥6 mm sized polyps as a threshold cut off, the authors weaken the concept of the advanced adenoma threshold size of 10 mm, which has been advocated as a surrogate for cancer risk and polyp detection by many CRC screening authorities in recent years (Gastrointest Endosc Clin N Am 2000;12:1–9). As the editorial by Ransohoff (JAMA 2004;14:1772–1774) accompanying the Cotton et al paper points out, the critical debate is over the significance of intermediate-sized polyps 6–9 mm in terms of the need for therapeutic intervention. Their use of the 6-mm threshold thus exaggerates the benefit/advantages of colonoscopy in terms of finding intermediate lesions whose removal may ultimately be of little clinical significance to the patient (JAMA 2004;291:1713–1719). The authors’ reporting of findings as “lesions” rather than adenomas vs nonadenomas further exaggerates the value of conventional colonoscopy. At least half of “lesions” <5 mm and as many as a third of lesions 6–9 mm are in fact not neoplasms or adenomas histologically and the patient thus derives no benefit from their removal at colonoscopy. The final intangible and probably unavoidable study design bias against CTC is the pressure of the 2-hour reading period before the colonoscopy. Although it may seem that 2 hours should be adequate to read a virtual colonoscopy, in clinical practice these studies are added to the existing Radiology Department workload and the participating radiologist was likely under considerable pressure to review the data and complete a probably complicated research case record report form. The likelihood is that errors of perception and interpretation were enhanced in order to not delay the patient or keep the colonoscopist waiting in the endoscopy suite. It remains a fact that all major studies of CTC published by radiologists have shown far better performance than this study published by gastroenterologists. It is also unclear why CTC should be held to a higher performance standard than the other 3 approved CRC screening tests, ie, FOBT, flexible sigmoidoscopy, or double contrast barium enema. None perform as well as colonoscopy, yet they remain standard tests. Assessment of CTC should be based on all the published evidence including comparison with all the existing approved CRC screening tests, not just colonoscopy. CTC radiologists are puzzled at the seemingly uncritical acceptance of the Cotton results and are aware that many gastroenterologists are even now ready to accept CTC. Moreover, as Cotton et al. pointed out, the imminent arrival of low prep laxative-free techniques (Am J Roentgenol 2003;181:799–805) for colon cleansing and the development of computer-aided detection techniques (Radiology 2002;222:327–336) to provide consistent, confident automated computer marking of suspicious lesions will likely add to both patient acceptance and the clinical performance in community practice. The results of Cotton et al notwithstanding, the prospects for CTC remain bright. Gastroenterologists, radiologists, and especially patients ultimately will all be winners. It won’t be long!