Considerable Interobserver Variability Research Articles

The Impact of Colon Cleanliness Assessment on Endoscopists' Recommendations for Follow-Up Colonoscopy

Repeat colonoscopy is advocated for low-quality preparations. However, there are few data on how endoscopists assess the quality of bowel preparation. We aimed to investigate, in a visually reproducible manner, endoscopists' assessment of colon cleanliness, as reflected by their subsequent recommendations for follow-up. Gastroenterologists attending the Israeli Gastroenterology Association meeting were presented with photographs depicting varying degrees of colon cleanliness at a hypothetical screening colonoscopy. Endoscopists were requested to denote their recommendation for the timing of a follow-up procedure for each of the different preparations, both when no polyps were detected and when two small adenomas were found. Seventy-eight gastroenterologists were included. There was considerable interobserver variability in endoscopists' assessment of preparation adequacy, and recommended follow-up timing ranged from more than 5 yr to immediate repeat colonoscopy for identical preparations. Interestingly, even when repeat colonoscopy was not considered necessary, most endoscopists recommended progressively shorter follow-up intervals in line with reduced preparation quality (mean interval dropping from 9.2+/-1.7 to 6.3+/-2.8 to 2.5+/-2 yr, P<0.001 for trend). Similar findings were observed when two adenomas were hypothetically found on the index procedure, although follow-up intervals were shorter. No correlation was found between endoscopists' clinical experience or acquaintance with clinical guidelines and their actual recommendation. Clinical decisions derived from colon cleanliness assessment vary considerably among endoscopists, and there is little agreement on what constitutes a disqualifying preparation. Moreover, when confronted with an intermediate-quality preparation, most gastroenterologists recommend a shorter follow-up interval, rather than repeating the procedure. Further studies are required to validate this management approach and to standardize the assessment of preparation quality.

Management strategies for Barrett's esophagus

Barrett's esophagus is a premalignant condition, with dysplasia usually preceding the development of adenocarcinoma. Two main outstanding issues are firstly, the prediction of a subset of patients at higher risk of developing dysplasia and secondly, the treatment to try and avoid neoplastic progression. Whilst it is tempting to rely on the presence of low-grade dysplasia to predict subsequent malignant potential, low-grade dysplasia on its own is often an unreliable marker of such risk. As well as the difficulty in reproducing biopsies from the same site in the Barrett's segment, there is considerable interobserver variability and the histopathologist must be meticulous in excluding inflammatory changes secondary to severe gastro-esophageal reflux because these can mimic dysplasia. Low-grade dysplasia may also regress after adequate acid suppression.1 In Barrett's esophagus, the metaplastic cells have a higher proliferative rate. This activity is increased by acid exposure in vitro and in vivo via mitogen-activated protein kinase pathways which transmit growth regulatory signals to enhance cellular proliferation and decrease apoptosis.2 Mucosal acid exposure has been shown to promote epithelial proliferation,3,4 and increased epithelial proliferation in patients with Barrett's esophagus has been associated with a stepwise progression of dysplasia to adenocarcinoma.3,5,6 Conversely, acid suppression with proton-pump inhibitors (PPIs) has been shown to stabilize proliferative cell activity in Barrett's esophagus4,6,7 and may prevent dysplasia and reduce cancer risk.1,8 Also, cyclooxygenase-2 (COX-2) is over-expressed in the early transformation of esophageal epithelium in Barrett's esophagus and in the transition from low- to high-grade dysplasia and adenocarcinoma.9 There is speculation that chemoprevention could reduce the incidence of dysplasia and adenocarcinoma,8–10 and that trials of chemoprevention with antisecretory therapy combined with COX-2 inhibitors are therefore warranted.10 In this issue of the Journal, there are two papers about Barrett's esophagus. Kim et al. address the prevalence of Barrett's esophagus in an Asian country (Korea) and the associated demographics and risk factors (age, male sex, cigarette smoking, and acid regurgitation). Male sex has been identified previously as a major risk factor for the development of high-grade dysplasia and adenocarcinoma.11 The present study by Kim and colleagues also documents the predictive value of macroscopic markers, such as severe esophagitis, Barrett's ulcer, nodularity, and stricture formation. Interestingly, there were only two patients with severe esophagitis identified in the Korean study and only one patient with adenocarcinoma, but no dysplasia was seen. Whether this reflects the relatively small numbers in this series, the retrospective analysis or a different natural history remains to be elucidated. There is no doubt that smaller studies tend to overestimate the cancer risk.12,13 Larger patient series suggest that the risk of adenocarcinoma is more in the order of 1/176 patient-years.11 The second article by Amano et al. looks at factors affecting squamous epithelialization (and by inference, regression) of Barrett's esophagus. Intestinal mucin phenotype and maturation, COX-2 expression, and cellular proliferation were all identified as predominant predictors of poor squamous re-epithelialization. PPIs (albeit in low doses for short periods) were only effective in causing regression when the intestinal phenotype was not present. This may allow immuno-histochemical identification of a subgroup of patients with a higher malignant potential. But it may also mark a group that requires high-dose, long-term PPIs to heal reflux esophagitis and to maintain healing, and thereby decrease continued cellular proliferation. Whilst optimal medical management of Barrett's esophagus is uncertain,14 PPIs are an attractive class of medications; they have few side effects and are readily available. Accordingly, it may be appropriate to initiate long-term, high-dose PPIs as soon as the diagnosis is established, or even prior to endoscopy if the severity of reflux symptoms is such that the diagnosis is entertained.15 Although biomarkers would be a convenient method to target surveillance of high risk subgroups of Barrett's esophagus, chemoprevention with PPIs and/or COX-2 inhibitors may be a more cost-effective option for management of Barrett's esophagus in the long term. It must be emphasized, however, that PPIs need to be prescribed at standard or even high dose indefinitely in order for this strategy to be effective.

Grading of dysplasia in Barrett's oesophagus: substantial interobserver variation between general and gastrointestinal pathologists

To determine interobserver variation in grading of dysplasia in Barrett's oesophagus (BO) between non-expert general pathologists and expert gastrointestinal pathologists on the one hand and between expert pathologists on the other hand. In this prospective multicentre study, non-expert and expert pathologists graded biopsy specimens of 920 patients with endoscopic BO, which were blindly reviewed by one member of a panel of expert pathologists (panel experts) and by a second panel expert in case of disagreement on dysplasia grade. Agreement between two of three pathologists was established as the final diagnosis. Analysis was performed by kappa statistics. Due to absence of intestinal metaplasia, 127/920 (14%) patients were excluded. The interobserver agreement for dysplasia [no dysplasia (ND) versus indefinite for dysplasia/low-grade dysplasia (IND/LGD) versus high-grade dysplasia (HGD)/adenocarcinoma (AC)] between non-experts and first panel experts and between initial experts and first panel experts was fair (kappa = 0.24 and kappa = 0.27, respectively), and substantial for differentiation of HGD/AC from ND/IND/LGD (kappa = 0.62 and kappa = 0.58, respectively). There was considerable interobserver variability in the interpretation of ND or IND/LGD in BO between non-experts and experts, but also between expert pathologists. This suggests that less subjective markers are needed to determine the risk of developing AC in BO.

Night Vision Goggles for Surveillance of Barrett’s Esophagus: A Paradigm Shift is Coming!

Night Vision Goggles for Surveillance of Barrett’s Esophagus: A Paradigm Shift is Coming!

Multiplex short tandem repeat DNA analysis confirms the accuracy of p57 KIP2 immunostaining in the diagnosis of complete hydatidiform mole

Detailed histopathologic examination remains to be the basis for the diagnosis of hydatidiform mole (HM). However, poor sampling, necrosis, and earlier uterine evacuation can lead to uncertainty in the diagnosis. Also, the criteria are subjective, resulting in considerable interobserver variability. The p57(KIP2) gene is paternally imprinted and maternally expressed, and the presence of its protein product serves as a surrogate marker for the nuclear maternal genome. Because a complete HM (CHM) is the only type of conceptus lacking a maternal contribution, p57(KIP2) immunostaining is correspondingly absent, whereas it is present in CHM mimics. Although analysis of DNA microsatellite polymorphisms is a reliable method for the diagnosis and classification of HM, it is not universally available. To assess the relative accuracy of p57(KIP2) immunostaining and molecular diagnosis by nuclear DNA microsatellite polymorphisms in discriminating CHM from its mimics, we analyzed archival tissue from 33 case patients (7 with a definitive diagnosis of CHM, 16 with a possible diagnosis of HM, and 10 with normal placentas) by both methods. Concordant results were obtained in all cases, and p57(KIP2) immunostaining accurately identified all cases of CHM from the groups with a definitive or possible diagnosis of HM. p57(KIP2) immunohistochemistry is a time- and cost-effective means of distinguishing CHM from its mimics in challenging cases.

Reliability of interpretation of CT examination of the larynx in patients with glottic laryngeal carcinoma

Reports in literature suggest that tumor volume, cartilage invasion, and cartilage sclerosis are independent prognostic factors for tumor control in radiotherapy as primary treatment of laryngeal carcinomas. Assessment of the interobserver variability and prognostic value in the measurement of tumor volume, cartilage invasion, and sclerosis. Retrospective analysis of 55 CT scans by three independent observers. Volume measurements and determination of cartilage invasion, cartilage sclerosis, and tumor localization were calculated. Correlation between the prognostic factors and radiotherapy was calculated for each observer. Values for interobserver agreement varied substantially; cartilage invasion (kappa value: -0.02 to 0.66), cartilage sclerosis (kappa value: 0.13 to 0.57), tumor localization of subsites (kappa value: 0.03 to 0.60), and tumor volume (correlation: 0.34 to 0.73). The found interobserver variation makes it impossible to establish accurate prognostic factors. Determination of tumor volume, cartilage invasion, and cartilage sclerosis on the basis of CT imaging shows considerable interobserver variation; clinical significance appears to be limited. B-3a.

Diagnostic accuracy and interobserver variability of plain radiography, subtraction arthrography, nuclear arthrography, and bone scintigraphy in the assessment of aseptic femoral component loosening

Plain radiography, subtraction arthrography, nuclear arthrography, and bone scintigraphy are imaging techniques commonly used to identify aseptic femoral component loosening. Controversy exists about the relative utility of these techniques. We evaluated the diagnostic accuracy and interobserver reliability of the four techniques in 78 consecutive patients (mean age 70 years, range 29-88 years) referred for evaluation of their femoral hip prostheses. The standard evaluation protocol consisted of plain radiography followed by subtraction arthrography, nuclear arthrography, and bone scintigraphy. Surgery or the subsequent clinical course of the patient was used as gold standard. Overall, plain radiography had a sensitivity and specificity of 81 and 74%, respectively. Subtraction arthrography had a sensitivity of 47% and a specificity of 78%. Nuclear arthrography had a sensitivity of 69% and a specificity of 76%, and bone scintigraphy had a sensitivity of 88% with a specificity of 50%. We found considerable interobserver variability in all four techniques. Multivariate regression analysis revealed that bone scintigraphy and nuclear arthrography together made a significant contribution to the diagnosis when used in combination with plain radiography and are, when plain radiography is inconclusive, useful additional diagnostic techniques for the detection of femoral component loosening.

Analysis of Regional Left Ventricular Function by Cineventriculography, Cardiac Magnetic Resonance Imaging, and Unenhanced and Contrast-Enhanced Echocardiography: A Multicenter Comparison of Methods

To define the use of cineventriculography, cardiac magnetic resonance imaging (cMRI), and unenhanced and contrast-enhanced echocardiography for detection of left ventricular (LV) regional wall motion abnormalities (RWMA). Detection of RWMA is integral to the evaluation of LV function. In 100 patients, cineventriculography and unenhanced and contrast-enhanced echocardiography were performed. Fifty-six of the patients underwent additional cMRI. RWMA were assessed referring to a 16-segment model for cMRI, unenhanced and contrast echocardiography. Cineventriculography was evaluated on a 7-segment model. Hypokinesia in one or more segments defined presence of RWMA. Interobserver agreement among three readers was determined within each imaging modality. Intermethod agreement between imaging modalities was analyzed. A standard of truth for the presence of RWMA was obtained by an independent expert panel decision (EPD) based on clinical data, electrocardiogram, coronary angiography, and blinded information from the imaging modalities. Sixty-seven patients were found to have an RWMA by EPD. Interobserver agreement expressed as kappa coefficient was 0.41 (range 0.37 to 0.44) for unenhanced echocardiography, 0.43 (range 0.29 to 0.79) for cMRT, 0.56 (range 0.44 to 0.70) for cineventriculography, and 0.77 (range 0.71 to 0.88) for contrast echocardiography. Contrast enhancement compared to unenhanced echocardiography improved agreement of echocardiography related to cMRI (kappa 0.46 vs. 0.29) and related to cineventriculography (kappa 0.59 vs. 0.28). Accuracy to detect EPD-defined RWMA was highest for contrast echocardiography, followed by cMRI, unenhanced echocardiography, and cineventriculography. Analysis of RWMA is characterized by considerable interobserver variability even using high-quality imaging modalities. Interobserver agreement on RWMA and accuracy to detect panel-defined RWMA is good using contrast echocardiography.

Diagnosing Pneumonia by Medical History and Physical Examination

Diagnosing Pneumonia by Medical History and Physical Examination

Diagnosis: To Biopsy or Not to Biopsy: Assessing the Role of Surgical Lung Biopsy in the Diagnosis of Idiopathic Pulmonary Fibrosis

Idiopathic pulmonary fibrosis (IPF) is a chronic disorder that is associated with a poorer prognosis than subacute idiopathic interstitial pneumonias (IIPs). IPF can be differentiated from other IIPs on the basis of its histologic pattern of usual interstitial pneumonia (UIP). Although a surgical lung biopsy specimen showing a UIP pattern is required for the definitive diagnosis of IPF, clinical criteria can be used in the absence of a lung biopsy specimen to make a likely diagnosis of IPF. The predictive value of these criteria largely depends on the expertise of the clinician and radiologist, but considerable interobserver variability exists even when evaluations are performed by experts in the field. Moreover, these criteria lead to misdiagnosis in about 25 to 35% of cases. Interobserver variability is reduced and diagnostic accuracy is improved in cases in which a diagnosis is made with a high degree of confidence. Diagnostic accuracy is also higher when the diagnosis is made by a core group of experts rather than by a referring center. The decision on whether or not to perform a surgical lung biopsy is difficult. It is clearly indicated in cases in which clinical or radiologic findings are atypical or when the diagnosis is made with a low degree of certainty.

Interobserver and Intraobserver Variability in the Assessment of Pulmonary Nodule Size on CT Using Film and Computer Display Methods 1

A critical element in determining biologic behavior of pulmonary nodules is volume and temporal volume change. We evaluate variability in nodule volume among readers and measuring methods. 55 small (<2 cm) lung nodules were measured in long- and short-axis dimensions independently by 4 radiologists, using 3 methods: 1) hard copy, 2) GE Advantage Windows workstation (GE Healthcare, Milwaukee, WI), 3) Siemens IMACS workstation (Siemens Medical Systems, Iselan, NJ). Nodule margin was recorded as smooth, lobulated, or spiculated. Volume was calculated from diameter measurements. Variability in nodule volume was evaluated within each reader, between readers, and across measurement tools. Mean nodule short-axis diameter was 5.3 mm; mean long-axis diameter 7.2 mm. There was statistically significant variation among readers and measurement method for nodule volume. Volume was significantly larger using hard-copy measurements (51.9%-54.1% variation; P < .0001) than either workstation, and not different between workstations. There was greater intraobserver variability in volume using the hard-copy method, and no difference between workstation methods. Volumes based on measurements from one reader were consistently lower than those from other readers (P = < .001, .003, and .02); volume was consistently larger for another reader (P < .0001, .03, and .12). Reader agreement for nodule margin was good to excellent. Considerable interobserver and intraobserver variability in measuring nodules exists using hard-copy and computer tools. Since a small change in diameter indicates a much larger change in volume, this may be significant when using early repeat CT to follow small pulmonary nodules. Computer-aided diagnostic tools that reproducibly measure nodule volume are strongly needed.

Vertebral Rotation and Thoracic Torsion in Adolescent Idiopathic Scoliosis

As a result of the increased appreciation of the three-dimensional nature of scoliosis and modern spinal instrumentation's improved corrective capabilities, there has been renewed interest in the correction and measurement of vertebral rotation. Computed tomography (CT), the gold standard for accuracy, is limited in its clinical utility owing to cost, radiation exposure, and the effects of postural changes on scoliosis curves and vertebral rotation. Consequently, the Perdriolle and Nash-Moe techniques remain the standard measurements for providing a reasonable estimate of pre- and postoperative vertebral rotation because of their simplicity. However, these techniques have considerable interobserver variability, and pedicle screw instrumentation obscures the landmarks necessary for utilizing these techniques for postoperative vertebral rotation assessment. The purpose of the present study was to assess the utility of alternate radiographic measures to assess vertebral rotation and thoracic torsion when compared with conventional measures on pre- and postoperative radiographs and CT evaluation. We reviewed the preoperative, immediate postoperative, and final follow-up radiographs, as well as the pre- and postoperative CT scans, of 19 patients (average age 15 years, 6 months) with Lenke 1 curves (average 55 degrees , range 47-66 degrees ), all treated with anterior spinal fusion. Coronal and sagittal Cobb angles as well as vertebral rotation (Perdriolle and Nash-Moe) at the superior uninstrumented, superior instrumented, apical, inferior instrumented, and inferior uninstrumented vertebrae were measured on all films, and vertebral rotation was assessed on the CT scans by a previously described method. Additionally, several measures of thoracic torsion (as a proxy for vertebral rotation and overall deformity improvement) were assessed. These included the rib-vertebral angle difference (RVAD), apical rib hump prominence (RH), apical vertebral body-rib ratio (AVB-R), and apical rib spread difference (ARSD). The postoperative main thoracic curve averaged 26 degrees (range 16-39 degrees , 52% correction) and 29 degrees (range 22-40 degrees , 47% correction) at final follow-up. For apical derotation, the postoperative CT improved from -11.5 degrees to -6.6 degrees and correlated significantly with the Cobb main thoracic curves (42% correction, r = 0.48, P = 0.003). There was weakly positive, but statistically significant, correlation between the pre- and postoperative CT scans and the corresponding Perdriolle and Nash-Moe measures of segmental rotation (r = 0.32-0.40, all P < 0.0001). The RVAD demonstrated poor correlation with the main thoracic curve values and correction, Perdriolle rotation and correction, and CT rotation and correction (r = -0.22-0.37, all P > 0.20). The apical RH demonstrated good correlation with the main thoracic curve (r = 0.65, P < 0.0001), apical Perdriolle rotation (r = 0.57, P < 0.0001), and CT apical rotation (r = 0.53, P = 0.002). We also found moderate correlation between the AVB-R and the main thoracic Cobb, apical Perdriolle, and CT (r = 0.57, 0.59, and 0.49, respectively; all P < 0.005). Similar relationships were found with the ARSD (r = 0.51, 0.47, and 0.43, respectively; all P < 0.02). The RH, AVB-R, and the ARSD-measures of thoracic torsion-demonstrated moderate to good overall correlation with the main thoracic curve Cobb angles, apical Perdriolle rotation, and apical CT rotation. These should be useful as clinical measures for assessing three-dimensional deformity correction on plane radiographs, especially for the intraoperative evaluation of vertebral derotation and thoracic symmetry restoration.

P57 KIP2 immunohistochemistry in early molar pregnancies: emphasis on its complementary role in the differential diagnosis of hydropic abortuses

Morphologic examination still forms the main diagnostic tool in the differential diagnosis of molar pregnancies. However, the criteria are subjective and show considerable interobserver variability among pathologists. Once a diagnosis of molar pregnancy is made, DNA ploidy studies help to differentiate a triploid partial mole from diploid complete mole (CM). However, with earlier diagnosis and therapeutic evacuation of molar pregnancies, the differentiation of molar pregnancies from early nonmolar placentation is becoming increasingly difficult. The p57(KIP2) gene ( CDKN1C ) is strongly paternally imprinted and expressed from the maternal allele. Because CM lacks a maternal genome, p57(KIP2) immunostaining is correspondingly absent, whereas hydropic abortuses and partial mole show positive staining. We compared the use of p57(KIP2) staining in the differential diagnosis of 68 morphologically challenging cases of early first-trimester hydropic placentas. Diagnosis based on p57(KIP2) staining was compared with the original diagnosis based on morphology and DNA ploidy analysis. Concordant results were obtained in 65 of 68 cases studied. In 2 of 3 cases with a discordant diagnosis, microsatellite DNA genotyping analysis agreed with the results of p57(KIP2) staining, confirming that positive p57(KIP2) staining is a highly sensitive and specific marker for excluding CM in this setting. In addition, p57(KIP2) staining has the advantage of differentiating hydropic abortuses from CMs, a distinction not made by ploidy analysis. p57(KIP2) staining can be used in concert with ploidy studies to refine the diagnosis of early molar pregnancies.

A Comparison of Radiographic and Scintigraphic Techniques to Assess Aseptic Loosening of the Acetabular Component in a Total Hip Replacement

The diagnosis of a loose total hip prosthesis is often established with use of radiographic and nuclear medicine techniques, but there is controversy about the relative utility of plain radiography, subtraction arthrography, nuclear arthrography, and bone scintigraphy. In this retrospective study, we evaluated the sensitivity, specificity, and interobserver reliability of these imaging modalities in patients suspected of having aseptic loosening of the acetabular component. From 1994 to 1999, eighty-six consecutive patients with pain after a total hip arthroplasty were evaluated for possible loosening of the components. The imaging evaluation included plain radiography followed by a one-day protocol that included bone scintigraphy, subtraction arthrography, and nuclear arthrography. For this study, two experienced nuclear medicine physicians and two experienced radiologists, all of whom were blinded with respect to the clinical pretest data and the clinical outcome, retrospectively interpreted the diagnostic images. The sensitivity and the specificity of each imaging modality were established by comparing the findings obtained with each technique with those found at surgery or during the subsequent clinical course of the patient. Interobserver variability was determined with the intraclass correlation coefficient. Plain radiography had a sensitivity of 85% (95% confidence interval, 71 to 94) and a specificity of 85% (95% confidence interval, 66 to 96) in detecting aseptic loosening of the acetabular component, but it had only fair interobserver variability (intraclass correlation coefficient, 0.37). For subtraction arthrography, the sensitivity was 72% (95% confidence interval, 57 to 84), the specificity was 70% (95% confidence interval, 50 to 86), and there was good interobserver variability (intraclass correlation coefficient, 0.71). For nuclear arthrography, the sensitivity was 57% (95% confidence interval, 41 to 71), the specificity was 67% (95% confidence interval, 46 to 84), and there was fair interobserver variability (intraclass correlation coefficient, 0.24). For bone scintigraphy, the sensitivity was 83% (95% confidence interval, 69 to 92), the specificity was 67% (95% confidence interval, 46 to 84), and there was moderate interobserver variability (intraclass correlation coefficient, 0.43). Plain radiography had the highest diagnostic accuracy in the evaluation of aseptic loosening of the acetabular component. The diagnostic accuracy was increased when plain radiography was combined with bone scintigraphy or subtraction arthrography. However, we found considerable interobserver variability in image interpretation, even with experienced radiologists and nuclear medicine physicians.

An alternative measure of sleep fragmentation in clinical practice: the sleep fragmentation index

Background and purpose Micro-arousals (MA) are commonly considered as sleep components reflecting sleep fragmentation. However, their elucidation is time-consuming, with considerable inter-observer variability. The aim of our study was to investigate the usefulness of a sleep fragmentation index (SFI) to detect sleep disruption in a large sample of patients. Patients and methods Five-hundred ninety-eight polysomnographic studies made in controls and patients were examined. The SFI was calculated as the total number of awakenings and sleep stage shifts divided by total sleep time. Results In the whole group a significant correlation was found between the SFI and the MA index (MAI) ( P<0.001) with good agreement across a wide range of values. When patients were stratified according to final diagnosis a significant relation was noted for patients with insomnia ( P<0.001), parasomnia ( P<0.001), circadian schedule disorders ( P<0.001) and sleep related breathing disorders ( P< 0.001). Lower values were found in controls ( P<0.01) and in patients with periodic limb movement disorder and/or restless legs syndrome ( P<0.05). In 111 patients having two consecutive recording nights, a good reproducibility was present with no differences between nights ( P=ns) and with significant correlation ( P<0.001). Conclusions The SFI seems to be an accurate, reproducible and easy method to detect sleep fragmentation in patients with sleep disorders. Further studies are needed to validate the usefulness of this tool in clinical practice.

A critical analysis of the accuracy, reproducibility, and clinical utility of histologic endometrial dating in fertile women

Objective To refine or redefine the traditional histologic criteria used to date the secretory phase endometrium. Design Randomized, observational study. Setting Academic clinical research center. Patient(s) One hundred and thirty healthy, regularly cycling, fertile volunteers, aged 18 to 35 years. Intervention(s) Patients were randomized to undergo endometrial sampling and measurement of serum estradiol and progesterone 1 to 14 days after the midcycle urinary luteinizing hormone surge. Three gynecologic histopathologists objectively scored each tissue specimen for 32 distinct histologic features and dated the endometrium using traditional histologic criteria. Main outcome measure(s) The 32 features were evaluated for [1] temporally dependent variation, [2] the amplitude of variations in score observed across the secretory phase, and [3] interobserver variability. Additionally, traditional dating criteria were analyzed. Result(s) The traditional endometrial histologic dating criteria are much less temporally distinct and discriminating than originally described, due to considerable intersubject, intrasubject, and interobserver variability. Neither traditional dating criteria nor any combination of the best performing histologic features identified by our objective and systematic analyses could reliably distinguish any specific cycle day or narrow interval of days. Conclusion(s) Histologic endometrial dating does not have the accuracy or the precision necessary to provide a valid method for the diagnosis of luteal phase deficiency or to otherwise guide the clinical management of women with reproductive failure.

Computerized diagnostic decision support system for the classification of preinvasive cervical squamous lesions

Previous studies have revealed considerable interobserver and intraobserver variation in the histological classification of preinvasive cervical squamous lesions. The aim of the present study was to develop a decision support system (DSS) for the histological interpretation of these lesions. Knowledge and uncertainty were represented in the form of a Bayesian belief network that permitted the storage of diagnostic knowledge and, for a given case, the collection of evidence in a cumulative manner that provided a final probability for the possible diagnostic outcomes. The network comprised 8 diagnostic histological features (evidence nodes) that were each independently linked to the diagnosis (decision node) by a conditional probability matrix. Diagnostic outcomes comprised normal; koilocytosis; and cervical intraepithelial neoplasia (CIN) I, CIN II, and CIN III. For each evidence feature, a set of images was recorded that represented the full spectrum of change for that feature. The system was designed to be interactive in that the histopathologist was prompted to enter evidence into the network via a specifically designed graphical user interface (i-Path Diagnostics, Belfast, Northern Ireland). Membership functions were used to derive the relative likelihoods for the alternative feature outcomes, the likelihood vector was entered into the network, and the updated diagnostic belief was computed for the diagnostic outcomes and displayed. A cumulative probability graph was generated throughout the diagnostic process and presented on screen. The network was tested on 50 cervical colposcopic biopsy specimens, comprising 10 cases each of normal, koilocytosis, CIN I, CIN II, and CIN III. These had been preselected by a consultant gynecological pathologist. Using conventional morphological assessment, the cases were classified on 2 separate occasions by 2 consultant and 2 junior pathologists. The cases were also then classified using the DSS on 2 occasions by the 4 pathologists and by 2 medical students with no experience in cervical histology. Interobserver and intraobserver agreement using morphology and using the DSS was calculated with κ statistics. Intraobserver reproducibility using conventional unaided diagnosis was reasonably good (κ range, 0.688 to 0.861), but interobserver agreement was poor (κ range, 0.347 to 0.747). Using the DSS improved overall reproducibility between individuals. Using the DSS, however, did not enhance the diagnostic performance of junior pathologists when comparing their DSS-based diagnosis against an experienced consultant. However, the generation of a cumulative probability graph also allowed a comparison of individual performance, how individual features were assessed in the same case, and how this contributed to diagnostic disagreement between individuals. Diagnostic features such as nuclear pleomorphism were shown to be particularly problematic and poorly reproducible. DSSs such as this therefore not only have a role to play in enhancing decision making but also in the study of diagnostic protocol, education, self-assessment, and quality control.

Interobserver variability in the radiological assessment of response to chemotherapy in glioma.

To assess the interobserver variability in the radiologic assessment of response to chemotherapy in patients with recurrent glioma. Five clinicians with experience in the treatment and follow-up of patients with glioma measured tumor size in 20 pairs of CT and 20 pairs of MRI scans of 35 patients who had been treated with chemotherapy for recurrent glioma. Tumor size was defined as the product of the two largest perpendicular enhancing tumor diameters on the postcontrast images. To assess the interobserver variability in the measurements of tumor size, and in the classification according to the widely used Macdonald response criteria, intraclass correlation coefficients (ICC) and weighted kappa values were calculated. Substantial interobserver agreement was noted in the manual, two-dimensional measurements of tumor size on CT and MRI in patients treated with chemotherapy for recurrent glioma (overall ICC 0.64). Classification of response to chemotherapy according to the Macdonald criteria resulted in moderate interobserver agreement (overall weighted kappa 0.51). In 65% of evaluated CT and in 55% of evaluated MRI studies, no complete consensus was found for the categorical tumor response measurement. The radiologic assessment of response to chemotherapy in patients with recurrent glioma is susceptible to considerable interobserver variability. This underlines the difficulties that arise in scoring response to chemotherapy by conventional radiologic techniques.

Interobserver variation in duplex scanning of infrainguinal arterial bypass grafts

Objective: to determine the degree of interobserver variation of color-flow duplex scanning of infrainguinal arterial bypass grafts. Methods: two experienced vascular technologists randomly assessed bypass grafts in 32 consecutive patients, using a color-flow duplex scan. In pre-defined segments the highest peak systolic velocity (PSVmax) and end-diastolic velocity (EDV) were measured and a peak systolic velocity ratio (PSV ratio) was calculated. Results were analyzed as continuous variables (Bland and Altman plots and Intraclass Correlation Coefficient = ICC) and also as categorical data (weighted Kappa coefficient) for the PSV ratio 1–2.5, ≥2.5–4, ≥4.0. Results: the ICC for the PSVmax, PSV ratio and EDV indicated “almost perfect” agreement for all three parameters. However, the Bland and Altman plots showed impressive interobserver variation for the higher values of all three parameters. For the PSV ratio categories a weighted kappa of 0.31 was calculated, indicating only fair agreement. Substantial variation was found for the categories with PSV ratios ≥2.5–4.0 and ≥4.0. Conclusion: though performing accurately for the lower values of the assessed parameters, duplex scanning shows considerable interobserver variation for the clinically significant higher values. Particularly in the PSV ratio interval ≥2.5–4.0, most relevant for clinical decision-making, the interobserver variability is unacceptable.Eur J Vasc Endovasc Surg 25, 224–228 (2003)

Radiological evaluation of oncologic treatment response: current update

Radiological evaluation of oncologic treatment response: current update