Interobserver Reproducibility Study Research Articles

Impact of scapula anatomical landmark positioning on scapular orientation using CT-based 3D models: an intra-observer repeatability and inter-observer reproducibility study

HypothesisThe primary objective of this study was to assess intra-observer repeatability and inter-observer reproducibility of the 3D coordinates of a set of scapula anatomical landmarks obtained by manual positioning on conventional supine CT-scan based scapula surface models. The secondary objective was to assess intra-observer repeatability and inter-observer reproducibility of the resulting 3D scapular orientation. It was hypothesized that the 3D scapular orientation reliability would be better or similar than established reliability of intra-operative baseplate positioning (i.e. version and inclination) in reverse total shoulder arthroplasty (rTSA). MethodsThree anatomical landmarks, i.e. acromial angle (AA), inferior angle (IA), and trigonum scapulae (TS), were manually positioned on 81 scapula surface models using updated landmark definitions. These models were obtained by the segmentation of CT-scan images acquired from patients undergoing elective rTSA procedures at the Department of Surgery at Geneva University Hospitals between May 2022 and December 2023. This procedure was repeated 3 times per scapula by 3 independent observers. A set of parameters corresponding to the 3D landmark coordinates (expressed in an average scapula coordinate system) and the 3D scapular orientation (expressed as three angles related to the retraction/protraction, lateral/medial rotation, and internal/external rotation) were computed. Intra-observer repeatability and inter-observer reproducibility were assessed for each of these parameters using an intra-class correlation as a relative reliability metric. Standard error of measurement (SEM) was also reported as an absolute reliability metric. ResultsIntra-observer ICC and inter-observer ICC ranged from poor to moderate for AA, poor to excellent for IA, and poor to excellent for TS. However, low SEM, ranging from 0.4 mm to 2.4 mm for intra-observer repeatability and from 0.2 mm to 1.8 mm for inter-observer reproducibility, were obtained for every coordinate of each anatomical landmark. This results in poor to moderate intra-observer ICC and inter-observer ICC for scapula angular orientation. Again, low SEM, ranging from 0.4° to 0.8° for intra-observer repeatability and 0.4° to 0.6° for inter-observer reproducibility, were obtained, arguing for reliable measurements. Discussion and ConclusionThis study demonstrates that manual positioning of the three scapula anatomical landmarks recommended by the International Society of Biomechanics can be performed in a reliable manner across measures and observers on surface bone models obtained from CT-scan images. However, a clear and standardized definition of these landmarks is needed to ensure consistency across measurements.

Assessment of fetal cardiac function in early fetal life: feasibility, reproducibility, and early fetal nomograms

BackgroundFetal cardiology has shown a rapid development in the last decades. Fetal echocardiography is not only used for the detection of structural anomalies but also to assess fetal cardiac function. Assessment of the fetal cardiac function is performed mostly at second and third trimester. The study of fetal cardiac function at the end of first timester has not been properly investigated and there is a lack of reference values at early gestational weeks. ObjectiveTo assess whether the measurement of time related parameters of cardiac function in the left ventricle of the fetal heart is feasible and reproducible at the end of the first trimester. If possible, we provide nomograms of these parameters from 11 to 13.6 gestational weeks. Study designWe conducted a prospective observational study from March to September 2022. The study was carried out in two hospitals (Hospital Universitari Dexeus, Barcelona and Hospital VITAHS 9 Octubre, Valencia). The scans were performed by 3 specialists in fetal medicine. Exclusion criteria were: fetal cardiac rhythm abnormalities, abnormal nuchal translucency, abnormal ductus venosus, fetal malformations, stillbirth, estimated fetal weight< 10 centile, diabetes and gestational hypertensive disorders. The cardiac function parameters studied in the left ventricle were: isovolumetric contraction time, isovolumetric relaxation time, ejection time, filling time, cycle time, myocardial performance index, ejection time fraction and filling time fraction. We study the feasibility and intra- and inter-observer reproducibility of this parameters using the interclass correlation coefficient (ICC). Nomograms were made and the percentiles of the values of the different parameters were calculated. Results409 cases were recruited but only 296 could be included in the statistical analysis once exclusion criteria were applied. The intra-observer reproducibility study was excellent (ICC>0.900) and the inter-observer reproducibility study was good (ICC>0.700). Data regression analysis showed that cycle time, filling time, isovolumetric contraction time and filling time fraction increased with gestational age, whereas ejection time fraction decreased with gestational age and myocardial performance index (mean 0.43±0.08), isovolumetric relaxation time (mean 0.04±0.01) and ejection time (mean 0.16±0.01) remained constant from 11 to 13 weeks. ConclusionsThe study of fetal cardiac function is feasible and reproducible at 11 to 13+6 gestational weeks. Nomograms of the studied parameters are provided.

7943 Modification of the “Aagl Endometriosis Classification” to a 3-Stage System: Diagnostic Accuracy & Inter-Observer Reproducibility Study

<h3>Study Objective</h3> We aim to assess the performance of a modified AAGL (mAAGL) endometriosis classification system and its inter-observer reproducibility. <h3>Design</h3> Retrospective diagnostic accuracy & inter-observer reproducibility study. <h3>Setting</h3> 2 gynecology units in metropolitan Sydney, Australia. <h3>Patients or Participants</h3> Patients with suspected endometriosis who underwent laparoscopy between January 2016 and October 2021 (n=272). <h3>Interventions</h3> Patients with complete surgical data included. The original 4-stage American Association of Gynecologic Laparoscopists (AAGL) staging system was modified to three stages by merging AAGL stages 2 and 3 and surgical complexity levels B and C. Three observers independently apportioned an AAGL stage 1-3, blinded to surgical skill level (A, B and C). End points were diagnostic performance of each observer to predict level of surgical complexity for each stage, i.e., mAAGL stage 1 to predict level A, mAAGL stage 2 to predict level B, mAAGL stage 3 to predict level C. Inter-observer reproducibility was assessed using OR of a higher grade, between each pair of observers. <h3>Measurements and Main Results</h3> 272 surgical cases analyzed. The diagnostic accuracy for all observers to predict each level in terms of sensitivity, specificity, PPV and NPV were: stage 1 to predict level A: 98.0-98.7%, 60.2-64.2%, 75.0-76.9% & 96.3-97.5%; stage 2 to predict level B: 31.1-36.9%, 97.0-98.2%, 86.5-92.6% & 69.8 – 71.9%; stage 3 to predict level C: 90.0-95.0%, 90.1-91.7%, 41.8 - 47.5% & 99.1-99.6%. Inter-observer reproducibility between observer 1 & 2 was 0.81, between observer 1 & 3 was 0.69 and between observer 2 & 3 was 0.43. Overall percentage agreement between observers 1 & 2 was 93.4%, between 1 & 3 was 93.4% & between 2 & 3 was 96.3%. <h3>Conclusion</h3> The mAAGL system appears to perform well for both low and high stage endometriosis disease. The 4-stage AAGL staging system could be improved by merging AAGL stages 2 and 3 and in turn using a 3-stage system.

OP06.11: Modification of the four‐stage “AAGL endometriosis classification system” to a three‐stage system: diagnostic accuracy and interobserver reproducibility study

OP06.11: Modification of the four‐stage “AAGL endometriosis classification system” to a three‐stage system: diagnostic accuracy and interobserver reproducibility study

CD117, BAP1, MTAP, and TdT Is a Useful Immunohistochemical Panel to Distinguish Thymoma from Thymic Carcinoma.

Simple SummaryThymic epithelial tumors, including thymomas and thymic carcinomas are malignant neoplasms that occur in the prevascular (anterior) mediastinum. Thymic carcinomas have a worse outcome than thymomas, and, therefore, management and treatment may differ. However, the morphologic distinction between thymomas and thymic carcinomas can be challenging, as has also been shown in reproducibility studies. We aimed to identify immunohistochemical markers that aid in the distinction between thymomas and thymic carcinomas. We found that a panel of CD117 with a cut-off of 10% tumor cell expression, BAP1, mTAP, and TdT was able to predict 88.9% of thymic carcinomas and 77.8% of thymomas (among the studied types A and B3 thymomas and micronodular thymomas with lymphoid stroma). Only a small subset of thymic carcinomas (11.1%) and thymomas (22.2%) could not be predicted with that model.Background: The morphologic distinction between thymic carcinomas and thymomas, specifically types B3, A, and occasionally micronodular thymomas with lymphoid stroma (MNTLS) can be challenging, as has also been shown in interobserver reproducibility studies. Since thymic carcinomas have a worse prognosis than thymomas, the diagnosis is important for patient management and treatment. This study aimed to identify a panel of immunohistochemical (IHC) markers that aid in the distinction between thymomas and thymic carcinomas in routine practice. Materials and Method: Thymic carcinomas, type A and B3 thymomas, and MNTLS were identified in an institutional database of thymic epithelial tumors (TET) (1963–2021). IHC was performed using antibodies against TdT, Glut-1, CD5, CD117, BAP1, and mTAP. Percent tumor cell staining was recorded (Glut-1, CD5, CD117); loss of expression (BAP1, mTAP) was considered if essentially all tumor cells were negative; TdT was recorded as thymocytes present or absent (including rare thymocytes). Results: 81 specimens included 44 thymomas (25 type A, 11 type B3, 8 MNTLS) and 37 thymic carcinomas (including 24 squamous cell carcinomas). Using BAP1, mTAP, CD117 (cut-off, 10%), and TdT, 88.9% of thymic carcinomas (95.7% of squamous cell carcinomas) and 77.8% of thymomas could be predicted. Glut-1 expression was not found to be useful in that distinction. All tumors that expressed CD5 in ≥50% of tumor cells also expressed CD117 in ≥10% of tumor cells. In four carcinomas with homozygous deletion of CDKN2A, mTAP expression was lost in two squamous cell carcinomas and in a subset of tumor cells of an adenocarcinoma and was preserved in a lymphoepithelial carcinoma. Conclusion: A panel of immunostains including BAP1, mTAP, CD117 (using a cut-off of 10% tumor cell expression), and TdT can be useful in the distinction between thymomas and thymic carcinomas, with only a minority of cases being inconclusive.

Reproducibility of Systolic Strain in Mice Using Cardiac Magnetic Resonance Feature Tracking of Black-Blood Cine Images.

Mouse models are widely utilized to enhance our understanding of cardiac disease. The goal of this study is to investigate the reproducibility of strain parameters that were measured in mice using cardiac magnetic resonance (CMR) feature-tracking (CMR42, Canada). We retrospectively analyzed black-blood CMR datasets from thirteen C57BL/6 B6.SJL-CD45.1 mice (N=10 female, N=3 male) that were imaged previously. The circumferential, longitudinal, and radial (Ecc, Ell, and Err, respectively) parameters of strain were measured in the mid-ventricular region of the left ventricle. Intraobserver and interobserver reproducibility were assessed for both the end-systolic (ES) and peak strain. The ES strain had larger intraclass correlation coefficient (ICC) values when compared to peak strain, for both the intraobserver and interobserver reproducibility studies. Specifically, the intraobserver study showed excellent reproducibility for all three ES strain parameters, namely, Ecc (ICC 0.95, 95% CI 0.83-0.98), Ell (ICC 0.90, 95% CI 0.59-0.97), and Err (ICC 0.92, 95% CI 0.73-0.97). This was also the case for the interobserver study, namely, Ecc (ICC 0.92, 95% CI 0.60-0.98), Ell (ICC 0.76, 95% CI 0.33-0.93), and Err (ICC 0.93, 95% CI 0.68-0.98). Additionally, the coefficient of variation values were all <10%. The results of this preliminary study showed excellent reproducibility for all ES strain parameters, with good to excellent reproducibility for the peak strain parameters. Moreover, all ES strain parameters had larger ICC values than the peak strain. In general, these results imply that feature-tracking with CMR42 software and black-blood cine images can be reliably used to assess strain patterns in mice.

Selection of Representative Histologic Slides in Interobserver Reproducibility Studies: Insights from Expert Review for Ovarian Carcinoma Subtype Classification

Background: Observer studies in pathology often utilize a limited number of representative slides per case, selected and reported in a nonstandardized manner. Reference diagnoses are commonly assumed to be generalizable to all slides of a case. We examined these issues in the context of pathologist concordance for histologic subtype classification of ovarian carcinomas (OCs). Materials and Methods: A cohort of 114 OCs consisting of 72 cases with a single representative slide (Group 1) and 42 cases with multiple representative slides (148 slides, 2–6 sections per case, Group 2) was independently reviewed by three experts in gynecologic pathology (case-based review). In a follow-up study, each individual slide was independently reviewed in a randomized order by the same pathologists (section-based review). Results: Average interobserver concordance varied from 100% for Group 1 to 64.3% for Group 2 (86.8% across all cases). Across Group 2, 19 cases (45.2%) had at least one slide classified as a different subtype than the subtype assigned from case-based review, demonstrating the impact of intratumoral heterogeneity. Section-based concordance across individual sections from Group 2 was comparable to case-based concordance for those cases indicating diagnostic challenges at the individual section level. Findings demonstrate the increased diagnostic complexity of heterogeneous tumors that require multiple section sampling and its impact on pathologist performance. Conclusions: The proportion of cases with multiple representative slides in cohorts used in validation studies, such as those conducted to evaluate artificial intelligence/machine learning tools, can influence diagnostic performance, and if not accounted for, can cause disparities between research and real-world observations and between research studies. Case selection in validation studies should account for tumor heterogeneity to create balanced datasets in terms of diagnostic complexity.

Cytology Reporting System for Lung Cancer from the Japan Lung Cancer Society and Japanese Society of Clinical Cytology: An Interobserver Reproducibility Study and Risk of Malignancy Evaluation on Cytology Specimens

Introduction: The classification of lung carcinoma is based on small biopsies and/or cytology in 80% of patients with non-small cell carcinoma. However, there is no widely accepted classification system for respiratory cytology. The Japan Lung Cancer Society (JLCS) and Japanese Society of Clinical Cytology (JSCC) have proposed a new four-tiered cytology reporting system for lung carcinoma with the following categories: (1) “negative for malignancy,” (2) “atypical cells,” (3) “suspicious for malignancy,” and (4) “malignancy.” Objective: The aim of this work was to perform an interobserver reproducibility study to confirm the utility of the four-tiered reporting system on respiratory cytological samples. Methods: We analyzed 90 cytological samples obtained with bronchoscopy. Seven observers classified these cases into each category by reviewing one Papanicolaou-stained slide per case according to the three-, four-, and five-tiered reporting systems. Results: The interobserver agreement was fair in the three- (κ = 0.50), four- (κ = 0.45), and five-tiered (κ = 0.45) reporting systems. However, the four-tiered reporting system provided more precise information than the three-tiered reporting system in patient management. The risk of malignancy in the four-tiered reporting system was also stratified well: 19.3% for “negative for malignancy,” 45.6% for “atypical cells,” 74.7% for “suspicious for malignancy,” and 88.1% for “malignancy.” Conclusions: The reporting system proposed by the JLCS and JSCC was designed to enhance the communication between clinicians and pathologists and among different institutions. It is simple and applicable to cytological diagnosis of any respiratory diseases. We propose establishing an international classification for respiratory cytology, harmonizing the reporting systems proposed by different countries.

European guidelines on quality criteria for diagnostic radiographic images of the lumbar spine \u2013 an intra- and inter-observer reproducibility study

BackgroundThe Commission of the European Communities has published guidelines to be used as a gold standard for quality assessment of diagnostic radiographic images. Image quality and radiation dose must be monitored and optimally balanced for diagnostic purposes on patients. The objective of the current study was to assess intra- and inter-observer reproducibility in less experienced observers using the proposed European Guidelines on Quality Criteria for Diagnostic Radiographic Images in a quality assessment of lumbar spine radiographs in primary chiropractic practice in Denmark.MethodsTwo observers initially evaluated lumbar spine radiographs randomly selected from fifty chiropractic clinics, all connected to the national PACS server (KirPACS) in Denmark. All evaluations were performed twice by both observers using a four-week interval and for compliance with the European Quality Criteria for Diagnostic Radiographic Images. Inter- and intra-observer reproducibility was calculated using kappa statistics. In the interpretation of the kappa coefficient, the standards for strength of agreement reported by Landis and Koch were followed.ResultsThe strength of the inter-observer agreement of general image quality at baseline ranged from moderate agreement (k = 0.47) to substantial agreement (k = 0.68). After four weeks, the inter-observer agreement still ranged from moderate agreement (k = 0.59) to substantial agreement (k = 0.71), but with increased agreement for both kappa coefficients. In relation to intra-observer agreement of general image quality, the strength for observer A ranged from moderate (k = 0.58) to substantial (k = 0.72) and the strength for observer B overall was substantial (k = 0.63–0.75).ConclusionThe European Guidelines on Quality Criteria for Diagnostic Radiographic Images are considered a gold-standard and used in a method for quality assurance within the Danish chiropractic profession. The inter-rater and intra-rater agreements in this study, using the CEC-criteria, were found mostly acceptable. With appropriate attention to clear understanding of the individual criteria and sufficient training, this method is found to be reliable, even using less experienced observers, to carry out Diagnostic Radiographic Image Quality-assurance in primary care settings.

Clinical Practice Impact of the Boston Bowel Preparation Scale in a European Country

Introduction: The diagnostic acuity of colonoscopy requires a careful evaluation of the colonic mucosa, so an adequate bowel cleansing is a key element of the procedure. It is internationally recommended that an evaluation of the quality of the intestinal preparation should be included in the colonoscopy report. The Boston Bowel Preparation Scale (BBPS) is a bowel cleanliness rating scale that has gained some preponderance in this context. However, its application implies some degree of subjective appreciation, and it is important to conduct interobserver reproducibility studies in different contexts. The objective of the present study was to evaluate the reliability of the BBPS in the Portuguese gastroenterological community. Methods: A prospective study involving Portuguese gastroenterologists with clinical practice in several contexts, and using different methods of evaluation of the intestinal preparation. Participants were invited to answer a questionnaire encompassing 93 static images and 12 video segments of colonoscopy examinations, with different levels of intestinal preparation. For each item, the participant gastroenterologist assigned a score of 0-3, according to the BBPS criteria. A statistical analysis was performed with SPSS 20.0 software, using the intraclass correlation coefficient (ICC). Results: From 45 invited gastroenterologists, 36 replied (mean age 39 ± 9 years). Fifteen (41%) had more than 10 years of colonoscopy experience and 20 (56%) performed more than 40 examinations per month. Twenty-seven (77%) usually used the BBPS in their daily practice. Statistical analysis revealed a strong interobserver correlation (ICC = 0.783) in the application of the BBPS, even in those gastroenterologists who did not use the scale in their daily routine (ICC = 0.775) and those with less years of experience (ICC = 0.820). The correlation in the videos was slightly lower than that observed in the static images (ICC = 0.74 vs. ICC = 0.78). Conclusion: The application of the BBPS in the Portuguese gastroenterology community is reproducible and can represent a way to harmonize the colonoscopy reports, contributing to its correct interpretation and subsequent patient orientation.

Bethesda Interobserver Reproducibility Study-2 (BIRST-2): Bethesda System 2014

In concert with the 2014 update to the Bethesda System for Reporting Cervical Cytology, a Web-based image interobserver study was performed to evaluate concordance with the "expert panel" interpretation, as was done during the Bethesda 2001 update. The aim was to identify cytomorphologic features and Bethesda reporting categories that represent sources of poor interobserver agreement and see how the trends compared to the first Bethesda Interobserver Reproducibility Study (BIRST). Participants were recruited online through national and international cytopathology professional societies. Study participants evaluated 84 previously unpublished web images chosen from the third Bethesda Atlas image set, prior to the release of the atlas. These images spanned all reporting categories and included typical and borderline cytomorphology. Demographic information was collected on level of training, practice patterns, and experience of the participants. Participation was restricted to those correctly answering 2 basic cytopathology questions, ensuring minimal knowledge of gynecologic cytopathology. A total of 1290 unique individuals attempted access to this Web-based study and 833 correctly answered the two qualifying questions. Of these, 518 respondents completed the survey. Participant origin included: 59% United States, 41% international; 48% cytotechnologists, 41% pathologists, 5% fellows, and 6% other. Practice types were: 39% academic institutions, 29% private hospitals, and 16% commercial laboratories. Overall, the mean participant agreement with the exact Bethesda panel interpretation was 62.8%. The best agreement was found for negative for intraepithelial lesion or malignancy (NILM; 74%) and low-grade squamous intraepithelial lesion (LSIL; 86%) categories. Squamous cell carcinoma (SCC) (63%), high-grade squamous intraepithelial lesion (HSIL; 60%), atypical squamous cells of undetermined significance (ASC-US; 62%) and atypical squamous cells, cannot exclude high-grade squamous intraepithelial lesion (ASC-H; 60%) showed slightly lower concordance with the panel interpretations. Cervical glandular lesions were more problematic (33%). Anal samples performed similarly to their gynecologic counterparts. There was similar diagnostic agreement across participant certifications and practice type (academic versus non-academic). Performance was higher for United States and other North America-based participants (P = 0.0104). This significance may be attributed to a language bias, as the survey was only offered in English. Similar to the BIRST-1 study conducted in 2001, the most important factor for diagnostic agreement by cytotechnologists, pathologists, and trainees was the a priori difficulty of an image rather than participant training, certification, or experience. Participants showed better general diagnostic agreement with the expert panel interpretations of the material in BIRST-2 than in BIRST-1. Agreement was highest for Bethesda categories of NILM, LSIL, HSIL, and SCC. Concordance for even the borderline ASC-US and ASC-H categories exhibited remarkable improvement in the BIRST-2.

P2.01-047 Intra- and Inter-Observer Reproducibility Study of PD-L1 Biomarker in Non-Small Cell Lung Cancer (NSCLC) - The DREAM STUDY

PD-L1 expression in NSCLC correlates with increased response to pembrolizumab, supporting its role as a predictive biomarker but the reproducibility of pathologists’ scoring of PD-L1 requires further investigation. The primary objective of the DREAM study was to assess the reproducibility of scoring PD-L1 staining by evaluating intra- and inter-observer reproducibility for the assessment of PD-L1 expression in NSCLC. The secondary objective was to assess the impact of training on reproducibility. The study was a blinded, pathologist reproducibility study of scoring PD-L1 expression in NSCLC cases stained with PD-L1 22C3 pharmDx™ kit using the Dako Automated Link 48 Platform. Two pathologists previously trained and certified by Dako scored 789 specimens to form the gold standard. From these specimens 60 were randomly selected to evaluate a 1% cut-point and 60 for a 50% cut-point. Both sample sets were designed to include 50% positive/negative specimens and 20-30 close to each cut-point. Ten pathologists were randomly assigned to two subgroups. Subgroup 1 analyzed all samples on two consecutive days. Subgroup 2 performed the same assessments, except they received a one hour training session prior to the second assessment. The overall percent agreement (OPA) for the analysis of the intra-observer reproducibility was 89.7% (95% CI: 85.7; 92.6) for the 1% cut-point sample set and 91.3% (95% CI: 87.6; 94.0) for the 50% cut-point. The OPAs for inter-observer reproducibility of all ten pathologists were 84.2% (95% CI: 82.8; 85.5) and 81.9% (95% CI: 80.4; 83.3) for the 1% and 50% cut-point sample sets, respectively. There was substantial agreement at both the 1% cut-point (prevalence-adjusted bias-adjusted kappa 0.68 (95% CI: 0.65; 0.71)) and the 50% cut-point (prevalence-adjusted bias-adjusted kappa 0.64 (95% CI: 0.61; 0.67)). Training was found to have no or very little impact on the inter- or intra-observer reproducibility in subgroup 2. The OPAs for the inter-observer reproducibility assessments were 82.0% and 82.3% for the first and second assessments of the 1% cut-point sample set, respectively, and 78.3% and 81.7% for the first and second assessments of the 50% cut-point sample set, respectively. The exploratory analyses showed that the sensitivity and specificity of the pathologists assessments, compared with the gold standard assessment, were 84.3% and 91.3%, respectively, for the 1% cut-point and 56.3% and 94.0%, respectively, for the 50% cut-point. There is high intra-observer reproducibility and substantial inter-observer agreement in pathologists’ assessment of PD-L1 expression in NSCLC at 1% and 50% cut-points.

The carotid artery as an alternative site for dynamic autoregulation measurement: an inter-observer reproducibility study

The internal carotid artery (ICA) has been proposed as an alternative site to the middle cerebral artery (MCA) to measure dynamic cerebral autoregulation (dCA) using transcranial Doppler ultrasound (TCD). Our aim was to test the inter-operator reproducibility of dCA assessment in the ICA and the effect of interaction amongst different variables (artery source × operator × intra-subject variability). Two operators measured blood flow velocity using TCD at the ICA and MCA simultaneously on each side in 12 healthy volunteers. The autoregulation index (ARI) was estimated by transfer function analysis. A two-way repeated measurements ANOVA with post-hoc Tukey tested the difference between ARI by different operators and interaction effects were analysed based on the generalized linear model. In this healthy population, no significant differences between operator and no interaction effects were identified amongst the different variables. This study reinforced the validity of using the ICA as an alternative site for the assessment of dCA. Further work is needed to confirm and extend our findings, particularly to disease populations.

Diagnosis of Gleason pattern 5 prostate adenocarcinoma on core needle biopsy: an interobserver reproducibility study among urologic pathologists.

Accurate recognition of Gleason pattern 5 (GP5) prostate adenocarcinoma on needle biopsy is critical as it is associated with disease progression and adverse clinical outcome. Despite important implications of this diagnosis, interobserver variation in the diagnosis of GP5 has not been adequately studied. Digital images of 66 prostate adenocarcinoma cases that potentially contained a GP5 component were distributed to 16 urologic pathologists who were asked to classify whether GP5 was present. Each image was initially classified into 1 of 4 morphologic subpatterns by 2 coauthors (R.B.S. and M.Z.): solid nests (15), comedocarcinoma (8), single cells and/or cords (35), and variant morphology (8). Additional features captured included: size (large: >20 cells, medium: 10 to 20 cells, and small: <10 cells) and distribution of nuclei (uniform vs. nonuniform) for nests pattern; intraluminal coagulative tumor necrosis, karyorrhectic debris, and amorphous material for comedocarcinoma pattern; and quantity (≤5, 6 to 10, and >10) and distribution (clustered vs. intermixed with adjacent well-formed glands) for single cells/cords pattern. Interobserver reproducibility of a diagnosis of GP5 was assessed and the morphologic subpatterns and features were correlated with the consensus diagnosis (defined as 75% agreement). Interobserver reproducibility for overall diagnostic agreement was fair (κ=0.376). Among subpatterns, comedocarcinoma had highest reproducibility (κ=0.499), followed by variant morphology (κ=0.443), single cells/cords (κ=0.369), and nests (κ=0.347). All cases with the following features achieved consensus for GP5: large nests regardless of nuclear distribution; coagulative necrosis with or without karyorrhectic debris; single cells/cords >10 or 6 to 10 in a cluster; and signet ring-like cells in single cells or within nests pattern. A majority of cases with the following features achieved consensus against GP5: medium-size nests; exclusive intraluminal amorphous material; single cells/cords ≤5; and Paneth cell change. Remaining morphologic features did not reach consensus for or against GP5. A majority (86%) of participants would diagnose a small focus of GP5 only when it is present in >1 level. The diagnostic reproducibility of GP5 within certain morphologies was only fair among urologic pathologists. However, the diagnosis of GP5 was more reproducible when certain restrictive morphologic and quantitative criteria were applied. These findings suggest that additional studies are needed to find highly reproducible features of GP5 associated with documented aggressive clinical outcome.

Intra- and inter-observer reproducibility study of gestational age estimation using three common foetal biometric parameters: Experienced versus inexperienced sonographer

AimTo assess reproducibility of estimating gestational age (GA) of foetus using femur length (FL), biparietal diameter (BPD) and abdominal circumference (AC) within experienced and inexperienced sonographers and between the two. Patients and methodsTwo sets of GA estimates each were obtained for FL, BPD and AC by the two observers in 20 normal singleton foetuses. The first estimates for the three biometric parameters were made by the experienced sonographer. Subsequently, the inexperienced sonographer, blind to the estimates of the first observer obtained his own estimates for the same biometric parameters. After a time interval of ten minutes the process was repeated for the second set of GA estimates. All the gestational age estimates were made following standard protocol. Statistical analysis was performed by Pearson's and intraclass correlations, coefficient of variation and Bland–Altman plots. Statistical inferences were drawn at p < 0.05. ResultsThe Pearson's and intraclass correlations of between GA estimates within and between both observers from measurement of FL, BPD and AC were very high and statistically significant (p < 0.05). Coefficient of variation for duplicate measurements for GA estimates within observers and between observers were quite negligible. Between observers, the first and second GA estimates from FL measurements showed the least variation. Estimates from BPD and AC measurements showed greater degree of variation between the observers. ConclusionReproducibility of GA estimation using FL, BPD and AC within experienced and inexperienced sonographers and between the two was excellent. Therefore, a fresh Nigerian radiography graduate with adequate exposure in obstetric ultrasound can correctly determine the gestational age of foetus in routine obstetric ultrasound without supervision.

The Interobserver Reproducibility of Thyroid Fine-Needle Aspiration Using the UK Royal College of Pathologists’ Classification System

The overall interobserver reproducibility of thyroid fine-needle aspiration (FNA) has not been comprehensively assessed. A blinded 6-rater interobserver reproducibility study was conducted of 200 thyroid FNA cases using the UK System, which is similar to The Bethesda System for Reporting Thyroid Cytology: Thy1, nondiagnostic; Thy2, nonneoplastic; Thy3a, atypia, probably benign; Thy3f, follicular lesion; Thy4, suspicious of malignancy; and Thy5, malignant. There was good interobserver agreement for the Thy1 (κ = 0.69) and Thy5 (κ = 0.61), moderate agreement for Thy2 (κ = 0.55) and Thy3f (κ = 0.51), and poor agreement for Thy3a (κ = 0.11) and Thy4 (κ = 0.17) categories. Combining categories implying surgical management (Thy3f, Thy4, and Thy5) achieved good agreement (κ = 0.72), as did combining categories implying medical management (Thy1, Thy2, and Thy3a; κ = 0.72). The UK thyroid FNA terminology is a reproducible and clinically relevant system for thyroid FNA reporting. This study demonstrates that international efforts to harmonize and refine thyroid cytology classification systems can improve consistency in the clinical management of thyroid nodules.

Educational impact of discussing results among nuclear medicine physicians participating in interobserver reproducibility studies

In nuclear medicine, reproducibility studies allow the assessment of the robustness of a technique. They could also be used in continuing education. This study evaluates this educational impact. Two series of clinical data and I-123 fluoropropyl-carbomethoxy-iodophenyl-nortropane single photon emission tomography images from 12 patients were prepared before initiating the study. Each series covered similarly a wide spectrum of clinical situations and images. Nuclear medicine physicians having experience with I-123 fluoropropyl-carbomethoxy-iodophenyl-nortropane single photon emission tomography were recruited and assigned two similar groups regarding their level of experience and type of institutions from which they were issued. Clinical data and images were transmitted by e-mail. For each case, observers had to choose among three answers: normal, equivocal, and abnormal. Answers were returned anonymously. Before the second series was sent, observers from group A had a collegiate discussion on the results of the first series of cases. Observers from group B received the second series without any information regarding analysis of series 1. In group A, median agreement increased from 75% (series 1) to 100% (series 2); in group B it was 75% (series 1) and 87% (series 2). In group A, a 100% agreement was observed for four cases (series 1) and for eight cases (series 2). In group B, a 100% agreement was observed for four cases for both series. These preliminary results, obtained in a small sample of observers, suggest that participating and discussing results of interobserver reproducibility studies seems to have a positive educational impact and therefore improves interobserver reproducibility.

Should we stop performing nuclear medicine procedures?

Should we stop performing nuclear medicine procedures?

The important role of audits, surveys and interobserver reproducibility studies in continuing education in nuclear medicine

The important role of audits, surveys and interobserver reproducibility studies in continuing education in nuclear medicine

51. Quantification of the availability of dopamine transporters: an intra- and inter-observer reproducibility study

s of the 31st Annual Meeting of the British Nuclear Medicine Society, Manchester, UK, 29 April to 1 May 2003 - TECHNOLOGISTS I