Inter-grader Variability Research Articles

DSMT-Net: Dual Self-Supervised Multi-Operator Transformation for Multi-Source Endoscopic Ultrasound Diagnosis.

Pancreatic cancer has the worst prognosis of all cancers. The clinical application of endoscopic ultrasound (EUS) for the assessment of pancreatic cancer risk and of deep learning for the classification of EUS images have been hindered by inter-grader variability and labeling capability. One of the key reasons for these difficulties is that EUS images are obtained from multiple sources with varying resolutions, effective regions, and interference signals, making the distribution of the data highly variable and negatively impacting the performance of deep learning models. Additionally, manual labeling of images is time-consuming and requires significant effort, leading to the desire to effectively utilize a large amount of unlabeled data for network training. To address these challenges, this study proposes the Dual Self-supervised Multi-Operator Transformation Network (DSMT-Net) for multi-source EUS diagnosis. The DSMT-Net includes a multi-operator transformation approach to standardize the extraction of regions of interest in EUS images and eliminate irrelevant pixels. Furthermore, a transformer-based dual self-supervised network is designed to integrate unlabeled EUS images for pre-training the representation model, which can be transferred to supervised tasks such as classification, detection, and segmentation. A large-scale EUS-based pancreas image dataset (LEPset) has been collected, including 3,500 pathologically proven labeled EUS images (from pancreatic and non-pancreatic cancers) and 8,000 unlabeled EUS images for model development. The self-supervised method has also been applied to breast cancer diagnosis and was compared to state-of-the-art deep learning models on both datasets. The results demonstrate that the DSMT-Net significantly improves the accuracy of pancreatic and breast cancer diagnosis.

Extracting spacing-derived estimates of rod density in healthy retinae.

Quantification of the rod photoreceptor mosaic using adaptive optics scanning light ophthalmoscopy (AOSLO) remains challenging. Here we demonstrate a method for deriving estimates of rod density and rod:cone ratio based on measures of rod spacing, cone numerosity, and cone inner segment area. Twenty-two AOSLO images with complete rod visualization were used to validate this spacing-derived method for estimating density. The method was then used to estimate rod metrics in an additional 105 images without complete rod visualization. The spacing-derived rod mosaic metrics were comparable to published data from histology. This method could be leveraged to develop large normative databases of rod mosaic metrics, though limitations persist with intergrader variability in assessing cone area and numerosity.

Repeatability of Choroidal Thickness Measurements in Healthy Subjects using RTVue XR Optical Coherence Tomography.

To determine the intrasession, intraobserver, interobserver, and repeatability of choroidal thickness measurements in the healthy subjects imaged on enhanced depth imaging system of RTVue XR spectral domain optical coherence tomography (OCT). In this prospective, cross-sectional study, seventy eyes of 70 healthy volunteers with no known ocular disease were imaged using high-density scanning protocol of RTVue XR OCT. In a single imaging session, three sequential 12 mm macular-enhanced depth horizontal line scans were obtained through the fovea. Two experienced examiners measured the subfoveal choroidal thickness (SFCT), choroidal thickness at 500 μm nasally and temporally from the fovea in all the eyes, using the manual calipers provided in the software. The graders were masked to each other's measurement readings. The coefficient of repeatability (CR) and intraclass correlation coefficient (ICC) were used to measure the reliability within graders. Intergrader variability was assessed using Bland-Altman method and 95% limits of agreement (LoA). Intragrader CR for grader one was 4.11 μm (95% confidence interval [CI], -2.84-11.06) for SFCT and 5.73 μm (95% CI, -3.71-15.16) for the grader two. Intragrader ICC of grader one ranged from 0.996 for SFCT to 0.994 for temporal choroidal thickness. Intragrader ICC of grader two ranged from 0.993 for temporal choroidal thickness to 0.991 for SFCT. Intergrader CR ranged from 5.24 μm (95% CI, -4.66-15.15) for SFCT to 5.89 μm (95% CI, -7.27-19.04) for temporal choroidal thickness. Intergrader 95% LoA for SFCT, nasal and temporal choroidal thickness were, -15.84-12.15 μm, -15.99-17.7 μm, and - 19.12-15.57 μm, respectively. Choroidal thickness measurements can be quantified with good repeatability using RTVue XR OCT, which would be useful in patients with chorioretinal diseases.

Artificial Intelligence for Retinopathy of Prematurity: Validation of a Vascular Severity Scale against International Expert Diagnosis

To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee. Validation study of an AI-based ROP vascular severity score. A total of 34 ROP experts from the ICROP3 committee. Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts. The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.

Management of Type 2 Retinopathy of Prematurity or Less in Infants Aged 45 Weeks Postmenstrual Age or Older.

This study aimed to identify the degree of concordance between fluorescein angiograms (FA) and fundus photographs (FP) in assessing the severity and potential need for treatment in infants 45 weeks or older postmenstrual age (PMA) with type 2 or less retinopathy of prematurity (ROP). An observational retrospective case series performed at Associated Retinal Consultants, William Beaumont Hospital in Royal Oak, Michigan. All infants born between 2006 and 2016 with stage 1 to 3 ROP that did not meet type 1 ROP criteria (type 2 or less) who received ablative laser therapy during or after age 45 weeks PMA. Pretreatment FP and FA images were randomized and sent to nine expert retina specialist graders to assess severity and inter-grader variability. A total of 10 babies (19 eyes) were enrolled in this study, and 53 FAs and 27 FPs of these 19 eyes were selected to be interpreted by the nine graders. The number of eyes deemed to be abnormal and warranted for treatment was higher with FA, whereas more eyes were deemed "normal" with FP. Although still controversial, knowledge of these findings may encourage retina specialists to closely examine infants with mild ROP older than age 45 weeks PMA and consider ablative laser therapy under certain conditions (even if not meeting type 1 Early Treatment for ROP criteria). [Ophthalmic Surg Lasers Imaging Retina. 2021;52:636-641.].

Automated segmentation of choroidal layers from 3-dimensional macular optical coherence tomography scans

BackgroundChanges in choroidal thickness are associated with various ocular diseases, and the choroid can be imaged using spectral-domain optical coherence tomography (SD-OCT) and enhanced depth imaging OCT (EDI-OCT). New methodEighty macular SD-OCT volumes from 80 patients were obtained using the Zeiss Cirrus machine. Eleven additional control subjects had two Cirrus scans done in one visit along with enhanced depth imaging (EDI-OCT) using the Heidelberg Spectralis machine. To automatically segment choroidal layers from the OCT volumes, our graph-theoretic approach was utilized. The segmentation results were compared with reference standards from two independent graders, and the accuracy of automated segmentation was calculated using unsigned/signed border positioning/thickness errors and Dice similarity coefficient (DSC). The repeatability and reproducibility of our choroidal thicknesses were determined by intraclass correlation coefficient (ICC), coefficient of variation (CV), and repeatability coefficient (RC). ResultsThe mean unsigned/signed border positioning errors for the choroidal inner and outer surfaces are 3.39 ± 1.26 µm (mean ± standard deviation)/− 1.52 ± 1.63 µm and 16.09 ± 6.21 µm/4.73 ± 9.53 µm, respectively. The mean unsigned/signed choroidal thickness errors are 16.54 ± 6.47 µm/6.25 ± 9.91 µm, and the mean DSC is 0.949 ± 0.025. The ICC (95% confidence interval), CV, RC values are 0.991 (0.977–0.997), 2.48%, 14.25 µm for the repeatability and 0.991 (0.977–0.997), 2.49%, 14.30 µm for the reproducibility studies, respectively. Comparison with existing method(s)The proposed method outperformed our previous method using choroidal vessel segmentation and inter-grader variability. ConclusionsThis automated segmentation method can reliably measure choroidal thickness using different OCT platforms.

An automated computational image analysis pipeline for histological grading of cardiac allograft rejection.

AimAllograft rejection is a serious concern in heart transplant medicine. Though endomyocardial biopsy with histological grading is the diagnostic standard for rejection, poor inter-pathologist agreement creates significant clinical uncertainty. The aim of this investigation is to demonstrate that cellular rejection grades generated via computational histological analysis are on-par with those provided by expert pathologistsMethods and resultsThe study cohort consisted of 2472 endomyocardial biopsy slides originating from three major US transplant centres. The ‘Computer-Assisted Cardiac Histologic Evaluation (CACHE)-Grader’ pipeline was trained using an interpretable, biologically inspired, ‘hand-crafted’ feature extraction approach. From a menu of 154 quantitative histological features relating the density and orientation of lymphocytes, myocytes, and stroma, a model was developed to reproduce the 4-grade clinical standard for cellular rejection diagnosis. CACHE-grader interpretations were compared with independent pathologists and the ‘grade of record’, testing for non-inferiority (δ = 6%). Study pathologists achieved a 60.7% agreement [95% confidence interval (CI): 55.2–66.0%] with the grade of record, and pair-wise agreement among all human graders was 61.5% (95% CI: 57.0–65.8%). The CACHE-Grader met the threshold for non-inferiority, achieving a 65.9% agreement (95% CI: 63.4–68.3%) with the grade of record and a 62.6% agreement (95% CI: 60.3–64.8%) with all human graders. The CACHE-Grader demonstrated nearly identical performance in internal and external validation sets (66.1% vs. 65.8%), resilience to inter-centre variations in tissue processing/digitization, and superior sensitivity for high-grade rejection (74.4% vs. 39.5%, P < 0.001).ConclusionThese results show that the CACHE-grader pipeline, derived using intuitive morphological features, can provide expert-quality rejection grading, performing within the range of inter-grader variability seen among human pathologists.

Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning.

The clinical application of breast ultrasound for the assessment of cancer risk and of deep learning for the classification of breast-ultrasound images has been hindered by inter-grader variability and high false positive rates and by deep-learning models that do not follow Breast Imaging Reporting and Data System (BI-RADS) standards, lack explainability features and have not been tested prospectively. Here, we show that an explainable deep-learning system trained on 10,815 multimodal breast-ultrasound images of 721 biopsy-confirmed lesions from 634 patients across two hospitals and prospectively tested on 912 additional images of 152 lesions from 141 patients predicts BI-RADS scores for breast cancer as accurately as experienced radiologists, with areas under the receiver operating curve of 0.922 (95% confidence interval (CI) = 0.868-0.959) for bimodal images and 0.955 (95% CI = 0.909-0.982) for multimodal images. Multimodal multiview breast-ultrasound images augmented with heatmaps for malignancy risk predicted via deep learning may facilitate the adoption of ultrasound imaging in screening mammography workflows.

Validation of virtual reality orbitometry bridges digital and physical worlds

Clinical science and medical imaging technology are traditionally displayed in two dimensions (2D) on a computer monitor. In contrast, three-dimensional (3D) virtual reality (VR) expands the realm of 2D image visualization, enabling an immersive VR experience with unhindered spatial interaction by the user. Thus far, analysis of data extracted from VR applications was mainly qualitative. In this study, we enhance VR and provide evidence for quantitative VR research by validating digital VR display of computed tomography (CT) data of the orbit. Volumetric CT data were transferred and rendered into a VR environment. Subsequently, seven graders performed repeated and blinded diameter measurements. The intergrader variability of the measurements in VR was much lower compared to measurements in the physical world and measurements were reasonably consistent with their corresponding elements in the real context. The overall VR measurements were 5.49% higher. As such, this study attests the ability of VR to provide similar quantitative data alongside the added benefit of VR interfaces. VR entails a lot of potential for the future research in ophthalmology and beyond in any scientific field that uses three-dimensional data.

Clinical utility of circulating genetically abnormal cells within low-dose computed tomography lung cancer screening: A correlative MCPND trial study.

e15536 Background: Recent screening trial results using low-dose computed tomography (LDCT) have been shown to reduce lung cancer mortality in high-risk patients. However, high false-positive rates, inter-grader variability, and radiation injuries are challenging, and therefor highlight the necessity for complementary biomarkers. The diagnostic performance of circulating genetically abnormal cells (CACs) was respectively evaluated in samples collected from patients with pulmonary nodules before and after lung lobectomy within the Multicenter Chinese Pulmonary Nodule Detection (MCPND) trial. We also propose a deep learning that detects related genetically abnormal cells in excisions of symptomatic lesions to identify the source of CACs. Methods: 269 plasma samples (n = 179, before surgery; n = 90, after surgery) from 179 participants, including 125 patients with lung cancer and 54 donors with benign nodular lesions, were analyzed by using a FISH–based assay for 3p22.1, 3q29, 10q22_3 and CEP10. Diagnostic performance of CAC was evaluated in a blinded validation group. 109 paraffin-embedded specimens (45 benign nodular lesions, 64 malignancies) obtained from participants were tested in the same way to identify the genetically abnormal cells. Results: The diagnostic performance of CACs for lung cancer detection was 91% for sensitivity and 81% for specificity. For all participants, CACs had a positive predictive value of 92% and 80% for negative predictive value, respectively. Area under the ROC curve was 0.89 (P &lt; 0.01; 95%CI (0.82, 0.96)). 90 plasma samples were drawn from patients within 10 days after surgery. Of these samples, 58 participants’ CACs level reduced and 25 of them remained stable. Moreover, of 64 malignant specimens, genetically abnormal cells were detected in 60 specimens. However, only 4 specimens were detected having genetically abnormal cells in 45 benign nodular lesions. Conclusions: This validation study indicated that the number of genetically abnormal cells in circulation decreased after surgical resection, which suggested CACs could be released from lung cancer. CACs have diagnostic and prognostic value and could assist to improve the diagnostic accuracy in early stage lung cancer after low-dose computed tomography. Clinical trial information: ChiCTR1900026233.

End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.

With an estimated 160,000 deaths in 2018, lung cancer is the most common cause of cancer death in the United States1. Lung cancer screening using low-dose computed tomography has been shown to reduce mortality by 20-43% and is now included in US screening guidelines1-6. Existing challenges include inter-grader variability and high false-positive and false-negative rates7-10. We propose a deep learning algorithm that uses a patient's current and prior computed tomography volumes to predict the risk of lung cancer. Our model achieves a state-of-the-art performance (94.4% area under the curve) on 6,716 National Lung Cancer Screening Trial cases, and performs similarly on an independent clinical validation set of 1,139 cases. We conducted two reader studies. When prior computed tomography imaging was not available, our model outperformed all six radiologists with absolute reductions of 11% in false positives and 5% in false negatives. Where prior computed tomography imaging was available, the model performance was on-par with the same radiologists. This creates an opportunity to optimize the screening process via computer assistance and automation. While the vast majority of patients remain unscreened, we show the potential for deep learning models to increase the accuracy, consistency and adoption of lung cancer screening worldwide.

Feasibility of Diagnosing Both Severity and Features of Diabetic Retinopathy in Fundus Photography

Diabetic retinopathy (DR) diagnosis methods in the literature are usually criticized as being limit in diagnosing DR-related features or being lack of interpretability. To deal with these issues, this paper investigates the feasibility of diagnosing both DR severity levels and the presence of DR-related features in a two-step procedure. Specifically, this paper first analyzes the quality of annotations in DR grading by measuring inter-grader variability. Cosine similarity is considered to evaluate the inter-grader variability of the presence of DR-related features, and quadratic weighted Cohen's kappa is employed to assess the inter-grader variability of DR severity levels. Next, different annotation methods as follows are compared to DR severity prediction performance using logistic regression: 1) single annotations by single grader (SASG); 2) single annotations from multiple graders (SAMG); 3) multiple annotations by voting (MAV); and 4) double annotations with adjudication of disagreement (DAAD). Based on the comparison results, the feasibility of diagnosing both DR severity and features is investigated. In the experiments, 1589 fundus images graded by three retinal specialists and four general ophthalmologists are considered. The results demonstrate that retinal specialists are more consistent than general ophthalmologists in grading both the presence of DR-related features and DR severity. The SASG and MAV should be avoided if possible while the DAAD is the good option when prediction performance is the highest priority and the SAMG is especially beneficial when both prediction performance and grading costs are considered. The upper limit performance of DR severity prediction gets accuracy 95.6% and kappa 0.962. When DR-related feature prediction achieves average cosine similarity 0.823, it is potential to get accuracy 91.2% and kappa 0.905 for DR severity prediction in real applications. These results together suggest the potential of diagnosis of both DR severity and the presence of DR-related features in a two-step procedure.

RETOUCH: The Retinal OCT Fluid Detection and Segmentation Benchmark and Challenge.

Retinal swelling due to the accumulation of fluid is associated with the most vision-threatening retinal diseases. Optical coherence tomography (OCT) is the current standard of care in assessing the presence and quantity of retinal fluid and image-guided treatment management. Deep learning methods have made their impact across medical imaging, and many retinal OCT analysis methods have been proposed. However, it is currently not clear how successful they are in interpreting the retinal fluid on OCT, which is due to the lack of standardized benchmarks. To address this, we organized a challenge RETOUCH in conjunction with MICCAI 2017, with eight teams participating. The challenge consisted of two tasks: fluid detection and fluid segmentation. It featured for the first time: all three retinal fluid types, with annotated images provided by two clinical centers, which were acquired with the three most common OCT device vendors from patients with two different retinal diseases. The analysis revealed that in the detection task, the performance on the automated fluid detection was within the inter-grader variability. However, in the segmentation task, fusing the automated methods produced segmentations that were superior to all individual methods, indicating the need for further improvements in the segmentation performance.

Influence of Optic Disc Size on Identifying Glaucomatous Optic Neuropathy

The aim of this study was to use a large group of observers to test prior research findings that suggest optic disc size, clinical evaluation of the neuroretinal rim (ISNT rule), and practitioner characteristics influence the accuracy of differentiating normal from glaucomatous optic nerves. Participant observers were optometrists, optometry students, and vision scientists/researchers attending the 2013 American Academy of Optometry Annual Meeting. Each observer viewed and judged six sets of stereoscopic photographs of normal and clinically confirmed glaucomatous optic nerves of different sizes presented in random order. Observers were queried on whether each nerve was glaucomatous or normal, whether the nerve followed the ISNT rule, and whether further evaluation with advanced imaging techniques was indicated. Of the 261 observers who participated, 59% were practicing optometrists, 7% were vision scientists, and 34% were residents or students. Of practicing optometrists and vision scientists, half (49%) had more than 15 years of experience, whereas 11% had less than 2 years of experience. Diagnostic accuracy differed based on optic nerve size: average-sized nerves were correctly identified by 90% of subjects, whereas small nerves and large nerves were correctly identified by 42% and 62%, respectively. Notably, only 9% of subjects correctly identified the small glaucomatous nerve, and only 34% correctly identified the large normal nerve. No practitioner characteristics were associated with diagnostic accuracy. Accurate identification of glaucomatous optic neuropathy was significantly influenced by optic disc size. This was particularly evident for the large normal nerve and the small glaucomatous nerve. The ISNT rule provided value for differentiating normal from glaucomatous nerves, but its subjective interpretation resulted in considerable intergrader variability. These findings agree with other studies utilizing smaller numbers of observers but larger numbers of optic nerve presentations that disc size and the ISNT rule have value for enhancing accuracy of optic nerve assessment.

Effects of light variation on Schwalbe's line-based anterior chamber angle metrics measured with cirrus spectral domain optical coherence tomography.

Evaluate the impact of variations in ambient lighting conditions on the reproducibility/reliability of Schwalbe's Line (SL)-based anterior chamber angle (ACA) metrics using spectral domain optical coherence tomography (SD-OCT). Images were taken at Doheny Eye Centers-UCLA clinic, which were randomized, masked and graded twice by dual reading centre graders. Twenty-five normal/healthy participants with open angles Inferior angles were imaged using Cirrus SD-OCT under five light levels (foot-candles (fc) measured at camera-eye interface with Sper light-meter) with the instrument's corneal illumination function set to default level of 50 (CIdef) and low 5 (CIlow). Each eye was imaged 20 times, totaling 500 assessments. SL-angle-opening-distance (SL-AOD) and SL-trabecular-iris-space-area (SL-TISA) measured using custom ImageJ software. Intra-/inter-grader variability analyses were completed using Statistical-Package-for-Social-Science and Bland-Altman plots demonstrated limits of agreement for comparisons. Light level demonstrated a linear relationship with angle size, thus differences from highest to lowest light levels were compared. Decreasing light from 1.0 → 0.0fc at CIdef decreased SL-AOD from 394 µm +/- 137 µm → 356 µm +/- 137 µm (mean percent difference (MPD) = 10.71%,P < 0.001) and SL-TISA from 297 µm(2) +/- 114 µm(2) → 261 µm(2) +/- 109 µm(2) (MPD = 13.7%, P < 0.001). Decreasing from 1.0 → 0.0fc at CIlow decreased SL-AOD from 366 µm +/- 136 µm → 329 µm +/- 122 µm (MPD = 10.9%, P < 0.001) and SL-TISA from 271 µm(2) +/- 113 µm(2) → 234 µm(2) +/- 98 µm(2) (MPD = 15.8%, P < 0.001). There was 7.9/11.4% (both P < 0.001) difference for SL-AOD/SL-TISA between CIdef → CIlow at 1.0fc, and 7.7/9.4% (both P < 0.001) difference at 0.0fc. Intra-/inter-grader results showed high reproducibility for all metrics (MPD = 0.33-4.4%; CV = 0.96-1.36; PCC = 0.93-0.95(P < 0.001); R2 = 0.94-0.98). Bland-Altman plots did not demonstrate bias, with repeat-ability and agreement among measurements. Using Cirrus SD-OCT, we found that SL-based ACA morphometrics are exquisitely sensitive to changes in ambient illumination and also corneal illumination by the OCT instrument. Consistently imaging in the darkest room possible is recommended (≤0.2 fce).

Diagnosis of Dry Eye by Tear Meniscus Measurements Using Anterior Segment Swept Source Optical Coherence Tomography.

To evaluate the diagnostic power of tear meniscus measurements using anterior segment swept source optical coherence tomography (SS-OCT) for dry eye and suspected dry eye. Fifty subjects (27 men and 23 women; mean age 43.3 ± 13.7 years), comprising 26 eyes of 26 healthy subjects and 24 eyes of 24 patients with dry eye or suspected dry eye according to Japanese diagnostic criteria, were enrolled at The University of Tokyo Graduate School of Medicine. Subjects underwent SS-OCT, and the central upper and lower tear meniscus heights (TMHs) and areas (TMAs) and the lower tear meniscus volume (TMV) were examined. Intergrader variability and interimage variability, calculated using the pooled coefficient of variation and intraclass correlation coefficient, were used to assess repeatability of measurements in the dry eye group. To diagnose dry eye using tear meniscus measurements by SS-OCT, sensitivity, specificity, and cutoff values of optical coherence tomography (OCT) measurements were determined using receiver operating characteristic analysis. The TMH, TMA, and TMV measured by OCT were significantly lower in the dry eye group than in the control group (P < 0.001). Cutoff values of the lower TMH, TMA, and TMV were 191 μm, 12,360 μm, and 0.0473 mm, respectively. Intergrader repeatability and interimage repeatability, measured as the intraclass correlation coefficient, were >80% for all tear meniscus parameters, with acceptable repeatability. Significant correlations between tear meniscus measurements by OCT and vital staining scores, Schirmer test values, and tear film breakup time were observed (P < 0.05). SS-OCT is a noninvasive and practical method for quantitative evaluation of tear fluid and has the potential for detecting dry eye and suspected dry eye.

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Measurable competence derived from comprehensive and advanced training in grading digital images is critical in studies using a reading center to evaluate retinal fundus images from infants at risk for retinopathy of prematurity (ROP). Details of certification for nonphysician trained readers (TRs) have not yet been described. To describe a centralized system for grading ROP digital images by TRs in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study. Multicenter observational cohort study conducted from July 1, 2010, to June 30, 2014. The TRs were trained by experienced ROP specialists and certified to detect ROP morphology in digital retinal images under supervision of an ophthalmologist reading center director. An ROP reading center was developed with standard hardware, secure Internet access, and customized image viewing software with an electronic grading form. A detailed protocol for grading was developed. Based on results of TR gradings, a computerized algorithm determined whether referral-warranted ROP (RW-ROP; defined as presence of plus disease, zone I ROP, and stage 3 or worse ROP) was present in digital images from infants with birth weight less than 1251 g enrolled from May 25, 2011, through October 31, 2013. Independent double grading was done by the TRs with adjudication of discrepant fields performed by the reading center director. Digital retinal images. Intragrader and intergrader variability and monitoring for temporal drift. Four TRs underwent rigorous training and certification. A total of 5520 image sets were double graded, with 24.5% requiring adjudication for at least 1 component of RW-ROP. For individual RW-ROP components, the adjudication rate was 3.9% for plus disease, 12.4% for zone I ROP, and 16.9% for stage 3 or worse ROP. The weighted κ for intergrader agreement (n = 80 image sets) was 0.72 (95% CI, 0.52-0.93) for RW-ROP, 0.57 (95% CI, 0.37-0.77) for plus disease, 0.43 (95% CI, 0.24-0.63) for zone I ROP, and 0.67 (95% CI, 0.47-0.88) for stage 3 or worse ROP. The weighted κ for grade-regrade agreement was 0.77 (95% CI, 0.57-0.97) for RW-ROP, 0.87 (95% CI, 0.67-1.00) for plus disease, 0.70 (95% CI, 0.51-0.90) for zone I ROP, and 0.77 (95% CI, 0.57-0.97) for stage 3 or worse ROP. These data suggest that the e-ROP system for training and certifying nonphysicians to grade ROP images under the supervision of a reading center director reliably detects potentially serious ROP with good intragrader and intergrader consistency and minimal temporal drift.

Low Paneth cell numbers at onset of gastrointestinal graft-versus-host disease identify patients at high risk for nonrelapse mortality

AbstractAcute graft-versus-host disease (GVHD) of the gastrointestinal (GI) tract is an often lethal complication of allogeneic hematopoietic cell transplant. Clinical severity correlates with outcomes, but histopathologic grading is primarily used to confirm the clinical diagnosis. One barrier to using histopathologic grading to predict clinical outcomes is inter-grader variability among transplant centers. Recent experimental models have shown that the loss of Paneth cells, which are located in the small intestine and help regulate the GI microbiome by secreting antimicrobial peptides, correlates with clinical GVHD severity. Because Paneth cells are easy to identify and quantify by light microscopy, we evaluated the mean number of Paneth cells per high-powered field (hpf) in 116 duodenal biopsies obtained at diagnosis of GI GVHD at 2 different centers with their clinical outcomes. Paneth cell counts were reproducible between centers (r2 = 0.81; P < .0001). Lower numbers of Paneth cells at diagnosis correlated with clinically more severe GI GVHD (P < .0001) and less likelihood of response to GVHD treatment (P < .0001). A threshold of 4 Paneth cells per hpf stratified patients according to nonrelapse mortality (28% vs 56%; P = .004). We conclude that the enumeration of duodenal Paneth cells is a readily available index of disease severity that provides important information regarding GVHD prognosis.

Two-wavelength fundus autofluorescence and macular pigment optical density imaging in diabetic macular oedema

To evaluate the application of 488 and 514 nm fundus autofluorescence (FAF) and macular pigment optical density (MPOD) imaging in diabetic macular oedema (DMO) and to demonstrate the typical imaging features. A hundred and twenty-five eyes of 71 consecutive patients with diabetic retinopathy who underwent examination at a specialist university clinic employing a modified Heidelberg Retina Angiograph, using two different light sources of 488 and 514 nm wavelength, were retrospectively reviewed. MPOD images were calculated using modified Heidelberg Eye Explorer software. All images were evaluated by two independent masked graders. Features from FAF and MPOD images were correlated with optical coherence tomography (OCT) imaging findings and inter-grader variability, sensitivity and specificity were calculated using OCT as reference. Sixty-seven eyes had DMO on OCT. The inter-grader variability was 0.84 for 488 nm FAF, 0.63 for 514 nm FAF and 0.79 for MPOD imaging. Sensitivity and specificity for detection of DMO were 80.6 and 89.7% for 488 nm FAF; 55.2 and 94.8% for 514 nm FAF; and 80.6 and 91.4% for MPOD imaging. In 488 nm FAF and MPOD imaging, DMO was better visualised in comparison with 514 nm FAF imaging, P<0.01. MPOD revealed displacement of macular pigment by intraretinal cysts. MPOD imaging, and particularly its combination with 488 nm and 514 nm FAF, provides a valuable addition to OCT in the evaluation of DMO and is clinically useful in rapid en-face assessment of the central macula.

Assessment of patient communication skills during OSCE: Examining effectiveness of a training program in minimizing inter-grader variability

ObjectiveTo assess effectiveness of a training program in reducing inter-grader variability in grading communication skills during an objective structured clinical exam (OSCE). MethodsGlobal communication (GC) skills are assessed by standardized participants (SP) and faculty during each OSCE using a 6 item rubric. Despite criteria delineated in the GC rubric, inter-grader variability was observed. During 2008–2009 academic year, a training program was implemented before each OSCE to achieve more consistent interpretation and grading in GC skills. GC grades between SP and faculty for 2nd and 3rd level student OSCEs during 2008–2009 (post-training) were compared to 2007–2008 (pre-training). Data was analyzed using repeated measures ANOVA. Results274 and 281 students participated in OSCEs during 2007–2008 and 2008–2009 academic years, respectively. Training significantly (P<.001) decreased grader variability between SPs and faculty. There was a greater mean difference between faculty vs. SP before training (faculty 14.68, SP 15.87) compared to after training (faculty 13.51, SP 13.78). Mean GC scores for both faculty and SPs also decreased significantly after training. Conclusion and practice implicationsA training program may be necessary to reduce inter-rater variability in assessment of OSCE communication skills if it is to be truly helpful to student pharmacists learning to counsel patients.