Sensitivity and specificity of computer vision classification of eyelid photographs for programmatic trachoma assessment

Matthew C Kim,Zerihun Tadesse,Bruce D Gaynor,Thomas M Lietman,Jeremy D Keenan,Alexander M Ryner,Travis C Porco,Sun Y Cotter,Abdou Amza,Kazunori Okada,Ryan E Wiegand

doi:10.1371/journal.pone.0210463

Matthew C Kim, Zerihun Tadesse + Show 9 more

Open Access

https://doi.org/10.1371/journal.pone.0210463

Copy DOI

Journal: PloS one	Publication Date: Feb 11, 2019
Citations: 15	License type: CC BY 4.0

Affiliation: University of California, San Francisco

Abstract

Background/aimsTrachoma programs base treatment decisions on the community prevalence of the clinical signs of trachoma, assessed by direct examination of the conjunctiva. Automated assessment could be more standardized and more cost-effective. We tested the hypothesis that an automated algorithm could classify eyelid photographs better than chance.MethodsA total of 1,656 field-collected conjunctival images were obtained from clinical trial participants in Niger and Ethiopia. Images were scored for trachomatous inflammation—follicular (TF) and trachomatous inflammation—intense (TI) according to the simplified World Health Organization grading system by expert raters. We developed an automated procedure for image enhancement followed by application of a convolutional neural net classifier for TF and separately for TI. One hundred images were selected for testing TF and TI, and these images were not used for training.ResultsThe agreement score for TF and TI tasks for the automated algorithm relative to expert graders was κ = 0.44 (95% CI: 0.26 to 0.62, P < 0.001) and κ = 0.69 (95% CI: 0.55 to 0.84, P < 0.001), respectively.DiscussionFor assessing the clinical signs of trachoma, a convolutional neural net performed well above chance when tested against expert consensus. Further improvements in specificity may render this method suitable for field use.

Highlights

Millions of people are currently blind because of trachoma worldwide, a result of infection by ocular strains of Chlamydia trachomatis. [1, 2] This infection is treatable using single-dose azithromycin, and mass administration of azithromycin forms the basis of the World Health Organization’s strategy for trachoma control. [3] Stakeholders base decisions on starting programs, stopping mass treatment, and declaring control on the clinical signs of trachoma
Is it possible for an automated algorithm to clinically grade active trachoma from photographs collected in the field? We note that automated image processing is becoming useful in many medical imaging applications. [5,6,7] Our application differs from most in that we use images collected under field conditions, and in that we are conducting classifications of a subclinical condition with an ultimate goal of guiding, not individual treatment, but community-wide mass administration of azithromycin for a public health control campaign
Neural networks have long been useful for diagnostic tests in medicine, and in ophthalmological applications in particular. [10,11,12] Here, we test the hypothesis that a convolutional neural network [13] can classify trachoma photographs substantially better than chance

Summary

Introduction

Millions of people are currently blind because of trachoma worldwide, a result of infection by ocular strains of Chlamydia trachomatis. [1, 2] This infection is treatable using single-dose azithromycin, and mass administration of azithromycin forms the basis of the World Health Organization’s strategy for trachoma control. [3] Stakeholders base decisions on starting programs, stopping mass treatment, and declaring control on the clinical signs of trachoma. [1, 2] This infection is treatable using single-dose azithromycin, and mass administration of azithromycin forms the basis of the World Health Organization’s strategy for trachoma control. Is it possible for an automated algorithm to clinically grade active trachoma from photographs collected in the field? [5,6,7] Our application differs from most in that we use images collected under field conditions (under differing lighting conditions and camera angles and distances), and in that we are conducting classifications of a subclinical condition with an ultimate goal of guiding, not individual treatment, but community-wide mass administration of azithromycin for a public health control campaign. [10,11,12] Here, we test the hypothesis that a convolutional neural network [13] can classify trachoma photographs substantially better than chance Neural networks have long been useful for diagnostic tests in medicine, and in ophthalmological applications in particular. [10,11,12] Here, we test the hypothesis that a convolutional neural network [13] can classify trachoma photographs substantially better than chance

Methods

Results

Conclusion