Limits on improving the positive predictive value of the Welch Allyn SureSight for preschool vision screening

Abstract

To describe our experience using the Welch Allyn SureSight (Welch Allyn, Inc., Skaneateles Falls, NY) when vision screening a large population of preschool children. Additionally, we explore the usefulness of altering referral criteria to create high specificity for remote field screening in instances in which over-referral is costly. Preschool children were screened for amblyogenic factors with the SureSight. Referred children received a gold standard examination with American Association for Pediatric Ophthalmology and Strabismus Vision Screening Committee-established failure criteria. Referral criteria were made increasingly more stringent to lower the rate of referral, and the effect on positive predictive value (PPV) was determined. A total of 15,749 children were screened, with reliable screening obtained in >99%. Rowatt-modified Vision in Preschoolers Study referral criteria produced a referral rate of 7.3% and a PPV of 48.2%. The PPV was >70% for children with unreliable screenings and with estimated refractive errors exceeding the instrument's range. Altering referral criteria improved PPV until referral rate reached 4% (PPV approximately 64%); further refinement past this level did not increase PPV and, hence, inappropriately limited sensitivity. A good test instrument should have high PPV and a referral rate approaching the population disease. Although altering referral criteria to decrease referral rate also decreases sensitivity, it should improve PPV. The presence of an asymptotic limit to PPV means additional improvement in PPV cannot be obtained with this technology. Thus, SureSight's usefulness for high-specificity screening is limited; however, the current referral criteria are sufficient for large screening programs and provide an acceptable referral rate and PPV.