Vision Screening

Abstract

I have concerns about the Vision in Preschoolers (VIP) Study.1The Vision in Preschoolers Study GroupComparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision in Preschoolers Study.Ophthalmology. 2004; 111: 637-650Abstract Full Text Full Text PDF PubMed Scopus (233) Google Scholar A major question is, why do the results disagree with those of dozens of other masked prospective studies of the sensitivity, specificity, and predictive values for the PhotoScreener (Medical Technologies, Inc., Riviera Beach, FL) and other photoscreeners?This study, though comprehensive, has numerous flaws in its methods and reporting. It was designed to compare 11 preschool vision screening tests.My concerns are the following: 1The test population was not representative of the general population at risk for amblyopia. There were 48% black, 22% Hispanic, and only 18% white children in the screening population. This selection of dark-eyed subjects prejudices against testing modalities that rely upon light reflection from the fundus, such as photoscreeners. It is more difficult to read photoscreening images from darkly pigmented eyes. This may explain why the 3 photoscreening instruments had lower sensitivity scores than the tests that did not rely on fundus light reflection.2Children who were not testable were determined to have failed the screening and were added to the failure group, which skews the statistics and lowers specificity. For example, 84 of 1444 (5.8%) MTI photoscreening images and 37 of 1438 (2.6%) iScreen photoscreening images (iScreen, Memphis, TN) were unreadable and were therefore counted as failures, rather than excluding them from the data analysis.3The failure criteria for the SureSight Vision Screener (Welch Allyn, Skaneateles Falls, NY) were changed arbitrarily by the VIP Study rather than using the manufacturer’s failure criteria, which had a high specificity. The VIP Study did not change any other manufacturer’s failure criteria.4Photoscreeners were tested last on all the patients to reduce light-induced effects on subsequent tests, but this may have induced a fatigue bias and affected the accuracy of fixation of the photoscreening subjects.5The VIP Study researchers found that 5.8% of their photoscreening photographs were unreadable. Donahue et al2Donahue S.P. Johnson T.M. Leonard-Martin T.C. Screening for amblyogenic factors using a volunteer lay network and the MTI PhotoScreener initial results from 15,000 preschool children in a statewide effort.Ophthalmology. 2000; 107: 1637-1644Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar have demonstrated that inexperienced lay operators have 10% unreadable photos and, with proper training, decrease their unreadable percentages to <4%. A 6% unreadable rate is unacceptable in a study such as the VIP. The photographers (all eye care practitioners) should have been able to take enough photographs to ensure at least a readable image, especially as they were well funded and the cost of film was not a consideration in retaking pictures.With the high numbers of unreadable photographs, it is likely that the quality of photographs for analysis was low (with eccentric fixation of small degrees). This, as well as other factors explained above, would explain why the sensitivity of the photoscreening instruments found in the VIP Study (37%) was found to be less than the sensitivity determined in numerous high-quality masked prospective studies in a normal population of children (sensitivities ranged from 77% to 87%).3Ottar W.L. Scott W.E. Holgado S.I. Photoscreening for amblyogenic factors.J Pediatr Ophthalmol Strabismus. 1995; 32: 289-295PubMed Google Scholar, 4Weinand F. Graf M. Demming K. Sensitivity of the MTI Photoscreener for amblyogenic factors in infancy and early childhood.Graefes Arch Clin Exp Ophthalmol. 1998; 236: 801-805Crossref PubMed Scopus (26) Google Scholar6The biggest flaw in the statistics is the omission of the positive predictive value (PPV) and negative predictive value (NPV) results for each screening test. Positive predictive value and NPV are the best tests of a screening instrument’s value in practice, as they put specificity and sensitivity in the context of disease prevalence. I do not understand how a peer-reviewed study could be allowed to omit these data.When you analyze the PPV and NPV for the instruments studies, the following data emerge: (1) noncontact retinoscopy, PPV = 73%, NPV = 85%; (2) SureSight manufacturer’s criteria, PPV = 47%, NPV = 91%; (3) SureSight (revised VIP Study criteria), PPV = 71%, NPV = 86%; (4) PhotoScreener, PPV = 71%, NPV = 79%; (5) iScreen, PPV = 71%, NPV = 79%; (6) Power Refractor II (Plusoptix, Nuremburg, Germany), PPV = 68%, NPV = 91%; (7) Retinomax (Nikon, Inc., Melville, NY), PPV = 72%, NPV = 86%; (8) Stereo Smile II (Stereo Optical, Inc., Chicago, IL), PPV = 66%, NPV = 80%.It appears that the only statistically different number is for the SureSight (using the manufacturer’s criteria). All the other screening tests are very similar in their predictive values once you factor in the prevalence of disease. Large community-based studies of the PhotoScreener in hundreds of thousands of preschool children have shown PPVs of 66% to 92%.2Donahue S.P. Johnson T.M. Leonard-Martin T.C. Screening for amblyogenic factors using a volunteer lay network and the MTI PhotoScreener initial results from 15,000 preschool children in a statewide effort.Ophthalmology. 2000; 107: 1637-1644Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar, 5Arnold R.W. Gionet E.G. Jastrzebski A.I. et al.The Alaska Blind Child Discovery Project rationale, methods and results of 4000 screenings.Alaska Med. 2000; 42: 58-72PubMed Google ScholarWith these data, perhaps the decision to eliminate further testing of photoscreening instruments was not appropriate for the third phase of the VIP Study.In conclusion, the numerous faults in data analysis, patient selection, and testing in the VIP Study may explain why it found such discrepant sensitivities and specificities in the 3 photoscreening instruments relative to the other previous studies. I have concerns about the Vision in Preschoolers (VIP) Study.1The Vision in Preschoolers Study GroupComparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision in Preschoolers Study.Ophthalmology. 2004; 111: 637-650Abstract Full Text Full Text PDF PubMed Scopus (233) Google Scholar A major question is, why do the results disagree with those of dozens of other masked prospective studies of the sensitivity, specificity, and predictive values for the PhotoScreener (Medical Technologies, Inc., Riviera Beach, FL) and other photoscreeners? This study, though comprehensive, has numerous flaws in its methods and reporting. It was designed to compare 11 preschool vision screening tests. My concerns are the following: 1The test population was not representative of the general population at risk for amblyopia. There were 48% black, 22% Hispanic, and only 18% white children in the screening population. This selection of dark-eyed subjects prejudices against testing modalities that rely upon light reflection from the fundus, such as photoscreeners. It is more difficult to read photoscreening images from darkly pigmented eyes. This may explain why the 3 photoscreening instruments had lower sensitivity scores than the tests that did not rely on fundus light reflection.2Children who were not testable were determined to have failed the screening and were added to the failure group, which skews the statistics and lowers specificity. For example, 84 of 1444 (5.8%) MTI photoscreening images and 37 of 1438 (2.6%) iScreen photoscreening images (iScreen, Memphis, TN) were unreadable and were therefore counted as failures, rather than excluding them from the data analysis.3The failure criteria for the SureSight Vision Screener (Welch Allyn, Skaneateles Falls, NY) were changed arbitrarily by the VIP Study rather than using the manufacturer’s failure criteria, which had a high specificity. The VIP Study did not change any other manufacturer’s failure criteria.4Photoscreeners were tested last on all the patients to reduce light-induced effects on subsequent tests, but this may have induced a fatigue bias and affected the accuracy of fixation of the photoscreening subjects.5The VIP Study researchers found that 5.8% of their photoscreening photographs were unreadable. Donahue et al2Donahue S.P. Johnson T.M. Leonard-Martin T.C. Screening for amblyogenic factors using a volunteer lay network and the MTI PhotoScreener initial results from 15,000 preschool children in a statewide effort.Ophthalmology. 2000; 107: 1637-1644Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar have demonstrated that inexperienced lay operators have 10% unreadable photos and, with proper training, decrease their unreadable percentages to <4%. A 6% unreadable rate is unacceptable in a study such as the VIP. The photographers (all eye care practitioners) should have been able to take enough photographs to ensure at least a readable image, especially as they were well funded and the cost of film was not a consideration in retaking pictures.With the high numbers of unreadable photographs, it is likely that the quality of photographs for analysis was low (with eccentric fixation of small degrees). This, as well as other factors explained above, would explain why the sensitivity of the photoscreening instruments found in the VIP Study (37%) was found to be less than the sensitivity determined in numerous high-quality masked prospective studies in a normal population of children (sensitivities ranged from 77% to 87%).3Ottar W.L. Scott W.E. Holgado S.I. Photoscreening for amblyogenic factors.J Pediatr Ophthalmol Strabismus. 1995; 32: 289-295PubMed Google Scholar, 4Weinand F. Graf M. Demming K. Sensitivity of the MTI Photoscreener for amblyogenic factors in infancy and early childhood.Graefes Arch Clin Exp Ophthalmol. 1998; 236: 801-805Crossref PubMed Scopus (26) Google Scholar6The biggest flaw in the statistics is the omission of the positive predictive value (PPV) and negative predictive value (NPV) results for each screening test. Positive predictive value and NPV are the best tests of a screening instrument’s value in practice, as they put specificity and sensitivity in the context of disease prevalence. I do not understand how a peer-reviewed study could be allowed to omit these data.When you analyze the PPV and NPV for the instruments studies, the following data emerge: (1) noncontact retinoscopy, PPV = 73%, NPV = 85%; (2) SureSight manufacturer’s criteria, PPV = 47%, NPV = 91%; (3) SureSight (revised VIP Study criteria), PPV = 71%, NPV = 86%; (4) PhotoScreener, PPV = 71%, NPV = 79%; (5) iScreen, PPV = 71%, NPV = 79%; (6) Power Refractor II (Plusoptix, Nuremburg, Germany), PPV = 68%, NPV = 91%; (7) Retinomax (Nikon, Inc., Melville, NY), PPV = 72%, NPV = 86%; (8) Stereo Smile II (Stereo Optical, Inc., Chicago, IL), PPV = 66%, NPV = 80%. It appears that the only statistically different number is for the SureSight (using the manufacturer’s criteria). All the other screening tests are very similar in their predictive values once you factor in the prevalence of disease. Large community-based studies of the PhotoScreener in hundreds of thousands of preschool children have shown PPVs of 66% to 92%.2Donahue S.P. Johnson T.M. Leonard-Martin T.C. Screening for amblyogenic factors using a volunteer lay network and the MTI PhotoScreener initial results from 15,000 preschool children in a statewide effort.Ophthalmology. 2000; 107: 1637-1644Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar, 5Arnold R.W. Gionet E.G. Jastrzebski A.I. et al.The Alaska Blind Child Discovery Project rationale, methods and results of 4000 screenings.Alaska Med. 2000; 42: 58-72PubMed Google Scholar With these data, perhaps the decision to eliminate further testing of photoscreening instruments was not appropriate for the third phase of the VIP Study. In conclusion, the numerous faults in data analysis, patient selection, and testing in the VIP Study may explain why it found such discrepant sensitivities and specificities in the 3 photoscreening instruments relative to the other previous studies. Vision Screening: Author ReplyOphthalmologyVol. 113Issue 7PreviewDr Freedman is correct that many other studies of the PhotoScreener (MTI, Riviera Beach, FL) have had a higher sensitivity than that found in the Vision in Preschoolers (VIP) Study. The VIP Study utilized the referral criteria we developed for our statewide volunteer-led vision-screening program. When I developed and tested those criteria in 1997, I made the decision to set the referral criteria to produce a high positive predictive value. I did that because photoscreening was not well established at the time, and I felt that large numbers of false-negative results would doom the program in the minds of volunteers. Full-Text PDF