Assessment of the American Board of Ophthalmology’s Maintenance of Certification Part 4 (Improvement in Medical Practice)

Abstract

There is disagreement as to whether Maintenance of Certification is creating value for physicians and their patients. To our knowledge, this report provides the first measures of the effectiveness of Part 4 of this activity in assisting ophthalmologists with quality improvement in their practices. To evaluate the effectiveness of the American Board of Ophthalmology's quality improvement program-Maintenance of Certification Part 4 (Improvement in Medical Practice)-in assisting its diplomates with quality improvement in their practices. A retrospective analysis was conducted of the performance of 1046 American Board of Ophthalmology diplomates on Practice Improvement Modules between September 1, 2012, and December 31, 2014. The mean baseline scores for each process or outcome measure on a medical record abstraction were calculated before and after the practice improvement activity. Paired t tests were used to assess improvement before and after the activity. Diplomates' comments and ratings of the usefulness of the activity in assisting them with quality improvement were also analyzed. Diplomate performance on process and outcome measures before and after a performance improvement activity and diplomate satisfaction that the activity met the stated goals of assistance with quality improvement in their practices. The 1046 American Board of Ophthalmology diplomates completed 1408 Practice Improvement Modules. When measures with participation by at least 20 diplomates were analyzed, there was improvement in 24 of 30 individual process measures (80.0%; 95% CI, 61.4%-92.3%) and in 7 of 18 individual outcome measures (38.9%; 95% CI, 17.3%-64.3%) chosen for improvement by diplomates. Analysis of the mean results for each diplomate on process measures chosen for improvement showed gains occurring in 9 of 12 modules and, for outcomes chosen for improvement, in 6 of 12 modules with at least 20 participants. A total of 826 of 1115 modules (74.1%) assessed by diplomates were rated from good to excellent; positive comments outnumbered negative ones by a ratio of 5:1. Quantitative and qualitative analyses, while limited by self-report that has not been validated, suggest that the American Board of Ophthalmology's Maintenance of Certification Part 4 can help diplomates improve quality on process, and to a lesser extent, outcome measures. Findings of this study may provide a basis to improve this activity.