Assessment of Accuracy of a Physician Ratings Website in One Metropolitan Area

Abstract

Patients frequently use online physician ratings websites (PRWs) to identify physicians for care. PRWs provide physician information and reviews. However, the accuracy of PRWs is uncertain. We investigated the accuracy and validity of Healthgrades with respect to endocrine surgery. We identified factors associated with reported board certification inaccuracy, higher ratings, greater quantity of ratings. The search term "endocrine surgery specialist" was used and the search was limited to a 25-mile radius around Philadelphia, PA. Data was collected on physician sex, age, board certification, surgical specialty, quantity of ratings, average rating, response to comments, and provision of a self-description. Descriptive statistics were performed to examine surgeon characteristics, ratings, and reported board certifications. Board certification accuracy was determined by searching the corresponding American Board website and calculating a kappa statistic. Logistic regression was performed to identify factors associated with board certification inaccuracy, higher average ratings, and higher quantity of ratings. A total of 300 physicians were identified. Eighty-four percent of listed board certifications were accurate; the kappa statistic for accuracy of board certification was 0.634. Providing a response to comments and greater quantity of ratings were associated with higher average ratings. Provision of a self-description, male sex, and younger age were identified as factors associated with higher quantity of ratings. A wide range of specialties are identified as endocrine surgery specialists. The reliability of board certification reporting was moderate. Increased surgeon involvement with the Healthgrades site was inconsistently associated with higher average ratings and higher quantity of ratings but lower accuracy.