What Can SCORE Web Portal Usage Analytics Tell Us About How Surgical Residents Learn?

Abstract

The Surgical Council on Resident Education (SCORE) was established in 2004 with 2 goals: to develop a standardized, competency-based curriculum for general surgery residency training; and to develop a web portal to deliver this content. By 2012, 96% of general surgery residency programs subscribed to the SCORE web portal. Surgical educators have previously described the myriad ways they have incorporated SCORE into their curricula. The aim of this study was to analyze user data to describe how and when residents use SCORE. Using analytic software, we measured SCORE usage from July, 2013 to June, 2016. Data such as IP addresses, geo-tagging coordinates, and operating system platforms were collected. The primary outcome was the median duration of SCORE use. Secondary outcomes were the time of day and the operating system used when accessing SCORE. Descriptive statistics were performed, and a p < 0.05 was deemed statistically significant. There were 42,743 total SCORE subscribers during the study period (75% resident and 25% faculty) with a mean of 14,248 subscribers per year. The overall median duration of SCORE use was 11.9minute/session (interquartile range [IQR]: 6.8). Additionally, there was a significant increase in session length over the 3 academic years; 10.1 (IQR: 6.4), 11.9 (IQR: 7.2), and 13.2minute/session (IQR: 5.4) in 2013 to 2014, 2014 to 2015, and 2015 to 2016, respectively (p < 0.001). SCORE usage was highest in November to February at 21.0minute/session (14.2) compared to July to October and March to June (12.3 [IQR: 3.2] and 9.6minute/session [IQR: 2.2]), respectively (p < 0.001). This seasonal trend continued for all 3 years. We observed an increased number of sessions per day over the 3 years: median of 1500 sessions/d (IQR: 1115) vs 1706 (IQR: 1334) vs 1728 (IQR: 1352), p < 0.001. (Fig.). Most SCORE sessions occurred at night: 38,011 (IQR: 4532) vs 17,529 (IQR: 19,850) during the day (p < 0.001). Windows was the most frequently used operating system at 48.9% (p < 0.001 vs others). SCORE usage has increased significantly over the last 3 years, when measured by number of sessions per day and length of time per session. There are predictable daily, diurnal, and seasonal variations in SCORE usage. The annual in-training examination is a prominent factor stimulating SCORE usage.