Leveraging Providers’ Preferences to Customize Instructional Content in Information and Communications Technology – Based Training Interventions: Retrospective Analysis of a Mobile Phone – Based Intervention in India

Abstract

Background: Many public health programs and interventions across the world increasingly rely on using Information and Communications Technology (ICT) tools to train and sensitize health professionals. However, the effects of such programs on provider knowledge, practice, or patient health outcomes have been inconsistent. One of the reasons for the varied effectiveness is low and varying levels of provider engagement, which, in turn, could be because of the form and mode of content used. Tailoring instructional content could improve engagement, but it is expensive and logistically demanding to do so in traditional training methods. ICT-based training interventions make it feasible to customize content to cater to providers’ preferences. Objective: This study aimed to discover preferences among providers over form (articles or videos), mode (featuring peers or experts), and length (short or long) of the instructional content; to quantify the extent to which differences in these preferences can explain variation in provider engagement with ICT-based training interventions; and to compare the power of content preferences to explain provider engagement against that of demographic variables. Methods: We used data from a mobile phone–based intervention focused on improving tuberculosis diagnostic practices among 24,949 private providers from 5 specialties and 1734 cities over 1 year. Engagement time was used as the primary outcome to assess provider engagement. A k-mean clustering was conducted to segment providers based on the proportion of engagement time spent on content formats, modes, and lengths to discover their content preferences. The identified clusters were used to predict engagement time using a linear regression model. Then, we compared the accuracy of the cluster-based prediction model with that based on demographic variables of providers (e.g., specialty and geographic location). Results: The average engagement time across all providers was 7.5 min (median 0, IQR 0-1.58). A total of 69.75% (17,401/24,949) of providers did not consume any content. The average engagement time for providers with nonzero engagement time was 24.8 min (median 4.9, IQR 2.2-10.1). We identified 4 clusters of providers with distinct preferences for the form, mode, and length of content. These clusters explained a substantially higher proportion of the variation in engagement time compared with demographic variables (32.9% vs 1.0%) and yielded a more accurate prediction for the engagement time (root mean square error: 4.29 vs 5.21 and mean absolute error: 3.30 vs 4.26). Conclusions: Providers participating in a mobile phone–based digital campaign have inherent preferences for instructional content. Targeting providers based on individual content preferences could result in higher provider engagement when compared with targeting providers based on demographic variables.