Ask Your Doctor to Prescribe a YouTube Video: An Augmented Intelligence Approach to Assess Understandability of YouTube Videos for Patient Education

Abstract

Healthcare information in video format may be more understandable for users, offering the promise of improved health literacy, patient-physician interactions, self-care and outcomes. However, while recognizing the value of YouTube videos for patient education, most existing work in health communication has evaluated video understandability manually which is not scalable, replicable or efficient. In this study, we draw on the Patient Education Material Assessment Tool (PEMAT), a systematic approach for audio-visual educational materials assessment, to develop an automated video classification solution that is also generalizable. Extracting video features and metadata from YouTube, we develop an algorithmic approach employing PEMAT-based patient education constructs, annotations from domain experts and co-training methods from machine learning to assess the understandability of diabetes videos for patient education. The co-training approach significantly improves the video understandability classification performance in comparison to three widely used benchmark machine learning models. We further examine the impact of understandability on several dimensions of collective engagement with videos. A challenge in evaluating collective engagement with understandable videos is that there could be content that is not medically validated but yet engage users. Hence, we consider the simultaneous impact of understandability and validated medical information in a video on collective engagement by conducting a multiple-treatment propensity score based matching approach that allows us to implement a quasi-randomization research design. While confirming common assessments of the relationship between user engagement and understandability of education materials, our analysis quantifies these effects using actual viewing data in the specific context of understandability of complex medical information encoded in patient education videos found on YouTube. Implications for research and practice are discussed.