Supporting Self-Directed Learning: Development of a Faculty Evaluation Scale

Abstract

Construct: We sought to assess medical education faculty members’ ability to support students in the development of self-regulated learning skills. Background: It is broadly agreed that medical students should become self-directed lifelong learners to succeed as physicians. To that end, many instruments have been developed that measure student attributes of self-directed learning (SDL). By contrast, no such analogous valid instrument is available to assess faculty familiarity in pedagogical strategies to promote SDL among students. Approach: An item bank with 45 items was created and its content-related validity evaluated by a panel of twenty experts. The items were selected and validated in the framework of Item Response Theory (IRT). The unidimensionality of all items within four constructs was assessed by using modified parallel analysis and 2 parameter IRT model calibration. The final version of the Recognition of Learning-Oriented Teaching Strategies (RoLOTS) instrument was validated by using the IRT marginal reliability as well as 2PL model calibrations. Different Item Functioning (DIF) in student-educators and gender was examined. Findings: The final version of RoLOTS included 20 items, with five in each of four subdomains: building content knowledge; the emotional and motivational aspects of learning; leveraging the social nature of the learning process; and metacognitive processes that promote student regulation of the learning process. Construct validity and reliability of each of the four domains were well supported by the results. Significant DIF was not detected in student-educator and gender. Conclusions: The RoLOTS successfully evaluated whether a faculty member is familiar with pedagogical tools to promote medical students’ self-directed learning, which can be used as a first step in needs-based professional development. Further research is needed to provide more validity and reliability evidence among groups with diverse SDL experience.