Formative Feedback Tool Research Articles

Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study

This paper presents a study on the impact of ChatGPT as a formative feedback tool on the writing skills of undergraduate ESL students. Since artificial intelligence-driven automated writing evaluation tools positively impact students’ writing, ChatGPT, a generative artificial intelligence-propelled tool, can be expected to have a more substantial positive impact. However, very little empirical evidence regarding the impact of ChatGPT on writing is available. The current mixed methods intervention study tried to address this gap. Data were collected from tertiary level ESL students through three tests and as many focus group discussions. The findings indicate a significant positive impact of ChatGPT on students' academic writing skills, and students’ perceptions of the impact were also overwhelmingly positive. The study strengthens and advances theories of feedback as a dialogic tool and ChatGPT as a reliable writing tool, and has practical implications. With proper student training, ChatGPT can be a good feedback tool in large-size writing classes. Future researchers can investigate the impact of ChatGPT on various specific genres and micro aspects of writing.

Engaging, motivating, and supporting students through feedback

Learning is iterative. It takes time and effort. Real progress in any subject requires we engage, motivate, and support learners as they take next steps to improve their work. Brent Duckor and Carrie Holmberg show how teachers can use progress guides as a formative feedback tool to re-engage and support students where they are, while keeping the focus on concrete steps they can take to move forward in the learning cycle. Drawing from research and case study evidence, they highlight how students and teachers can move conversations forward in safe and respectful ways. Without relying on graded judgments, the progress guide approach preserves the goals of continuous improvement and recenters learning for all students.

Development of a formative feedback tool for transanal total mesorectal excision.

Transanal total mesorectal excision (TaTME) is a novel procedure in the treatment of rectal cancer. Current training models for TaTME suggest a period of proctored cases, but no structured feedback tool exists to guide operators during the learning phase. The objective of this study therefore was to develop a formative feedback tool for the critical steps of the transanal portion of TaTME. A two-round Delphi study by TaTME experts was conducted to determine the items to be included in the formative feedback tool. Participants rated each step from a prepared list using a Likert scale from 1 (Not relevant) to 5 (Very relevant) with the option to suggest additional steps. Responses to the first round were presented in the second round, where participants rated the revised list of steps. Consensus was defined as > 80% of participants rating the step as 4 or 5 (out of 5). Items were combined when appropriate to avoid redundancy. Rating anchors describing performance (on a 5-point scale) were then developed for each step. The final tool was recirculated and participants rated the finished product on its feasibility and usefulness. Twenty-six TaTME experts were contacted for participation. Fifteen experts (58%) participated in the first round of the study, and eleven (42%) participated in the second round. The majority (14, 93%) had completed fellowship training in colorectal surgery. The first round of the Delphi study contained 34 items, and 32 items met inclusion criteria after the second round. Redundant items were combined into 15 items that comprised the final tool. Out of eight respondents to the feasibility survey, all believed the feedback tool enhances the feedback of learners and would use it for training purposes if available. This work describes the development of a novel consensus-based formative feedback tool specific to TaTME.

Using Individual Assessments as a Tool for Formative Feedback on Emotional Intelligence Training Programs in Healthcare Learners.

Emotional Intelligence (EI) is a skillset that influences and impacts an individual's ability to create, foster, and maintain strong relationships. In healthcare settings optimal patient centered care exists when teamwork, critical thinking, selfless service, integrity, and emotional intelligence are effectively practiced. While various methods exist to teach EI in the preprofessional and professional settings, the assessment of the efficacy of these types of training remains elusive. We propose a novel use of EI assessments to determine the effectiveness of EI programs and suggest that the information obtained can help shape and improve future EI education. Volunteer participants involved in the 2020-2021 Feagin Leadership Program (FLP) at Duke University were recruited for this study. FLP is a one year program that aims to train healthcare leadership skills, with a special emphasis on EI. It is comprised of various stages of healthcare learners with a desire to improve their healthcare leadership skills. All participants took both an EI self-assessment (SSEIT) and EI ability assessment (MSCEIT) both before and after a dedicated 5-hour EI educational session. Individuals must have completed both a pre- and post-test for at least one assessment to be included in the study. Apart from standard descriptive statistics, Wilcoxon sign rank tests were utilized to determine the effectiveness of the educational session by comparing pre- and post-tests within each assessment. A Spearman's rank correlation coefficient was used to compare the results of the SSEIT and MSCEIT. A total of 32 FLP scholars initially participated in which 18 completed all assessments. Average age was 29 years old and consisted of medical students (n = 16), residents (n = 7), fellows (n = 7), advanced practice provider (n = 1) and a researcher (n = 1). Group analysis of the SSEIT pre and post scores were 131 (±13, range 98-149) and 136(± 13, 106-105), respectively which were statistically significant. Pre and post MSCEIT scores were 102 (±20, range 32-141) and 103 (±12, range 80-121), which were not significant. The EI branches with the highest score on each test was Managing Own Emotions and Understanding Emotions for the SSEIT and MSCEIT respectively while Perceiving Emotions was the lowest for both assessments. Comparison of the SSEIT and MSCEIT demonstrated a moderate correlation that was statistically significant. In our study participants felt their EI improved following the EI educational session, however this did not appear translate into their actual ability. This could be a function of self-report bias or a limitation of the EI assessments. More studies in this space are needed to make this determination. Additionally, the strengths of this specific program were within the strategic use of emotions therefore in the future more attention should be placed on experiential use of emotions, specifically perceiving emotions. As EI education and training becomes more prevalent it is important to not only accurately assess an individual's EI ability but also the effectiveness of the education being presented. We propose that EI assessments can be utilized as a tool to measure the effectiveness of EI education and receive formative programmatic feedback.

The application of the tracer method with peer observation and formative feedback for professional development in clinical practice: ascoping review.

IntroductionThe tracer method, commonly used for quality assessment, can also be used as a tool for peer observation and formative feedback on professional development. This scoping review describes how, by whom, and with what effect the tracer method is applied as a formative professional development instrument between healthcare professionals of equal status and aims to identify the types of scientific evidence for this use of the tracer method.MethodsThe authors searched four electronic databases for eligible articles, which were screened and assessed for eligibility by two independent researchers. From eligible studies, data were extracted to summarize, collate, and make a narrative account of the findings.ResultsThe electronic search yielded 1757 unique studies, eight of which were included as valid and relevant to our aim: five qualitative, two mixed methods, and one quantitative study. Seven studies took place in hospitals and one in general practice. The tracer method was used mainly as a form of peer observation and formative feedback. Most studies evaluated the tracer method’s feasibility and its impact on professional development. All but one study reported positive effects: participants described the tracer method generally as being valuable and worth continuing.DiscussionAlthough the body of evidence is small and largely limited to the hospital setting, using the tracer method for peer observation and formative feedback between healthcare professionals of equal status appears sufficiently useful to merit further rigorous evaluation and implementation in continuous professional development in healthcare.Supplementary InformationThe online version of this article (10.1007/s40037-021-00693-6) contains supplementary material, which is available to authorized users.

AcaWriter: A Learning Analytics Tool for Formative Feedback on Academic Writing

Written communication is an important skill across academia, the workplace, and civic participation. Effective writing incorporates instantiations of particular text structures - rhetorical moves - that communicate intent to the reader. These rhetorical moves are important across a range of academic styles of writing, including essays and research abstracts, as well as in forms of writing in which one reflects on learning gained through experience. However, learning how to effectively instantiate and use these rhetorical moves is a challenge. Moreover, educators often struggle to provide feedback supporting this learning, particularly at scale. Where effective support is provided, the techniques can be hard to share beyond single implementation sites. We address these challenges through the open-source AcaWriter tool, which provides feedback on rhetorical moves, with a design that allows feedback customization for specific contexts. We introduce three example implementations in which we have customized the tool and evaluated it with regard to user perceptions, and its impact on student writing. We discuss the tool's general theoretical background and provide a detailed technical account. We conclude with four recommendations that emphasize the potential of collaborative approaches in building, sharing and evaluating writing tools in research and practice.

Plickers: An ICT Tool for Formative Assessment and Feedback

Plickers: An ICT Tool for Formative Assessment and Feedback

Pharmacy Students’ Utilization of an Online Tool for Immediate Formative Feedback on Reflective Writing Tasks

Objective. To assess pharmacy students' perceptions of the benefits and utility of a novel online reflective-writing tool. Methods. After completing a required Academic Writing Analytics (AWA) workshop, Master of Pharmacy students submitted a reflective writing assignment in the AWA web application. A six-item survey was administered to students prior to and immediately after using AWA. Results. Sixty students volunteered to participate in the study; however, only 39 students provided a pseudonym that allowed their pretest and posttest to be matched. A comparison of students' responses on the pretest with those on the posttest, which was administered four weeks after the workshop, suggest a noticeable increase in agreement with AWA's benefits as an effective, online tool for improving their reflective learning skills. Conclusion. This novel online tool has the potential to assist pharmacy students with self-critiquing and improving their reflective writing assignments prior to submission. Furthermore, as the elements of reflection are transferable, this tool has the potential to be used in other educational contexts.

A Tool for Formative Assessment and Learning in a Graphics Design Course: Adaptive Comparative Judgement

Improving graphics education may begin with understanding best practices for providing, receiving, and improving formative feedback. Challenges related to anonymity, efficiency, and validity in peer critique settings all contribute to a difficult-to-implement process. This research investigates university-level computer graphics students while engaged in adaptive comparative judgement (ACJ), as a formative learning, assessment, and feedback tool, during several open-ended graphics design projects. A control group of students wrote feedback on papers in small group critiques while the experimental group students participated in ACJ, acting as judges of peer work and providing and receiving feedback to, and from, their peers. Relationships between the paper-based group approach and the ACJ approach and student achievement were explored. Further, this paper discusses the potential benefits, and challenges, of using ACJ as a formative assessment and peer feedback tool as well as student impressions of both approaches toward peer formative assessment and feedback.

A REFLECTION ON USING FACE-TO-FACE PEER REVIEW AS A METHOD OF PROVIDING FORMATIVE FEEDBACK

Abstract –Face-to-face peer review was introduced into a third-year engineering course as a mechanism for providing formative feedback on written reports and oral presentations. 
 The design of the peer review exercises are a work-in-progress, and in this paper we present our reflections on the first experience of using peer review in this course. As authors, we are the course lecturer and three students, and so we present our reflections from both instructor and student perspectives.
 Through our reflections, we identified that peer review was a valuable tool for formative feedback. We suggested that student engagement could be increased by improving the structure how peer review was implemented in the course.

Using adaptive comparative judgment for student formative feedback and learning during a middle school design project

While design-based pedagogies have increasingly been emphasized, the assessment of design projects remains difficult due to the large number of potentially “correct” solutions. Adaptive comparative judgment (ACJ), an approach based on assessors/judges working through a series of paired comparisons and selecting the better of two items, has demonstrated high levels of inter-rater reliability with design projects. Efforts towards using ACJ for assessing design have largely centered on summative assessment. However, evidence suggests that ACJ may be a powerful tool for formative assessment and design learning when undertaken by students. Therefore, this study investigated middle school students participated in ACJ at the midpoint and conclusion of a design project, both receiving and providing feedback to/from their peers through the ACJ process. Findings demonstrated promise for using ACJ, as a formative assessment and feedback tool, to improve student learning and achievement.

The use and analysis of anti‐plagiarism software: Turnitin tool for formative assessment and feedback

AbstractThis analysis investigates the efficiency of the Turnitin software as a formative writing tool. The inquiry is especially looking into undergraduate and postgraduate students’ experiences while using Turnitin. The perceptions and experiences of students will be prioritized in the study with the purpose of determining ways to improve Turnitin from students’ point of view. Turnitin obtains text matches or similarity index values of 3,173 assignments submitted on subjects uploaded between 2012 and 2014 by university students. We statistically analyzed the similarity index values or levels of plagiarism percentage between the first and the last assignments, using the two‐sample Kolmogorov–Smirnov test, and we found that there was a significant improvement (p = 0.002). Hence, our results demonstrated that using Turnitin as a formative writing tool, allows students to prepare an assignment in an academically acceptable way, during the second half of the semester, with less plagiarism. The results found in this study suggests an insignificant difference between the draft version and final version of the same assignment (p = 0.192). Similarity index values are also different for different courses, such as writing based project subject and mathematics based engineering subject have different values (p < 0.0001). We also observed that students seem to be able to fool Turnitin tool by uploading images of the assignments instead of the text. Nevertheless, the nature of the subject, individual talent, learning approach, time contribution, and the exclusion of consecutive word count may affect the plagiarism percentage. Our results also indicate that there is a substantial benefit in using Turnitin as an educational writing tool rather than a punitive tool, as the use of Turnitin, promotes student learning outcomes with significantly improved academic skills. Thus, this paper provides an insight into avoiding high levels of plagiarism by using Turnitin as a preemptive tool.

Establishing meaningful benchmarks: the development of a formative feedback tool for advanced laparoscopic suturing.

Time and accuracy are commonly used metrics to assess laparoscopic skills in a simulated environment. However, they do not provide trainees with meaningful information about how to improve their skills. The objective of this study was to provide preliminary validity evidence for the creation a formative feedback tool (FFT) for advanced laparoscopic suturing skills. Videotapes of senior surgical residents (PGY3-5; SRs) and minimally invasive surgeons (MIS) performing 3 advanced laparoscopic suturing tasks were analyzed: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). A FFT was created based on a grounded theory analysis of interviews with MIS surgeons about the key technical aspects of each task. The FFT was used to assess the videotaped performances of SRs and MIS surgeons by two blinded independent raters. The FFT is composed of three parts: NH contains 10 items, UT 18, and CS 20. Each item was classified according to seven key surgical principles: depth perception, safety, bimanual dexterity, exposure, tissue handling, instrument manipulation, and forward planning. The videotaped performance of SR and MIS surgeons was graded on a 3-point Likert scale ("does well," "needs some improvement," and "does poorly") and scores were calculated as a sum of the points. ICCs for all three tasks were high (NH 0.90, UT 0.87, and CS 0.90). FFT score correlated strongly with combined time and accuracy measurements for UT (0.82, p<0.01) and CS (0.81, p<0.01), and moderately for NH (0.65, p<0.01). MIS surgeons performed significantly better than SRs on UT (p=0.02) and CS (p=0.05), while scores on NH were similar (p=0.57). A comprehensive tool for providing feedback about advanced laparoscopic suturing skills was developed. The FFT demonstrates evidence for validity as a measure of suturing skills and experience, and provides meaningful information to trainees about how to improve their skills and engage in more deliberate and efficient practice.

Deliberate Practice with Standardized Patient Actors and Development of a Formative Feedback Tool for Advance Care Planning Facilitators (S762)

Actor-based simulation represents an important method for formative feedback and setting minimum standards for performance. We used actor-based simulation to train advance care planning (ACP) facilitators and developed a formative feedback tool for future application in the workplace setting. Participants were surveyed pre-course regarding past experience (EXP). The course includes four 30-minute (prep-simulate-debrief) simulation scenarios. The observational tool derived from a mini-clinical examination for residents includes 18 yes/no observable critical actions, 2-3 case-specific critical actions (together = CL), and a global rating for scale of competence (GRSCOMP) (“needs further instruction,” “close supervision,” “minimal supervision,” “competent to perform independently”) and communication skill (GRSCOMM) (“unsatisfactory,” “satisfactory,” “good,” “very good,” “outstanding”). Each participant performed at least one direct interaction with the patient actor. Trained faculty observers rated real time or by video recording. Inter-rater reliability (IRR) was calculated on randomly selected paired ratings. Simulation performance based on observational tool ratings was explored for construct validity using Spearman’s rho correlations. 69/83 individuals consented to participate during 7 courses over 13 months ending May 2016, including 43 nurses, 16 social workers, and 10 others. 112 total simulations were conducted and we randomly selected 48 paired ratings for IRR. The absolute rater agreement for critical actions across all scenarios was 82% and IRR was 0.71 (Gwet’s AC1 statistic) and 0.58 (Brennan-Prediger’s kappa). We found a positive correlation between CL and GRSCOMP (rho=0.48, p<0.05) and between GRSCOMM and GRSCOMP (rho=0.73, p<0.05). We did not find a correlation between run order in the day and GRSCOMP (rho 0.03, p=0.66) or between EXP and GRSCOMP (rho=0.11, p=0.16). We achieved good inter-rater reliability using the observational rating tool. We suggest the observational checklist and rating scale are adequate for formative feedback.

Deliberate Practice with Standardized Patient Actors and the Development of Formative Feedback for Advance Care Planning Facilitators.

Multimodal curricular assessment after adding standardized patient (SP) actor-based simulation to an advance care planning (ACP) facilitator training course and development of a formative feedback tool. ACP represents a highly valued service requiring more and better trained facilitators. Participants were primarily nurses and social workers in a large multisite health system. The course included a precourse video demonstration of ACP, traditional lectures, and four 30-minute simulations with SPs. Knowledge was tested with a multiple choice question (MCQ) test. In addition to standard postcourse/postsimulation evaluations, learners were surveyed pre/post/30-90 days delayed for self-perceived confidence. A linear mixed-effects model was used to analyze changes over time. Trained faculty rated performance in simulations with an observational mini-clinical examination (mini-CEX)-type rating form with a checklist, global competency, and global communication rating. Inter-rater reliability (IRR) was calculated on randomly selected paired ratings. Sixty-seven individuals consented to participate. MCQ scores improved from 83% ± 10% to 92% ± 8% (p < 0.001). Paired learner surveys of self-confidence across six domains were available for 65 pre, 65 post, and 40 delayed with a mean positive change on a 0 to 10 point scale from pre-post (2.32 ± 1.65; p < 0.001) and predelayed (2.34 ± 1.96; p < 0.001) time frames. For the faculty observation ratings of simulation performance, the average raw agreement for critical actions was 82% and IRR was 0.71. Learner feedback and self-assessment suggest that actor-based simulation contributed to improved confidence in conducting ACP. The mini-CEX observation form is adequate for formative feedback, with further testing needed to make judgments of competence.

The MacTRAUMA TTL Assessment Tool: Developing a Novel Tool for Assessing Performance of Trauma Trainees: Initial Reliability Testing

To develop a novel assessment tool for trainees-led trauma resuscitation. Assess psychometric properties of the proposed tool. Evaluate feasibility and utility of the tool. Trauma resuscitation is a structured and complex process involving unique sets of skills. There is currently no published structured formative evaluation tool for trauma trainees. Therefore, many trauma trainees rely upon limited, unstructured feedback on their performance. We developed a tool to assess trainee performance while leading a trauma resuscitation and to assist faculty in providing trainee feedback after the encounter. This study was conducted in a level I trauma centre in Ontario, Canada. Principles of learning theories, literature review, and clinical expert opinions were used to design a tool to assess clinical competence required to lead the resuscitation. In total, 5 critical domains were identified. High-fidelity simulation-based environment was used to test interrater reliability using intraclass correlation coefficients. To gauge feasibility, practicality, and utility of the tool, an online survey was sent to raters and trainees at the end of the study. We found "excellent" agreement for "initial critical assessment" domain (0.80) and "moderate to good" agreement for the "communication and leadership" (0.67) and "clinical performance" domains (0.53). "Poor" agreement was identified for the "decision-making" domain (0.33). The coefficients for individual items reached "good" agreement for 5 items, and "moderate" agreement for 8 items. Intraclass correlation coefficients for the remaining 7 items were "fair" or "poor." Most raters agreed that items in the medical training domain were not applicable. Feedback from raters and trainees confirmed the feasibility and acceptability of the tool for formative feedback, in addition to some suggestions to enhance the tool. MacTrauma TTL assessment tool is a novel tool for formative feedback for trainees' performance during trauma resuscitation. Initial psychometric property testing is promising. Further reliability and validity testing of the modified tool is needed. The tool has been shown to be feasible and acceptable by both trainees and faculty as a formative assessment tool.

Animation and multiple-choice questions as a formative feedback tool for legal education

ABSTRACTTwo-dimensional animation when combined with multiple-choice questions affords an interesting and innovative formative feedback tool for engaging law students in problem-based learning. This article investigates methods for making animation a more accessible medium for legal academics. The article also describes, evaluates and reflects upon the results of a focus group and a survey of online law student perceptions on the combined use of animation and multiple-choice questions as a formative feedback tool. Ethical issues involving delinquent and guilty clients were used as the context in which to describe an animation workflow and explore student attitudes to animation. A website was created within which animations created by academics were included to provide feedback to scaffold student understanding of legal ethics. Students viewed the animations as a very positive learning experience, in the sense of making the lesson more interesting to learn, assisting them to learn and help visualise the ethical problems. Student comments highlighted design features which, when refined, may improve the quality of both the animations and the student experience in studying law using animation.

Digital homework for kinematics and dynamics of machinery

A digital homework program applicable to the study of kinematics and dynamics of machinery has been developed and successfully incorporated into an ongoing kinematics and dynamics of machinery course providing students with regulated self-practice, formative feedback, and partial credit scoring. The use of digital homework in upper-level engineering courses is sparsely documented in the literature but can provide homework assignments that give immediate feedback for self-assessment of learning while significantly reducing the time required of the instructor for grading. This paper presents a procedure for creating a digital homework set generally applicable to engineering courses using a commercially available course management tool. The paper then documents a specific implementation into a junior-level kinematics and dynamics of machinery course. This implementation is assessed over four semesters based on its comparison with traditional homework as perceived by students. The responses indicate that students were acquainted with digital homework from pre-engineering courses and did not regard the digital format as a hindrance. However, students perceived no benefit to the immediate feedback and, therefore, preferred the traditional format of upper-level courses. This perception is attributed to students’ tendency to make use of the homework feedback much later after the assignment and primarily at the point of exam preparation. However, this seems to be counter to the role of homework as a formative feedback tool that students use in real time to gauge and regulate learning activities. The authors conclude that the presented digital homework is both efficient and appropriate as a teaching supplement to an upper-level kinematics and dynamics of machinery course in the light of the role of homework as a tool to regulate learning of course material. Examples of the implemented digital homework problems are provided to serve as a resource for other instructors.

Editor's Spotlight/Take 5: Objective Structured Assessments of Technical Skills (OSATS) Does Not Assess the Quality of the Surgical Result Effectively.

H ow we train our residents to become orthopaedists affects all of us, whether as educators now, or patients of these young surgeons later. So naturally we ask, given time and resource constraints in our training programs, are residents competent when they graduate? And how do we determine competence, anyway? The Accreditation Council for Graduate Medical Education and the American Board of Orthopaedic Surgery’s Milestones program provide some more objective definitions, and perhaps more than before, we now assess competence rather than merely track time in service [6]. Against this backdrop of more rigorous assessments in surgical education, surgical educators have developed some more objective tools. In particular, the Objective Structured Assessment of Technical Skills (OSATS) has gained wider acceptance, with recent iterations including global rating scales and detailed checklists [7]. A recent review [3] concluded that the OSATS is generally a valid tool for formative feedback to the learner, but evidence of OSATS’s validity remains insufficiently robust to use this tool for high-stakes examinations and certifications. Why might there be problems with the OSATS’s validity? First, we must remember that validity is a reflection of how well something conforms to reality, and depending on what aspect is measured, validity can be divided into different types. The OSATS, as an evaluation tool for technique and skill, performs well in terms of construct validity; that is, the test seems to measure accurately the things it sets out to measure [2]. However, it is less clear that we are measuring the most important things in the first place (content validity), and we just do not know to what degree the measurements predict important realworld endpoints, like the effective performance of actual surgery (concurrent validity). How might we measure the technical proficiency of a surgical learner in the context of actual (or simulated) surgery? When piecing together intraarticular fracture fragments, we strive Note from the Editor-In-Chief: In ‘‘Editor’s Spotlight,’’ one of our editors provides brief commentary on a paper we believe is especially important and worthy of general interest. Following the explanation of our choice, we present ‘‘Take Five,’’ in which the editor goes behind the discovery with a oneon-one interview with an author of the article featured in ‘‘Editor’s Spotlight.’’ The author certifies that he, or any members of his immediate family, has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article. All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request. The opinions expressed are those of the writers, and do not reflect the opinion or policy of CORR or The Association of Bone and Joint Surgeons. This comment refers to the article available at: DOI: 10.1007/s11999-015-4603-4. M. D. Wongworawat MD (&) Clinical Orthopaedics and Related Research1, 1600 Spruce Street, Philadelphia, PA 19013, USA e-mail: mwongworawat@clinorthop.org Editor’s Spotlight/Take 5 Published online: 4 January 2016 The Association of Bone and Joint Surgeons1 2015

Measuring the Promise: A Learning-Focused Syllabus Rubric

To enrich the resources for measuring the impact of educational development work, we have created a rubric to assess the degree to which a syllabus achieves a learning orientation. The rubric provides qualitative descriptions of components that distinguish learning-focused syllabi and uses a quantitative scoring system that places syllabi on a spectrum from content-focused to learning-focused. It is flexible enough to accommodate a diverse range of levels, disciplines, institutions, and learning environments, yet nuanced enough to provide summative information to developers using the tool for assessment purposes and formative feedback to instructors interested in gauging the focus of their syllabi.