Longitudinal Integrated Clerkship Students Research Articles

Characterization of Distinctive Teaching Practices in Longitudinal Integrated Clerkships: Perspectives From Students and Faculty

Phenomenon : Longitudinal integrated clerkships (LICs) are novel curricula that place medical students in long-term learning and coaching relationships with faculty and require adaptation of teaching practices on the behalf of faculty to maximize learning outcomes. An understanding of how teaching in an LIC model differs from teaching trainees in more traditional models is critical to ensuring curricular innovation success through faculty development. Approach : A qualitative approach was used to describe the teaching practices of faculty and learning experiences of student participants in longitudinal integrated clerkships in different clinical and community settings. Forty-five faculty and 20 students participated in focus groups. Thematic analysis of focus group data was used to identify differences and similarities between groups, sites, and specialties. Findings : Two groupings of themes emerged in thematic analysis: (1) precepting strategies distinctive to the longitudinal integrated clerkship model and (2) precepting strategies enhanced when employed in the LIC model. Distinct to the LIC model, preceptors and students described the importance of understanding the curricular structure and supporting students in longitudinal care of patients. Enhanced in the LIC model are the strategies of relationship-based teaching, support of autonomy, feedback, and support of longitudinal growth in skills. Insights : Students and faculty across LIC sites were broadly aligned in their opinions of best practices for teaching in an LIC model. The longitudinal relationship between student and faculty in an LIC distinguishes this model from traditional block rotations and a distinctive approach to successful teaching is demonstrated. Preceptors use time afforded to build trusting relationships with students, which created opportunity for novel teaching approaches and enhanced otherwise effective teaching strategies. A focus on orientation to the curricular model and support of longitudinal relationships with patients may serve as an anchor for faculty development efforts in the development of an LIC.

Narratives, Sensemaking, and Didactic Relationships in Longitudinal Integrated Clerkship Education.

To understand what longitudinal integrated clerkship (LIC) participants found meaningful and valuable about their experiences while grounded in a communicated narrative sensemaking (CNSM) framework. Semi-structured interviews were conducted with a cohort of 3 LIC students and 7 of their 12 preceptors. Interviews were designed to elicit narratives ie time-ordered accounts of events, experiences, or reflections. Narratives were identified and coded to understand aspects of experiences. Participants made sense of their experiences by narrating stories of LIC length allowing relationships to grow over time in a way that gave students a holistic perspective on patient care, built deep relationships between students and preceptors, and grew student confidence. Participants value longitudinal integration in LICs because it allows for unique perspectives on patient care and builds deep meaningful mentor relationships with preceptors. Relationships in LICs must be managed carefully because much LIC learning is facilitated by relationships. The clerkship is a crucial aspect of medical education of which sense will be made by physicians during their whole lives.

Recruiting students to rural longitudinal integrated clerkships: a qualitative study of medical educationists’ experiences across continents

IntroductionMany health systems struggle in the provision of a sustainable and an efficient rural health workforce. There is evidence to suggest that Longitudinal Integrated Clerkships (LIC) placing student learners in rural community settings have positively impacted the provision of rural health care services The recruitment and engagement of students in rural LIC have significant challenges. This study explored best practice methods of recruiting and supporting the transition of medical students into rural LIC.MethodsThe study took place during the 2021 Consortium of Longitudinal Integrated Clerkships Conference, a virtual event hosted by Stellenbosch University, South Africa. Participants consisted of delegates attending the Personally Arranged Learning Session (PeArLS) themed ‘Secrets to success’. The session was recorded with the participants’ consent and the recordings were transcribed verbatim. Data was uploaded to NVivo software and coded and analyzed using constant comparative analysis. Salient themes and patterns were identified.ResultsThirteen attendees participated in the PeArLS representing a range of countries and institutions. Strategically marketing the LIC brand, improving the LIC program profile within institutions by bridging logistics, and the need to scaffold the transition to the rural LIC learning environment emerged as key themes for success. The attendees highlighted their experiences of using peer groups, early exposure to rural LIC sites, and student allocation strategies for promotion. Unique learning styles adopted in LIC models, student anxiety and the importance of fostering supportive relationships with stakeholders to support students in their transition to the LIC environment were discussed.DiscussionThis PeArLS highlighted successful systems and processes implemented in rural settings across different countries to recruit and manage the transition of medical students to rural LIC. The process proved to be a quick and efficient way to elicit rich information and may be of benefit to educationists seeking to establish similar programs or improve existing rural LIC.

Clinical Assessment Specialists: A New Faculty Role to Improve Clerkship Assessment.

Ambulatory clerkships, including longitudinal integrated clerkships (LICs), face challenges to assessment, including time pressure and clinical demands on preceptors. High-quality clinical assessment is critical to implementing competency-based medical education, generating valid grades, and supporting learning. This importance is further heightened with the new pass/fail scoring for US Medical Licensing Exam Step 1, discontinuation of US Medical Licensing Exam Step 2 Clinical Skills, and the growing concern for bias in assessment. The Kaiser Permanente Bernard J. Tyson School of Medicine's LIC spans the first 2 years with 50 students per class. In 2021-2022, the authors created a new faculty role, the clinical assessment specialist (CASp). CASps are highly trained clinical teachers who directly observe clerkship students in the ambulatory setting, provide feedback, and complete competency-based assessment forms. CASps completed 186 assessments of first-year (Y1) LIC students and 333 assessments of second-year (Y2) LIC students. Y2 students achieved average higher milestones and were rated as requiring less supervision compared to Y1 students. Y1 students rated CASps more favorably than Y2 students. Preceptors rated the contribution of CASps similarly across both years. Clerkship directors described benefits including identification of at-risk students and value of augmenting preceptor assessments. The CASp role may offer an innovative way to generate valid assessment of student performance, offset clinical pressures faced by preceptors, identify at-risk students, and mitigate bias, especially in an LIC. Future studies may examine assessment validity, including use in summative assessment. CASps are an innovative approach to clinical clerkship assessment.

The spacing effect: Improving electrocardiogram interpretation.

Accurate electrocardiogram (ECG) interpretation is critical for safe patient care, making this skill a necessary competency for medical school graduation. Improved long-term memory retention with repeated exposure to material is one of the most evidenced-based components of adult learning science. This curricular innovation aimed to determine if implementing spaced repetition and retrieval practice using ECG quizzes during the principal clinical year would improve ECG interpretation skills among medical students enrolled in a Longitudinal Integrated Clerkship (LIC). The curricular innovation applied the spacing effect and retrieval practice. Cognitive science demonstrates enhanced long-term retention through repeated interval exposure to learned material. Studies of spaced retrieval indicate that memory retention is enhanced through tests involving effortful recall. LIC students in an intervention group were exposed to the spacing effect with periodic ECG quizzes throughout their clinical clerkship year. The results of the 17-item post-test for 140 students were analysed: LIC intervention, N= 54; block control, N= 62; and LIC control, N = 24. The ANOVA test was significant (p < 0.001). Games-Howell post hoc testing showed that the mean score in the LIC intervention group was significantly higher compared with the LIC control group (p < 0.001) and the block control group (p < 0.001). There was no significant difference between the LIC control and block control groups (p = 0.59). Spaced repetition of material through ECG quizzes improved ECG interpretation skills on an ECG post-test and mitigates the forgetting curve, maintaining student competency in ECG interpretation.

An exploratory study of goal orientations of traditional block and longitudinal integrated clerkship students

Background Achievement goal theory links goal setting, motivation, and learning and describes three orientations: ‘mastery’ (seeking learning), ‘performance’ (seeking positive judgments), and ‘performance-avoidance’ (avoiding negative judgments). Mastery orientation is considered most adaptive. The authors investigated goal orientations of traditional block clerkship (TBC) and longitudinal integrated clerkship (LIC) students. Methods This was an exploratory study conducted at one US medical school. Three hundred and twenty students completed an anonymous survey consisting of three tools with validation evidence: Patterns of Adaptive Learning Survey, Task-choice Goal Measures, and Questionnaire Goal Choice Items. The authors analyzed the data using regression analyses, Chi-square, and Wilcoxon’s rank-sum tests. Results While all students rated mastery items most highly on the five-point Likert scale (mean 4.58/5.00), LIC students rated performance-orientation lower (β = −0.36, p = .04), chose personal mastery-orientation items more frequently (92% vs. 64.4%, p = .005), and perceived their learning environment as promoting less performance (β = −0.60, p = .002) and performance-avoidance (β = −0.78, p < .001) compared to TBC students. Conclusions LIC and TBC students differed in their report of personal and clerkship goal orientations. These differences may inform educational design and future research to promote students’ mastery orientation.

The future integrated care workforce.

The future integrated care workforce.

The Perception of Surgery Program Directors of Residency Applicants From Longitudinal Integrated Curriculum Medical Schools: A Survey of the Association of Program Directors in Surgery Members.

Research statement This study explores whether longitudinal integrated clerkship (LIC) students are competitive general surgery applicants and if they are perceived as adequately prepared for general surgery residency compared to traditional block rotation (BR) students. Background/relevance of the study There is increasing interest in LIC models of clinical education versus BR models. LIC students have been shown to perform similarly on examinations to BR students. However, while LICs seem well suited for students pursuing primary care specialties, little is known about how this approach impacts clinical education for surgery. Design and methods An electronic survey was prepared and approved by the Association of Program Directors in Surgery (APDS) and our university's institutional review board (IRB). Ten multiple-choice questions were administered along with an option for narrative comments. Surveys were sent over a one-month period to members of APDS Listserv. Returned emails were de-identified, and the results were tabulated. Results From 43 responses, the majority identified as program directors (PDs) (65%) and reported being somewhat familiar or very familiar with LICs (90%). When asked about the statement "LIC students are prepared for surgical residency," 22% "disagreed" or "strongly disagreed."When asked"How would you rank a LIC prospective applicant in comparison to a BR student?"35% responded that they would rank the LIC student lower or not at all. Of the respondents, 47% reported that they have current residents who were LIC students. Most of these residents (65%)are graded as "average"for current performance. Conclusions The results suggest that medical students who are trained using LICs may be disadvantaged when applying to general surgery residencies. Interpretation is limited by the small number of respondents, and it only reflects the opinions of active APDS Listserv members. Further study is needed to confirm these findings and elucidate the basis of perceived deficiencies in LICs. Students from these schools should be advised to obtain additional surgery experience.

Paint Me a Picture: Adding Patient Imaging Experiences and Radiology to a Longitudinal Integrated Clerkship.

Purpose: Despite the increasing prevalence of Longitudinal Integrated Clerkships (LICs), there is a paucity of published LIC models which directly integrate radiology into longitudinal clinical experiences, instead favoring protected time for radiology didactics or a more traditional block rotation (BR). 1–3 Such models miss the opportunity to better capture longitudinal clinical and professional relationships and the integration of system-based care which defines LICs. We sought to explore if the core constructs of LICs could be expanded to incorporate radiology learning objectives in a pilot LIC Radiology program. Approach: As a pilot program, LIC students were encouraged to identify patients during their clinical experiences (clinics, urgent care, emergency department, or inpatient immersion) in which radiology examinations were ordered and accompany the patient to the radiology studies/procedures. Students were required to meet individually with the radiologist to review the clinical indications and imaging findings of the examination for 25 patients for the pilot year. Student participation was tracked via Qualtrics surveys. Students were given access to, and expected to asynchronously review, radiology didactic material used in parallel by the BR student clerkship. Radiology faculty champions were identified in each imaging subspecialty followed by LIC and pilot faculty development with a requirement for student feedback evaluations via Qualtrics surveys. Outcomes: Two hundred thirty-seven unique radiology encounters were logged by the 9 LIC students (mean = 26.33 encounters per student, maximum = 33 encounters by one student). Eight out of nine reached the target of 25 radiology encounters, and 1/9 had 24 encounters. One hundred percent of students completed at least 2 encounters with radiographs, CT scans, and MRI examinations; 88.9% (8/9) students completed at least 1 encounter with ultrasound. There was variable exposure to nuclear medicine (66.7%, 6/9) and interventional radiology (22.2%, 2/9). Modalities encountered were radiographs (31.7%, 75/237), CT scan (24.9%, 59/237), MRI (24.1%, 57/237), ultrasound (9.3%, 22/237), nuclear medicine (8.4%, 20/237), and interventional radiology (1.7%, 4/237). Twelve unique radiology faculty completed 84 feedback evaluations (mean 9.3/student). According to faculty evaluators, 100% of the students were prepared for the interaction and were at (60.7%, 51) or above (39.3%, 33) the expected level of training. Discussion: Our data suggests that incorporating a radiology curriculum into an LIC by leveraging established LIC patients has merit and is effective. While students were not required to have a specific modality distribution during the pilot year, students reviewed multiple imaging modalities for patients with whom they had an established relationship. The Association of University Radiologists recommended learning objectives for a core radiology rotation include a focus on skills associated with the working dynamic of radiology as a consultative service, such as “compare and contrast the benefits and limitations of different radiology modalities including: plain film, CT, US, MR, nuclear medicine” and “describe the procedure for ordering a radiologic exam at your institution.” 4 Our findings suggest that directly integrating radiology into the patient-centered framework of an LIC offers an opportunity to meet these learning objectives with results comparable to traditional BR models. Given the positive feedback and early success of the pilot, the course has subsequently been expanded from 2 to 4 credits and students must now reach an encounter threshold of 40. Ongoing assessment of this model will include future logistics, student satisfaction and performance, and faculty feedback to track achievement of learning objectives. Significance: Directly integrating radiology as a part of LIC can meet radiology learning objectives while preserving a diverse experience and solidifying the LIC construct. This model could be applicable at other institutions to further patient-centered education within an LIC framework and could conceivably replace models which fall beyond LIC constructs. Acknowledgments: The authors wish to thank Dr. Deborah Engle for supporting development of assessment strategies in longitudinal integrated clerkships. The authors also wish to thank Melissa Graham who helped with survey development, distribution, and collection.

Medical Student Identity Construction Within Longitudinal Integrated Clerkships: An International, Longitudinal Qualitative Study.

Longitudinal integrated clerkships (LICs) have been implemented worldwide to increase authentic student participation in patient care over time. Studies have shown benefits of the model include the ability of LICs to attract future practitioners to underserved areas, student engagement in advocacy, and development of an "ethic of caring." Less is known, however, about how LICs impart their benefits, although LICs may strengthen professional identity. As such, this study aimed to explore medical student professional identity construction through time within LICs internationally. This was a longitudinal qualitative study from 2019 to 2020, involving 33 students across 4 medical schools in the United Kingdom, Ireland, and the United States. The authors explored participating students' identity construction during LICs. Data collection involved 3 stages: individual, semistructured interviews at entry (n = 33) and exit of the LIC (n = 29), and audio diaries throughout. Data were analyzed inductively using a reflexive thematic approach. Three themes were identified. Longitudinal relationships with patients and preceptors encouraged patient care ownership and responsibility; LIC students identified as patient advocates; and longitudinal relationships shaped students' social consciences. Themes were underpinned by continuity of relationships between students and patients, and students and tutors. Though continuity is lauded as the cornerstone of LICs, these findings suggest that this is too broad a way of understanding the benefits of LICs. Instead, continuity should be seen as facilitating the development of important relationships within diverse communities of practice, which leads medical students to construct professional identities as responsible, compassionate advocates for underserved populations. This was the case across the institutions, nations, and LIC types in this study. By highlighting the ways in which LICs influence identity, these findings offer important insight relating to the future development and delivery of LICs.

A is for advocacy: How introducing student advocacy assessment impacts longitudinal integrated clerkship students and clinical supervisors

To improve health, physicians are increasingly called to advocate. Yet medical schools have only recently focused on health advocacy skill-building. Limited work to date addresses assessing medical student advocacy on behalf of patients. We describe how students and clinical supervisors (CS) in two urban longitudinal integrated clerkships (LIC) experience patient advocacy and how introducing a new advocacy assessment impacts them. Using a thematic approach, we analyzed transcripts of focus groups during 2018–2019. Seventeen of 24 (71%) students and 15 of 21 (71%) CS participated in the focus groups. We describe how students perceive their advocacy role as they accompany the patient, amplify their voice, and facilitate connection. The rationale for advocacy assessment includes that it (1) adds a novel dimension to the written and verbal assessment, (2) drives student learning, (3) aligns with the institutional goal to promote equity, and (4) impacts CS teaching and clinical practice. Challenges are the ambiguity of expectations, pressure to ‘perform,’ and a moral overlay to advocacy assessment. Findings demonstrate how educational alliances between students and CS and longitudinal relationships between LIC students and patients offer a constructive opportunity for advocacy assessment. We describe suggestions to hone and expand the reach of advocacy assessment.

Disrupting the consultation: students empowering patients in a longitudinal clerkship

ABSTRACT Although well-established worldwide as a method of clinical medical education, Longitudinal Integrated Clerkships (LICs) are green shoots in the UK medical education landscape. The first comprehensive LIC in the UK was introduced in Dundee, Scotland in 2016. Substantial work has been carried out to evaluate the experiences of students and primary care tutors involved in the Dundee LIC, but the experiences of the patients LIC students cared for had not been evaluated. The purpose of this study was to explore the experiences of these patients, particularly the impact the involvement of a LIC student might have on their experience of healthcare. The study is a cross-sectional qualitative study involving semi-structured interviews with five patients who had experienced several contacts with LIC students. An interpretive phenomenological approach was taken. We describe the presence of the student as a disruptive force leading to the empowerment of patients. Students disrupted the status quo in the consultation by altering both the structure of the interaction and the doctor–patient relationship. The student–patient relationship was a powerful enabler of patient empowerment through the provision of education and information to the patient and through increasing patient centredness in the consultation. The positive social interaction provided by the student–patient relationship led to a reframing of patients’ perceptions of the medical profession, challenging their perceptions of occupational hierarchy and power of the medical profession.

Impact of Longitudinal Integrated Clerkships on Student Outcomes.

To the Editor: As fourth-year UK medical students, we read Beard and colleagues’ article 1 with great interest. The study focuses solely on how longitudinal integrated clerkships (LICs) can improve patient satisfaction, but we wish to stress our belief that LICs can also greatly improve student outcomes. Studies have shown that medical students appreciate LICs; they report taking more active roles in patient care, with increased engagement. 2 Indeed, even simpler aspects of LICs, such as access to patient care calendars, enable students to gain an in-depth perspective of patients’ interactions with health services, while highlighting the range of services available. Beard and colleagues acknowledge that LICs resulted in improved physician–patient communication through better patient education and coordination of care. Medical educators should therefore recognize that students on LICs who observe this improved communication will likely develop superior professional standards from the outset of their clinical exposure. Patients should benefit in turn as students transition into more competent medical professionals. Additionally, student dissatisfaction and burnout while on clinical placements is strongly associated with feelings of being a burden on both the medical team and patients, 3 leading to decreased engagement, attendance, and development of clinical skills. By implementing LICs and highlighting the benefits of their longitudinal presence to students, students may feel more optimistic and engaged toward their placements. Only 9 of 37 UK medical schools offer LICs. 4 We propose that more widespread integration of LICs, particularly in the early years of UK medical education, would enable students to better develop their clinical skills and appreciate the patient experience. In future studies, outcomes of LIC and traditional block students could also be compared with evaluate the potentially profound, multifaced benefits of LICs to both patients and students.

The impact of clerkship model and clinical setting on medical student’s participation in the clinical workplace: A comparison of rural LIC and rural block rotation experience

Purpose To compare the participation of rural Longitudinal Integrated Clerkship (LIC) and rural Block Rotation (BR) students in surgical clinical activities. Methods Third-year medical students from Deakin University’s Rural Clinical Schools were invited to record their participation during clinical encounters with ten common surgical conditions, using a descriptive five-point progressive entrustment scale. Participation levels of LIC and BR students were compared for differences, according to clinical task and context. Results LIC students recorded greater active participation across all clinical tasks. Highest levels of active participation occurred in General Practice, a setting only LIC students were exposed to at this course stage. BR students recorded the majority of their surgical encounters in the hospital inpatient setting, where their involvement was predominantly observational. Both groups recorded high levels of participation in the Emergency Department. Conclusions Active participation in clinical encounters with surgical patients was enhanced by participation in a LIC program and cannot be attributed to the rural context alone. Student participation is influenced by clinical context, presenting the opportunity to reconsider the design of clerkships to include models that facilitate active student participation. Further research is required to investigate the learner, supervisor and contextual factors influencing entrustment decisions within clerkships.

The future of health professions education: Emerging trends in the United States.

The future of health professions education: Emerging trends in the United States.

Not all who wander are lost: evaluation of the Hull York medical school longitudinal integrated clerkship

ABSTRACT Longitudinal integrated clerkships (LICs) are increasingly available within the United Kingdom, but gaps in knowledge remain regarding their efficacy and the influence of local context. In 2019–20, the Hull York Medical School ran a pilot LIC for 6 fourth-year medical students. This work describes the longitudinal qualitative programme evaluation. LIC students participated in two focus groups, one after four months, and another at the end of the programme. In total, 16 faculty were also interviewed regarding their experiences in developing, implementing and running the LIC. Students’ GP supervisors were difficult to engage in detailed evaluation due to the COVID-19 pandemic, and so were briefly surveyed at the end of the LIC. All data were pooled and analysed together using reflexive thematic analysis. Two major themes were identified: ‘Trajectory of the LIC’, describing the learning curve students and faculty encounter, and ‘Institutional decision making’, describing the need for clarity regarding the programme’s purpose. The programme was largely positively received, but areas for improvement locally, and transferrable recommendations, were identified. Aligning assessment to programme aims is an important area for future development, alongside balancing structured with unstructured time, and supporting students as they navigate a J-shaped learning curve.

The University of Vermont Larner College of Medicine.

The University of Vermont Larner College of Medicine.

Validating an Obstetrics and Gynaecology Longitudinal Integrated Clerkship Curriculum at the University of Toronto: A Four-Year Review.

The University of Toronto launched a longitudinal integrated clerkship (LIC) pilot project in 2014. Our aim was to evaluate LIC student outcomes in obstetrics and gynaecology compared with those of traditional block students. LIC and block students underwent identical evaluations, including written and oral exams, as well as clinical assessments from the ward. LIC student scores in each domain were compared with those of the traditional block students over 4 years. There were no differences in overall pass/fail status between LIC and block students. Thus, our LIC design could be appropriately modeled at other teaching institutions contemplating introducing an LIC stream.

It Takes a Village: Utilizing a Community-based Longitudinal Integrated Clerkship Model at a Regional Medical Campus to Provide the Core Emergency Medicine Clerkship Experience.

The objective was to evaluate the efficacy of a required emergency medicine (EM) experience embedded in a new community-based longitudinal integrated clerkship (LIC) for participating students and faculty. We developed and implemented a community-based LIC EM experience. The experience included 10 clinical shifts, one emergency medical services prehospital care shift, four didactic sessions, one reflection, and an emergency medicine simulation session. Students' outcomes were assessed using end-of-year surveys, focus groups, an emergency medicine subject examination, simulation performance, and clinical evaluations of the students by faculty. Faculty were recruited, underwent faculty development, and were evaluated by students. Faculty perceptions were gathered from a focus group and faculty retention rates were collected. Three cohorts of LIC students (total N=61) have completed their core emergency medicine experience in our community-based LIC. Among students, 76% to 95% rated the overall quality of teaching as very good or excellent and 66% to 100% rated the quality of learning in their community-based setting as very good or excellent. All students who passed the EM subject examination achieved the clinical competencies of the experience based on clinical evaluations. Among faculty, we have retained greater than 95% each year and they have reflected positively on their precepting experience. Implementing a required EM experience within a community-based LIC model can be an effective learning experience for students and perceived positively by community-based faculty. This model may offer an opportunity to expand clinical learning experiences in EM.

Same but Different: Exploring Mechanisms of Learning in a Longitudinal Integrated Clerkship.

Longitudinal integrated clerkships (LICs) are a widely used method of delivering clerkship curricula. Although there is evidence that LICs work and core components of LIC training have been identified, there is insufficient understanding of which components are integral to why they work. To address this question, this research explored how students experienced the first year of an LIC program. The aim was to use participants' understanding of their learning experiences to identify potential mechanisms of the LIC curriculum model. Thirty-two interviews were conducted with 13 University of Toronto students, 7 LIC and 6 block rotation students from the same site, from October 2014 to September 2015. A thematic analysis was performed iteratively to explore participants' understanding of their key learning experiences and outcomes. Participants in both cohorts described their key learning outcome as integration and application of knowledge during patient care. Experiences supporting this outcome were articulated as longitudinal variable practice and continuity of relationships with preceptors and patients. Critically, these experiences manifested differently for the 2 cohorts. For block students, these learning experiences appeared to reflect the informal curriculum, whereas for LIC students, learning experiences were better supported by the LIC formal curriculum. The results illustrate the importance of learning experiences that support longitudinality and continuity. By also emphasizing variability and knowledge integration, they align with literature on expert development. Notably, many of the learning experiences identified resulted from informal learning and thus support going beyond the formal curriculum when evaluating the effectiveness of curricula.