Medical Knowledge Test Research Articles

Assessment of Resident and AI Chatbot Performance on the University of Toronto Family Medicine Residency Progress Test: Comparative Study.

Large language model (LLM)-based chatbots are evolving at an unprecedented pace with the release of ChatGPT, specifically GPT-3.5, and its successor, GPT-4. Their capabilities in general-purpose tasks and language generation have advanced to the point of performing excellently on various educational examination benchmarks, including medical knowledge tests. Comparing the performance of these 2 LLM models to that of Family Medicine residents on a multiple-choice medical knowledge test can provide insights into their potential as medical education tools. This study aimed to quantitatively and qualitatively compare the performance of GPT-3.5, GPT-4, and Family Medicine residents in a multiple-choice medical knowledge test appropriate for the level of a Family Medicine resident. An official University of Toronto Department of Family and Community Medicine Progress Test consisting of multiple-choice questions was inputted into GPT-3.5 and GPT-4. The artificial intelligence chatbot's responses were manually reviewed to determine the selected answer, response length, response time, provision of a rationale for the outputted response, and the root cause of all incorrect responses (classified into arithmetic, logical, and information errors). The performance of the artificial intelligence chatbots were compared against a cohort of Family Medicine residents who concurrently attempted the test. GPT-4 performed significantly better compared to GPT-3.5 (difference 25.0%, 95% CI 16.3%-32.8%; McNemar test: P<.001); it correctly answered 89/108 (82.4%) questions, while GPT-3.5 answered 62/108 (57.4%) questions correctly. Further, GPT-4 scored higher across all 11 categories of Family Medicine knowledge. In 86.1% (n=93) of the responses, GPT-4 provided a rationale for why other multiple-choice options were not chosen compared to the 16.7% (n=18) achieved by GPT-3.5. Qualitatively, for both GPT-3.5 and GPT-4 responses, logical errors were the most common, while arithmetic errors were the least common. The average performance of Family Medicine residents was 56.9% (95% CI 56.2%-57.6%). The performance of GPT-3.5 was similar to that of the average Family Medicine resident (P=.16), while the performance of GPT-4 exceeded that of the top-performing Family Medicine resident (P<.001). GPT-4 significantly outperforms both GPT-3.5 and Family Medicine residents on a multiple-choice medical knowledge test designed for Family Medicine residents. GPT-4 provides a logical rationale for its response choice, ruling out other answer choices efficiently and with concise justification. Its high degree of accuracy and advanced reasoning capabilities facilitate its potential applications in medical education, including the creation of exam questions and scenarios as well as serving as a resource for medical knowledge or information on community services.

Cognitive perspectives on maintaining physicians' medical expertise: II. Acquiring, maintaining, and updating cognitive skills.

Over the course of training, physicians develop significant knowledge and expertise. We review dual-process theory, the dominant theory in explaining medical decision making: physicians use both heuristics from accumulated experience (System 1) and logical deduction (System 2). We then discuss how the accumulation of System 1 clinical experience can have both positive effects (e.g., quick and accurate pattern recognition) and negative ones (e.g., gaps and biases in knowledge from physicians' idiosyncratic clinical experience). These idiosyncrasies, biases, and knowledge gaps indicate a need for individuals to engage in appropriate training and study to keep these cognitive skills current lest they decline over time. Indeed, we review converging evidence that physicians further out from training tend to perform worse on tests of medical knowledge and provide poorer patient care. This may reflect a variety of factors, such as specialization of a physician's practice, but is likely to stem at least in part from cognitive factors. Acquired knowledge or skills gained may not always be readily accessible to physicians for a number of reasons, including an absence of study, cognitive changes with age, and the presence of other similar knowledge or skills that compete in what is brought to mind. Lastly, we discuss the cognitive challenges of keeping up with standards of care that continuously evolve over time.

Knowledge of pressure injury in medical and surgical nurses in a tertiary level hospital: A cross-sectional study

BackgroundNurses play a vital role in pressure injury prevention (PIP) but require foundational knowledge to ensure appropriate PIP strategies are enacted. AimsTo describe and compare medical and surgical nurses’ knowledge of pressure injury (PI) in a tertiary level hospital in China, and to identify predictors of PI knowledge among these groups. DesignA cross-sectional survey was conducted between June and December 2020. MethodsRegistered nurses from nine medical and fifteen surgical wards in a tertiary hospital were invited. The survey was composed of two parts; demographic and professional characteristics; and the Chinese translated version of the Pressure Ulcer Knowledge Assessment Tool 2.0 (PUKAT 2.0) where the total score ranged from 0 to 25; higher scores imply more knowledge. Medical and surgical nurses' knowledge test scores were compared using independent t-test. Multiple linear regression analysis was used to determine factors predictive of nurses’ knowledge. ResultsIn total, 423 nurses from 24 wards participated the study and 404 nurses (95.5%) completed the knowledge test (Surgical n = 236, 58.4%; Medical n = 168, 41.6%). The PUKAT 2.0 mean score was 11.6 ± 3.0 (Surgical 12.2 ± 3.0; Medical 10.7 ± 2.8) with 335 (82.9%) nurses scoring <60%. Multiple linear regression showed working in surgical wards, nurse-in-charge position and previous PI training were significant predictors of knowledge scores. ConclusionKnowledge is a precursor to safe practice. Nurses demonstrated poor knowledge of PIP. Pressure injury related education may help improve nurses' knowledge but the extent to which it is used in place and impacts patients’ outcome requires more investigation.

A Validity Framework for Effective Analysis and Interpretation of Milestones Data.

A Validity Framework for Effective Analysis and Interpretation of Milestones Data.

Two Indias: The structure of primary health care markets in rural Indian villages with implications for policy

We visited 1519 villages across 19 Indian states in 2009 to (a) count all health care providers and (b) elicit their quality as measured through tests of medical knowledge. We document three main findings. First, 75% of villages have at least one health care provider and 64% of care is sought in villages with 3 or more providers. Most providers are in the private sector (86%) and, within the private sector, the majority are ‘informal providers' without any formal medical training. Our estimates suggest that such informal providers account for 68% of the total provider population in rural India. Second, there is considerable variation in quality across states and formal qualifications are a poor predictor of quality. For instance, the medical knowledge of informal providers in Tamil Nadu and Karnataka is higher than that of fully trained doctors in Bihar and Uttar Pradesh. Surprisingly, the share of informal providers does not decline with socioeconomic status. Instead, their quality, along with the quality of doctors in the private and public sector, increases sharply. Third, India is divided into two nations not just by quality of health care providers, but also by costs: Better performing states provide higher quality at lower per-visit costs, suggesting that they are on a different production possibility frontier. These patterns are consistent with significant variation across states in the availability and quality of medical education. Our results highlight the complex structure of health care markets, the large share of private informal providers, and the substantial variation in the quality and cost of care across and within markets in rural India. Measuring and accounting for this complexity is essential for health care policy in India.

Using an Adaptive, Self-Directed Web-Based Learning Module to Enhance Residents' Medical Knowledge Prior to a New Clinical Rotation.

The effect of self-directed adaptive learning on internal medicine residents' knowledge prior to a new clinical rotation is not known. We developed an adaptive, online, self-directed spaced repetition module and determined the effect on medical knowledge acquisition. We randomized postgraduate year 1 internal medicine residents into two groups. The intervention group (n = 27) received an electronic version of the clinical rotation curriculum as portable document format (PDF) files and participated in the online module, delivered via Moodle, a free, open-source learning management system. The non-intervention group (n = 27) only received the PDF files. All residents participated in a medical knowledge test at baseline and 3 months later. Both groups were similar at study baseline in terms of age, trainee type, years since graduation, results at United States Medical Licensing Examination (USMLE) Step 1, 2, In-Training Examination (ITE), and pre-intervention evaluation. There was a statistically significant improvement in scores on the post-intervention medical knowledge assessment for the intervention group when compared with the non-intervention group (24.2 ± 15.4% vs. 8.6 ± 9.9%, p < 0.001). An online, self-directed, adaptive spaced repetition-learning module can offer a simple and effective method to increase the medical knowledge present at the start of residents' clinical rotations.

Resident Competency and Proficiency in Combined Spinal-Epidural Catheter Placement Is Improved Using a Computer-Enhanced Visual Learning Program: A Randomized Controlled Trial.

Physician educators must balance the need for resident procedural education with clinical time pressures as well as patient safety and comfort. Alternative educational strategies, including e-learning tools, may be beneficial to orient novice learners to new procedures and speed proficiency. We created an e-learning tool (computer-enhanced visual learning [CEVL] neuraxial) to enhance trainee proficiency in combined spinal-epidural catheter placement in obstetric patients and performed a randomized controlled 2-center trial to test the hypothesis that use of the tool improved the initial procedure performed by the anesthesiology residents. Anesthesiology residents completing their first obstetric anesthesiology rotation were randomized to receive online access to the neuraxial module (CEVL group) or no access (control) 2 weeks before the rotation. On the first day of the rotation, residents completed a neuraxial procedure self-confidence scale and an open-ended medical knowledge test. Blinded raters observed residents performing combined spinal-epidural catheter techniques in laboring parturients using a procedural checklist (0-49 pts); the time required to perform the procedure was recorded. The primary outcome was the duration of the procedure. The CEVL group had significantly shorter mean (±standard deviation) procedure time compared to the control group 22.5 ± 4.9 vs 39.5 ± 7.1 minutes (P < .001) and had higher scores on the overall performance checklist 36.4 ± 6.6 vs 28.8 ± 7.1 (P = .012). The intervention group also had higher scores on the open-ended medical knowledge test (27.83 ± 3.07 vs 22.25 ± 4.67; P = .002), but self-confidence scores were not different between groups (P = .64). CEVL neuraxial is a novel prerotation teaching tool that may enhance the traditional initial teaching of combined spinal-epidural procedures in obstetric anesthesiology. Future research should examine whether the use of web-based learning tools impacts long-term provider performance or patient outcomes.

A holistic remediation intervention for struggling undergraduate medical students affords sustained Progress Test performance recovery

This article was migrated. The article was not marked as recommended. Introduction: Despite meeting the high standards required to study medicine, a proportion of students struggle to perform consistently to a satisfactory standard. Available literature promotes using more holistic remediation approaches to address any underpinning, often complex multiple interlinked factors associated with chronic student underperformance. Our holistic Level Two remediation intervention draws on principles from Self-Regulated Learner theory, Spaced-Retrieval Practice theory and Motivational Interviewing, and also explores health and wellbeing factors. In this study, we explored the effectiveness of our holistic "Level Two" remediation intervention, offered to chronically struggling Yr1-5 undergraduate medical students, by analysing Progress Test performances. Methods: For Year 1-5 students that were offered Level Two remediation, Progress Test performances pre- and post a Level Two remediation intervention were analysed between 2008-2017. Results: Over this period, 5.9% Year 1-5 students were offered Level Two remediation, with an associated 93% offer acceptance rate. Highest incidence of Level Two remediation occurred in Year 2, then declined significantly over Years 3-5. For all years of study, a significant and sustained enhancement of medical knowledge test performance occurred following remediation. This enhancement did not display any significant gender or ethinicity bias. Conclusions: We conclude that our holistic Level Two remediation pathway correlates with a significant and sustained enhancement of struggling medical students' academic performance.

An analysis of reflective writing early in the medical curriculum: The relationship between reflective capacity and academic achievement

Objective: To examine the relationship between reflection, gender, residency choice, word count, and academic achievement among medical students.Methods: A modified version of the Reflection Evaluation for Learners’ Enhanced Competencies Tool (REFLECT) was developed and used for this study (Cronbach’s alpha of 0.86 with an intraclass correlation coefficient [ICC] of 0.68). This was applied to writing samples about professionalism in gross anatomy from first-year medical students between 2005 and 2011. Four analysts reviewed and scored written reflections independently. Composite reflection scores were compared with gender, residency choice, length of written reflection, NBME® Gross Anatomy and Embryology Subject Examination scores, and final gross anatomy course.Results: Total of 319 written reflections were evaluated. Female students who pursued medicine specialties had the highest composite reflection scores (87 [27.2%]). Word count frequently correlated with reflection score (p < 0.0001). Students who performed well on the NBME® Gross Anatomy and Embryology Subject Examination tended to achieve high anatomy course grades (p < 0.0001). There was no statistically significant relationship between composite reflection scores and NBME® Gross Anatomy and Embryology Subject Examination scores (p = 0.16) or anatomy course grades (p = 0.90).Conclusions: This study suggests there are likely no correlations between reflective capacity and academic performance on tests of medical knowledge administered early in the medical curriculum.

A model of cardiopulmonary bypass staged training integrating technical and non-technical skills dedicated to cardiac trainees.

To develop a standardized simulation-based curriculum to teach medical knowledge and technical, communication and critical thinking skills necessary to initiate and wean from cardiopulmonary bypass (CPB) to junior cardiac trainees (CTs) in France. Performance on post-curricular tests was compared between CTs who participated in the new curriculum to those who did not. The simulation-based curriculum was developed by content and education experts. Simulations sequentially taught the skills necessary for initiating and weaning from CPB as well as managing crises by adding fidelity and complexity to scenarios. Nine CTs were randomly assigned to the new curriculum (n=5) or the traditional curriculum (n=4). Skills were assessed using tests of medical knowledge and technical, communication (GRS) and critical thinking (SCT) skills. A two-sample Wilcoxon rank-sum test compared average scores between the two groups. Alpha of 0.05 was set to indicate statistically significant differences. The resutls revealed that CTs in the new curriculum significantly outperformed CTs in the traditional curriculum on technical (18.2 vs 14.8, p=0.05) and communication (3.5 vs 2.2, p=0.013) skills. There was no significant difference between CTs in the new curriculum in the Script Concordance Test (16.5 vs 14.8, p=0.141) and knowledge tests (26.9 vs 24.6, p=0.14) compared to CTs in the traditional curriculum. Our new curriculum teaches communication and technical skills necessary for CPB. The results of this pilot study are encouraging and relevant. They give grounds for future research with a larger panel of trainees. Based on the current distribution of scores, a sample size of 12 CTs per group should yield significant results for all tests.

The survey on disaster medical rescue capacity of medical staffs in urban community health service centers of Jilin province

Objective To investigate the medical staffs' grasp of disaster medical rescue knowledge and skill in urban community health service centers of Jilin province.Methods The questionnaire about disaster medical rescue knowledge and skill was designed and 313 medical staffs of 20 urban community health service centers selected randomly from 9 cities/prefecture in Jilin province were investigated.Results The Cronbach's alpha value of the questionnaire was 0.838.The total score of disaster medical rescue knowledge and skill test for the medical staff was 17.83 ± 3.78.The results was compared by grouping,according to whether concerned the knowledge and information of disaster medical rescue.The difference was significant.Conclusions The disaster medical rescue capacity of medical staffs in urban community health service centers of Jilin province should be improved. Key words: Community health service center; Disaster; Medical rescue

Using automatic item generation to create multiple-choice test items

Many tests of medical knowledge, from the undergraduate level to the level of certification and licensure, contain multiple-choice items. Although these are efficient in measuring examinees' knowledge and skills across diverse content areas, multiple-choice items are time-consuming and expensive to create. Changes in student assessment brought about by new forms of computer-based testing have created the demand for large numbers of multiple-choice items. Our current approaches to item development cannot meet this demand. We present a methodology for developing multiple-choice items based on automatic item generation (AIG) concepts and procedures. We describe a three-stage approach to AIG and we illustrate this approach by generating multiple-choice items for a medical licensure test in the content area of surgery. To generate multiple-choice items, our method requires a three-stage process. Firstly, a cognitive model is created by content specialists. Secondly, item models are developed using the content from the cognitive model. Thirdly, items are generated from the item models using computer software. Using this methodology, we generated 1248 multiple-choice items from one item model. Automatic item generation is a process that involves using models to generate items using computer technology. With our method, content specialists identify and structure the content for the test items, and computer technology systematically combines the content to generate new test items. By combining these outcomes, items can be generated automatically.

P03.01. Characteristics of residents and training sites influence successful completion of the Integrative Medicine in Residency program

Methods Resident and residency site characteristics were examined to determine factors influencing completion of the Integrative Medicine in Residency (IMR) curriculum for the 2011 graduating class. Completion criteria included finishing ≥80% of the online courses plus a final score on the medical knowledge test of ≥70%. Resident characteristics used as predictors included demographics; medical school type [US MD, DO, foreign medical graduates (FMG)]; and, responses to a post-match survey administered before the residency began in 2008 on previous participation in CAM courses or experiences, importance of the IMR in choosing residency, and interests in learning IM and applying IM after graduation. Site characteristics included: extent of IM in the residency culture (e.g., faculty practicing IM consultations; IM consultation on site; an IM 4th year fellowship at the site); faculty characteristics (i.e., faculty leader IM fellowship trained; faculty leader with designated IM teaching time); and curriculum delivery (i.e., using the IMR plus other IM teaching/rotation/electives; monthly IM case conferences; resident IM clinical application).

Integrative Medicine in Residency Education: Developing Competency Through Online Curriculum Training

The Integrative Medicine in Residency (IMR) program, a 200-hour Internet-based, collaborative educational initiative was implemented in 8 family medicine residency programs and has shown a potential to serve as a national model for incorporating training in integrative/complementary/alternative medicine in graduate medical education. The curriculum content was designed based on a needs assessment and a set of competencies for graduate medical education developed following the Accreditation Council for Graduate Medical Education outcome project guidelines. The content was delivered through distributed online learning and included onsite activities. A modular format allowed for a flexible implementation in different residency settings. TO ASSESS THE FEASIBILITY OF IMPLEMENTING THE CURRICULUM, A MULTIMODAL EVALUATION WAS UTILIZED, INCLUDING: (1) residents' evaluation of the curriculum; (2) residents' competencies evaluation through medical knowledge testing, self-assessment, direct observations, and reflections; and (3) residents' wellness and well-being through behavioral assessments. The class of 2011 (n = 61) had a high rate of curriculum completion in the first and second year (98.7% and 84.2%) and course evaluations on meeting objectives, clinical utility, and functioning of the technology were highly rated. There was a statistically significant improvement in medical knowledge test scores for questions aligned with content for both the PGY-1 and PGY-2 courses. The IMR program is an advance in the national effort to make training in integrative medicine available to physicians on a broad scale and is a success in terms of online education. Evaluation suggests that this program is feasible for implementation and acceptable to residents despite the many pressures of residency.

University of New Mexico School of Medicine

University of New Mexico School of Medicine

Evaluation of a Web‐based Asynchronous Pediatric Emergency Medicine Learning Tool for Residents and Medical Students

To examine the effectiveness of an asynchronous learning tool consisting of web-based lectures for trainees covering major topics pertinent to pediatric emergency medicine (PEM) and to assess resident and student evaluation of this mode of education. PEM faculty and fellows created a 21-lecture, web-based curriculum. These 20-minute online lectures used Microsoft PowerPoint with the voice-over feature. A 75-question test was created to assess the effectiveness of the web-based learning model, administered online before and after the rotation in the pediatric emergency department (PED). All fourth-year medical students and residents (across all specialties) rotating through the PED were required to complete 10 of the 21 lectures during their 1-month rotation. The main outcome variable was difference in score between pre- and post-rotation tests of participants who viewed no lectures and those who viewed at least one lecture. Evaluation of the program was assessed by anonymous survey using 5-point discrete visual analog scales. Responses of 4 or 5 were considered positive for analysis. One hundred eleven residents and fourth-year medical students participated in the program. An initial 32 completed testing before implementation of the on-line lectures (March 2007-August 2007), and another five did not complete the on-line lectures after implementation (September 2007-February 2008). Seventy-one completed testing and on-line lectures, and all but three completed at least 10 on-line lectures during their rotation. Fourteen of 111 trainees did not complete the pre- or post-test (including two who viewed the lectures). The mean change in score was a 1% improvement from pre-test to post-test for trainees who viewed no lectures and a 6.2% improvement for those who viewed the lectures (mean difference = 5.2%, 95% confidence interval = 2.5% to 7.9%). In the linear regression model, the estimate of the coefficient was 0.43 (p < 0.001), meaning that, for each lecture viewed, post-test score rose by 0.43%. Sixty-nine of 75 test items (92%) had a point biserial correlation greater than 0.15. Thirty of the 72 trainees who completed the online lectures and testing (42%) returned surveys. All were comfortable using the Internet, and 87% (26/30) found the web-site easy to use. All felt that their educational goals were met, and 100% felt that the format would be useful in other areas of education. Although not a replacement for traditional bedside teaching, the use of web-based lectures as an asynchronous learning tool has a positive effect on medical knowledge test scores. Trainees were able to view online lectures on their own schedules, in the location of their choice. This is helpful in a field with shift work, in which trainees rarely work together, making it difficult to synchronously provide lectures to all trainees.

Are multiple choice tests fair to medical students with specific learning disabilities?

The purpose of multiple choice tests of medical knowledge is to estimate as accurately as possible a candidate's level of knowledge. However, concern is sometimes expressed that multiple choice tests may also discriminate in undesirable and irrelevant ways, such as between minority ethnic groups or by sex of candidates. There is little literature to establish whether multiple choice tests may also discriminate against students with specific learning disabilities (SLDs), in particular those with a diagnosis of dyslexia, and whether the commonly-used accommodations allow such students to perform up to their capability. We looked for evidence to help us determine whether multiple choice tests could be relied upon to test all medical students fairly, regardless of disability. We analyzed the mean scores of over 900 undergraduate medical students on eight multiple-choice progress tests containing 1,000 items using a repeated-measures analysis of variance. We included disability, gender and ethnicity as possible explanatory factors, as well as year group. There was no significant difference between mean scores of students with an SLD who had test accommodations and students with no SLD and no test accommodation. Virtually all students were able to complete the tests within the allowed time. There were no significant differences between the mean scores of known minority ethnic groups or between the genders. We conclude that properly-designed multiple-choice tests of medical knowledge do not systematically discriminate against medical students with specific learning disabilities.

Design, Implementation, and Outcome of a Hands-on Arthrocentesis Workshop

During a 4-week rheumatology elective at our institution, opportunities for internal medicine residents to perform arthrocentesis were limited, particularly for sites other than the knee. Consequently, residents were inadequately prepared and had less self-confidence to perform such procedures. To overcome these educational deficiencies, an arthrocentesis workshop was developed. We report our quality improvement data that was collected during the first year of workshop implementation. We devised a structured half-day arthrocentesis workshop for rheumatology fellows as well as rotating internal medicine residents. This program consisted of a one hour lecture immediately followed by a hands-on workshop that used mannequin models for 5 anatomic sites. A self-assessment questionnaire and medical knowledge test were administered before and after each session. The accuracy of the self-assessment questionnaire was analyzed by comparing responses to an external objective measure of knowledge in the same content area. Finally, an optional postworkshop survey addressed resident satisfaction. Thirty-eight trainees participated in the workshop between July 2006 and June 2007. There were statistically significant improvements in self-confidence in 9 content areas (P < 0.0002), cognitive testing (P < 0.0001) and in self-assurance of procedural skill at all anatomic sites. A high degree of discordance was found between the perceived level of competence and the actual performance on the medical knowledge test during the preworkshop analysis. In contrast, the postworkshop analysis displayed modestly higher concordance. All residents completing a postworkshop survey believed that it was a useful exercise, and 96% stated that they would change their practice habits. The arthrocentesis workshop provided a solid foundation from which trainees can learn key concepts of joint injection, increase their self-confidence and refine their motor skills. The accuracy of resident self-reported confidence is poor and should therefore be used only to complement other means of competency assessment and medical knowledge acquisition.

Quality of care evaluation in France

Monitoring and enhancing physician performance is a major challenge. Several countries have developed programs for maintenance of certification, which may include written tests of medical knowledge, tests of clinical skills, and ratings by patients and peers, among other elements. The importance of

Didactic Teaching Conferences for IM Residents

Didactic teaching conferences are a cornerstone of residents' training in internal medicine, yet these programs have received little formal evaluation. This study determines the factors associated with residents' attendance at didactic teaching conferences and the relationship of attendance to residents' scores on medical certification examinations. The attendance of 81 residents was recorded at 199 conferences at one university hospital's internal medicine residency program during the 1996-97 academic year. Characteristics of the conferences were recorded, including the date, whether lunch was provided, the daily census on the medicine general wards, daily ambient temperature, and conference type. Residents' scores on the United States Medical Licensing Examination (USMLE) Step 2 and American Board of Internal Medicine (ABIM) certification examination were collected. Residents' attendance at conferences was 34% overall or 54% excluding excused absences. Adjusting for the conferences' and residents' characteristics, attendance declined by one third as the year progressed. Providing lunch at noon conferences enhanced residents' attendance (odds ratio [OR] 1.26 overall and OR 1.64 for residents on inpatient rotations). Higher attendance was not associated with improvement in standardized medical knowledge test scores. Absenteeism and waning attendance through the year have important implications for structuring didactic internal medicine teaching. Providing lunch improves attendance, but this incentive should be weighed against the potential burdens of the pharmaceutical industry's funding of these lunches. The lack of relationship between attendance and residents' adjusted board scores calls for a better understanding of the value of this high-intensity medical education intervention.