Online Question Banks Research Articles

The Comparative Performance of Large Language Models on the Hand Surgery Self-Assessment Examination.

Generative artificial intelligence (AI) models have emerged as capable of producing human-like responses and have showcased their potential in general medical specialties. This study explores the performance of AI systems on the American Society for Surgery of the Hand (ASSH) Self-Assessment Exams (SAE). ChatGPT 4.0 and Bing AI were evaluated on a set of multiple-choice questions drawn from the ASSH SAE online question bank spanning 5 years (2019-2023). Each system was evaluated with 999 questions. Images and video links were inserted into question prompts to allow for complete AI interpretation. The performance of both systems was standardized using the May 2023 version of ChatGPT 4.0 and Microsoft Bing AI, both of which had web browsing and image capabilities. ChatGPT 4.0 scored an average of 66.5% on the ASSH questions. Bing AI scored higher, with an average of 75.3%. Bing AI outperformed ChatGPT 4.0 by an average of 8.8%. As a benchmark, a minimum passing score of 50% was required for continuing medical education credit. Both ChatGPT 4.0 and Bing AI had poorer performance on video-type and image-type questions on analysis of variance testing. Responses from both models contained elements from sources such as PubMed, Journal of Hand Surgery, and American Academy of Orthopedic Surgeons. ChatGPT 4.0 with browsing and Bing AI can both be anticipated to achieve passing scores on the ASSH SAE. Generative AI, with its ability to provide logical responses and literature citations, presents a convincing argument for use as an interactive learning aid and educational tool.

Artificial intelligence in orthopaedic education: A comparative analysis of ChatGPT and Bing AI's Orthopaedic In‐Training Examination performance

AbstractBackgroundThis study evaluated the performance of generative artificial intelligence (AI) models on the Orthopaedic In‐Training Examination (OITE), an annual exam administered to U.S. orthopaedic residency programs.MethodsChatGPT 3.5 and Bing AI GPT 4.0 were evaluated on standardised sets of multiple‐choice questions drawn from the American Academy of Orthopaedic Surgeons OITE online question bank spanning 5 years (2018–2022). A total of 1165 questions were posed to each AI system. The performance of both systems was standardised using the latest versions of ChatGPT 3.5 and Bing AI GPT 4.0. Historical data of resident scores taken from the annual OITE technical reports was used as a comparison.ResultsAcross the five datasets, ChatGPT 3.5 scored an average of 55.0% on the OITE questions. Bing AI GPT 4.0 scored higher with an average of 80.0%. In comparison, the average performance of orthopaedic residents in national accredited programs was 62.1%. Bing AI GPT 4.0 outperformed ChatGPT 3.5 and Accreditation Council for Graduate Medical Education examinees, and analysis of variance analysis demonstrated p < 0.001 among groups. The best performance was by Bing AI GPT 4.0 on OITE 2020.ConclusionGenerative AI can provide a logical context across answer responses through its in‐depth information searches and citation of resources. This combination presents a convincing argument for the possible uses of AI in medical education as an interactive learning aid.

In-depth analysis of ChatGPT’s performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions

ChatGPT has garnered attention as a multifaceted AI chatbot with potential applications in medicine. Despite intriguing preliminary findings in areas such as clinical management and patient education, there remains a substantial knowledge gap in comprehensively understanding the chances and limitations of ChatGPT’s capabilities, especially in medical test-taking and education. A total of n = 2,729 USMLE Step 1 practice questions were extracted from the Amboss question bank. After excluding 352 image-based questions, a total of 2,377 text-based questions were further categorized and entered manually into ChatGPT, and its responses were recorded. ChatGPT’s overall performance was analyzed based on question difficulty, category, and content with regards to specific signal words and phrases. ChatGPT achieved an overall accuracy rate of 55.8% in a total number of n = 2,377 USMLE Step 1 preparation questions obtained from the Amboss online question bank. It demonstrated a significant inverse correlation between question difficulty and performance with rs = -0.306; p < 0.001, maintaining comparable accuracy to the human user peer group across different levels of question difficulty. Notably, ChatGPT outperformed in serology-related questions (61.1% vs. 53.8%; p = 0.005) but struggled with ECG-related content (42.9% vs. 55.6%; p = 0.021). ChatGPT achieved statistically significant worse performances in pathophysiology-related question stems. (Signal phrase = “what is the most likely/probable cause”). ChatGPT performed consistent across various question categories and difficulty levels. These findings emphasize the need for further investigations to explore the potential and limitations of ChatGPT in medical examination and education.

Educational Limitations of ChatGPT in Neurosurgery Board Preparation.

Objective This study evaluated the potential of Chat Generative Pre-trained Transformer (ChatGPT) as an educational tool for neurosurgery residents preparing for the American Board of Neurological Surgery (ABNS) primary examination. Methods Non-imaging questions from the Congress of Neurological Surgeons (CNS) Self-Assessment in Neurological Surgery(SANS) online question bank were input into ChatGPT. Accuracy was evaluated and compared to human performance across subcategories. To quantify ChatGPT's educational potential, the concordance and insight of explanations were assessed by multiple neurosurgical faculty. Associations among these metrics as well as question length were evaluated. Results ChatGPT had an accuracy of 50.4% (1,068/2,120), with the highest and lowest accuracies in the pharmacology (81.2%, 13/16) and vascular (32.9%, 91/277) subcategories, respectively. ChatGPT performed worse than humans overall, as well as in the functional, other, peripheral, radiology, spine, trauma, tumor, and vascular subcategories. There were no subjects in which ChatGPT performed better than humans and its accuracy was below that required to pass the exam. The mean concordance was 93.4% (198/212) and the mean insight score was 2.7. Accuracy was negatively associated with question length (R2=0.29, p=0.03) but positively associated with both concordance (p<0.001, q<0.001) and insight (p<0.001, q<0.001). Conclusions The current study provides the largest and most comprehensive assessment of the accuracy and explanatory quality of ChatGPT in answering ABNS primary exam questions. The findings demonstrate shortcomings regarding ChatGPT's ability to pass, let alone teach, the neurosurgical boards.

CyberQ: Generating Questions and Answers for Cybersecurity Education Using Knowledge Graph-Augmented LLMs

Building a skilled cybersecurity workforce is paramount to building a safer digital world. However, the diverse skill set, constantly emerging vulnerabilities, and deployment of new cyber threats make learning cybersecurity challenging. Traditional education methods struggle to cope with cybersecurity's rapidly evolving landscape and keep students engaged and motivated. Different studies on students' behaviors show that an interactive mode of education by engaging through a question-answering system or dialoguing is one of the most effective learning methodologies. There is a strong need to create advanced AI-enabled education tools to promote interactive learning in cybersecurity. Unfortunately, there are no publicly available standard question-answer datasets to build such systems for students and novice learners to learn cybersecurity concepts, tools, and techniques. The education course material and online question banks are unstructured and need to be validated and updated by domain experts, which is tedious when done manually. In this paper, we propose CyberGen, a novel unification of large language models (LLMs) and knowledge graphs (KG) to generate the questions and answers for cybersecurity automatically. Augmenting the structured knowledge from knowledge graphs in prompts improves factual reasoning and reduces hallucinations in LLMs. We used the knowledge triples from cybersecurity knowledge graphs (AISecKG) to design prompts for ChatGPT and generate questions and answers using different prompting techniques. Our question-answer dataset, CyberQ, contains around 4k pairs of questions and answers. The domain expert manually evaluated the random samples for consistency and correctness. We train the generative model using the CyberQ dataset for question answering task.

Perceptions and use of evidence-based revision methods among undergraduate medical students.

To assess awareness, perceptions and use related to evidence-based revision methods by undergraduate medical students. The descriptive cross-sectional study was conducted in three medical colleges of Rawalpindi, Islamabad, Pakistan, from December 01, 2019, to January 31, 2020, after approval from the ethics review committee of Army Medical College, Rawalpindi. The sample comprised undergraduate medical students of either gender. Data was collected online using a 10-item standardised questionnaire. Students were asked about the revision methods they used routinely and their perceptions of conventional and evidence-based revision methods. Data was analysed using SPSS 23. Of the total 136 respondents, 92(67%) were females and 44(32.3%) were male students. The response from preclinical and clinical years was 67 (50.7%) and 69(49.2%), respectively. Highlighting was the most widely used revision method among students (70 (51%) students), followed by re-reading important points (65 (47.7%) students). 126 (92%) students had the opinion that conventional revision methods were effective learning tools. Only 52 (38.2%) students were aware of the term 'evidence-based revision methods'. Digital tools based on principles of evidence-based revision were used by a minority of students which included use of online question banks by 21 (15.4%) students, osmosis by 40 (29.4%) students, sketchy pharma by 35 (25%)students, flashcards by 19 (14%) and picmonic by only 3 (2.2%). More than 114 (80%) students responded that they wanted evidence-based revision methods to be incorporated into curriculum, and 116 (85%) students said they would like to have a workshop on these techniques. Most medical students were not aware of and were not using evidence-based revision methods, and relied on conventional revision tools. They were, however, eager to learn about newer revision strategies.

Multiple Choice Question Difficulty Level Classification with Multi Class Confusion Matrix in the Online Question Bank of Education Gallery

Multiple Choice Question Difficulty Level Classification with Multi Class Confusion Matrix in the Online Question Bank of Education Gallery

THU001 Novel Interactive Text-messaging Curriculum For Endocrinology Board Review

Abstract Disclosure: P. Majety: None. A. Ajayi: None. A. Modest: None. M. Vamvini: Grant Recipient; Self; MV was supported by F32-DK126432, and Joslin Diabetes Center P&amp;F. J. Freed: None. Background: The ABIM certification exam is one of the measures to ensure that physicians have the clinical skills for good care delivery. The five-year average pass rate for the ABIM Endocrinology &amp; Metabolism subspecialty exam is 86%, relatively lower when compared to other subspecialties. The pass rate for first time test takers significantly decreased from a usual range of 84-91% over 2017-2020 to a nadir of 74% in 2021 compared to a mean of 84% for other medicine subspecialties. This represents the lowest pass rate in 2021 for any internal medicine subspecialty exam. Objectives: The primary purpose of our study was to assess the feasibility of a unique text messaging curriculum for endocrinology fellows and its utility in improving their knowledge. We also assessed their study habits and resources available to them for ABIM subspecialty exam preparation. Methods: In 2021, endocrinology fellows from 51 programs across the country were invited to participate in our curriculum. They completed a pre-test, joined a texting group via Remind application and received 1 multiple choice question daily (total n = 78). At the end of curriculum, fellows received a post-test questionnaire, and a feedback survey. After 15 weeks, they completed a post-test and survey. Paired results from pre- and post-test were compared. Results: A total of 89 fellows from 27 programs responded. Of these, 82 fellows, predominantly females (n=60; 73%), filled out the pre-test. On an average, 42 fellows (SD = 12) responded to the questions daily. Thirty fellows completed the post-test. The median number of correct responses on the pre-test was 5 (IQR 3-6), compared to 8 (IQR 6-9) in the post-test. There was a significant improvement (p-value &amp;lt;0.0001) in fellows’ performance in the post-test when compared with the pre-test following our intervention.Final feedback survey was filled out by 33 of them. Ninety-seven percent responded that they use question banks as a resource for exam preparation followed by PowerPoint presentations (52%), journals (48%), online videos (39%), textbooks (33%) and lecture notes (33%). 23 fellows (70%) preferred mobile friendly question bank applications; 5 fellows (15%) preferred online question banks available through company websites only. None of them preferred paper-based question banks.Nineteen of the 33 fellows (58%) found the curriculum very useful. 17 fellows (52%) said it served as a discussion tool sometimes, while 9 fellows (27%) said they used it as a discussion tool often. Thirty two out of 33 fellows (97%) responded that they would like to continue to receive a question daily for the rest of their fellowship. Conclusions: Text-messaging based curriculum for exam preparation is feasible and can improve test performance. Fellows find receiving a daily high yield multiple choice question via text-message as a useful tool for exam preparation. Presentation: Thursday, June 15, 2023

Evaluating the Performance of ChatGPT in Ophthalmology: An Analysis of Its Successes and Shortcomings

Foundation models are a novel type of artificial intelligence algorithms, in which models are pretrained at scale on unannotated data and fine-tuned for a myriad of downstream tasks, such as generating text. This study assessed the accuracy of ChatGPT, a large language model (LLM), in the ophthalmology question-answering space. Evaluation of diagnostic test or technology. ChatGPT is a publicly available LLM. We tested 2 versions of ChatGPT (January 9 "legacy" and ChatGPT Plus) on 2 popular multiple choice question banks commonly used to prepare for the high-stakes Ophthalmic Knowledge Assessment Program (OKAP) examination. We generated two 260-question simulated exams from the Basic and Clinical Science Course (BCSC) Self-Assessment Program and the OphthoQuestions online question bank. We carried out logistic regression to determine the effect of the examination section, cognitive level, and difficulty index on answer accuracy. We also performed a post hoc analysis using Tukey's test to decide if there were meaningful differences between the tested subspecialties. We reported the accuracy of ChatGPT for each examination section in percentage correct by comparing ChatGPT's outputs with the answer key provided by the question banks. We presented logistic regression results with a likelihood ratio (LR) chi-square. We considered differences between examination sections statistically significant at a P value of < 0.05. The legacy model achieved 55.8% accuracy on the BCSC set and 42.7% on the OphthoQuestions set. With ChatGPT Plus, accuracy increased to 59.4% ± 0.6% and 49.2% ± 1.0%, respectively. Accuracy improved with easier questions when controlling for the examination section and cognitive level. Logistic regression analysis of the legacy model showed that the examination section (LR, 27.57; P= 0.006) followed by question difficulty (LR, 24.05; P < 0.001) were most predictive of ChatGPT's answer accuracy. Although the legacy model performed best in general medicine and worst in neuro-ophthalmology (P < 0.001) and ocular pathology (P= 0.029), similar post hoc findings were not seen with ChatGPT Plus, suggesting more consistent results across examination sections. ChatGPT has encouraging performance on a simulated OKAP examination. Specializing LLMs through domain-specific pretraining may be necessary to improve their performance in ophthalmic subspecialties. Proprietary or commercial disclosure may be found after the references.

A Longitudinal Evaluation of a Multimodal POCUS Curriculum in Pediatric Residents.

Introduction: Pediatric residency programs often do not include a point of care ultrasound (POCUS) curriculum. We analyzed a novel POCUS curriculum for pediatric residents that incorporated an online question bank (QB), in addition to a traditional teaching model of didactic instruction and hands-on learning experience. Methods: Four high-yield POCUS topics were chosen: Focused Assessment by Sonography for Trauma (FAST), soft tissue, lung, and cardiac. Residents completed online multiple-choice quizzes before and after each of four in-person learning sessions, taught by ultrasound faculty and fellows. At the end of the academic year participants completed a knowledge retention quiz. Confidence surveys were administered to participants throughout the course of the study. Differences in means were compared by Student's t-test. Results: Learners demonstrated post-intervention score improvement for each of the four modules. Retention testing demonstrated retained improvement for the soft tissue and cardiac modules, but not for the FAST module. Self-reported confidence increased across all four topics. Conclusion: A multimodal POCUS curriculum utilizing a combination of an online QB and in-person teaching demonstrated lasting knowledge for pediatric trainees.

Blended learning in undergraduate dental education: a global pilot study

ABSTRACT Aims To explore the global trends in blended learning in undergraduate dental education during the COVID pandemic and during the recovery phase by engaging with the students and faculty and evaluate the implications for dental education in the post-COVID era. Methods It was a pilot cross-sectional study which employed a convenience sampling technique to recruit representatives of dental faculty and undergraduate students in 80 dental institutions globally. A previously validated questionnaire consisting of a combination of closed and open-ended items was used for data collection. Responses to these online questionnaires were processed and analysed using the R statistical computing environment. Results A total of 320 dental students and 169 faculty members from 47 different dental institutions participated in the study. Video and Live Online Tutorials were considered to be the most effective method of online learning followed by online question banks by both groups. Significant differences were noted between faculty and students regarding time spent and effectiveness of online teaching and learning, respectively, both before and after the start of COVID. The results highlight the faculty need to engage more closely with the students to address their learning needs. Finally, the participants provided several recommendations regarding the future development of teaching and learning strategies as well as assessments in the post-pandemic era. Conclusions This is the first study which explores blended learning in dental education with participants from multiple institutions in different regions of the globe. Compared to the faculty, students considered online learning to be less interactive and preferred learning activities and all assessments to be delivered face-to-face. The results underscore the need to adapt teaching practices to suit the learning needs of the students.

The learning preferences of aspiring orthopaedic surgeons in the UK.

The workload of orthopaedic trainees is rapidly increasing at the expense of training. Trainees are expected to assimilate large quantities of information efficiently. This prospective cohort study explores the learning styles, resource preferences and educational needs of aspiring orthopaedic trainees. A 21-item questionnaire was distributed to delegates of an orthopaedic teaching series. Data were obtained pertaining to demographics, visual, aural, read/write and kinaesthetic learning styles, study materials used and teaching exposure. Participants had a strong preference for visual (48.0%) and kinaesthetic (43.0%) learning modalities. Most participants prepared for written exams using online question banks (85.9%), clinical exams using question banks (37.5%) and discussion with colleagues (27.3%), and surgical procedures intraoperatively (43.8%). Only 12.4% of participants felt the teaching they received was consistently adapted to their visual, aural, read/write or kinaesthetic learning preference. The surgical landscape is rapidly changing. It is vital that trainers take into consideration how budding orthopaedic surgeons learn and make the relevant accommodations, to ensure optimal learning.

Impact of an Online Question Bank on Resident In-Training Exam Performance.

In-training exams (ITEs) are administered annually to Obstetrics and Gynecology (OBGYN) residents and have been demonstrated to correlate with success on licensing examinations. Our study objective was to determine the impact of a question bank and mock exam on the performance of Council on Resident Education in Obstetrics and Gynecology (CREOG) ITEs. Secondarily, we investigated the correlation between the extent of question bank usage and performance on the exam. Pre-post intervention study of resident performance on CREOG ITE before and after implementation of the question bank and mock ITE at Indiana University in 2018. Performance was measured as year-to-year improvement in percent correct on ITE exams. Scores were excluded if a resident did not have a prequestion bank score report or if they did not sit for all eligible ITE exams. There were 51 OBGYN residents at Indiana University during the study period, with 38 available for analysis (75%). Before implementation, average year-to-year improvement for PGY1-2, PGY2-3 and PGY3-4 classes were 0.60%, 1.0% and -1.6%, respectively. After implementation, all resident classes had significant improvements in ITE scores of 6.6% (P < .01), 9.0% (P < .01), and 7.2% (P < .01), respectively. There was a moderate program-wide correlation between the number of questions completed and the percent improvement on the ITE of R = 0.36 (P = .046). Our study demonstrated that access to a question bank and mock ITE significantly improved CREOG ITE performance of OBGYN residents at Indiana University.

Analysis of the Foot and Ankle Questions on the Orthopaedic In-Training Examination Over a 10-Year Period (2012-2021)

Category: Other; Ankle; Hindfoot; Midfoot/Forefoot Introduction/Purpose: The Orthopaedic In-Training Examination (OITE) was first administered in 1963 by the American Academy of Orthopaedic Surgeons to evaluate and monitor the training and knowledge of orthopaedic surgery residents. After various iterations of the exam, the current OITE consists of 275 questions covering foot and ankle, hip and knee, shoulder, and elbow, spine, sports medicine, trauma, and basic science. The most recent investigation of the foot and ankle OITE questions evaluated content, references, and treatment methods included on the exams from 2006-2010. This study evaluates OITE test questions over a 10-year period from 2012-2021 for topic, taxonomy, treatment tested, imaging modality, and references cited. The goal of this analysis is to inform and optimize foot and ankle education among orthopaedic residency and fellowship programs. Methods: OITE questions between 2012 and 2021 were obtained through the AAOS 'ResStudy' online question bank. Each question was manually evaluated by each author for foot and ankle content. The total number of questions pertaining to foot and ankle were recorded for each year and the topic of each question was classified under broader topic areas: trauma, deformity, tendon disorder, diabetic foot, sports, great toe, lesser toe, pediatric foot, heel pain, arthritis, and tumor. Imaging techniques were classified as radiographs, magnetic resonance imaging (MRI), computerized tomography, and clinical photo or video. Question taxonomy was classified using criteria in a previous OITE question analysis with three groups2. Taxonomy 1 (T1): knowledge (requires recall of orthopaedic practice or literature); T2: comprehension (interpretation of information and/or diagnosis); T3: application (formulation of a treatment plan from provided information). References provided by AAOS were identified by journal title and publication date or textbook title and editors. Results: Overall, foot and ankle questions comprised of 11.5% (308/2679) of all the graded OITE questions. In recent years, the proportion of foot and ankle questions has decreased. From 2012-2017 the average proportion of foot and ankle questions was 12.6% per test, and from 2018-2021, the average proportion was 9.8%. The most common question taxonomy from 2012-2017 was T3 (application) comprising 60.9% of the question, however, from 2018-2021, T3 questions made up only 29.4% of foot and ankle questions. In total, 59.1% (182/308) of the questions required knowledge of treatment modalities, most commonly open reduction internal fixation and arthrodesis procedures. Imaging modalities were tested 67.9% (209/308) of the time, with radiographs appearing in 47.4% (146/308) of all questions. The most common journals cited were Foot and Ankle International, Journal of the American Academy of Orthopaedic Surgeons, and the Journal of Bone and Joint Surgery American Volume. Conclusion: The proportion of foot and ankle questions on the OITE has decreased over the past 10 years and there has been a reduced emphasis on application questions with a more balanced distribution among knowledge, comprehension and application questions recently. Residents and residency programs should ensure proficiency in interpreting radiographs, which are tested on almost half of all foot and ankle OITE questions.

Medical students' perspectives on how COVID-19 has impacted their otolaryngology educational experience: A nationwide survey study.

ObjectivesTo evaluate how medical students perceive COVID-19 has impacted various otolaryngology educational experiences and appreciate useful adaptations/resources that can be implemented moving forward.MethodsWe circulated a web-based questionnaire among pre-clinical/clinical medical students. Participants were asked to rate their experiences in the realms of research, clinical exposure, faculty interaction, interest group (IG) activities, advising, and USMLE preparation considering the COVID-19 pandemic. Qualitative responses were mapped into thematic domains.ResultsCompleted responses from 201 medical students demonstrated an effect across all educational activities. Pre-clinical students experienced a negative impact on their IG involvement (77%; 86/111) and clinical students reported a diminished clinical experience (60%; 40/67), with 54% (36/67) citing below-expectation surgical exposure. Overall, 51% (90/178) of students reported a perceived decrease in beneficial otolaryngology career advising, decreased research involvement (57%, 101/178), and reduced faculty interaction (79%, 141/178). Faculty mentorship programs, curated video resources, and online question banks were selected as valuable resources during COVID-19 (21%, 20%, and 13% of selections, respectively) that may offer ongoing benefit. Free-response feedback also indicated that COVID-19 negatively impacted students’ mental well-being and inhibited opportunities to complete away sub-internships. As for supplemental resources, students appreciated virtual grand rounds/teaching, interest group events, online resources, and meetings with attendings/residents.ConclusionsAccording to students, COVID-19 negatively impacted a broad array of otolaryngology educational experiences including research, faculty interaction, and interest group involvement. Students also indicated several well-received adaptations, such as virtual mentorship programs, curated video materials, remote research opportunities, and online question banks. This feedback helps contextualize selection committee decisions and, importantly, assists clerkship directors and undergraduate medical educators to optimize their curriculums.

Impact of Question Bank Use for In-Training Examination Preparation by OBGYN Residents – A Multicenter Study

To examine the impact of access to and utilization of a commercially available question bank (TrueLearn) for in-training examination (ITE) preparation in Obstetrics and Gynecology (OBGYN). This was a retrospective cohort study examining the impact of TrueLearn usage on ITE examination performance outcomes. Produced by the educational arm of the American College of Obstetricians and Gynecologists, the Council on Resident Education in Obstetrics and Gynecology (CREOG) exam is a multiple-choice test given to all residents annually. Residency programs participating in this study provided residency program mean CREOG scores from the year prior (2015), and the first (2016) and second (2017) years of TrueLearn usage. Programs also contributed resident-specific CREOG scores for each resident for 2016 and 2017. This data was combined with each resident's TrueLearn usage data that was provided by TrueLearn with residency program consent. The CREOG scores consisted of the CREOG score standardized to all program years, the CREOG score standardized to the same program year (PGY) and the total percent (%) correct. TrueLearn usage data included number of practice questions completed, number of practice tests taken, average number of days between successive tests, and percent correct of answered practice questions. OBGYN Residency Training Programs. OBGYN residency programs that purchased and utilized TrueLearn for the 2016 CREOG examination were eligible for participation (n = 14). Ten residency programs participated, which consisted of 212 residents in 2016 and 218 residents in 2017. TrueLearn was used by 78.8% (167/212) of the residents in 2016 and 84.9% (185/218) of the residents in 2017. No significant difference was seen in the average CREOG scores available on a per- program level before versus after the first year of implementation either using the CREOG score standardized to all PGYs (mean difference 1.0; p = 0.58) or standardized to the same PGY (mean difference 3.1; p = 0.25). Using resident-level data, there was no significant difference in mean CREOG score standardized to all PGYs between users and non-users of TrueLearn in 2016 (mean, 199.4 vs 196.7; p = 0.41) or 2017 (mean, 198.2 vs 203.4; p = 0.19). The percent of practice questions answered correctly on TrueLearn was positively correlated with the CREOG score standardized to all PGYs (r = 0.47 for 2016 and r = 0.60 for 2017), as well as with the CREOG total percent correct (r = 0.47 for 2016 and r = 0.61 for 2017). Based on a simple linear regression, for every 500 practice questions completed, the CREOG score significantly increased for PGY-2 residents by an average (±SE) of 7.3 ± 2.8 points (p = 0.013); the average increase was 0.7 ± 2.5 (p = 0.79) for PGY-3 residents and 5.8 ± 3.3 points (p = 0.09) for PGY-4 residents. Adoption of an online question bank did not result in higher mean CREOG scores at participating institutions. However, performance on the TrueLearn questions correlated with ITE performance, supporting predictive validity and the use of this question bank as a formative assessment for resident education and exam preparation.

Construction of Online and Offline mixed courses for Business English Writing in the New Era of Internet Economy

This paper gives some suggestion on the construction of online and offline mixed courses for business English writing in the new era of network economy. This paper focuses on analying the existing problems of application of mixed courses in Business English Writing and giving some specific suggestion including the constructon of students‘online teaching and training practice. In the construction of students’online -learning resources library, this paper suggests that workers should build the electrontic textbook database, optimize online course recording and construct various online question banks, which can be helpful to the teaching of mixed course of bussiness English writing.

Expanding Free Open-Access Medical Education.

During my medical school cardiology placement, I remember our clinical tutor printing us out the cardiovascular examination checklist from Geeky MedicsÓ as our learning tool for the term. When I would study later for my pre-clinical exams, AmbossÓ and UpToDateÓ were the first and most useful resources that I looked at, rather than my course lectures. When I would study for finals, the advice from seniors was to get through the core lectures from the university quickly, and then to move onto online question banks. My experience was not unique; I would rather confidently say it characterised the learning and revision approach of most of my peers. The growing use of FOAM resources by students and the concerns justify the need for academic institutes to take an active role in developing them. Faculties should consider investing in developing FOAM resources to empower their faculty in adopting a new era and dominant pedagogy in medical education. This will not only benefit their own students, who are increasingly turning to such resources for their education, but also help these faculties become more active members in the global community of medical educators.

The impact of COVID-19 on surgical education: perspectives from Canadian general surgery residents.

SummaryMost institutions have mitigated the impact of the COVID-19 pandemic on residency education by transitioning to web-based educational platforms and using innovative solutions, such as surgical video libraries, telehealth clinics, online question banks via social media platforms, and procedural simulations. Here, we assess the perceived impact of COVID-19 on Canadian surgical residency education and discuss the unique challenges in adapting to a virtual format and how novel training methods implemented during the pandemic may be useful in the future of surgical education.

Student Perspectives on Remote Medical Education in Clinical Core Clerkships During the COVID-19 Pandemic.

PurposeTo assess student perceptions of remote learning curricula implemented by clinical clerkships at a single US medical school during the COVID-19 pandemic.MethodStudents enrolled in core clinical clerkships at the UC San Diego School of Medicine from March to April 2020 were sent an anonymous online survey, assessing components of remote learning curricula via quantitative rating and free-response questions.ResultsOf 132 students enrolled, 96 (73%) completed the survey. Online question banks, remote progress reviews, and telehealth sessions were rated as the most valuable resources, while textbooks were rated least valuable. Most students felt that the number of resources was adequate (69/94, 74%), the quantity of required content was appropriate (77/93, 83%), and there was a good balance of rigidity versus flexibility in the schedule (81/94, 86%). Most students felt “prepared” or “somewhat prepared” for the shelf examination (61/91, 67%) and for the next year of medical training (73/94, 78%). In narrative reflections, students valued the flexibility afforded by remote curricula but desired some weekly structure. Students enjoyed expanded access to learning resources and participating in interactive remote sessions. Students expressed anxiety about the pandemic and uncertainty regarding medical education, but transparent communication from clerkship directors helped assuage these concerns.ConclusionsStudent participation in direct patient care is fundamental to medical school core clerkships and was negatively impacted by the COVID-19 pandemic. Strategic incorporation of select remote learning components into clinical clerkships may permit a favorable student experience even when opportunities for on-site participation are limited.Electronic supplementary materialThe online version of this article (10.1007/s40670-020-01114-9) contains supplementary material, which is available to authorized users.