Question Of Style Research Articles

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis.

The United States Medical Licensing Examination (USMLE) has been critical in medical education since 1992, testing various aspects of a medical student's knowledge and skills through different steps, based on their training level. Artificial intelligence (AI) tools, including chatbots like ChatGPT, are emerging technologies with potential applications in medicine. However, comprehensive studies analyzing ChatGPT's performance on USMLE Step 3 in large-scale scenarios and comparing different versions of ChatGPT are limited. This paper aimed to analyze ChatGPT's performance on USMLE Step 3 practice test questions to better elucidate the strengths and weaknesses of AI use in medical education and deduce evidence-based strategies to counteract AI cheating. A total of 2069 USMLE Step 3 practice questions were extracted from the AMBOSS study platform. After including 229 image-based questions, a total of 1840 text-based questions were further categorized and entered into ChatGPT 3.5, while a subset of 229 questions were entered into ChatGPT 4. Responses were recorded, and the accuracy of ChatGPT answers as well as its performance in different test question categories and for different difficulty levels were compared between both versions. Overall, ChatGPT 4 demonstrated a statistically significant superior performance compared to ChatGPT 3.5, achieving an accuracy of 84.7% (194/229) and 56.9% (1047/1840), respectively. A noteworthy correlation was observed between the length of test questions and the performance of ChatGPT 3.5 (ρ=-0.069; P=.003), which was absent in ChatGPT 4 (P=.87). Additionally, the difficulty of test questions, as categorized by AMBOSS hammer ratings, showed a statistically significant correlation with performance for both ChatGPT versions, with ρ=-0.289 for ChatGPT 3.5 and ρ=-0.344 for ChatGPT 4. ChatGPT 4 surpassed ChatGPT 3.5 in all levels of test question difficulty, except for the 2 highest difficulty tiers (4 and 5 hammers), where statistical significance was not reached. In this study, ChatGPT 4 demonstrated remarkable proficiency in taking the USMLE Step 3, with an accuracy rate of 84.7% (194/229), outshining ChatGPT 3.5 with an accuracy rate of 56.9% (1047/1840). Although ChatGPT 4 performed exceptionally, it encountered difficulties in questions requiring the application of theoretical concepts, particularly in cardiology and neurology. These insights are pivotal for the development of examination strategies that are resilient to AI and underline the promising role of AI in the realm of medical education and diagnostics.

Behind the Researcher, the Forgotten Artist: the Question of Style in Management Science through the Prism of Nietzsche's Critique of Language

Behind the Researcher, the Forgotten Artist: the Question of Style in Management Science through the Prism of Nietzsche's Critique of Language

TYPES OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS IN MEDICAL EDUCATION AND THEIR EFFECT ON PASSING RATE OF STUDENTS

Background: The most popular written exam in health professions education is the multiple-choice style question (MCQ). This study sought to understand the effects of multiple-choice writing errors on students' academic performance in medical education. Methods: This descriptive study was done from December 2017 to June 2019 in Azad Jammu Kashmir (AJK) Medical College, Muzaffarabad. Ten block examinations were included. The item review committee reviewed all MCQs for flaws. Two tests from each class of MBBS were taken. The original tests containing all items were labeled as flawed tests and the result of each flawed test was evaluated. The students were graded into high, moderate and low achievement groups with scores of more than 79.9, between 50–79.9 and less than 50 percent respectively. Flawed items were then removed from the tests by the review committee and the scores of each test (standard test) were determined and compared with flawed tests and its effects were assessed in three achieving groups of students. Optical mark reading (OMR) classic-4 software was used for post-exam analysis and data were analyzed by using SPSS-25. Results: The passing rate ranged from 68.18% to 90.82% in flawed and 75.54% to 93.69% in standard tests. Most standard tests (7) had higher passing rates than flawed tests. Conclusion: Tests containing in-house developed MCQs have frequent item writing flaws and their inclusion in assessment did affect the passing rates of students but were not statistically significant. Pak J Physiol 2023;19(4):49–52

Assessing the Capability of ChatGPT, Google Bard, and Microsoft Bing in Solving Radiology Case Vignettes.

Background The field of radiology relies on accurate interpretation of medical images for effective diagnosis and patient care. Recent advancements in artificial intelligence (AI) and natural language processing have sparked interest in exploring the potential of AI models in assisting radiologists. However, limited research has been conducted to assess the performance of AI models in radiology case interpretation, particularly in comparison to human experts. Objective This study aimed to evaluate the performance of ChatGPT, Google Bard, and Bing in solving radiology case vignettes (Fellowship of the Royal College of Radiologists 2A [FRCR2A] examination style questions) by comparing their responses to those provided by two radiology residents. Methods A total of 120 multiple-choice questions based on radiology case vignettes were formulated according to the pattern of FRCR2A examination. The questions were presented to ChatGPT, Google Bard, and Bing. Two residents wrote the examination with the same questions in 3 hours. The responses generated by the AI models were collected and compared to the answer keys and explanation of the answers was rated by the two radiologists. A cutoff of 60% was set as the passing score. Results The two residents (63.33 and 57.5%) outperformed the three AI models: Bard (44.17%), Bing (53.33%), and ChatGPT (45%), but only one resident passed the examination. The response patterns among the five respondents were significantly different ( p = 0.0117). In addition, the agreement among the generative AI models was significant (intraclass correlation coefficient [ICC] = 0.628), but there was no agreement between the residents (Kappa = -0.376). The explanation of generative AI models in support of answer was 44.72% accurate. Conclusion Humans exhibited superior accuracy compared to the AI models, showcasing a stronger comprehension of the subject matter. All three AI models included in the study could not achieve the minimum percentage needed to pass an FRCR2A examination. However, generative AI models showed significant agreement in their answers where the residents exhibited low agreement, highlighting a lack of consistency in their responses.

ChatGPT-4 Performance on USMLE Step 1 Style Questions and Its Implications for Medical Education: A Comparative Study Across Systems and Disciplines.

We assessed the performance of OpenAI's ChatGPT-4 on United States Medical Licensing Exam STEP 1 style questions across the systems and disciplines appearing on the examination. ChatGPT-4 answered 86% of the 1300 questions accurately, exceeding the estimated passing score of 60% with no significant differences in performance across clinical domains. Findings demonstrated an improvement over earlier models as well as consistent performance in topics ranging from complex biological processes to ethical considerations in patient care. Its proficiency provides support for the use of artificial intelligence (AI) as an interactive learning tool and furthermore raises questions about how the technology can be used to educate students in the preclinical component of their medical education. The authors provide an example and discuss how students can leverage AI to receive real-time analogies and explanations tailored to their desired level of education. An appropriate application of this technology potentially enables enhancement of learning outcomes for medical students in the preclinical component of their education.

OLDER ADULT VOLUNTEERS AS MENTORS IN GERIATRIC EDUCATION

Abstract Health professions education that involves older adults has been shown to be effective in improving students’ clinical skills and attitudes towards older adults. With the goal of introducing healthcare students to geriatric care through first-hand experiences, the “Seniors Assisting in Geriatrics Education (SAGE)” Program involves older adult volunteers age 65+ in virtual person-centered health promotion visits with teams of three to four students. Through an application process, mentors are selected and receive orientation materials before visits with their assigned team. Students represent eight disciplines: dietetics, medicine, nursing, pharmacy, physical therapy, physician assistant studies, social work, and speech language pathology. Students practice basic clinical skills and screening techniques while working as a team and receiving feedback and input from their mentor. In 2022, we surveyed older adult volunteers to evaluate their mentor experience using open-ended and Likert style questions on a scale from strongly agree to strongly disagree. Survey responses (n=99) show that most mentors agreed students collaborated well (94%), health promotion topics for each visit were beneficial (83% to 85%), and they were satisfied with their SAGE experience (96%). Mentors identified opportunities for improvement related to student and mentor preparation, in-person versus online visits, and establishing rapport between mentors and teams. Results were used to enhance mentor training and engagement and to create community partnerships for recruitment and mentor diversity. By prioritizing recruitment of and training for volunteers, older adults can actively and effectively participate in meaningful educational experiences with health professions students.

Assessing the Impact of Race, Sexual Orientation, and Gender Identity on USMLE Style Questions: Preliminary Results of a Randomized Controlled Trial

Abstract Introduction/Objective Medical students are expected to graduate with cultural competence, which improves the quality of care of different minority groups and may help reduce health disparities. Incorporation of diversity, equity, and inclusion into medical teaching is critical to the development of culturally competent physicians; however, centuries of medical pedagogy have tied certain pathologies to racial and sexual/gender minority patients. This not only encourages stereotyping to arrive at the correct answer but also narrows the differential diagnosis developed by the future physician. We are surveying US and non-US third- and fourth-year medical students and first-year residents to demonstrate to what extent inclusion of race, sexual orientation, and gender identity data in question stems influences diagnosis and clinical decision-making. Methods/Case Report The survey consists of three sections. The first section collects demographic information, including age, race, gender identity, region, type of medical degree, current level of training, completion of examinations, and MCAT score. The second section presents one of two sets of twelve USMLE style questions with (experimental) or without (control) key demographic data, including race, sexual orientation, and gender identity. The third section presents four questions in order to assess participants’ understanding of the higher burden of hypertension among Black patients and attitudes regarding the role of their medical education in the development of biases. Participants were randomized to the set of twelve questions in the second section. Results (if a Case Study enter NA) Preliminary results from 25 medical trainees associated with Loyola University Chicago demonstrate that the presence of demographic data including race, sexual orientation, and gender identity impact answer choice on standardized questions. Conclusion The preliminary results of this study are significant because they suggest that race, sexual orientation, and gender identity are still being used in pathology education to highlight certain disease processes, potentially in a manner that is out of proportion to actual disease prevalence.

Capturing the patient’s voice to inform advanced clinical decision support for guideline-based cancer symptom management.

510 Background: Electronic patient reported outcomes (ePROs) have become routine in cancer care; however, ePROs alone do not provide sufficient information to enable informed symptom management. Informed symptom management requires an awareness of patient context to reduce inappropriate or erroneous advice. In this project we asked: how can information be collected directly from patients to provide clinicians with detailed, contextually-informed, patient-specific, evidence-based recommendations to manage distressing symptoms, especially among patients with complex comorbidities. Methods: The literature states that text messages are the most effective way to engage patients in clinical data collection. We texted patients a link that led them to a Web-based questionnaire that asked them about their common cancer symptoms. Prototypes were tested with 9 cancer patients through structured interviews and think aloud sessions from which themes were identified. Feedback from these interviews was used to develop a patient data collection questionnaire that we iteratively refined through think aloud sessions with 9 cancer patients to achieve universal patient acceptance. We also engaged 28 oncology clinicians in semi-structured interviews to assess the clinical impact of the additional contextual information on the symptom recommendations they received. Results: Formative evaluation identified four themes: 1) make instructions brief, 2) use intuitive navigation buttons with labels, 3) focus on multiple-choice style questions, and 4) extract data from the medical record for patient verification instead of free text entry by patients (e.g., medications). Developers incorporated “more detailed” questions into the patient data collection questionnaire; additional questions were presented if the patient indicated moderate or greater severity of a given symptom. These detailed questions focused on characteristics of the symptoms (e.g. neuropathic or somatic pain), remedies a patient had already initiated for self treatment (e.g., laxatives for constipation), and comorbid conditions that could impact the safety of recommendations (e.g., fall risk). Clinicians indicated that the additional data collected from patients increased the value of the tailored recommendations for enhanced symptom management. Conclusions: Additional data collection from patients using standard text messaging enables clinical decision support algorithms to provide more individually tailored cancer symptom management recommendations. This study is limited in that the design enhancements have been tested with a small number of end-users in a laboratory setting. Acknowledgements: This project was 100% funded using Cancer Moonshot funding through Phase I and II SBIR contracts from the National Cancer Institute, Contract #75N91018C00022 and #75N91020C00019, respectively.

Writing Differently: On the Constraints and Possibilities of Presenting Research Rooted in Feminist Epistemologies

AbstractIn this article, we adopted the genre of response letter, answering an editorial letter proposing to reject our submission. The rejection letter itself is fictionalized, but collated from various real reviews our texts have received throughout our academic career. Our aim is to both highlight the mechanisms pushing academic writing toward conformity, dullness, and irrelevance and to point toward the possibilities of Writing Differently: of crafting academic texts aligned with feminist sensibilities, conveying meaning as well as feeling, embedded in context, and open to difference. We discuss some texts by authors who have managed to break free from the constraints of the dominant style and published beautiful, meaningful texts, which challenge the orthodoxy of academic journal articles. We argue that the form of writing matters; that the question of style is, at its heart, the question of epistemology, what can be known, how it can be known, and how can such knowledge be shared. In addition, it also concerns the knowing subject and is thus a deeply feminist issue. We end our text by inviting the readers to join the growing ranks of academics crossing the boundaries of traditional journal articles, and to explore how Writing Differently enables new insights to be discovered and communicated.

Selected-Lesion SRS as a Novel Strategy in Treatment of Patients with Multiple Brain Metastases

With the increasing use of intracranial SRS (SRS) for treatment of patients with >10-15 brain metastases treatment; debate remains in the literature about how these patients should be treated. While many advocate for treatment of all lesions with either SRS or whole brain radiation therapy (WBRT), several groups are considering selected-lesion SRS (SL-SRS) where only a subset of intracranial lesions are treated. However, the current practice patterns of SL-SRS are not known. A survey of 19 questions was created using open-ended and multiple-choice style questions on SL-SRS practices and indications. The survey was distributed to providers in the US and internationally who perform SRS frequently. Ten out of 50 institutions provided responses reflecting the practices of 16 providers. Descriptive statistics was used to compare answers to each question when applicable including percentages and ranges. SL-SRS was performed at 8/10 institutions, (5 out of 6 US institutions and 3 out of 4 international institutions). Only 2 institutions had established clinical indications for SL-SRS (one in the US and one internationally) and one additional US institution reported clinical trials that require SL-SRS to study efficacy of CNS penetrating targeted therapies. One program reported research protocols for untreated brain metastases that would take priority over SL-SRS (program outside the US). Size of the lesion was cited as the most important factor (90%) when deciding to treat any single lesion. Next, lesion location and focal signs/symptoms were both considered moderately important. 80% ranked distance from prior SRS as the least important factor. Perilesional edema was also less important at most programs (90%). Lesion location and presence of symptoms were also considered important. There were several factors that would encourage providers to consider SL-SRS in a patient. Prior WBRT; progressing systemic disease and CNS-penetrating drug option available; and progressing systemic disease and immunotherapy option available were the most common responses. Most respondents cited "specific request by medical oncology" as well as "cooperative studies in this topic" as factors that might push them towards SL-SRS. Several institutions specified factors beyond the listed options. One institution reported that for patients with >20 lesions, they treated the largest lesions with SRS then follow with WBRT, termed the "pre-WBRT boost." Progression of untreated lesions was the most common reason why providers would bring back patients for additional treatment. The responses to this survey demonstrate that patients with >15-20 intracranial lesions, prior WBRT, and worsening systemic disease with CNS penetrating systemic therapies available are being considered for SL-SRS.

Selected-Lesion Stereotactic Radiosurgery (SL-SRS) as a Novel Strategy in the Treatment of Patients With Multiple Brain Metastases.

With the diminishing use of whole-brain radiotherapy (WBRT), there is increasing debate regarding the maximum number of brain metastases that should be treated with stereotactic radiosurgery (SRS). In patients with >10-15 lesions, some groups are proposing a new approach - selected-lesion SRS (SL-SRS) - where only a subset of intracranial lesions are chosen for irradiation. This study is an initial look into this practice. This is a cross-sectional exploratory survey study. A survey of 19 questions was created by the International Radiosurgery Research Foundation (IRRF) using open-ended and multiple-choice style questions on SL-SRS practices and indications with the goal of qualitatively understanding how SL-SRS is being implemented worldwide. The survey was distributed to physiciansin the United States (US) and internationally who are members of the IRRFand who perform SRS frequently.Ten out of 50 IRRF institutions provided responses reflecting the practices of 16 physicians. SL-SRS is being performed at 8/10 institutions. The most common reasons for using SL-SRS included patients with prior WBRT, patients with progressing systemic disease with central nervous system (CNS)-penetrating or immunotherapies available, specific requests from medical oncology, and cooperative studies using this approach. Lesion size was cited as the most important factor when choosing to irradiate any single lesion. The majority of respondents reported 30 mm and 40 mm as size cutoffs (by largest dimension) for treatment of a lesion in eloquent and non-eloquent locations, respectively. Eloquence of lesion location and attributable symptoms were also considered important. Progression of untreated lesions was the most common reason reported for bringing patients back for additional treatment. The responses to this survey show that SL-SRS is being used, allowing for small/asymptomatic brain metastases to be left safely unirradiated. It is currently used in patients who have >10-15 lesions with prior WBRT, those with progression of extracranial disease but with acceptable systemic treatment options, and those with poor functional status. The incorporation of this new approach into clinical trials should be considered for the safe study of the efficacy of new CNS-penetrating systemic therapies.

ChatGPT versus the neurosurgical written boards: a comparative analysis of artificial intelligence/machine learning performance on neurosurgical board-style questions.

"ChatGPT versus the neurosurgical written boards: a comparative analysis of artificial intelligence/machine learning performance on neurosurgical board–style questions" published on 24 Mar 2023 by American Association of Neurological Surgeons.

Women’s discursive agency in transitional justice policy-making: A feminist institutionalist approach

AbstractScholars have studied how women’s domestic and transnational civil society activism addresses the gendered nature of transitional justice. In contrast, they have paid scant attention to women’s impact on transitional justice policy-making in institutions. We leverage the feminist institutionalist perspective that makes visible gendered norms, rules, and discourses in institutions. Homing in on women’s influence in parliaments where women are outnumbered by men and marginalised by adversarial discourse, we develop a conceptualisation of women’s discursive agency. Foregrounding discourse in women’s ability to drive change, women’s agency is enacted through their linguistic communication style and substantive normative positions that constitute micro- and macro-level structures of domination. Quantitative and qualitative discourse analysis is applied to a corpus of parliamentary questions about transitional justice in the Croatian parliament from 2004 to 2020. Our results show that women adopt the adversarial style of questioning, which they use to broaden the scope of entitlements and press for reparations for female and male victims. They overcome constraints posed by partisanship and ideology, while constraints of nationalism are less easily broken. The article advances feminist transitional justice by demonstrating how women’s language contributes to dismantling men’s policy domination in institutions, with implications for mixed-sex interactions in non-institutional domains.

Exploring the Potential and Limitations of Chat Generative Pre-trained Transformer (ChatGPT) in Generating Board-Style Dermatology Questions: A Qualitative Analysis.

This article investigates the limitations of Chat Generative Pre-trained Transformer (ChatGPT), a language model developed by OpenAI, as a study tool in dermatology. The study utilized ChatPDF, an application that integrates PDF files with ChatGPT, to generate American Board of Dermatology Applied Exam (ABD-AE)-style questions from continuing medical education articles from the Journal of the American Board of Dermatology. A qualitative analysis of the questions was conducted by two board-certified dermatologists, assessing accuracy, complexity, and clarity. Out of 40 questions generated, only 16 (40%) were deemed accurate and appropriate for ABD-AE study preparation. The remaining questions exhibited limitations, including low complexity, lack of clarity, and inaccuracies. The findings highlight the challenges faced by ChatGPT in understanding the domain-specific knowledge required in dermatology. Moreover, the model's inability to comprehend the context and generate high-quality distractor options, as well as the absence of image generation capabilities, further hinders its usefulness. The study emphasizes that while ChatGPT may aid in generating simple questions, it cannot replace the expertise of dermatologists and medical educators in developing high-quality, board-style questions that effectively evaluate candidates' knowledge and reasoning abilities.

Physician Disclosure and Patient Awareness of Resident Involvement During Podiatric Surgery: Do Patients Deserve to Know Who is Holding the Scalpel?

An anonymous questionnaire consisting of multiple-choice style questions was created at www.surveymonkey.com to assess physicians' perspectives regarding podiatric resident involvement disclosures in the surgical setting. The survey link was sent to podiatric residency program directors, attendings, residents, and fellows through email, social media outlets, and direct contact. For the purpose of this study, residents and fellows will be grouped together as trainees. A total of 201 participants were surveyed, including 63 program directors, 61 attendings, and 77 trainees. Of the respondents, 89.5% reported that at least 1 resident introduces themselves to the patient as a participant in the case. Additionally, 99% of responses revealed that trainees are performing at least 50% of the case. However, surgeons perceive that only 25% of patients are truly aware of this advanced level of involvement. This study suggests that 89.5% of podiatric physicians do in fact disclose trainee surgical participation. However, it is perceived that patients are still unaware of the extent of trainee involvement. Further studies are warranted which would account for both patient and physician understanding of this controversial topic.

Pulling It Out of a Hat: Picabia, Lanson and Man Ray's Cover for Littérature

The magician’s top hat, standardized handwriting, calligraphic flourish and ink blot of Man Ray’s cover for the new series of Littérature in 1922 open questions of style in both fashion and writing. The cover’s emblem-like visual components resonate not only with questions of style and cultural sophistication in such magazines as Monsieur and La Gazette du bon ton, but also with contemporary Dada squabbles with Cubism, and with a Paris Dada bugbear” Gustave Lanson’s magisterial text, Histoire de la littérature française (1894). In his 1924 Manifeste André Breton deftly capitalized on Lansonism in order to turn it against its own premises and devices. Finally, Man Ray’s emblem points to questions at the time of authenticity, authority and falsity.

SMK MUHAMMADIYAH BANDA ACEH STUDENTS' PERCEPTION ON THE TOEIC COMPETENCY TEST

The purpose of this study is to describe how students at SMK Muhammadiyah Banda Aceh feel about the TOEIC competency test and pinpoint any obstacles they may have encountered while taking the test. Using the purposive sampling technique, 30 children from class 2 at SMK Muhammadiyah Banda Aceh were chosen as the sample for this study. Two questionnaires were used to gather data. The analysis findings suggest that most students have a favorable opinion on the TOEIC test. They believed that taking the test helped them improve their English, and they would suggest it to others. However, they also struggle to read test texts, comprehend challenging terms and settings, and respond to questions. Additionally, some students lacked confidence and were unfamiliar with the test question styles. This study found that the TOEIC test has a good effect, but also highlights the difficulties that students experience. The recommendations include improving students' comprehension of exam instructions, addressing issues by understanding challenging terms and circumstances, and offering assistance to increase students' confidence. The TOEIC test and more efficient methods of English language instruction were predicted to benefit from the findings of this study.

Performance of Generative Large Language Models on Ophthalmology Board–Style Questions

To investigate the ability of generative artificial intelligence models to answer ophthalmology board style questions DESIGN: Experimental study. This study evaluated three large language models (LLMs) with chat interfaces, Bing Chat (Microsoft) and ChatGPT 3.5 and 4.0 (OpenAI), using 250 questions from the Basic Science and Clinical Science (BCSC) Self-Assessment Program (SAP). While ChatGPT is trained on information last updated in 2021, Bing Chat incorporates more recently indexed internet search to generate its answers. Performance was compared to human respondents. Questions were categorized by complexity and patient care phase, and instances of information fabrication or non-logical reasoning were documented. Primary outcome: response accuracy. performance in question subcategories and hallucination frequency. Human respondents had an average accuracy of 72.2%. ChatGPT-3.5 scored the lowest (58.8%), while ChatGPT-4.0 (71.6%) and Bing Chat (71.2%) performed comparably. ChatGPT-4.0 excelled in workup-type questions (OR = 3.89, 95% CI 1.19-14.73, p = 0.03) compared with diagnostic questions, but struggled with image interpretation (OR = 0.14, 95% CI 0.05-0.33, p < 0.01) when compared with single step reasoning questions. Against single step questions, Bing Chat also faced difficulties with image interpretation (OR = 0.18, 95% CI 0.08-0.44, p < 0.01) and multi-step reasoning (OR = 0.30, 95% CI 0.11-0.84, p = 0.02). ChatGPT-3.5 had the highest rate of hallucinations or non-logical reasoning (42.4%), followed by ChatGPT-4.0 (18.0%) and Bing Chat (25.6%). LLMs (particularly ChatGPT-4.0 and Bing Chat) can perform similarly with human respondents answering questions from the BCSC SAP. The frequency of hallucinations and non-logical reasoning suggest room for improvement in the performance of conversational agents in the medical domain.

Automated grading of anatomical objective structured practical examinations using decision trees: An artificial intelligence approach.

An Objective Structured Practical Examination (OSPE) is an effective and robust, but resource-intensive, means of evaluating anatomical knowledge. Since most OSPEs employ short answer or fill-in-the-blank style questions, the format requires many people familiar with the content to mark the examinations. However, the increasing prevalence of online delivery for anatomy and physiology courses could result in students losing the OSPE practice that they would receive in face-to-face learning sessions. The purpose of this study was to test the accuracy of Decision Trees (DTs) in marking OSPE questions as a first step to creating an intelligent, online OSPE tutoring system. The study used the results of the winter 2020 semester final OSPE from McMaster University's anatomy and physiology course in the Faculty of Health Sciences (HTHSCI 2FF3/2LL3/1D06) as the data set. Ninety percent of the data set was used in a 10-fold validation algorithm to train a DT for each of the 54 questions. Each DT was comprised of unique words that appeared in correct, student-written answers. The remaining 10% of the data set was marked by the generated DTs. When the answers marked by the DT were compared to the answers marked by staff and faculty, the DT achieved an average accuracy of 94.49% across all 54 questions. This suggests that machine learning algorithms such as DTs are a highly effective option for OSPE grading and are suitable for the development of an intelligent, online OSPE tutoring system.

Improving undergraduate nursing students' clinical judgment: A peer mentoring project

Integrating various opportunities for nursing students to rehearse their use of nursing clinical judgment (NCJ) is needed to prepare for the licensure exam and clinical practice. Two exceptional nursing students were selected to provide peer mentoring sessions for nursing students enrolled in nursing courses designed to assist students with development of cognitive problem-solving skills. Two peer mentoring sessions were conducted to provide rehearsal of next generation style questions including case studies. This quality improvement project warrants further research to explore whether this innovative approach to coaching and mentoring has an impact on students’ use of NCJ.