Correct Responses Research Articles

Evaluation of a Large Language Model on the American Academy of Pediatrics' PREP Emergency Medicine Question Bank

Background Large language models (LLMs), including ChatGPT (Chat Generative Pretrained Transformer), a popular, publicly available LLM, represent an important innovation in the application of artificial intelligence. These systems generate relevant content by identifying patterns in large text datasets based on user input across various topics. We sought to evaluate the performance of ChatGPT in practice test questions designed to assess knowledge competency for pediatric emergency medicine (PEM). Methods We evaluated the performance of ChatGPT for PEM board certification using a popular question bank used for board certification in PEM published between 2022 and 2024. Clinicians assessed performance of ChatGPT by inputting prompts and recording the software's responses, asking each question over 3 separate iterations. We calculated correct answer percentages (defined as correct in at least 2/3 iterations) and assessed for agreement between the iterations using Fleiss' κ. Results We included 215 questions over the 3 study years. ChatGPT responded correctly to 161 of PREP EM questions over 3 years (74.5%; 95% confidence interval, 68.5%–80.5%), which was similar within each study year (75.0%, 71.8%, and 77.8% for study years 2022, 2023, and 2024, respectively). Among correct responses, most were answered correctly on all 3 iterations (137/161, 85.1%). Performance varied by topic, with the highest scores in research and medical specialties and lower in procedures and toxicology. Fleiss' κ across the 3 iterations was 0.71, indicating substantial agreement. Conclusion ChatGPT provided correct answers to PEM responses in three-quarters of cases, over the recommended minimum of 65% provided by the question publisher for passing. Responses by ChatGPT included detailed explanations, suggesting potential use for medical education. We identified limitations in specific topics and image interpretation. These results demonstrate opportunities for LLMs to enhance both the education and clinical practice of PEM.

Evaluation of a distribution, education and awareness intervention for child passenger safety in Lebanon: a low-income and middle-income country setting

BackgroundThe Eastern Mediterranean Region suffers disproportionately from paediatric traffic-related injuries. Despite governmental laws, Lebanon—an eastern Mediterranean country—has low child restraint (CR) use prevalence. This study examined the impact of using...

Developmental patterns of inhibition and fronto-basal-ganglia white matter organisation in healthy children and children with attention-deficit/hyperactivity disorder.

There is robust evidence implicating inhibitory deficits as a fundamental behavioural phenotype in children with attention-deficit/hyperactivity disorder (ADHD). However, prior studies have not directly investigated the role in which white matter properties within the fronto-basal-ganglia circuit may play in the development of inhibitory control deficits in this group. Combining recent advancements in brain-behavioural modelling, we mapped the development of stop-signal task (SST) performance and fronto-basal-ganglia maturation in a longitudinal sample of children aged 9-14 with and without ADHD. In a large sample of 135 ADHD and 138 non-ADHD children, we found that the ADHD group had poorer inhibitory control (i.e., longer stop-signal reaction times) across age compared to non-ADHD controls. When applying the novel parametric race model, this group effect was driven by higher within-subject variability (sigma) and higher number of extreme responses (tau) on stop trials. The ADHD group also displayed higher within-subject variability on correct responses to go stimuli. Moreover, we observed the ADHD group committing more task-based failures such as responding on stop trials (trigger failures) and omissions on go trials (go failures) compared to non-ADHD controls, suggesting the contribution of attentional lapses to poorer response inhibition performance. In contrast, longitudinal modelling of fixel-based analysis measures revealed no significant group differences in the maturation of fronto-basal-ganglia fibre cross-section in a subsample (74 ADHD and 73 non-ADHD children). Finally, brain-behavioural models revealed that age-related changes in fronto-basal-ganglia morphology (fibre cross-section) were significantly associated with reductions in the variability of the correct go-trial responses (sigma.true) and skew of the stop-trial distribution (tauS). However, this effect did not differ between ADHD and typically developing children. Overall, our findings support the growing consensus suggesting that attentional deficits subserve ADHD-related inhibitory dysfunction. Furthermore, we show novel evidence suggesting that while children with ADHD are consistently performing worse on the SST than their non-affected peers, they appear to have comparable rates of neurocognitive maturation across this period.

Knowledge, attitude and perception of Italian dental students toward HPV-related oropharyngeal cancer and vaccination: a cross-sectional study

BackgroundHuman papillomavirus (HPV) infection is a leading cause of oropharyngeal squamous cell cancer (OPSCC). This study aimed to carry out a Knowledge, Attitude and Perception survey on HPV infection, HPV+ OPSCC, and HPV vaccination among Italian dental students.MethodsThrough an online self-administered questionnaire consisting of 82 questions, data on dental students’ sociodemographic characteristics, knowledge, attitudes and perceptions concerning HPV+ OPSCC, infection and vaccination were acquired. A statistical analysis, based on their year of attendance (early career, from 1st to 3rd year vs. late career, from 4th to 6th year) was also conducted.ResultsA total of 412 dental students completed the questionnaire. Knowledge of HPV+ OPSCC was reported by 61% of early-career students and 73% of late-career students, with high awareness of the HPV-OPSCC correlation in both groups (85% vs. 89%, respectively). The percentage of correct responses regarding HPV infection knowledge was 61% for early-career and 73% for late-career students, while vaccine knowledge was 70% and 78%, respectively. Over 90% of students acknowledged the dentist’s role in educating patients about HPV and OPSCC, and attitudes toward discussing HPV and vaccination were positive. However, only about half would recommend the vaccine to either gender. Statistically significant differences were found between early- and late-career groups across all knowledge sections (p < 0.001), while no significant differences emerged for perception (p = 0.076) or attitude (p = 0.147).ConclusionsThe study reveals encouraging results but highlights significant gaps in dental students’ knowledge, perceptions, and attitudes toward HPV+ OPSCC, infection, and vaccination. Addressing these gaps through targeted education and training in dental curricula could improve HPV prevention awareness and patient education, ultimately enhancing public health outcomes.

Emergency imaging protocols for pregnant patients: a multi-institutional and multi- specialty comparison of physician education.

Previous studies have demonstrated that radiologists and other providers perceive maging to be higher than they actually are. Thus, pregnant patients were less likely to receive ionizing radiation procedures. While it is imperative to minimize fetal radiation exposure, clinicians must remember that diagnostic studies should not be avoided due to fear of radiation, particularly if the imaging study can significantly impact patient care. Although guidelines do exist regarding how best to image pregnant patients, many providers are unaware of these guidelines and thus lack confidence when making imaging decisions for pregnant patients. This study aimed to gather information about current education, confidence in, and knowledge about emergency imaging of pregnant women among radiology, emergency medicine, and OB/GYN providers. We created and distributed an anonymous survey to radiology, emergency medicine, and OB/GYN providers to evaluate their knowledge and confidence in imaging pregnant patients in the emergent setting. This study included a questionnaire with the intent of knowing the correct answers among physicians primarily across the United States (along with some international participation). We conducted subgroup analyses, comparing variables by specialty, radiology subspecialty, and training levels. Based on the survey results, we subsequently developed educational training videos. 108 radiologists, of which 32 self-identified as emergency radiologists, ten emergency medicine providers and six OB/GYN clinicians completed the survey. The overall correct response rate was 68.5%, though performance across questions was highly variable. Within our 18-question survey, four questions had a correct response rate under 50%, while five questions had correct response rates over 90%. Most responding physicians identified themselves as either "fairly" (58/124, 47%) or "very" (51/124, 41%) confident. Amongst specialties, there were differences in performance concerning the knowledge assessment (p = 0.049), with the strongest performance from radiologists. There were no differences in knowledge by training level (p = 0.4), though confidence levels differed significantly between attending physicians and trainees (p < 0.001). This study highlights deficiencies in knowledge to support appropriate decision-making surrounding the imaging of pregnant patients. Our results indicate the need for improved physician education and dissemination of standardized clinical guidelines.

Toward diversification of acute stressors and precision stress research: A stage 2 Registered Report validating a reward-salient stress task in emerging adults.

Stress is one of, if not the, most ubiquitously studied risk factor across the health sciences. This is unlikely to change given that the primary drivers of mortality and disability are chronic, stress-mediated illnesses (often highly comorbid with psychopathology). We argue that an important limitation of stress research is the consistency with which the Trier Social Stress Test is used when the research questions are not specific to social stress. We advocate for precision stress research using qualitatively different stressors to facilitate exploration of how different types of stressors might differentially impact health outcomes, including psychopathology. This registered report validates a reward-salient stress task (a modified Anger Incentive Delay Task) in a sample of 101 emerging adults, over half of whom reported clinically relevant anxiety, hypo/mania, depression, and/or suicidal ideation, who participated in a study between 2020 and 2022. This task involves teaching participants a game where they can win money. Part way through, the "goal frustration" condition changes the rules such that correct responses to trials with anticipatory stimuli indicating the possibility to win money actually lose money on 56% of trials despite visual feedback indicating that responses were successful. Results consistently indicated that the Anger Incentive Delay Task successfully reduced positive emotions and motivation and increased negative emotions. The magnitude of these responses was predicted by individual differences in reward and punishment sensitivity. Given the breadth of psychopathologies that share both (a) stress and (b) reward and punishment sensitivity as risk factors, a reward-salient acute stress task is an important tool for precision psychopathology research. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Radiation information, protection and awareness of healthcare professionals in terms of occupational health and safety: a two–center survey study

Abstract The objective of this study was to assess the level of knowledge, protection and awareness of radiation among healthcare professionals. A total of 413 healthcare professionals from two major training and research hospitals in Istanbul took part in the study. The sample consisted of 26.6% physicians and 73.4% other healthcare professionals. There was considerable variation in the knowledge of radiation content in imaging techniques, with correct response rates of 89.3% for Computed Tomography, 80.1% for digital x-ray, and 52.8% for Magnetic Resonance Imaging. While 69.5% of respondents provided correct answers to questions regarding radiation protection principles, 64.4% demonstrated a lack of sufficient knowledge in this area. It is noteworthy that 48.4% of respondents incorrectly identified the medical imaging technique to which the public is most exposed. It was concluded that further training on radiation-related issues should be provided at the university level and through in-service training for healthcare professionals.

Experience of caring for patients with COVID-19 and educational achievement among Japanese resident physicians: a nationwide survey with general medicine in-training examination

BackgroundAs healthcare professional trainees, resident physicians are expected to help with COVID-19 care in various ways. Many resident physicians worldwide have cared for COVID-19 patients despite the increased risk of burnout. However, few studies have examined the experience with COVID-19 care among resident physicians and its effects on competency achievement regarding clinical basics and COVID-19 patient care.MethodThis nationwide, cross-sectional Japanese study used a clinical training environment questionnaire for resident physicians (PGY-1 and − 2) in 593 teaching hospitals during the General Medicine In-Training Examination in January 2021. The General Medicine In-Training Examination questions comprised four categories (medical interviews and professionalism; symptomatology and clinical reasoning; physical examination and clinical procedures; and disease knowledge) and a COVID-19-related question. We examined the COVID-19 care experience and its relationship with the General Medicine In-Training Examination score, adjusting for resident and hospital variables.ResultsOf the 6,049 resident physicians, 2,841 (47.0%) had no experience caring for patients with COVID-19 during 2020. Total and categorical General Medicine In-Training Examination scores were not different irrespective of the experience with COVID-19 patient care. For the COVID-19-related question, residents with experience in COVID-19 care showed a significant increase in correct response by 2.6% (95% confidence interval, 0.3–4.9%; p = 0.028).ConclusionsThe resident physicians’ COVID-19 care experience was associated with better achievement of COVID-19-related competency without reducing clinical basics. However, approximately half of the residents missed the critical experience of caring for patients during this unparalleled pandemic in Japan.

Reliability of the Revised Neurophysiology of Pain Questionnaire-Turkish in patients with neck pain: a cross-validation study

Background &amp; Objectives: The Neurophysiology of Pain Questionnaire (NPQ) measures the level of knowledge of patients and healthcare professionals regarding the neurophysiology of pain. The revised Turkish (Tr) version has low reliability for patients with chronic spinal pain. In this study, we investigated the reliability of the Revised NPQ-Tr in patients with neck pain. Methodology: The Revised NPQ-Tr was administered to 219 participants suffering from neck pain for at least 3 months, who were between the ages of 25 and 60 y and native Turkish speakers. Correct responses were awarded one point, while undecided or incorrectly marked responses were awarded 0 points. In order to analyse the validity of the test, the high-scoring and low-scoring groups were compared. Using the percentage of correct responses in the high and low scoring groups, we calculated the difficulty of the item (P-value) and the discrimination power of the item (r). Results: Responses to the questionnaire were highly correlated. Items 1 and 2 had high discrimination power (r &gt; 0.40), and items 3 and 5 had very good item quality (0.30 &lt; r &lt; 0.39). Among the items ranked between 1-7 and 9-12 in terms of difficulty of the item and discrimination power, only item 8 was found to be a difficult yet discriminating item (P &lt; 0.60). According to Cronbach's alpha (0.81) and Kuder-Richardson-20 (0.81) coefficients, the questionnaire had a high internal consistency and reliability, and a split-half correlation coefficient (0.802) determined its internal consistency and reliability. Conclusion: We conclude that the Neurophysiology of Pain Questionnaire-Turkish can be used on neck pain, and it is reliable and highly valid. Keywords: Chronic Pain, neck pain, pain knowledge, pain education, pain beliefs, Turkish language, reliability, validity. Citation: Yasarer O, Sari Z, Copuroglu OB, Eker TA, Aydogdu O. Reliability of the Revised Neurophysiology of Pain Questionnaire-Turkish in patients with neck pain: a cross-validation study. Anaesth. pain intensive care 2024;28(5):859−865; DOI: 10.35975/apic.v28i5.2552 Received: August 04, 2024; Reviewed: August 14, 2024; Accepted: August 17, 2024

Assessment of ChatGPT-4 in Family Medicine Board Examinations Using Advanced AI Learning and Analytical Methods: Observational Study.

This research explores the capabilities of ChatGPT-4 in passing the American Board of Family Medicine (ABFM) Certification Examination. Addressing a gap in existing literature, where earlier artificial intelligence (AI) models showed limitations in medical board examinations, this study evaluates the enhanced features and potential of ChatGPT-4, especially in document analysis and information synthesis. The primary goal is to assess whether ChatGPT-4, when provided with extensive preparation resources and when using sophisticated data analysis, can achieve a score equal to or above the passing threshold for the Family Medicine Board Examinations. In this study, ChatGPT-4 was embedded in a specialized subenvironment, "AI Family Medicine Board Exam Taker," designed to closely mimic the conditions of the ABFM Certification Examination. This subenvironment enabled the AI to access and analyze a range of relevant study materials, including a primary medical textbook and supplementary web-based resources. The AI was presented with a series of ABFM-type examination questions, reflecting the breadth and complexity typical of the examination. Emphasis was placed on assessing the AI's ability to interpret and respond to these questions accurately, leveraging its advanced data processing and analysis capabilities within this controlled subenvironment. In our study, ChatGPT-4's performance was quantitatively assessed on 300 practice ABFM examination questions. The AI achieved a correct response rate of 88.67% (95% CI 85.08%-92.25%) for the Custom Robot version and 87.33% (95% CI 83.57%-91.10%) for the Regular version. Statistical analysis, including the McNemar test (P=.45), indicated no significant difference in accuracy between the 2 versions. In addition, the chi-square test for error-type distribution (P=.32) revealed no significant variation in the pattern of errors across versions. These results highlight ChatGPT-4's capacity for high-level performance and consistency in responding to complex medical examination questions under controlled conditions. The study demonstrates that ChatGPT-4, particularly when equipped with specialized preparation and when operating in a tailored subenvironment, shows promising potential in handling the intricacies of medical board examinations. While its performance is comparable with the expected standards for passing the ABFM Certification Examination, further enhancements in AI technology and tailored training methods could push these capabilities to new heights. This exploration opens avenues for integrating AI tools such as ChatGPT-4 in medical education and assessment, emphasizing the importance of continuous advancement and specialized training in medical applications of AI.

The effect of knowledge and person-related factors on breast cancer susceptibility genes (BRCA1/2) testing perception in Turkish women.

Genetic testing for breast cancer susceptibility genes (BRCA1/2) plays a pivotal role in risk assessment and preventive interventions. However, individuals' awareness, knowledge, and attitudes toward genetic testing can vary across different societies. This study focuses on understanding Turkish women's knowledge, perceptions, and attitudes toward BRCA1/2 testing, considering demographic factors and awareness. In this cross-sectional study, 301 Turkish participants, including breast/ovarian cancer patients and their first-degree relatives, were surveyed. Information on sociodemographics, cancer history, awareness, knowledge, and perceptions was collected. The study aimed to assess knowledge levels about breast cancer inheritance and BRCA1/2 testing, describe perspectives about testing in women with a family history of breast or ovarian cancer, and determine associations between knowledge, personal factors, anxiety, and genetic testing perspectives. Results showed a wide range in correct responses (31.6%-96.7%) for knowledge items. No significant relationship between knowledge levels and positive perception was observed. However, participants answering a specific question incorrectly showed higher negative perceptions. While most participants recognized the benefits of genetic testing, concerns centered around passing the genes to future generations. Participants who were younger, more educated, had higher income, were employed, at an earlier disease stage, and were social media users demonstrated more positive attitudes. Negative perceptions were higher among younger patients, physicians, and healthcare professionals. Interestingly, anxiety in cancer patients did not correlate with either positive or negative perceptions. In conclusion, this study identifies participant-related factors influencing perceptions of hereditary genetic tests. Understanding these factors and addressing associated issues can enhance the utilization of genetic testing and promote preventive oncology applications.

Survey of Knowledge, Attitudes, and Levels of Confidence Regarding Age-Related Hyperkyphosis and Its Management among Thai Physiotherapists.

Thorough knowledge of the management of age-related hyperkyphosis is crucial to physiotherapists' effective handling of complex spinal deformities. This cross-sectional study investigated the knowledge, attitudes, and confidence of Thai physiotherapists regarding age-related hyperkyphosis. An online questionnaire with demographic, work-related, knowledge, attitude, and confidence questions was employed. The data analysis involved descriptive statistics, chi-square tests, and independent t-tests. Complete responses were obtained from 314 physiotherapists. The correct responses amounted to 68.79% for the definition, 90.00% for causes, 14.97% for age of onset, 22.38% for prevalence, and 64.65% for the gold-standard diagnosis of hyperkyphosis. Most of the diagnostic methods involved visual examination. The respondents reported that hyperkyphosis disrupts respiratory function. The aim of treatment, according to 88.85%, was to increase spine mobility. Corrective posture exercises were the most common treatment strategy, but the range of treatments was diverse. Many cited undergraduate education as their primary evidence source. Respondents had conflicting attitudes towards "age-related hyperkyphosis is a normal aging process" but mostly positive attitudes towards the need for physiotherapy treatment. Approximately 22% reported fair confidence in treating hyperkyphosis. This study highlights the importance of raising awareness and enhancing knowledge, attitudes, and confidence among physiotherapists to improve care for older adults with hyperkyphosis.

Awareness of chronic hepatitis b and its barriers to treatment in patient with HBV infection.

Objective: To determine knowledge of chronic hepatitis B virus (HBV) and barrier to treatment among patients with HBV infection. Study Design: Cross-sectional study. Setting: Department of Outpatient Clinics, Gastroenterology, Liaquat National Hospital. Period: March 2024 to June 2024. Methods: Patients of age 18 years and above of either gender and Hepatitis B surface antigen (HBsAg) positive more than six months were included in this study. self-structured questionnaire was developed after performing a detailed literature search. The criteria of adequate knowledge was to answer at least 50% of correct responses. Results: Total 247 patients were approached for this study, out of which 230 gave consent to the part of study and completed this survey. Hence data was analyzed for these 230 patients. Mean age of patients was 43.4 ± 12 years. More than half of them were males (57.4%). Around three-fourth of participants were diagnosed with HBV upon symptoms manifestation (72.6%). Nearly half had knowledge that HBV is a viral infection (55.2%). Using threshold of 4 and above, adequate knowledge was seen in 68.7% patients. Around quarter were vaccinated according to EPI schedule (23.5%). Half of patients said they did not get vaccination because of affordability issues (50.9%). Few also said that they were afraid of vaccine (13.2%). Conclusion: The present study analyzed that our HBV patients were not knowledgeable about the chronic HBV virus and patients were not willing for vaccination. Steps should be taken by policy makers and healthcare providers to address this gap.

Empatica E4 Assessment of Child Physiological Measures of Listening Effort During Remote and In-Person Communication.

Telepractice is a growing service model that delivers aural rehabilitation to deaf and hard-of hearing children via telecommunications technology. Despite known benefits of telepractice, this delivery approach may increase patients' listening effort (LE) characterized as an allocation of cognitive resources toward an auditory task. The study tested techniques for collecting physiological measures of LE in normal-hearing (NH) children during remote (referred to as tele-) and in-person communication using the wearable Empatica E4 wristband. Participants were 10 children (age range: 9-12 years old) who came to two tele- and two in-person weekly sessions, order counterbalanced. During each session, the children heard a short passage read by the clinical provider, completed an auditory passage comprehension task, and self-rated their effort as a part of the larger study. Measures of electrodermal activity and blood volume pulse amplitude were collected from the child E4 wristband. No differences in child subjective, physiological measures of LE or passage comprehension scores were found between in-person sessions and telesessions. However, an effect of treatment duration on subjective and physiological measures of LE was identified. Children self-reported a significant increase in LE over time. However, their physiological measures demonstrated a trend indicating a decrease in LE. A significant association between subjective measures and the passage comprehension task was found suggesting that those children who reported more effort demonstrated a higher proportion of correct responses. The study demonstrated the feasibility of collection of physiological measures of LE in NH children during remote and in-person communication using the E4 wristband. The results suggest that measures of LE are multidimensional and may reflect different sources of, or cognitive responses to, increased listening demand. https://doi.org/10.23641/asha.27122064.

Implementing the service-learning methodology in nursing education: A case study

IntroductionNurses play a crucial role in global health promotion, and innovative teaching strategies are vital to addressing modern educational challenges. Service-learning, a credit-based approach, integrates community service with academic learning, enhancing students' understanding of course content while fostering civic values. Although established in the United States of America and growing in Europe, most studies on service-learning focus on student learning outcomes, often overlooking its impact on the communities served. AimsThe purpose of this study is to test whether the service-learning practice of nursing students teaching a pharmacology curriculum topic to school children promotes knowledge of the topic in both the nursing students and the school children. DesignThis research involves two pretest-posttest quantitative analyses and two satisfaction surveys. ParticipantsSeventy-six nursing students participated during the 2022–2023 academic year, and 69 primary school pupils received the service-learning intervention. ResultsPost-test scores of nursing students were significantly higher than pre-test scores (p = 0.0009). The percentage of correct answers post-intervention was significantly higher than pre-intervention (p < 0.0001). Additionally, 98.7 % of nursing students found the service-learning experience beneficial for learning, and 94.7 % reported increased social awareness.For the school children, the percentage of correct responses after the service-learning experience was statistically significantly higher than before the activity (p < 0.0001). The percentage of correct answers from pupils in each primary grade (fourth, fifth, and sixth) was statistically significantly higher after the application of the learning experience compared to before (p < 0.0001, p < 0.0001, and p < 0.001, respectively). The satisfaction survey indicated high acceptance of the methodology among both nursing students and school pupils. ConclusionsService-learning enhances knowledge of the pharmacology topic in both nursing students and school children.

Eye-tracking training improves visuospatial working memory of children with attention-deficit/hyperactivity disorder and autism spectrum disorder.

Given the close connection between eye movement and frontal lobe functions and some evidence supporting the effect of eye-tracking training on enhancing cognitive performance mediated by the frontal lobe, this study aimed to explore if after-school eye-tracking training can improve the visuospatial working memory (VSWM) and cognitive flexibility performance in children with attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD). This study is a non-randomized cluster trial. Forty children from eight primary schools were selected, half receiving eye-tracking training for 20 sessions over 9 months, while the other half served as a waitlist control. They were matched on demographic characteristics and baseline cognitive performance. Their VSWM and cognitive flexibility were assessed at the beginning and end of the study. Results showed that children who received eye-tracking training, but not those on a waitlist, exhibited significant improvements in the total score and working memory span of the VSWM tests, and the correct responses in cognitive flexibility tests. Specifically, VSWM performance at higher span levels (5 or above) yielded a greater improvement. The findings suggest that eye-tracking training can be a feasible and effective after-school program for improving working memory and cognitive flexibility performance in children with ADHD and ASD. This study was prospectively registered at ClinicalTrials.gov (https://clinicaltrials.gov/, trial number: NCT05428657).

Are faculty members capable of finding the needed information? A study of selected universities of Jammu and Kashmir

We all need information and there is so much of it available in both physical and digital media. However, to find the needed information, capability of searching; knowledge of retrieval tools like catalogue; knowledge of Boolean operators, wildcards and truncations; and knowledge to different sources of information like bibliography, encyclopaedia, journals, etc. are equally important. The current study set out to investigate the faculty members’ searching capabilities in four of Jammu &amp; Kashmir's top institutions. The study used questionnaire as a tool to gather data from randomly identified population. After applying appropriate statistical techniques, the study revealed that the faculty members are not able to cross the half mark of correct answering for some basic questions posed to them. Eventually, the study suggests to conduct user awareness programmes since these programmes are significantly associated with the correct responses given by the faculty members in the universities studied.

Field-Testing Multiple-Choice Questions With AI Examinees: English Grammar Items

Field-testing is an essential yet often resource-intensive step in the development of high-quality educational assessments. I introduce an innovative method for field-testing newly written exam items by substituting human examinees with artificially intelligent (AI) examinees. The proposed approach is demonstrated using 466 four-option multiple-choice English grammar questions. Pre-trained transformer language models are fine-tuned based on the 2-parameter logistic (2PL) item response model to respond like human test-takers. Each AI examinee is associated with a latent ability θ, and the item text is used to predict response selection probabilities for each of the four response options. For the best modeling approach identified, the overall correlation between the true and predicted 2PL correct response probabilities was .82 (bias = 0.00, root mean squared error = 0.18). The study results were promising, showing that item response data generated from AI can be used to calculate item proportion correct, item discrimination, conduct item calibration with anchors, distractor analysis, dimensionality analysis, and latent trait scoring. However, the proposed approach did not achieve the level of accuracy obtainable with human examinee response data. If further refined, potential resource savings in transitioning from human to AI field-testing could be enormous. AI could shorten the field-testing timeline, prevent examinees from seeing low-quality field-test items in real exams, shorten test lengths, eliminate test security, item exposure, and sample size concerns, reduce overall cost, and help expand the item bank. Example Python code from this study is available on Github: https://github.com/hotakamaeda/ai_field_testing1

Performance of ChatGPT on Nursing Licensure Examinations in the United States and China: Cross-Sectional Study.

The creation of large language models (LLMs) such as ChatGPT is an important step in the development of artificial intelligence, which shows great potential in medical education due to its powerful language understanding and generative capabilities. The purpose of this study was to quantitatively evaluate and comprehensively analyze ChatGPT's performance in handling questions for the National Nursing Licensure Examination (NNLE) in China and the United States, including the National Council Licensure Examination for Registered Nurses (NCLEX-RN) and the NNLE. This study aims to examine how well LLMs respond to the NCLEX-RN and the NNLE multiple-choice questions (MCQs) in various language inputs. To evaluate whether LLMs can be used as multilingual learning assistance for nursing, and to assess whether they possess a repository of professional knowledge applicable to clinical nursing practice. First, we compiled 150 NCLEX-RN Practical MCQs, 240 NNLE Theoretical MCQs, and 240 NNLE Practical MCQs. Then, the translation function of ChatGPT 3.5 was used to translate NCLEX-RN questions from English to Chinese and NNLE questions from Chinese to English. Finally, the original version and the translated version of the MCQs were inputted into ChatGPT 4.0, ChatGPT 3.5, and Google Bard. Different LLMs were compared according to the accuracy rate, and the differences between different language inputs were compared. The accuracy rates of ChatGPT 4.0 for NCLEX-RN practical questions and Chinese-translated NCLEX-RN practical questions were 88.7% (133/150) and 79.3% (119/150), respectively. Despite the statistical significance of the difference (P=.03), the correct rate was generally satisfactory. Around 71.9% (169/235) of NNLE Theoretical MCQs and 69.1% (161/233) of NNLE Practical MCQs were correctly answered by ChatGPT 4.0. The accuracy of ChatGPT 4.0 in processing NNLE Theoretical MCQs and NNLE Practical MCQs translated into English was 71.5% (168/235; P=.92) and 67.8% (158/233; P=.77), respectively, and there was no statistically significant difference between the results of text input in different languages. ChatGPT 3.5 (NCLEX-RN P=.003, NNLE Theoretical P<.001, NNLE Practical P=.12) and Google Bard (NCLEX-RN P<.001, NNLE Theoretical P<.001, NNLE Practical P<.001) had lower accuracy rates for nursing-related MCQs than ChatGPT 4.0 in English input. English accuracy was higher when compared with ChatGPT 3.5's Chinese input, and the difference was statistically significant (NCLEX-RN P=.02, NNLE Practical P=.02). Whether submitted in Chinese or English, the MCQs from the NCLEX-RN and NNLE demonstrated that ChatGPT 4.0 had the highest number of unique correct responses and the lowest number of unique incorrect responses among the 3 LLMs. This study, focusing on 618 nursing MCQs including NCLEX-RN and NNLE exams, found that ChatGPT 4.0 outperformed ChatGPT 3.5 and Google Bard in accuracy. It excelled in processing English and Chinese inputs, underscoring its potential as a valuable tool in nursing education and clinical decision-making.

Performance of ChatGPT on Hand Surgery Board-Style Examination Questions

Background The performance of large-language models, such as ChatGPT, on medical and sub-specialty examinations has been preliminarily explored in fields such as radiology, obstetrics and gynecology, and orthopedic surgery. However, no literature assessing ChatGPT’s ability to answer hand surgery exam questions exists. This study’s purpose was to evaluate ChatGPT’s performance on hand surgery board-style examination questions. Methods All questions from the American Society for Surgery of the Hand (ASSH) Hand 100 Exam, Beginner, and Intermediate Assessment tools were entered into ChatGPT-3.5. Responses were regenerated two times to identify inconsistencies. Duplicate questions, questions with figures and/or videos, and questions that ChatGPT refused to provide a response to were excluded. ChatGPT’s correct response rate, answer modifications, and human accuracy were recorded. Results 117 questions from the 3 assessment tools were analyzed: 49 from the ASSH Hand 100, 32 from the Beginner, and 36 from the Intermediate Assessment tools. On ChatGPT’s initial attempt, 40.82% (20/49), 50.0% (16/32), 38.89% (14/36) of questions were correctly answered, respectively. Overall, ChatGPT correctly answered 50/117 (42.7%) of questions on the first try. ChatGPT excelled in topics (&gt;60% correct) of mass/tumor, nerve, wrist, and performed poorly (&lt;40% correct) on topics regarding anatomy/basic science/imaging, brachial plexus, congenital, elbow, tendon, and vascular disorders, trauma. On the Beginner and Intermediate Exams, humans correctly answered 56.64% and 62.73% of questions, respectively. Conclusions ChatGPT can correctly answer simpler hand surgery questions but performed poorly when compared to humans on higher-difficulty questions.