Competency Assessment Research Articles

A Mixed-Methods Investigation of First-Year Medical Students’ Professionalism Competency Development over the Gross Anatomy Course

Abstract Background While competency-based medical education (CBME) and competency assessment in clinical training are well documented, they are less commonly included in preclinical medical education. The gross anatomy laboratory is an opportune setting to incorporate competency assessment and reflection during preclinical medical education. This mixed-methods study determined how first-year medical student assessments of professionalism skills in the gross anatomy lab change over time and analyzed student reflections to contextualize the experiences they had developing the Professionalism competency. Methods First-year medical students at the University of Louisville completed self- and peer-assessments using the Professionalism Assessment Scale (PAS) at the beginning and end of their gross anatomy course (n = 83). The students also completed three competency development portfolio (CDP) entries throughout the course (n = 83). Qualitative thematic analysis with grounded theory was used to analyze comments related to professionalism skill development in the CDP reflections during the course. Results There was no statistical difference in PAS self-assessment scores from Time 1 (M = 4.81, SD = .209) to Time 2 (M = 4.85, SD = .217), p = .108. There was a statistically significant increase in PAS-peer-assessment scores from Time 1 (M = 4.89, SD = .165) to Time 2 (M = 4.93, SD = .127), p = .005. Thematic analysis of CDPs revealed that students believed they developed interpersonal relations/social skills, responsibility skills, and gross anatomy lab-specific skills over the gross anatomy course. Conclusions The Professionalism competency is inherently present and able to be assessed in the gross anatomy lab context for first-year medical students.

Development and validation of situational judgement test for assessment of behavioural competencies required for effective medical practice in Nigeria

Development and validation of situational judgement test for assessment of behavioural competencies required for effective medical practice in Nigeria

International medical graduates’ experiences of clinical competency assessment in postgraduate and licensing examinations: A scoping review protocol

An international medical graduate (IMG) is a doctor who has received their basic medical qualification from a medical school located in a different country from that in which they practice or intend to practice. IMGs are known to face difficulties in their working lives, including differential attainment in assessment. The objective of this review is to map key concepts and types of evidence in academic and gray literature relating to international medical graduates’ experiences of clinical competency assessment and to identify knowledge gaps on this topic by systematically searching, selecting, and synthesizing existing knowledge. All studies will relate to IMGs. The concept of interest will be IMGs’ experiences of assessment. The context will be postgraduate, licensing or credentialing medical assessments of clinical competence. This review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews. Seven electronic databases will be searched for literature published between 2009 and 2024: the Australian Education Index, British Education Index, ERIC, PubMed, PsycINFO, Scopus, and SocINDEX. Gray literature will be searched using Google, Google Scholar, and published reports from postgraduate training bodies and medical licensing organizations. Documents will be independently screened, selected, and extracted by two researchers using a piloted data-extraction tool. Data will be analyzed and presented in tables and in a narrative format. Trial resgistration: Scoping review registration: Open Science Framework: https://osf.io/8gdm7.

Sustainable entrepreneurship education: Impacts of the circular economy, health literacy, and well‐being

AbstractEducation systems that integrate sustainable entrepreneurship education, digitization, and the circular economy (CE) have an effect on well‐being and provide a holistic approach to the promotion of sustainable development. Higher education institutions have a valuable dual role in disseminating the CE as a new sustainability paradigm through both the teaching of students and the preparation of future teachers toward the incorporation of sustainability into other educational levels. This study focused on new health literacy (health competence) findings, with the aim to analyze the health literacy, well‐being, and socioeconomic status of university students in the era of new normality post‐pandemic (COVID‐19). The analysis was based on the health literacy questionnaire of Van der Vaart, WHO‐5, and the MacArthur Scale of socioeconomic status. The results showed that health literacy among students in Romania tended to improve during the pandemic based on their experience with digital media. This effect was particularly evident in the assessment of health information and digital health competence.

Numeracy Literacy Capability of Junior High School Students at Inamosol District

It is important for students to master numeracy literacy as part of the minimum competency assessment, in order to have the thinking skills needed in the 21st century. This research aims to describe the numeracy literacy abilities of junior high school students in mountainous areas. This research uses a mixed method. The data collected is quantitative in the form of numeracy literacy test results and qualitative data in the form of interview results to describe students’ weaknesses in numeracy literacy. The research results showed that student’s literacy skills still remain at levels 1 and 2. Less than 20% of students were at level 3, and there were no students at levels 5 and 6. The category of students is that intervention and basic literacy skills are needed. The qualitative analysis concluded that the cause of low literacy was low communication, mathematization, and representation skills. In terms of indicators of mathematical ability, there are various basic concepts that students have not mastered so they experience difficulty in solving mathematical problems. Apart from that the reasoning that occurs is still imitation in nature, students still tend to memorize and use standard and imperfect procedures. The strategy used only imitates the teacher’s strategy, while students have not used new strategies which causes literacy to still be at a lower level. Keywords: numeracy, literacy, numeracy literacy level, mountains areas

Validity Evidence for Endoscopic Ultrasound Competency Assessment Tools: A Systematic Review

Background and aims: Competent endoscopic ultrasound (EUS) performance requires a combination of technical, cognitive, and non-technical skills. Direct observation assessment tools can be employed to enhance learning and ascertain clinical competence; however, there is a need to systematically evaluate validity evidence supporting their use. We aimed to evaluate the validity evidence of competency assessment tools for EUS and examine their educational utility. Methods: We systematically searched five databases and grey literature for studies investigating EUS competency assessment tools from inception to May 2023. Data on validity evidence across five domains (content, response process, internal structure, relations to other variables, and consequences) were extracted and graded (maximum score 15). We evaluated educational utility using the Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI). Results: From 2081 records, we identified 5 EUS assessment tools from 10 studies. All tools are formative assessments intended to guide learning, with 4 employed in clinical settings. Validity evidence scores ranged from 3 to 12. The EUS and ERCP Skills Assessment Tool (TEESAT), Global Assessment of Performance and Skills in EUS (GAPS-EUS), and the EUS Assessment Tool (EUSAT) had the strongest validity evidence with scores of 12, 10, and 10, respectively. Overall educational utility was high given ease of tool use. MERSQI scores ranged from 9.5-12 (maximum score 13.5). Conclusions: The TEESAT, GAPS-EUS, and EUSAT demonstrate strong validity evidence for formative assessment of EUS and are easily implemented in educational settings to monitor progress and support learning.

Implementing an interactive oral task to assess undergraduate psychology students’ attainment of pre-professional competencies

Finding engaging and authentic ways to assess required competencies which are resistant to academic misconduct is a challenge for tertiary psychology education. While simulation-based tasks are commonly used to assess postgraduate students’ development of professional competencies, they are less commonly used in undergraduate programs to assess pre-professional competencies. Interactive oral assessments have been suggested to achieve authenticity while inhibiting academic misconduct, and to be feasible within an online learning environment. The current study presents students’ and markers’ perceptions of an interactive oral task implemented to assess fourth-year psychology students’ counselling and reflection skills. Results suggest that students and markers perceived the interactive oral task to be an authentic assessment of knowledge and skills required for future practice, which encouraged deeper learning than written assessments or exams, and was resistant to academic misconduct. Despite feeling anxious about the new assessment format, both students and markers reported finding the engaging nature of the task enjoyable and indicated that they would like to be involved in more interactive oral assessments in the future. The extent to which these findings reflect the current literature, and the role for interactive oral tasks in the innovative, authentic assessment of pre-professional competencies which promote academic integrity are discussed.

An evaluation of the impact of the Lambeth PCN pharmacist anticoagulation skills development project

Abstract Background Atrial Fibrillation (AF) is a prevalent heart condition linked to significant morbidity and mortality due to its association with stroke.1 Despite the clear benefits of anticoagulation in reducing stroke risk, its underuse and suboptimal monitoring remain a concern.2 Aim This study aims to evaluate the impact of a pharmacist-led training programme in Lambeth, London, focusing on improving pharmacists' competency and confidence in managing Direct Oral Anticoagulants (DOACs). It assesses if the training met pharmacists' needs, evaluates current monitoring patterns to determine if training was warranted, and identified areas for further training improvement. Methods The study involved 41 GP practices across 9 Primary Care Networks (PCNs) in Lambeth. Practices were eligible based on their involvement in the management of AF patients prescribed DOACs. All pharmacists at these practices who were actively engaged in anticoagulation management and willing to participate in pre- and post-training assessments were invited to take part. Recruitment involved direct communication through practice managers and emails detailing the study’s objectives and benefits. DOAC monitoring data was extracted from patient medication records and subsequently compared to the South East London anticoagulation guidelines3 to establish compliance percentages. A competency assessment was conducted on pharmacists using a pre- and post-training checklist framework devised by the Consultant Anticoagulant Team at Guy’s and St Thomas’ NHS Foundation Trust (GSTT). The Wilcoxon signed-rank test analysed this paired data, with a p < 0.05 set for statistical significance. Additionally, semi-structured interviews were conducted to gather insights on their training experience and perceptions. Transcripts were coded using NVIVO software to identify themes. Ethical approval was obtained from Kingston University London. Results Of the 2,987 AF patients across the 41 GP practices, 87.6% were on DOACs. Prescribing and monitoring trends were consistent across all GPs, with scope to improve compliance across all parameters, such as CHA₂DS₂-VASc score assessment prior to DOAC initiation. The competency assessment, completed by ten pharmacists, showed a significant improvement post-training, with mean scores rising from 3.3 to 4.1 (p< 0.00001). Improvements were seen across all domains, particularly in explaining the rationale for anticoagulation, showing CHA₂DS₂-VASc and HASBLED scores, and discussing modifiable HASBLED factors. Competency in patient communication remained highest, while managing particular cases like reversal procedures and patients with cancer showed lower competency, indicating areas for training improvement. The interviews, conducted with ten pharmacists, revealed three main themes: (1) Confidence and competence in anticoagulant prescribing and optimisation, (2) Evaluation of the training programme's effectiveness, and (3) Professional growth and career development. Pharmacists reported greater confidence in managing DOACs post-training, though challenges remained in patient engagement and collaboration with GPs lacking DOAC knowledge. Discussion / Conclusion The evaluation proved a positive impact on pharmacists' competency and confidence in DOAC management. Suboptimal monitoring practices in Lambeth highlighted the need for this training programme and its crucial role in improving patient outcomes. Limitations include a small interview sample and potential biases in self-reported evaluations. A longitudinal study is recommended to assess long-term impacts. Ongoing training and systematic data collection are essential, and similar training should be extended to other healthcare professionals.

Evaluating Pragmatic Competence in Language Learners Through an Assessment of Current Measurement Tools

This study reviews and evaluates current tools used to measure pragmatic competence in language learners. Pragmatic competence—the ability to use language appropriately in social contexts—is essential for effective communication in a second language. This article assesses various measurement tools, including discourse completion tests (DCTs), role-plays, self-assessment surveys, and technology-assisted assessments, analyzing their effectiveness and limitations. Findings suggest that each tool offers unique benefits, though a combination of traditional and digital methods provides the most comprehensive evaluation. The study concludes that blending these approaches allows for a more accurate assessment of pragmatic competence, supporting language educators in creating balanced evaluations for language learners.

Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task

Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the frameworks of CPS skills is still a challenging task. Traditional studies have relied on manual coding to decipher behavioral data for CPS, but such coding methods can be very time-consuming and cannot support real-time analyses. Scholars have begun to explore approaches for constructing automatic coding models. Nevertheless, the existing models built using machine learning or deep learning techniques depend on a large amount of training data and have relatively low accuracy. To address these problems, this paper proposes a prompt-based learning pre-trained model. The model can achieve high performance even with limited training data. In this study, three experiments were conducted, and the results showed that our model not only produced the highest accuracy, macro F1 score, and kappa values on large training sets, but also performed the best on small training sets of the CPS behavioral data. The application of the proposed prompt-based learning pre-trained model contributes to the CPS skills coding task and can also be used for other CSCW coding tasks to replace manual coding.

Assessment of Informatics Competency Among Nursing Faculty Members and Its Application in Educating Nursing Students at Abadan Nursing Faculty

Background: Nursing informatics in educational activities can serve as a strategy to bridge the gap between clinical practice and nursing education programs. Objectives: This study aimed to assess the informatics competency of nursing faculty members and its application in educating nursing students at Abadan Nursing Faculty. Methods: This descriptive-analytical research was conducted in 2023 at Abadan Nursing Faculty. The sample consisted of 73 faculty members from the nursing, operating room, and anesthesia departments, who were included in the study through census sampling. Data collection was performed using the standardized Nursing Informatics Competency Assessment Tool (NICAT) and an electronic teaching aids checklist. Data were analyzed using SPSS version 18, with independent t-tests, one-way analysis of variance (ANOVA), and Pearson’s correlation at a significance level of P < 0.05. Results: The mean score of faculty members’ nursing informatics competency at the Nursing Faculty was 110 out of a maximum of 150, placing it within the proficient range. The highest score for nursing informatics competency was related to the informatics literacy dimension (47.65), while the lowest was related to the information management skills dimension (19.54). Information management skills (r = 0.881, P < 0.001), computer literacy (r = 0.871, P < 0.001), and informatics literacy (r = 0.976, P < 0.001) were significantly correlated with nursing informatics competency. Faculty members reported the highest use of multimedia content development (mean score = 4.19) and the lowest use of robotic surgery (mean score = 1.71). Conclusions: Faculty members’ informatics competency was reported to be at a proficient level. However, due to advancements in technology and transformations in nursing care, it seems necessary to improve nursing faculty members’ abilities in all aspects, particularly in the information management skills dimension. It is recommended to hold faculty development programs regarding the integration of technology into the curriculum to elevate the quality of education.

Assessment of student-teachers' digital competencies across educational aspects: the current status and areas for improvement

In the present digital age, improved teacher digital competencies are increasingly becoming an important component of teachers' qualifications. The purpose of this study was to investigate the digital competencies level of student-teachers and probe the areas for improvement. The subject of this study was undergraduate student-teachers in their final year of Bachelor of Education degree programmes from the College of Education at one of the public Universities in Tanzania. Purposive and convenience sampling techniques were employed to get a sample of 549 participants. The study employed the UNESCO ICT Competency Framework for Teachers (ICT CFT) as a theoretical framework. The study employed a quantitative approach and descriptive cross-sectional design. Data were collected through an online survey that included six competency areas linked to teachers' professional practices. Data analysis procedures included descriptive and inferential analysis. The findings revealed that student-teachers perceived their digital competencies as moderately higher at the knowledge acquisition level and moderately low at the knowledge deepening level of ICT CFT. The results also demonstrated that student-teachers were more competent in integrating technology in educational aspects that are indirectly linked to teaching (e.g. teacher professional learning, and organization and administration) and less competent in educational aspects that are directly linked to classroom teaching (e.g. pedagogy, and curriculum and assessment). The study further established a positive and statistically significant association between student-teachers' digital competencies and knowledge acquisition and knowledge deepening levels. The results of the present study signify the importance of developing student-teachers' digital competencies across all educational aspects. However, in order to guarantee technology integration in classroom teaching, it is recommended that teacher education programmes focus more on improving student-teachers' digital competencies in competency areas that are directly linked to classroom pedagogical practices.

POLICY OF COMMON COMPETENCY ASSESSMENT IN TAIWAN’S HIGHER EDUCATION AND ITS APPLICATION IN UNIVERSITY AFFAIRS DATA

<p>This study focus on the competency assessments conducted through the University Career and Competency Assessment Network (UCAN), a platform widely utilized in Taiwan’s higher education system. Using a private university in Taiwan as a case study, the research tracked cohorts of students who enrolled between the 2018 and 2021 academic years. UCAN assessments were administered as pre-tests during the students’ first year and post-tests in their third year to measure changes in their common competencies. Institutional data obtained from these assessments were analyzed using one-way ANCOVA and independent sample t-tests to explore the impact of the Higher Education Sprout Project, which emphasizes innovative teaching, on students' competency development. The results from the longitudinal analysis reveal significant improvements in all eight core competencies, with the most notable self-efficacy gains observed in communication, problem-solving, and information technology skills.</p><p> </p><p><strong> Article visualizations:</strong></p><p><img src="/-counters-/soc/0726/a.php" alt="Hit counter" /></p>

Islamic-Centred Leadership: Qualifying Leaders For Islamic Banks

Islamic banking operates under the principles of Shari’ah law, which governs its financial practices and ethical standards. However, a major challenge facing the industry is the disconnect between the leadership skills required for Islamic banking and those currently in place. In light of this, a research study was conducted to evaluate the alignment between the competencies of leaders in Gulf countries’ Islamic banks and the Shari’ah law’s requirements. The study utilized a qualitative approach, including interviews with eight purposively selected HR managers from various banks in the region. The findings revealed a significant gap between the leadership competencies and the principles of trust and justice emphasized in Shari’ah law. This discrepancy highlights the need for a more thorough assessment of competencies to ensure that they align with the values and goals of Islamic banking. By bridging this gap, Islamic banks can effectively incorporate the principles of Shari’ah law into their leadership competencies and promote a more efficient and successful industry.

Training physicians in motivational communication: An integrated knowledge transfer study protocol.

While behavior change counseling (BCC) targeting health risk behaviors has shown efficacy for improving patient health outcomes, barriers to knowledge translation have resulted in poor uptake among health care providers (HCPs). This article outlines the development of a new BCC training framework for HCPs, from inception to readiness for efficacy testing. It provides an example of integrated knowledge translation (iKT) used in alignment with the obesity-related behavioral intervention trials model. (a) A modified Delphi process identified essential BCC skills for HCPs; (b) a survey assessed HCP attitudes and training needs; (c) an online competency assessment tool was developed using iKT mixed methods; (d) a training program was developed and refined using a logic model; and (e) the program was optimized using iterative rounds of participant feedback. A future proof-of-concept trial (f) will determine the program's readiness for full efficacy testing. A Delphi panel (n = 46) identified 11 core BCC competencies for HCPs, defining "motivational communication." The HCP survey (n = 80) showed willingness to devote 4 hr to introductory training in BCC. The Motivational Communication Competency Assessment Test (MC-CAT: an online, interactive evaluation tool) and a motivational communication training program (MOTIVATOR: accredited for continuing education by the Royal College of Physicians and Surgeons of Canada) were collaboratively developed with knowledge users. The optimization process (n = 11) provided key feedback, with minor changes being made to the program. In developing a new BCC framework, obstacles to BCC implementation were addressed through an iterative iKT process. This should improve eventual intervention uptake. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Implementing an obstetric simulation training protocol in a critical access hospital

BackgroundA high-fidelity (HF) simulation training protocol was implemented to improve nurses' knowledge, skills, and self-confidence to improve patient quality of care and safety, thus ultimately having an impact on decreasing maternal mortality rates. SampleThe sample consisted of 16 participants including nurses, a physician, emergency medical technicians (EMTs), and paramedics. MethodsThis project used a self-confidence and competence assessment prior to and following participation to determine statistical significance. In addition, review of the National League for Nursing Student Satisfaction and Self-Confidence in Learning tool data was done to determine the overall success of implementing a HF simulation training protocol. ResultsSuccess of the simulation and content was determined by a reported increase in self-confidence and competence post-simulation. The summative evaluation revealed that participants felt that implementing a HF simulation training protocol, in general, was useful for training. ConclusionWhile this project focused on obstetrics, the premise of the project can be applied to many subject areas to advance practice in rural health. Findings suggest that use of a high-fidelity simulation training protocol is beneficial.

Variations in Trauma Education Practices Across Emergency Medicine Residencies: Insights from a National Survey of Program Directors.

Care of the acutely injured trauma patient is integral to the practice of emergency medicine. It is currently unknown how most emergency medicine residencies structure their residents' trauma experience and little guidance for competency assessment is provided by the Residency Review Committee. Our study aimed to determine current emergency medicine residency practices in trauma resuscitation. We conducted a cross-sectional survey of members of the Council of Residency Directors in Emergency Medicine (CORD) listserv in April 2023. Frequency with percentage of item responses is reported and differences across trauma levels assessed via Fisher's exact test (α = 0.05). Fifty-seven program directors responded to the survey (21.9%), the majority of whom operate at Level I facilities. Significantly more Level II/ III centers send residents to other sites for trauma experience compared to Level I (p = 0.000). Residents participate in all key procedures (eg, airway management, central venous access) when managing traumas except thoracotomy where participation was notably lower and statistically different across levels (p = 0.000). Lastly, program directors were very confident their residents can lead traumas independently and few acknowledged citations for deficiency in trauma training. Trauma training and confirmation of competency is critical among EM residents who may serve as the sole lead in rural emergency departments. This study demonstrates that there is considerable variability in how residency programs structure trauma education, particularly with regards to the exposure to invasive procedures and the opportunity to lead trauma resuscitations. As the American Board of Emergency Medicine has introduced requirements for program directors to attest specifically to the competence of residents to lead trauma resuscitations, standardized and validated tools should be adopted to support this attestation and ensure competence regardless of the program hospital's trauma level.

Strengthening the Multiple-Choice Assessment: Improving Item-Writing Skills of Physician Assistant Educators.

Physician assistant (PA) educators are tasked with the challenging privilege of ensuring future PAs are knowledgeable and competent to practice medicine. Most enter academia without formal education in providing education. They learn as they go-a treacherous and steep learning curve. This study provides PA educators literature and skills to bolster the strength and validity of multiple-choice examination items to measure learner progress and identify knowledge gaps. Skills can be improved at the level of the individual educator and program or institution. Approaching item writing and item revision in a stepwise, evidence-based manner can improve the quality of the items, strengthening multiple-choice examinations. This process can yield improved assessments of students' medical knowledge and competence. This study aims to provide the PA educator with techniques to improve their item-writing skills.

Analysis of the Kampus Mengajar Program Cohort 7 on the Improvement of Literacy and Numeracy Abilities of Students at SD Negeri 1 Margototo

The Kampus Mengajar program, part of the Merdeka Belajar-Kampus Merdeka (MBKM) initiative, enables university students to engage in primary education, particularly in areas with limited access. This study analyzes the impact of the seventh cohort of the program on literacy and numeracy skills at SD Negeri 1 Margototo. Using a qualitative case study approach, data were gathered through observations, interviews, and documentation from teachers, students, and participating university students. The research findings indicate that there was an increase in the average scores from the pretest and post-test of the Minimum Competency Assessment (AKM), specifically a 42% improvement in literacy skills and a 44% improvement in numeracy skills. These findings suggest that the Kampus Mengajar program plays a significant role in enhancing the quality of primary education and serves as an effective collaborative model between university students and schools in underserved areas.

Postpartum Hemorrhage Management in Canadian Emergency Nursing Practice

Background: Postpartum hemorrhage (PPH) remains a leading cause of maternal mortality globally, claiming approximately 70,000 lives annually. Despite preventive efforts, high-income countries, including Canada, have reported increasing rates of PPH. The rising incidence and associated complications indicate improved emergency nursing practices in managing PPH are needed. Methods: PPH guidelines from the Society of Obstetricians and Gynecologists of Canada, Federation of International Gynecologists and Obstetricians, and the Canadian Association of Perinatal, and Women’s Health Nurses were compared to National Emergency Nurses Association guidelines. The competencies, medications, and obstetric maneuvers for PPH that may be relevant to emergency nurses were extracted and categorized for the emergency nursing HIRAID (History & Red flags, Assessment, Interventions, & Diagnosis) framework. Findings/Results: We found 13 discrete skills across the PPH guideline recommendations that are applicable to emergency nurses. There is near perfect overlap in medical recommendations, but gaps exist between FIGO and SOGC guidelines and NENA and CAPWHN guidelines. There are SOGC and FIGO guidelines that apply to emergency nurses that are not addressed in NENA guidelines, Discussion: There was significant overlap on treatment recommendations from maternal health bodies such as uterotonic drugs, hemorrhage control, and hemostatic resuscitation. There were significant gaps in NENA recommendations. Conclusion: The comparison of PPH guidelines highlights where NENA guidelines need further detail. These skills are particularly significant for providing nursing care in rural and remote areas. Standardized protocols for PPH management, interprofessional collaboration, and regular competency assessments are included in most guidelines but are conspicuously absent for emergency nurses. To address the gap, NENA could integrate national emergency nursing PPH competencies standards or develop recommendations for the care of maternal emergencies.