Competency Assessment Research Articles

Assessing the State of Training in Congenital Interventional Cardiology: A Global Survey of Program Directors.

This study aimed to evaluate the current state of congenital interventional cardiology training worldwide, with a focus on case volumes, competency assessment, and the need for ongoing mentorship during early career stages. A survey was conducted among program directors (PDs) of congenital interventional training programs across the globe. The survey gathered data on training pathways, case volumes, types of procedures performed, trainee competency assessment, and the role of ongoing mentorship. Of the 79 PDs who completed the survey, it was observed that training pathways and case volumes varied significantly, particularly between the United States and other countries. Most PDs reported an annual laboratory case volume of >500 congenital cardiac cases, with most cases being interventional. While trainees demonstrated competency in simple procedures (diagnostic cases, simple ASD closure), complex interventions (such as patent ductus arteriosus closure in premature infants) require ongoing mentorship for graduates. PDs recommended a minimum case volume of 400 total cases for trainees, including 250 interventional cases. In addition to case volumes, assessing trainee competency was deemed important, with clinical reasoning, judgment, skillset, teamwork, and complication management being key areas of evaluation. The study highlights the variability in congenital interventional cardiology training and the need for ongoing mentorship during the early career years. External mentorship programs, facilitated by national and international societies, are proposed to provide critical support for early career interventionalists thus enhancing patient care for congenital heart disease. Ultimately, the findings of this survey may serve as a framework for future training standards and guidelines in this specialized field.

Social-Emotional Competence among School-Aged Children in the Chinese Context: Validation of the Washoe County School District Social-Emotional Competency Assessment.

The present study aims to validate the 40-item and the brief 17-item Washoe County School District Social-Emotional Competency Assessment (WCSD-SECA), a self-report measure of social-emotional competencies, among Chinese school-aged children residing in Hong Kong. A total of 349 children (M = 9.86 years, SD = 1.22; 45.82% girls) and their parents (77.84% mothers) completed a set of questionnaires independently. The factor structure of both versions of the WCSD-SECA was assessed via confirmatory factor analyses. Structural equation models were then conducted to examine the predictive validity of the WCSD-SECA. The findings indicated that both the 40-item and the 17-item versions of the WCSD-SECA fit the data adequately. Both versions were also associated with self-reported positive and negative affect and parent-reported internalizing problems and externalizing problems. Additionally, social-emotional competencies as measured by the 40-item version were significantly associated with parent-reported prosocial behavior, whereas those as measured by the 17-item version were associated with prosocial behavior with a marginal significance. The findings demonstrated an adequate factor structure and predictive validity of the full version and the brief version of the WCSD-SECA in assessing social-emotional competencies. Hence, they serve as a useful tool for researchers, educators, and mental health practitioners to evaluate school-aged children's social-emotional competencies in the Chinese context.

Multi-modal recommender system for predicting project manager performance within a competency-based framework.

The evaluation of performance using competencies within a structured framework holds significant importance across various professional domains, particularly in roles like project manager. Typically, this assessment process, overseen by senior evaluators, involves scoring competencies based on data gathered from interviews, completed forms, and evaluation programs. However, this task is tedious and time-consuming, and requires the expertise of qualified professionals. Moreover, it is compounded by the inconsistent scoring biases introduced by different evaluators. In this paper, we propose a novel approach to automatically predict competency scores, thereby facilitating the assessment of project managers' performance. Initially, we performed data fusion to compile a comprehensive dataset from various sources and modalities, including demographic data, profile-related data, and historical competency assessments. Subsequently, NLP techniques were used to pre-process text data. Finally, recommender systems were explored to predict competency scores. We compared four different recommender system approaches: content-based filtering, demographic filtering, collaborative filtering, and hybrid filtering. Using assessment data collected from 38 project managers, encompassing scores across 67 different competencies, we evaluated the performance of each approach. Notably, the content-based approach yielded promising results, achieving a precision rate of 81.03%. Furthermore, we addressed the challenge of cold-starting, which in our context involves predicting scores for either a new project manager lacking competency data or a newly introduced competency without historical records. Our analysis revealed that demographic filtering achieved an average precision of 54.05% when dealing with new project managers. In contrast, content-based filtering exhibited remarkable performance, achieving a precision of 85.79% in predicting scores for new competencies. These findings underscore the potential of recommender systems in competency assessment, thereby facilitating more effective performance evaluation process.

Assessing“Breaking Bad News” Communication Competency in Midwifery Students

The communication task of conveying difficult news to patients by health care providers, described in the literature as “breaking bad news,” is one that is often performed poorly. There is, at the same time, a relatively small amount of research in the health sciences education literature devoted to the assessment of this particular competency. Like other health care providers, registered midwives in Canada are expected to have knowledge and skill in this domain of communication. In the Ontario Midwifery Education Program, senior students are expected to develop competence in managing situations involving loss during the childbearing year; however, there is little guidance for preceptors with respect to the assessment of performance for this skill. This article reviews the relevant literature on the assessment of “breaking bad news” and workplace assessment tools in health education, and proposes the use of a clinical encounter card that uses a modified Breaking Bad News Assessment Scale to assist and guide preceptors. Further investigation of the tool’s reliability, validity, and acceptability to preceptors is needed. This article has been peer reviewed.

Construction of a sustainable design competency assessment system for fashion designers in China

Construction of a sustainable design competency assessment system for fashion designers in China

Development of Teaching Factory Model to Enhance Creativity of Students in the Home Economics Department of the Faculty of Engineering, UNM, Indonesia

The research aims to (1) develop an appropriate 6M teaching factory model for practical learning in the Department of Family Welfare Education, Faculty of Engineering, Universitas Negeri Makassar with a concentration in Cosmetology, (2) produce a 6M teaching factory model that is valid, practical, and effective for practical learning in the Department of Family Welfare Education, Faculty of Engineering, Universitas Negeri Makassar with a concentration in Cosmetology. This study is an R & D (Research and Development)-based development research. The subjects used were 20 students from the Cosmetology concentration. The development method adapted in this study is derived from the Borg and Gall Model, which includes three main stages, namely: preliminary study covering field surveys, document analysis, and theoretical considerations; the design phase involving the creation of a teaching factory model draft and the improvement of the model; and the experimental phase that includes initial inspection, large-scale testing, and final refinement of the product. The results of this research are the creation of a teaching factory learning model called the TEFA 6M, along with operational tools including the TEFA 6M model book, Syllabus, modules, jobsheet, and LPPO. This model is designed to strengthen the practical abilities of students. The TEFA 6M model is divided into seven phases in one round, starting from pre-order, pre-production, production, post-production, packaging, to order delivery. One TEFA 6M model cycle combines soft skill and hard skill activities. The implementation of this model uses a block and rotating system and is concluded with a competency assessment. The developed TEFA 6M model has been validated by experts, revised, and tested on both small and large scales. Based on the t-test results, it is known that the Sig. (2-tailed) value of 0.000 is smaller than the probability of 0.05, so it can be concluded that there is a difference in test results between the experimental class posttest and the control class posttest. Furthermore, for the average score of the experimental group, a score of 78.38 was obtained, and 42.36 in the control group; thus, the 6M Teaching Factory Learning Model can be said to be effective in the experimental class in improving student competencies in the Facial Makeup Techniques course. The research indicates that the TEFA-6M model has met the standard of validity. Based on the evaluations carried out, it is known that this teaching approach is practical, in terms of management abilities by the lecturer, and effective, as seen from the level of student activity, student reactions to the functionality and implementation of the model, and from test results that assess task completion.

The design and operationalization of the teacher competence management framework: The method and practical tools to identify and close teacher competence gaps for the teaching of 21st century skills

The aim of the Framework for Teacher Competence Management is to provide school leadership with an empirically tested protocol on how to determine teachers’ professional development needs for teaching 21st century skills. Both quantitative and qualitative research methods were applied to collect and analyze data through three cycles of design-based implementation research over a period of three years, from 2017 to 2019, and by collaborating with staff from 15 schools. The Framework for Teacher Competence Management was developed through adapting a competence management model from the business environment to the school setting. The framework includes three steps: Competence identification, assessment and development; school developmental goals and education reform priorities; and the action and information flows between teachers and school leadership team members. The Framework for Teacher Competence Management has been operationalized through the design and validation of a method to identify teachers’ professional development needs and competence assessment tools, such as performance level descriptors, knowledge tests, and self-assessment questionnaires. The research outlines significant differences in teacher performance within a single school, emphasizing the necessity of gathering school-based evidence for planning more personalized professional development solutions to address professional development needs for different teacher groups.

Metabolic imaging of human cumulus cells reveals associations with pregnancy and live birth.

Can fluorescence lifetime imaging microscopy (FLIM) detect associations between the metabolic state of cumulus cell (CC) samples and the clinical outcome of the corresponding embryos? FLIM can detect significant variations in the metabolism of CC associated with the corresponding embryos that resulted in a clinical pregnancy versus those that did not. CC and oocyte metabolic cooperativity are known to be necessary for the acquisition of developmental competence. However, reliable CC biomarkers that reflect oocyte viability and embryo developmental competency have yet to be established. Quantitative measures of CC metabolism could be used to aid in the evaluation of oocyte and embryo quality in ART. A prospective observational study was carried out. In total, 223 patients undergoing IVF with either conventional insemination or ICSI at a tertiary care center from February 2018 to May 2020 were included, with no exclusion criteria applied. This cohort had a mean maternal age of 36.5 ± 4.4 years and an average oocyte yield of 16.9 (range 1-50). One to four CC clusters from each patient were collected after oocyte retrieval and vitrified. CC metabolic state was assessed using FLIM to measure the autofluorescence of the molecules NAD(P)H and FAD+, which are essential for multiple metabolic pathways. CC clusters were tracked with their corresponding oocytes and associated embryos. Patient age, Day 3 and Day 5/6 embryo morphological grades, and clinical outcomes of embryos with traceable fate were recorded. Nine FLIM quantitative parameters were obtained for each CC cluster. We investigated associations between the FLIM parameters and patient maternal age, embryo morphological rank, ploidy, and clinical outcome, where false discovery rate P-values of <0.05 were considered statistically significant. A total of 851 CC clusters from 851 cumulus-oocyte complexes from 223 patients were collected. Of these CC clusters, 623 were imaged using FLIM. None of the measured CC FLIM parameters were correlated with Day 3 morphological rank or ploidy of the corresponding embryos, but FAD+ FLIM parameters were significantly associated with morphological rank of blastocysts. There were significant differences for FAD+ FLIM parameters (FAD+ fraction engaged and short lifetime) from CC clusters linked with embryos resulting in a clinical pregnancy compared with those that did not, as well as for CC clusters associated with embryos that resulted in a live birth compared those that did not. Our data are based on a relatively low number of traceable embryos from an older patient population. Additionally, we only assessed CCs from 1 to 4 oocytes from each patient. Future work in a younger patient population with a larger number of traceable embryos, as well as measuring the metabolic state of CCs from all oocytes from each patient, would provide a better understanding of the potential utility of this technology for oocyte/embryo selection. Metabolic imaging via FLIM is able to detect CC metabolic associations with maternal age and detects variations in the metabolism of CCs associated with oocytes leading to embryos that result in a clinical pregnancy and a live birth versus those that do not. Our findings suggest that FLIM of CCs may be used as a new approach to aid in the assessment of oocyte and embryo developmental competence in clinical ART. National Institutes of Health grant NIH R01HD092550-03 (to C.R., and D.J.N.). Becker and Hickl GmbH and Boston Electronics sponsored research with the loaning of equipment for FLIM. D.J.N. and C.R. are inventors on patent US20170039415A1. N/A.

Surgical Competency Assessment Using Entrustable Professional Activities

Surgical Competency Assessment Using Entrustable Professional Activities

Associations of Internal Medicine Residency Milestone Ratings and Certification Examination Scores With Patient Outcomes

Despite its importance to medical education and competency assessment for internal medicine trainees, evidence about the relationship between physicians' milestone residency ratings or the American Board of Internal Medicine's initial certification examination and their hospitalized patients' outcomes is sparse. To examine the association between physicians' milestone ratings and certification examination scores and hospital outcomes for their patients. Retrospective cohort analyses of 6898 hospitalists completing training in 2016 to 2018 and caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019 at US hospitals. Primary outcome measures included 7-day mortality and readmission rates. Thirty-day mortality and readmission rates, length of stay, and subspecialist consultation frequency were also assessed. Analyses accounted for hospital fixed effects and adjusted for patient characteristics, physician years of experience, and year. Certification examination score quartile and milestone ratings, including an overall core competency rating measure equaling the mean of the end of residency milestone subcompetency ratings categorized as low, medium, or high, and a knowledge core competency measure categorized similarly. Among 455 120 hospitalizations, median patient age was 79 years (IQR, 73-86 years), 56.5% of patients were female, 1.9% were Asian, 9.8% were Black, 4.6% were Hispanic, and 81.9% were White. The 7-day mortality and readmission rates were 3.5% (95% CI, 3.4%-3.6%) and 5.6% (95% CI, 5.5%-5.6%), respectively, and were 8.8% (95% CI, 8.7%-8.9%) and 16.6% (95% CI, 16.5%-16.7%) for mortality and readmission at 30 days. Mean length of stay and number of specialty consultations were 3.6 days (95% CI, 3.6-3.6 days) and 1.01 (95% CI, 1.00-1.03), respectively. A high vs low overall or knowledge milestone core competency rating was associated with none of the outcome measures assessed. For example, a high vs low overall core competency rating was associated with a nonsignificant 2.7% increase in 7-day mortality rates (95% CI, -5.2% to 10.6%; P = .51). In contrast, top vs bottom examination score quartile was associated with a significant 8.0% reduction in 7-day mortality rates (95% CI, -13.0% to -3.1%; P = .002) and a 9.3% reduction in 7-day readmission rates (95% CI, -13.0% to -5.7%; P < .001). For 30-day mortality, this association was -3.5% (95% CI, -6.7% to -0.4%; P = .03). Top vs bottom examination score quartile was associated with 2.4% more consultations (95% CI, 0.8%-3.9%; P < .003) but was not associated with length of stay or 30-day readmission rates. Among newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries.

Enhancing the quality of competency assessment for elementary school students in modern education

The paper aims to explore strategies to enhance the quality of competency assessment for elementary school students in the context of modern education. Through a comprehensive study conducted from February 2022 to January 2024 in Hai Phong city, data were collected from 112 educational administrators and 564 teaching professionals. The research employed a mixed-methods approach, combining surveys and interviews to gather insights into current assessment practices and challenges. Findings underscored the critical role of competency assessments in facilitating student development. They revealed a need for tailored assessment methods aligned with contemporary educational standards and student-centered approaches. Additionally, the study highlighted the importance of continuous professional development for educators to improve their assessment literacy and competency evaluation skills. Overall, the research suggests that implementing targeted strategies can lead to more accurate and holistic assessment practices, ultimately fostering student growth and development in elementary education.

Wind Shear Operation-Based Competency Assessment Model for Civil Aviation Pilots

Wind Shear Operation-Based Competency Assessment Model for Civil Aviation Pilots

A Hybrid Multi-Attribute Decision-Making Approach Considering Correlations Among Indicators

Considering the fuzzy and uncertainty of the evaluation information of the evaluation object, the type of indicators are expanded into five types of mixed evaluation information, namely, exact number, interval number, triangular fuzzy number, hesitant fuzzy number, and probabilistic linguistic term sets, and different types of evaluation information are adopted according to the different characteristics and types of indicators, respectively. When the evaluation information is hybrid information, the generalized Shapley function based on fuzzy measurement is used to analyze the interaction between indicators and determine the weights of indicators, considering that the interaction between indicators is more complicated. In view of the fact that the conversion of mixed information into the same kind of information will lead to the problem of complicated calculation and missing information, the VIKOR method is used to comprehensively evaluate the evaluation objects and select the best ones. Finally, the validity and feasibility of the proposed method is verified by taking the assessment of data literacy level and competence of teachers in a university in a western province of China as an example.

An assessment on the competencies in local governance among the punong barangays in the Municipality of Hadji Panglima Tahil, Province of Sulu

The study evaluates the competencies of Punong Barangays in Hadji Panglima Tahil, Province of Sulu, in terms of delivery of basic services, executive functions, and legislative functions during the Fiscal Year 2023. The study involved 100 respondents, mostly female, married, aged 31+, with high school education and an average monthly income of 5,001-10,000. With 100 samples taken through non-probability sampling method via purposive sampling, and with the use of weighted mean, standard deviation, t-test for independent samples, One-way ANOVA, and Pearson’s r the findings indicate that Punong Barangays are generally competent in their duties and functions. The study supports the Antal Szerletics, Gerry Stoker, and B. Guy Peter’s model, which emphasizes the importance of establishing a good governance structure to guide and manage sustainability and responsibility. This structure allows leaders to set direction, establish goals, and create policies to execute strategies. Good governance can be conceived as a basket of practices, including professional civil service, anti-corruption policies, transparency and accountability, democratic decision-making, the rule of law, protection of human rights, and independent judiciary. The findings suggest that the Punong Barangays in Hadji Panglima Tahil, Province of Sulu are competent in their duties and functions, with age, gender, average monthly income, and educational attainment not significantly mediating their assessment of competencies.

Review on Competency Assessment Instrumentation in Computer-based Simulation

AbstractComputer-based simulation is utilised across various educational fields, employing diverse technologies to facilitate practical understanding of content and the acquisition of skills that can help close the gap between theory and practice. The possibility of providing scenarios that resemble on-the-job tasks, enables instructors to both train and assess the trainee’s comprehension of the tasks at hand. The practices as well as the technologies for the assessment of simulation-based training vary across disciplines. Our motivation is to address quality procedures from a cross-discipline perspective. There seems to be a lack of scientific investigation that takes one step back from the specific application and investigates how assessment instruments can be developed to fit training outcomes regardless of the professional discipline. This scoping literature review on empirical studies aims to do so by exploring how competency is assessed with computer-based simulation. Objectives to achieve this are: (1) apply established training research theory to structure a decomposition of assessment instruments; to (2) review approaches to assessments factored over this structure; and (3) discuss quality procedures taken in the creation of the reported instruments and then propose an approach to assessment instrumentation that can be applied independent of discipline, with the range of current technology, and for any focal outcome competency. By reviewing a spectrum of fields, we capture reported assessment practices across a range of currently employed technologies. This literature review combines the methods of a scoping review with the qualities of a systematic literature review while keeping to conventional reporting guidelines. This allowed us to provide insight into current approaches and research designs that applied measurements in the range from automated assessment to observer rating of simulation-based training in professional work settings. This study found that all reviewed studies measured skill-based outcomes with some variation and that there is more theoretical and empirical work to be done to close the gap on quality instrumentation and its validity evidence. Our contribution to the field of training research is the operationalized component structure and the synthesised approach to instrumentation that could offer researchers and practitioners guidance and inspiration to develop and conduct quality assessments in competency development.

Automated Competence Assessment Procedures in Entrepreneurship

This study endeavors to automate the assessment of competencies within the domain of entrepreneurship, specifically targeting the augmentation of entrepreneurial cognition and conduct within universities in German rural regions, like Lower Franconia. Employing methods, including literature analyses and expert interviews, we formulated and validated an entrepreneurship competence profile and accompanying self-assessment tool. The ensuing evaluative framework is poised for seamless integration into learning management systems, thereby facilitating intelligent competence monitoring within educational environments. Purpose: The aim of this thesis is to develop an automated competence assessment procedure in the field of entrepreneurship. This can be used in the university environment in the long term to promote and teach entrepreneurial thinking and behavior in order to sustainably improve the quality of learning outcomes and achieve targeted promotion of entrepreneurship. Methodology: Based on a relevant literature analysis, four guideline-based expert interviews were created and conducted. The results of the interviews were compiled and validated in a structured competence profile (entrepreneurship competence profile). Based on this competence catalog for entrepreneurs, an empirically valid self-test was created using standard psychometric questionnaire construction methods. Results: The entrepreneurship competence profile and a consequential empirically validated self-test for competence assessment were created. This test provides the basis for the long-term competence development of students and can further be embedded automatically into a learning management system (LMS) as part of intelligent competence monitoring, which allows for the recording of competencies for each student and the individual incorporation of gap closure into the curriculum. Originality/value: In previous research, there were no competence profiles or competence assessment procedures in the field of entrepreneurship that derived relevant competencies directly from actors within this environment. This work illustrates the development of a competence assessment procedure for entrepreneurs and shows which methods can be used to close prevailing research gaps in the field of intelligent competence monitoring.

Advancing STEAM education: a comprehensive assessment of competence

Advancing STEAM education: a comprehensive assessment of competence

Entrustable Professional Activities in Oral and Maxillofacial Surgery Education: A Faculty Development Construct

The formative and summative evaluations of oral and maxillofacial surgery residents have commonly involved the six core competencies of the Accreditation Council for Graduate Medical Education. Unfortunately, the assessment of a resident's competencies in these six core areas is often subjective such that the written feedback might not be supportive of the resident's learning and continuous professional development. Compounding this problem is that faculty are infrequently trained in providing feedback in these core competencies, thereby adding to the inadequacy of this exercise. Entrustable professional activities (EPAs) represent a unit of professional practice, defined as tasks or responsibilities to be entrusted to the unsupervised execution by a trainee once he or she has attained sufficient specific competence. It is the purpose of this article to review the concept of EPAs that represent a hopeful solution to the theoretical and abstract nature of exclusive competency-based training assessments in resident education. EPAs are specifically proposed for oral and maxillofacial surgery resident education while discussing their serving as a faculty development construct.

Exploring Medical Student Experiences With Direct Observation During the Pediatric Clerkship.

Direct observation (DO) enables assessment of vital competencies, such as clinical skills. Despite national requirement that medical students experience DOs during each clerkship, the frequency, length, quality, and context of these DOs are not well established. This study examines the quality, quantity, and characteristics of DOs obtained during pediatrics clerkships across multiple institutions. This multimethod study was performed at 6 U.S.-based institutions from March to October 2022. In the qualitative phase, focus groups and/or semistructured interviews were conducted with third-year medical students at the conclusion of pediatrics clerkships. In the quantitative phase, the authors administered an internally developed instrument after focus group discussions or interviews. Qualitative data were analyzed using thematic analysis, and quantitative data were analyzed using anonymous survey responses. Seventy-three medical students participated in 20 focus groups, and 71 (97.3%) completed the survey. The authors identified 7 themes that were organized into key principles: before, during, and after DO. Most students reported their DOs were conducted primarily by residents (62 [87.3%]) rather than attendings (6 [8.4%]) in inpatient settings. Participants reported daily attending observation of clinical reasoning (38 [53.5%]), communication (39 [54.9%]), and presentation skills (58 [81.7%]). One-third reported they were never observed taking a history by an inpatient attending (23 [32.4%]), and one-quarter reported they were never observed performing a physical exam (18 [25.4%]). This study revealed that students are not being assessed for performing vital clinical skills in the inpatient setting by attendings as frequently as previously believed. When observers set expectations, create a safe learning environment, and follow up with actionable feedback, medical students perceive the experience as valuable; however, the DO experience is currently suboptimal. Therefore, a high-quality, competency-based clinical education for medical students is necessary to directly drive future patient care by way of a competent physician workforce.

The Impact of a Nanotechnology-Based Training Program on the Development of Digital Competencies among High School Biology Teachers

This study aimed to investigate the impact of a training program based on nanotechnology applications on the development of digital competencies among high school biology teachers. There is a gap between teachers' current digital competencies and the skills needed to effectively leverage nanotechnology in biology education. The study found statistically significant differences between pre-test and post-test scores on a digital competency assessment after teachers completed the nanotechnology training program. The program encompassed fundamental nanoscience knowledge, biology-specific applications, digital skills training, and hands-on activities. Results showed a large effect size, indicating the substantial impact of the intervention on enhancing teachers' digital competencies. Recommendations highlight integrating nanotechnology into science curricula and emphasize biology connections, problem-solving, content creation, and ethical considerations regarding advanced nanomaterials. Promoting teacher motivation and scientific inquiry is vital. The research addresses an urgent need to equip biology educators with digital skills aligned with rapidly evolving science and technology.