Resident Assessment Research Articles

Should Obtaining Informed Consent Be Considered an Entrustable Professional Activity? Insights From Whether and How Attendings Entrust Surgical Trainees.

Because residents are frequently delegated the task of obtaining consent early in their training, the American Association of Medical Colleges describes "obtaining informed consent" as a core entrustable professional activity (EPA) for medical school graduates. However, prior studies demonstrated that residents frequently perform this task without receiving formal instruction or assessment of competency. This study sought to understand how attending physicians decide to delegate obtaining informed consent for surgical procedures to trainees. The authors conducted a survey of attending surgeons at a university-based health care system of 6 affiliated teaching hospitals (October-December 2020) to collect data about current entrustment practices and attendings' knowledge, experience, and attitudes surrounding the informed consent process. Summary statistics and bivariate analyses were applied. Eighty-five attending surgeons participated (response rate, 49.4%) from diverse specialties, practice types, and years in practice. Fifty-eight of 85 (68.2%) stated they "never" granted responsibility for the consent conversation to a trainee, and 74/81 (91.4%) reported they typically repeated their own consent conversation whenever a trainee already obtained consent. The most common reasons they retained responsibility for consent were ethical duty (69/82, 84.1%) and the patient relationship (65/82, 79.3%), while less than half (40/82, 48.8%) described concerns about trainee competency. Reflecting on hypothetical clinical scenarios, increased resident competency did not correspond with increased entrustment ( P = .27-.62). Nearly all respondents (83/85, 97.7%) believed residents should receive formal training; however, only 41/85 (48.2%) felt additional training and assessment of residents might change their current entrustment practices. Attendings view informed consent as an ethical and professional obligation that typically cannot be entrusted to trainees. This practice is discordant with previous literature studying residents' perspectives. Furthermore, resident competency does not play a predominant role in this decision, calling into question whether informed consent can be considered an EPA.

The Impact of Simulator Training of Vascular Anastomosis and Video Assessment for Surgical Residents

IntroductionThe rapid increase of minimally invasive surgery and the shortened training period for surgical residents has resulted in limited opportunities to acquire proficiency in open surgical techniques, such as vascular anastomosis. However, vascular anastomosis remains an essential skill in every surgery for bleeding control. This study aimed to validate the effectiveness of surgical education model for vascular anastomosis and assess the impact on the comprehension, skill, and confidence of surgical residents in performing vascular anastomosis. MethodsA total of 21 surgical residents with 1st to 3rd years of experience at Seoul National University Hospital participated in a four-week vascular anastomosis training program. The program included an educational lecture and the performance of an end-to-side anastomosis on a procedural model, with evaluations being conducted using the Objective Structured Assessment of Technical Skills (OSATS) and the End Product Rating Score (EPRS) in pre- and post-training surveys. ResultsSignificant improvement was observed in the OSATS score (from 9.22 ± 2.4 in week 1 to 12.87 ± 3.1 in week 4; P < 0.001) and the EPRS score (from 12.47 ± 4.1 in week 1 to 17.57 ± 2.2 in week 4; P < 0.001). Additionally, the surgical performance time significantly decreased from 20.99 ± 4.6 minutes to 16.33 ± 4.2 minutes (P = 0.019) ConclusionsSimulator training of in vitro vascular anastomosis, when accompanied by expert-led instruction, can effectively enhance the surgical proficiency, confidence, and overall surgical outcomes of residents, as inferred from the observed improvements in OSATS and EPRS scores. The results suggest that integration of this training model into surgical curricula could be a promising strategy for enhancing vascular surgical training.

Nurse perspectives on the assessment of emergency medicine residents: A qualitative study.

Multisource and 360-degree feedback are emerging methods in the medical education assessment literature. Nursing feedback in particular has been identified as a point of emphasis regarding what medical education leadership would most value for incorporating into their evaluations. There is no currently accepted tool for multisource feedback in the emergency department (ED). To answer this call, we conducted a qualitative study to elucidate nurse perspectives on evaluation of emergency medicine residents. We conducted individual semistructured interviews of ED nurses with experience working alongside residents via volunteer recruitment. Transcripts were coded independently by each author and then reviewed collaboratively to resolve differences. Codes were created by inductive thematic analysis and subsequently underwent constant comparison and classical content analysis. Recruitment continued until analysis of transcripts showed thematic saturation. We organized the interview transcript into chunks of information, arranging 407 quotes from the 10 interview transcripts into 17 codes and six themes regarding observable resident behaviors. Representative quotes and exemplar cases added detailed description to the meaning of these codes. The identified themes for observable resident behaviors included "general communication style," "medical knowledge," "efficiency," "patient communication," "nurse communication," and "professionalism." An additional 13 codes addressed the themes of barriers and catalysts for feedback. The identified themes on observable resident behaviors aligned well with prior literature and with the Accreditation Council for Graduate Medical Education (ACGME) Milestones. The interviews also highlighted nurse perspectives on barriers and catalysts for feedback that would be useful to incorporate into the design of an assessment tool.

Beyond competence and retention: Developing a comprehensive evaluation framework for a rural primary care nurse practitioner residency program.

With the rise in nurse practitioner (NP) residency programs, evaluations have largely focused on retention and competency completion for residents. There is a need for expanded evaluation to ensure the sustainability of NP residency programs, to ensure timely adaptations to address resident satisfaction, and to solidify a long-term pathway of NPs well prepared for rural practice. We created a family nurse practitioner (FNP) residency program with a comprehensive evaluation framework to prepare residents for practice in rural settings. The evaluation framework was developed through collaborative engagement of an external evaluation team, program leadership, and clinical site representatives. The evaluation framework of the FNP residency program combined resident assessment and holistic program evaluation, using a rapid continuous quality improvement (QI) approach. The evaluation considered three distinct perspectives: the resident, the peer coach, and the clinical site. The rapid continuous QI approach allowed program leadership to respond swiftly to programmatic challenges, improve the residency program in response to residents' reported experiences, and emphasize sustainability for continued program impact, while assessing residents' learning and performance. The program's data-driven evaluation approach has demonstrated its success in meeting the goals of the Health Resources and Services Administration funding by increasing the number of primary care providers in rural settings. The program's expansion and continued success have further validated the efficacy of this evaluation framework in assessing, improving, and ensuring the sustainability of APRN residency programs. This article calls for the adoption of similar evaluation strategies in future residency programs to promote their long-term success and impact in rural health care settings.

Dynamics of interethnic relations in the assessments of residents of a multiethnic city (the case of Yakutsk)

Dynamics of interethnic relations in the assessments of residents of a multiethnic city (the case of Yakutsk)

Utilizing the 4Ms framework to create a structure and process to support voluntary health assessments in affordable housing.

A growing number of older adults live in senior affordable housing, many with limited support systems and representing underserved or disadvantaged populations. Staff in these buildings are in a unique position to identify and address the healthcare and biopsychosocial needs of their residents and link them to services and supports. Staff in four affordable housing sites received training on the 4Ms approach to caring for older adults and conducting resident health assessments. They learned to collect comprehensive health information using a 4Ms Resident Health Risk Assessment (4Ms-RHRA) and results are entered into a customized electronic database. Embedded flags identify potential risk factors and initiate a follow-up process for documenting interventions and tracking referrals to healthcare and supportive services. Eighty-one percent of the 221 4Ms-RHRAs completed with residents (63% female, mean age 71.1 years, 73% live alone) were flagged for at least one concern (Mean = 2.2 flags). Items addressing What Matters were most frequently flagged: resident's "most important health issue" (55%) and Advance Care Planning (ACP: 48%). In response, staff provided Advance Directive forms and Five Wishes pamphlets to interested residents and reminded residents to review ACP documents annually. Training affordable housing staff, precepting faculty, and students to conduct health assessments based on the 4Ms framework and longitudinally track interventions related to resident-centered needs and manage long-term service and supports is a first step in creating an interprofessional workforce capable of addressing the complex needs of older individuals in affordable housing.

Gender Disparities in the Academic Performance of Neurology Residents in Portugal.

Implicit bias has been linked to gender disparities in medical careers, impacting not only access to leadership positions but also early career opportunities. We aimed to evaluate if there were differences in the assessment of Neurology residents according to gender. We collected publicly available grades and rankings of two major evaluations that residents are submitted to, one at the beginning and another at the end of the neurology residency program, the National Board Exam and neurology examination, respectively. The National Board Examination is a multiple-choice gender-blinded evaluation, while the neurology examination is an oral gender-unblinded evaluation. We found that 36.5% of women and 21.6% of men were in the first quartile of the National Board Examination ranking, which reflects a similar representation among top classifications when assessed through a gender-blinded examination. On the other hand, the percentage of men who were in the top classification of NE, a gender-unblinded evaluation, was more than twice as high compared to women (37.8% vs 18.3%, p < 0.05). The findings of the present study may imply that there could be a disparity in women's career progression among neurology residents in Portugal, although the recruitment seems balanced between genders.

Development of a Competency Assessment Framework for Pharmacy Residents Participating in High-Fidelity Simulation and Creation of the SIMPHARM Assessment Tool.

Simulations are used to improve professional practice across multiple health professions; however, the application of simulations in pharmacy curricula has been limited by a lack of evidence. To delineate the competencies of pharmacy residents needed to assess their progress while participating in a high-fidelity simulation training program and to create a reliable assessment tool based on these competencies. A literature review was conducted, followed by application of a modified Delphi method. An assessment tool based on the results of these activities was drafted (in French). A second modified Delphi process was carried out to assess the reliability of the tool, and the tool was modified accordingly. The literature search yielded a total of 2670 articles, of which 289 were included for analysis. The first modified Delphi process involved 19 experts in the first round and 10 experts in the second round. The Cronbach α was 0.866 (95% confidence interval [CI] 0.713-0.960), indicating good reliability. A total of 7 competencies were retained for inclusion in the SIMPHARM assessment tool: professionalism, leadership, teamwork, communication, critical thinking, preparation and packaging of medications, and pharmaceutical calculations. The second modified Delphi process involved 9 experts in the first round and 11 experts in the second round. The final Cronbach α was 0.877 (95% CI 0.741-0.960). To the authors' knowledge, this was one of the largest studies exploring pharmacy competencies in the context of simulations. This work yielded a reliable framework for the assessment of pharmacy residents' competencies. This assessment tool may help evaluators in assessing the competencies of pharmacy trainees after simulation training.

Effects of changing criteria on improving interRAI assessment for elder abuse: analysis of a national dataset from Aotearoa New Zealand

ObjectivesGlobally, one in six older adults in the community will be a victim of abuse (elder abuse; EA). Despite these horrific statistics, EA remains largely undetected and under-reported. Available screening...

Modern Trends, Examples and Methods for Assessing Physical Activity of City Residents

Introduction. A sufficient level of physical activity (PA) is crucial for maintaining population health. There is a wide range of methods and approaches to PA assessment, such as questionnaires that are a convenient tool for assessment of PA levels. Another modern way to assess PA is individual wearable devices, including accelerometers. Data obtained from accelerometers and other tools can contribute to questionnaire results used in PA studies. Purpose. This review was carried out with the aim of considering modern trends, examples and common methods of physical activity assessment of city residents. Materials and methods. Relevant information was collected using PubMed and E-Library databases. Google and Yandex search systems were used to obtain additional data. Results. This paper describes global trends in modern PA studies, presents the examples of Russian and foreign papers on PA of city population, as well as summarizes the main features of PA assessment tools. Notably, PA can be assessed using subjective and objective methods. International and national questionnaires are widely applied as a subjective assessment tool. At the same time, technologies for objective assessment based on digital systems, wearable devices and other approaches to PA monitoring are being developed. Methods for assessing PA can be selected in several stages using decision-making algorithms. Conclusions. Upon the obtained results we assume that modern study design and methods of assessment of FA level among city residents helps to identify factors influencing the formation and maintenance of healthy lifestyle in urban environment.

Exploring the Feasibility and Utility of Entrustable Professional Activities in Resident Assessment and Feedback: A Pilot Survey Study

Exploring the Feasibility and Utility of Entrustable Professional Activities in Resident Assessment and Feedback: A Pilot Survey Study

Radiation Oncology Education Collaborative Study Group Physics Curriculum Needs Assessment for Radiation Oncology Residents

Radiation Oncology Education Collaborative Study Group Physics Curriculum Needs Assessment for Radiation Oncology Residents

Standardizing Assessment of Gynecologic Brachytherapy Contours: A Pilot Study in Contour Competency Assessment for Radiation Oncology Residents

Standardizing Assessment of Gynecologic Brachytherapy Contours: A Pilot Study in Contour Competency Assessment for Radiation Oncology Residents

Validity and Reliability of the Residency and Immersion Program (PRIme) Assessment Instrument in Enhancing the Quality of New Principals

The Residency and Immersion Program (PRIme) is an induction for new school principals serving their first year. PRIme was implemented in 2014, and it requires a comprehensive evaluation for program improvement. Hence, this study aims to assess the validity and reliability of the PRIme assessment instrument. The assessment data of content validity by professional experts and face validity by new principals were analyzed using CVR, CVI, and FVI methods. Meanwhile, pilot study data was obtained from 53 respondents and further analyzed using the Rasch measurement model. The questionnaire includes 102 items for assessing the domains of input (courses, coaching, and mentoring), process (type of leadership practiced), and product (leadership and organizational management) in the implementation of PRIme. The Rasch analysis results showed that the items in the questionnaire have good content validity, construct validity, and reliability. Therefore, the Rasch analysis has proven that this questionnaire can be used as an instrument to assess PRIme in an effort to enhance the quality of new principals in Malaysia.

First Evidence on the Validity of the Complexity Index Derived from the Resident Assessment Instrument for Home Care in Home Care Patients

ObjectivesRecently, a Complexity Index (CI), based on the multidimensional complexity model and derived from the Resident Assessment Instrument for Home Care (interRAI HC) was proposed as a decision-support tool to help frontline health care professionals in their clinical evaluation to identify and analyze complex situations. This study aims to test the CI: (1) concurrent validity with another measure of complexity (ie, the COMID), (2) convergent validity with related constructs assessed by interRAI HC scales (eg, depression), (3) divergent validity (comparison between CI-COMID and scales-COMID correlations), and (4) predictive validity on coordination meetings. DesignA cross-sectional observational design was used for a secondary analysis of interRAI HC and COMID data collected in routine home care nursing practice (July-December 2021). Setting and ParticipantsParticipants were community-dwelling adults receiving home care, with full interRAI HC and COMID assessments (N = 3533). MethodsCorrelational analyses were conducted to test the concurrent validity of the CI (with the COMID) and the convergent and divergent validity of the CI (with interRAI HCSwitzerland scales, eg, Depression Rating Scale, Method for Assigning Priority Levels, and a Frailty Index). A receiver operating characteristic (ROC) analysis was conducted to test the discriminative ability of CI on specific professional team coordination meetings. ResultsResults showed that the CI correlated positively and strongly with the COMID (ρ = 0.691, P < .001, concurrent validity), positively with all the tested scales (P < .001, convergent validity), whereas the CI-COMID correlation was higher than the interRAI HC scales–COMID correlations (divergent validity). The ROC analysis showed the CI had a high area under the curve (AUC = 0.719, predictive validity). Conclusions and ImplicationsThe CI demonstrates good validity properties with a strong correlation with the COMID and a high predictive value for coordination meeting. It is distinct from the other interRAI HC scales and has its place among them to support the clinical analysis of complex situations.

Trained to cut? A literature review of veterinary surgical resident training.

A surgical residency trains veterinary graduates to a higher level of expertise in surgical procedures than is possible during veterinary school and prepares a resident to pursue board certification in surgery. The education of veterinary surgical residents has changed minimally since its inception in the twentieth century, and there are insufficient studies to determine if residency programs are producing surgeons with competence in each of the necessary procedural categories. The aims of this review were to report the current theory and methods used to provide surgical education to residents, to discuss the training most likely to create a competent, board-certified surgeon and to review assessment methods used during training. Several literature searches using broad terms such as "veterinary surgery residency," "veterinary surgery resident," and "veterinary surgical training" were performed using PubMed, CAB abstracts, and Google Scholar. Literature pertinent to theory, methods, training, and assessment of veterinary surgical residents was included. The reviewed literature demonstrated the need for research-based learning curves for specific procedures. Simulation training is known to facilitate deliberate practice and should be leveraged where possible to reach competency. The creation of validated assessment methods should be pursued as it enables assessment of competency instead of inferring its development from case logs. Understanding and supporting learner cognition and providing sufficient feedback remain important issues in the field. Surgical educators are urged to continue to search for innovative and evidence-based ways to train competent surgical residents.

Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education

ABSTRACT Introduction Since 2022, all Canadian post-graduate medical programs have transitioned to a Competence by Design (CBD) model within a Competency-Based Medical Education (CBME) framework. The CBME model emphasized more frequent, formative assessment of residents to evaluate their progress towards predefined competencies in comparison to traditional medical education models. Faculty members therefore have increased responsibility for providing assessments to residents on a more regular basis, which has associated challenges. Our study explores faculty assessment behaviours within the CBD framework and assesses their openness to opportunities aimed at improving the quality of written feedback. Specifically, we explore faculty’s receptiveness to routine metric performance reports that offer comprehensive feedback on their assessment patterns. Methods Online surveys were distributed to all 28 radiology faculty at Queen’s University. Data were collected on demographics, feedback practices, motivations for improving the teacher-learner feedback exchange, and openness to metric performance reports and quality improvement measures. Following descriptive statistics, unpaired t-tests and one-way analysis of variance were conducted to compare groups based on experience and subspecialty. Results The response rate was 89% (25/28 faculty). 56% of faculty were likely to complete evaluations after working with a resident. Regarding the degree to which faculty felt written feedback is important, 62% found it at least moderately important. A majority (67%) believed that performance reports could influence their evaluation approach, with volume of written feedback being the most likely to change. Faculty expressed interest in feedback-focused development opportunities (67%), favouring Grand Rounds and workshops. Conclusion Assessment of preceptor perceptions reveals that faculty recognize the importance of offering high-quality written feedback to learners. Faculty openness to quality improvement interventions for curricular reform relies on having sufficient time, knowledge, and skills for effective assessments. This suggests that integrating routine performance metrics into faculty assessments could serve as a catalyst for enhancing future feedback quality.

Excess Deaths in Assisted Living and Nursing Homes during the COVID-19 Pandemic in Alberta, Canada

ObjectivesAssisted living (AL) is a significant and growing congregate care option for vulnerable older adults designed to reduce the use of nursing homes (NHs). However, work on excess mortality in congregate care during the COVID-19 pandemic has primarily focused on NHs with only a few US studies examining AL. The objective of this study was to assess excess mortality among AL and NH residents with and without dementia or significant cognitive impairment in Alberta, Canada, during the first 2 years of the COVID-19 pandemic, relative to the 3 years before. DesignPopulation-based, retrospective cohort study. Setting and ParticipantsResidents who lived in an AL or NH facility operated or contracted by the Provincial health care system to provide publicly funded care in Alberta between January 1, 2017, and December 31, 2021. MethodsWe used administrative health care data, including Resident Assessment Instrument – Home Care (RAI-HC, AL) and Minimum Data Set 2.0 (RAI-MDS 2.0, NHs) records, linked with data on residents’ vital statistics, COVID-19 testing, emergency room registrations, and hospital stays. The outcome was excess deaths during COVID-19 (ie, the number of deaths beyond that expected based on pre-pandemic data), estimated, using overdispersed Poisson generalized linear models. ResultsOverall, the risk of excess mortality [adjusted incidence rate ratio (95% confidence interval)] was higher in ALs than in NHs [1.20 (1.14–1.26) vs 1.10 (1.07–1.13)]. Weekly peaks in excess deaths coincided with COVID-19 pandemic waves and were higher among those with diagnosed dementia or significant cognitive impairment in both, AL and NHs. Conclusions and ImplicationsFinding excess mortality within both AL and NH facilities should lead to greater focus on infection prevention and control measures across all forms of congregate housing for vulnerable older adults. The specific needs of residents with dementia in particular will have to be addressed.

Personalized Tablets for Residents in Long-Term Care to Support Recreation and Mitigate Isolation

ObjectivesThere is a digital divide in long-term care homes (LTCHs), with few residents having regular access to internet-connected devices. In this study, we provided long-term care residents with personalized and adapted tablets. We aimed to understand what factors influenced tablet use and the impact of tablet access on opportunities for social connection and recreation. DesignA pragmatic, mixed-methods multicenter, open-label, uncontrolled interventional study with assessment of outcomes at baseline and 3 months. Setting and ParticipantsA total of 58 resident-care partner dyads were recruited across 7 LTCHs in Ontario, Canada. The main inclusion criterion was having a care partner willing to participate, and we excluded residents who already had an internet-connected device. MethodsResident demographics, functional status assessments, and recreational engagement were captured using items from the Resident Assessment Instrument/Minimum Data Set. Care partners completed a questionnaire about relational closeness and site leads assessed resident quality of life before and approximately 3 months after tablet distribution. Interviews with 23 care partners and 7 residents post-implementation were completed and analyzed. ResultsThe median tablet use by participants was 7 minutes (interquartile range 27) per day on average over the study period. Predictors of higher tablet use were younger age, higher cognitive functioning, absence of hearing impairment, and having a care partner who lives farther away. There was no improvement on quantitative measures of quality of life, recreation, or relational closeness. In interviews, participants identified many different opportunities afforded by access to personalized tablets. Conclusions and ImplicationsSome LTCH residents without current access to the internet benefit from being provided a personal tablet and use it in a variety of ways to enrich their lives. There is a critical need to bridge the digital divide for this population.

Drug Burden Index Is a Modifiable Predictor of 30-Day Hospitalization in Community-Dwelling Older Adults With Complex Care Needs: Machine Learning Analysis of InterRAI Data.

Older adults (≥65 years) account for a disproportionately high proportion of hospitalization and in-hospital mortality, some of which may be avoidable. Although machine learning (ML) models have already been built and validated for predicting hospitalization and mortality, there remains a significant need to optimize ML models further. Accurately predicting hospitalization may tremendously affect the clinical care of older adults as preventative measures can be implemented to improve clinical outcomes for the patient. In this retrospective cohort study, a data set of 14198 community-dwelling older adults (≥65 years) with complex care needs from the International Resident Assessment Instrument-Home Care database was used to develop and optimize 3 ML models to predict 30-day hospitalization. The models developed and optimized were Random Forest (RF), XGBoost (XGB), and Logistic Regression (LR). Variable importance plots were generated for all 3 models to identify key predictors of 30-day hospitalization. The area under the receiver-operating characteristics curve for the RF, XGB, and LR models were 0.97, 0.90, and 0.72, respectively. Variable importance plots identified the Drug Burden Index and alcohol consumption as important, immediately potentially modifiable variables in predicting 30-day hospitalization. Identifying immediately potentially modifiable risk factors such as the Drug Burden Index and alcohol consumption is of high clinical relevance. If clinicians can influence these variables, they could proactively lower the risk of 30-day hospitalization. ML holds promise to improve the clinical care of older adults. It is crucial that these models undergo extensive validation through large-scale clinical studies before being utilized in the clinical setting.