Narrative Assessment Research Articles

Curriculum Changes and Trends 2010-2020: A Focused National Review Using the AAMC Curriculum Inventory and the LCME Annual Medical School Questionnaire Part II.

Medical school curricula have evolved from 2010 to 2020. Numerous pressures and influences affect medical school curricula, including those from external sources, academic medical institutions, clinical teaching faculty, and undergraduate medical students. Using data from the AAMC Curriculum Inventory and the LCME Annual Medical School Questionnaire Part II, the nature of curriculum change is illuminated. Most medical schools are undertaking curriculum change, both in small cycles of continuous quality improvement and through significant change to curricular structure and content. Four topic areas are explored: cost consciousness, guns and firearms, nutrition, and opioids and addiction medicine. The authors examine how these topic areas are taught and assessed, where in the curriculum they are located, and how much time is dedicated to them in relation to the curriculum as a whole. When examining instructional methods overall, notable findings include (1) the decrease of lecture, although lecture remains the most used instructional method, (2) the increase of collaborative instructional methods, (3) the decrease of laboratory, and (4) the prevalence of clinical instructional methods in academic levels 3 and 4. Regarding assessment methods overall, notable findings include (1) the recent change of the USMLE Step 1 examination to a pass/fail reporting system, (2) a modest increase in narrative assessment, (3) the decline of practical labs, and (4) the predominance of institutionally developed written/computer-based examinations and participation. Among instructional and assessment methods, the most used methods tend to cluster by academic level. It is critical that faculty development evolves alongside curricula. Continued diversity in the use of instructional and assessment methods is necessary to adequately prepare tomorrow's physicians. Future research into the life cycle of a curriculum, as well optional curriculum content, is warranted.

Keck School of Medicine of the University of Southern California.

Keck School of Medicine of the University of Southern California.

Western Michigan University Homer Stryker M.D. School of Medicine.

Western Michigan University Homer Stryker M.D. School of Medicine.

University of Arizona College of Medicine-Tucson.

University of Arizona College of Medicine-Tucson.

Incorporating oral language assessment into MTSS/RTI frameworks: The potential of personal narrative assessment

ABSTRACT In Multi-tiered System of Support (MTSS) and Response to Intervention (RTI) models, the reading achievement of English Learners (ELs) is assessed multiple times during the school year to monitor students’ progress toward meeting grade-level reading standards. Most MTSS/RTI frameworks do not include a comparable system for assessing oral language development. The English Learner Oral Narrative Scale (ELONS) was developed as an informal, criterion-referenced instrument to assess the personal narrative skills of ELs (i.e., skills in recounting personal experiences). The ELONS was piloted with 28 third grade ELs in a transitional bilingual education program to describe participants’ personal narrative skills and to explore relationships between narrative skills and reading achievement. Mean scores were at the Advanced level for English global narrative proficiency and vocabulary, grammar, and fluency; mean ratings for oral comprehension and pronunciation were at the Advanced High level. Twelve of 28 ELs produced Spanish narratives of sufficient length to score. Mean global and subskill scores were at the Advanced level except for fluency (Intermediate level). ELONS global proficiency scores showed moderate correlations to state assessments of English language proficiency and English reading. Implications for oral language assessment in MTSS/RTI frameworks and future research directions are discussed.

The Validity of Connecting Conversations: A Narrative Method to Assess Experienced Quality of Care in Nursing Homes from the Resident's Perspective.

It is important to assess experienced quality of care in nursing homes, as this portrays what is important to residents and helps identify what quality improvements should focus on. Connecting Conversations is a narrative method that assesses experienced quality of care from the resident’s perspective in nursing homes by having separate conversations with residents, family, and professional caregivers (triads) within a learning network. This study assessed the validity of performing the narrative method, Connecting Conversations. Trained nursing home staff (interviewers) performed the conversations in another nursing home than where they were employed. In total, 149 conversations were performed in 10 nursing homes. Findings show that experts deemed the narrative assessment method appropriate and complete to assess experienced quality of care (face validity). The questions asked appeared to capture the full construct of experienced quality of care (content validity). Additionally, there was a range in how positive conversations were and first results indicated that a nursing home scoring higher on satisfaction had more positive conversations (construct validity). More data are needed to perform additional construct validity analyses. In conclusion, Connecting Conversations shows promising results for its use as a valid narrative method to assess experienced quality of care.

Clinical considerations in the psychological evaluation of gestational surrogates: uses of narrative assessment

Potential risks and ethical considerations inherent in surrogacy arrangements place tremendous responsibility on mental health professionals performing the necessary assessments, yet there is little discussion and no consensus on the best testing protocol. In the U.S., the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) and the Personality Assessment Inventory (PAI) are the most common psychological measures used to screen gestational surrogates (GSs), although these self-report symptom inventories reveal tendencies to underreport, deny problems and present positive impressions that may conceal important clinical information. This study examines the utility of a multi-method approach in the psychological evaluation of GS candidates. A total of 43 women cleared to be GSs completed the PAI and an abbreviated protocol of a narrative construction assessment measure (The Thematic Apperception Test (TAT)) as part of the psychological screening process. Findings of the study show that utilising a multi-method approach in the psychological consideration of a GS can address the limitations inherent in a single-measure assessment approach. Including a narrative measure such as the TAT that is less affected by defensiveness found on self-report scales may allow for more subtleties in interpretation, helping assessors to identify women who may have particular psychological vulnerabilities. These might then be addressed through implications counselling.

Assessing an Eliciting Narrative Tool Used for Studying the Development of Persian-speaking Children’s Narrative Discourse Skills

Introduction: Developing children’s skills in producing oral narratives can reflect their linguistic and cognitive abilities. However, to evaluate these abilities appropriately, it is necessary to find and apply an efficient narrative assessment tool. This study primarily aimed to assess the reliability and validity of a picture story, as a narrative eliciting tool, based on Persian-speaking children’s narratives. This assessment is going to be done at the microstructure and macrostructure levels. Furthermore, to evaluate the power of the assessment tool, we explored the effect of age and gender variables on using different narrative elements at the microstructure and macrostructure levels. Materials and Methods: We used a picture story, “Frog, where are you?” to elicit oral narratives in 48 subjects, including 7-, 9-, and 11-year-old boys and girls. The reliability and validity of the tool were respectively assessed by test-retest and factor analysis. Results: The findings indicated a significantly high correlation between the evaluated features based on test-retest. Besides, factor analysis revealed four categories: sentence structures, references, conjunctions, measures of story length. They were valid indicators for assessing Persian-speaking children’s narratives. The results also showed a statistically significant difference among different age groups, but an insignificant effect of gender on using discursive features in the tales.Conclusion: The picture story “Frog, where are you?” can be used as a reliable and valid narrative eliciting tool for Persian data at the microstructure and macrostructure levels. Also, the age factor, but not the gender one, affects the stories recited by Persian children.

Attachment representations in children with disruptive behavior disorders: A special focus on insecurity in middle childhood.

According to the emotional and social problems encountered in disruptive behaviour disorder (DBD), there is little available information regarding the attachment processes during middle childhood. Therefore, this study aims to investigate the attachment representations to both caregivers in a sample of 84 children, aged 8-12 years (N = 42 with a diagnosis of DBD; N = 42 as a comparison group). Attachment is assessed through the Child Attachment Interview, a semi-structured interview that incorporates both narrative and behavioural assessments of attachment, permits the assessment of attachment representations using dimensional and categorical approaches, and allows for the detection of disorganisation to both parents individually. Our results point to the presence of high frequencies of insecure attachment (more than 80%) in the DBD group, with a higher percentage of dismissing attachment, and an over-representation of disorganised attachment (more than 50%). Higher levels of dismissal and lower levels of security are found in DBD than in the comparison group. No attachment differences with respect to both parents are found. Understanding how different aspects of attachment during a specific developmental phase plays a role within the broader emotional functioning of DBD has great potential to lead to innovations in evaluations and treatment processes.

“Today at Playcentre, we ….”: What values underpin narrative assessment in Playcentre?

Narrative assessments are widely used in Aotearoa New Zealand’s early childhood services, especially in the form of ‘Learning Stories’. This sociocultural approach to assessment foregrounds the importance of identifying what is valued learning within a particular context. What does this look like within the context of the parent co-operative Playcentre? A small-scale qualitative study brought together focus groups of Playcentre parents to share what Learning Stories are being written, to explore their experiences in writing and using Learning Stories, and to consider what underlying values shape narrative assessment in Playcentres. A major finding is that adult learning and relationship building are key aspects of Learning Stories in Playcentres. The authors argue that because of the positioning of parents as both learners and Kaiako in Playcentre, assessment in Playcentre is fundamentally different to that in teacher-led services.

Gender and Pain in Nineteenth-Century Cancer Care.

Gender and Pain in Nineteenth-Century Cancer Care.

Examination of Narrative Language at Microstructural and Macrostructural Levels in Spanish-Speaking Preschoolers.

Purpose The purpose of this study was to examine narrative language samples of Spanish-speaking preschoolers to analyze changes in microstructural and macrostructural skills in their first language (L1) from fall to spring, relationships between narrative and vocabulary skills in L1, and the extent to which fall skills predict spring performance. Method Participants included 40 Spanish-speaking children who were enrolled in community-based preschool programs. Narrative language retells from the fall and spring were examined, and microstructural and macrostructural components were analyzed using the Narrative Assessment Protocol-Spanish and the Narrative Scoring Scheme, respectively. Participants also completed an assessment of expressive vocabulary in Spanish. Results The results indicated different degrees of change in microstructural and macrostructural elements, change from fall to spring in some but not all elements measured, shifting patterns of association between L1 vocabulary and narrative skills, and variable prediction of spring scores. Conclusion Results from this study enhance professionals' understanding of Spanish-speaking preschoolers' narrative language skills in L1 and considerations for assessing and monitoring progress at different points in the academic year.

Democracy in Indonesian Strategic Narratives. A New Framework of Coherence Analysis

Indonesia’s rise, its democracy promotion, and engagement in the democratisation of regional institutions have often been analysed in scholarly literature in the recent years. Indonesia’s “democratic turn,” both internal and external, increased its relevance in international relations. The academic discussion focuses on the role and meaning of Indonesia’s contribution. The aim of this article is to broaden the debate by analysing the meaning of “democracy” in Indonesian narrative investigated from the perspective of the strategic narratives concept and to propose a new framework for the assessment of narrative based on its coherence. Therefore, the research question is – Are the strategic narratives of democracy articulated by Indonesia coherent? The analysis shows, and it is an argument, that with some exceptions strategic narratives are incoherent and this incoherence has two dimensions: intra- and inter-incoherence. The argument is substantiated by an analysis of Indonesian foreign policy strategies and political speeches.

The effectiveness of the adapted Box Clever language intervention programme in the development of vocabulary and narrative skills of deaf and hard of hearing children

ABSTRACT Children who are deaf or hard of hearing (DHH) in Sri Lanka have inadequate access to early amplification and language stimulation. As a result, they are at risk of impoverished language development: spoken language and sign language. Thirty DHH children who use Sri Lankan sign language between the ages of 4;4-7;2 years from three classrooms of a School for the Deaf were included in this study. The culturally-modified Box Clever language-enrichment programme was offered as a whole-class approach twice a week during a 12-week school term through the class teacher. Pre- and post-intervention language measures and statistical analyses of language scores were undertaken of receptive and expressive vocabulary on picture-naming tasks of target vocabulary. In addition, content and syntactic analyses of narrative skills were undertaken using the Elephant Tales Narrative Assessment, an informal assessment created by the researchers, as no local formal standardised assessments exist currently. There were promising results with statistically significant positive gains in receptive and expressive vocabulary skills on target vocabulary items post-intervention by all the participants. Positive qualitative differences were observed in the content within the target narrative assessment post-intervention by all participants with evidence of the emergence of two to three sign combinations by many of the children.

Development of the Narrative Assessment Protocol-2: A Tool for Examining Young Children's Narrative Skill.

Purpose Narrative skill, a child's ability to create a temporally sequenced account of an experience or event, is considered an important domain of children's language development. Narrative skill is strongly predictive of later language and literacy and is emphasized in curricula and educational standards. However, the need to transcribe a child's narrative and the lack of psychometrically justified scoring methods have precluded broad consideration of narrative skill among practitioners. We describe the development and validation of the Narrative Assessment Protocol-2 (NAP-2), an assessment of narrative skill for children ages 3-6 years, which uses event-based frequency scoring directly from a video recording of a child's narrative. Method The NAP-2 underwent a rigorous development process involving creation of four wordless picture books and associated scripts and identification of a broad item pool, including aspects of narrative microstructure and macrostructure. We collected two narratives from each of 470 children using the NAP-2 elicitation materials and scored each with the 60 items in the initial item pool. Results Cross-validated exploratory factor analyses indicated a single narrative skill factor. Rasch measurement analysis led to selection of 20 items that maintained high reliability while having good fit to the model and no evidence of differential item functioning across books and gender. Conclusions The NAP-2 offers a psychometrically sound and easy-to-use assessment of narrative skill for children ages 3-6 years. The NAP-2 is available freely online for use by speech-language pathologists, educational practitioners, and researchers. Supplemental Material https://doi.org/10.23641/asha.11800779.

Does a narrative retelling task improve the assessment of language proficiency in school-aged children born very preterm?

ABSTRACT Almost half of the children born very preterm (VP) experience language difficulties at school-age, specifically with more complex language tasks. Narrative retelling is such a task. Therefore, we explored the value of narrative retelling assessment in school-aged children born VP, compared to item-based language assessment. In 63 children born VP and 30 age-matched full-term (FT) controls Renfrew’s Bus Story Test and Clinical Evaluation of Language Fundamentals were assessed. The retelling of the Bus Story was transcribed and language complexity and content measures were analyzed with Computerised Language Analysis software. Narrative outcomes of the VP group were worse than that of the FT group. Group differences were significant for the language complexity measures, but not for the language content measures. However, the mean narrative composite score of the VP group was significantly better than their mean item-based language score, while in the FT group the narrative score was worse than the item-based score. Significant positive correlations between narrative and item-based language scores were found only in the VP group. In conclusion, in VP children narrative retelling appears to be less sensitive to detecting academic language problems than item-based language assessment. This might be related to the mediating role of attention in item-based tasks, that appears not to affect more spontaneous language tasks such as retelling. Therefore, in school-aged children born VP we recommend using narrative assessment, in addition to item-based assessments, because it is more related to spontaneous language and less sensitive to attention problems.

Coordinating Assessment of Spiritual Needs: A Cross-Walk of Narrative and Psychometric Assessment Tools Used in Palliative Care (S784)

Coordinating Assessment of Spiritual Needs: A Cross-Walk of Narrative and Psychometric Assessment Tools Used in Palliative Care (S784)

Assessment and evaluation of narratives in Guided Imagery and Music (GIM) (Perilli)

This is a review of the book "Assessment and evaluation of narratives in Guided Imagery and Music (GIM)" authored by Gabriella Giordanella Perilli.
 Title: Assessment and evaluation of narratives in Guided Imagery and Music (GIM) Author: Gabriella Giordanella Perilli Publication year: 2017 Publisher: Barcelona Publishers Pages: 94 ISBN: 9781945411144

Race- and Gender-Based Differences in Descriptions of Applicants in the Letters of Recommendation for Orthopaedic Surgery Residency

Background:Letters of recommendation (LOR) are an important component of trainee advancement and assessment. Examination of word use in LOR has demonstrated significant differences in how letter writers describe female and male applicants. Given the emphasis on increasing both gender and racial diversity among orthopaedic surgeons, we aimed to study gender and racial differences in LOR for applicants to orthopaedic surgery residencies.Methods:All applications submitted to a single, academic orthopaedic residency program in 2018 were included. Self-identified gender and race were recorded. The LOR were analyzed via a text analysis software program using previously described categories of communal, agentic, grindstone, ability, and standout words. We examined the relative frequency of word use in letters for (1) male and female applicants and (2) white and underrepresented in orthopaedics (UiO) applicants, with the subgroup analysis based on whether standardized (using the American Orthopaedic Association template) or traditional (narrative) LOR were used.Results:Two thousand six hundred twenty-five LOR were submitted for 730 applicants (79% men). Fifty-nine percent of applicants were self-identified as white, and 34% were self-identified as UiO. In traditional LOR, standout words (odds ratio [OR] 1.07; p = 0.01) were more likely to be used in letters for women compared with men, with no difference in any other word-use category. In standardized LOR, there were no gender-based differences in any word category. In traditional LOR, grindstone words (OR = 0.96; p = 0.02) were more likely to be used in letters for UiO than white applicants, whereas standout words (OR = 1.05; p = 0.04) were more likely to be used in letters for white candidates. In standardized LOR, there were no race-based differences in any word category use.Conclusions:Small differences were found in the categories of words used to describe male and female candidates and white and UiO candidates. These differences were not present in the standardized LOR compared with traditional LOR. It is possible that the use of standardized LOR may reduce gender- and race-based bias in the narrative assessment of applicants.

Which Emergency Medicine Milestone Sub-competencies are Identified Through Narrative Assessments?

IntroductionEvaluators use assessment data to make judgments on resident performance within the Accreditation Council for Graduate Medical Education (ACGME) milestones framework. While workplace-based narrative assessments (WBNA) offer advantages to rating scales, validity evidence for their use in assessing the milestone sub-competencies is lacking. This study aimed to determine the frequency of sub-competencies assessed through WBNAs in an emergency medicine (EM) residency program.MethodsWe performed a retrospective analysis of WBNAs of postgraduate year (PGY) 2–4 residents. A shared mental model was established by reading and discussing the milestones framework, and we created a guide for coding WBNAs to the milestone sub-competencies in an iterative process. Once inter-rater reliability was satisfactory, raters coded each WBNA to the 23 EM milestone sub-competencies.ResultsWe analyzed 2517 WBNAs. An average of 2.04 sub-competencies were assessed per WBNA. The sub-competencies most frequently identified were multitasking, medical knowledge, practice-based performance improvement, patient-centered communication, and team management. The sub-competencies least frequently identified were pharmacotherapy, airway management, anesthesia and acute pain management, goal-directed focused ultrasound, wound management, and vascular access. Overall, the frequency with which WBNAs assessed individual sub-competencies was low, with 14 of the 23 sub-competencies being assessed in less than 5% of WBNAs.ConclusionWBNAs identify few milestone sub-competencies. Faculty assessed similar sub-competencies related to interpersonal and communication skills, practice-based learning and improvement, and medical knowledge, while neglecting sub-competencies related to patient care and procedural skills. These findings can help shape faculty development programs designed to improve assessments of specific workplace behaviors and provide more robust data for the summative assessment of residents.