Innovative Assessment Strategies Research Articles

Exploring alternative assessments during COVID: Instructor experiences using oral exams

Research into conventional summative assessment methods, such as written exams, has been extensively documented in the literature. However, as academia evolves in response to changing dynamics, there is a growing demand for more robust summative assessment approaches. Oral exams have emerged as a noteworthy form of summative assessment with intrinsic formative qualities, enabling instructors to delve deeply into students’ comprehension within a meaningful learning framework. Considering the constraints imposed by the traditional written examination format during the COVID pandemic, two educators implemented oral assessments in their chemistry courses, one in general chemistry and the other in organic chemistry. This article presents a comprehensive account of their approach, course structure, rubrics, documentation procedures, and the challenges associated with implementing oral exams. Furthermore, the authors offer insights derived from perceived course outcomes, experiences, collaborative efforts, and reflections from this transformative process. Through candid exploration, this article delves into both the potential advantages and the hurdles associated with the adoption of oral exams in chemistry education. It serves as a valuable resource for educators seeking innovative assessment strategies.

Progress Testing in UK Medical Education: Evaluating Its Impact and Potential.

This comprehensive review critically examines the UK medical curriculum, with a particular focus on progress testing as an innovative assessment strategy. The curriculum, evolving from foundational sciences to practical applications, is encapsulated in the integrated curriculum model (ICM). This model adeptly combines theoretical knowledge with clinical practice, fostering cognitive, affective, and psychomotor skills among medical students. Central to this review is an exploration of progress testing. This method, grounded in constructivist learning theories, emphasises continuous assessment and professional development. Progress testing's regular, comprehensive examinations are instrumental in guiding students through the progressive stages of competence, as outlined in Miller's pyramid, from foundational knowledge to clinical proficiency. The review also addresses the broader impacts of progress testing on teaching approaches, student feedback, academic and pastoral support, and quality assurance. By aligning with the dynamic requirements of 21st-century medical training, progress testing not only nurtures well-rounded professionals but also ensures compliance with regulatory bodies like the General Medical Council. Its emphasis on continuous evaluation aligns with the practical realities of a medical career, driving curricular innovation and aligning with regulatory standards. The implementation of progress testing marks a significant advancement in medical education. Its continuous, holistic nature benefits both students and educators, nurturing a more engaged learning attitude and meeting evolving medical needs. The adoption of this assessment strategy is seen as pivotal in shaping competent medical professionals, ready to face the challenges of modern medical practice.

Assessment Practices in Continuing Professional Development Activities in Health Professions: A Scoping Review.

In continuing professional development (CPD), educators face the need to develop and implement innovative assessment strategies to adhere to accreditation standards and support lifelong learning. However, little is known about the development and validation of these assessment practices. We aimed to document the breadth and depth of what is known about the development and implementation of assessment practices within CPD activities. We conducted a scoping review using the framework proposed by Arksey and O'Malley (2005) and updated in 2020. We examined five databases and identified 1733 abstracts. Two team members screened titles and abstracts for inclusion/exclusion. After data extraction, we conducted a descriptive analysis of quantitative data and a thematic analysis of qualitative data. A total of 130 studies were retained for the full review. Most reported assessments are written assessments (n = 100), such as multiple-choice items (n = 79). In 99 studies, authors developed an assessment for research purpose rather than for the CPD activity itself. The assessment validation process was detailed in 105 articles. In most cases, the authors examined the content with experts (n = 57) or pilot-tested the assessment (n = 50). We identified three themes: 1-satisfaction with assessment choices; 2-difficulties experienced during the administration of the assessment; and 3-complexity of the validation process. Building on the adage "assessment drives learning," it is imperative that the CPD practices contribute to the intended learning and limit the unintended negative consequences of assessment. Our results suggest that validation processes must be considered and adapted within CPD contexts.

Synchronous self-assessment: First experience for higher education instructors

We present a qualitative study of four cases of university instructors (teacher educators) implementing synchronous self-assessment (SSA). SSA consists of an innovative assessment strategy during written exam situations, which highlights the students’ voice and agency, giving it greater weight in the power balance traditionally established between instructors and students in classroom assessment practices. In this article, we focus on the effects and pedagogical potential of this assessment strategy from instructors’ point of view. In our study, three instructors were novels in implementing this strategy; the fourth instructor had several years of experience with it. The four instructors agreed on basic design features for an end-of-semester exam offered in four groups of first-year students of the same shared program at a Bachelor’s degree for Kindergarten Educator and Primary School Teacher. The instructors were individually interviewed after the assessment session in their course and the exams were gathered for analysis. Content and discursive analysis was carried out on the data. Results show substantial differences in the evaluative artefacts (instructors’ exams) in terms of cognitive demand and formative assessment potential, and point to noticeable needs for professional development in pursuit of assessment literacy in Higher Education.

Novel bioavailability-based risk assessment of Cd in earthworms and leeches utilizing in vitro digestion/Caco-2 and MDCK cells.

In the present study, the oral bioavailability of cadmium (Cd) in earthworms and leeches was investigated through in vitro physiologically based extraction test (PBET) digestion/Caco2 and MDKC cell models. We are the first to create an innovative assessment strategy which has capacity to offer a more precise evaluation of Cd-associated health risks in traditional animal medicines (TAMs), by combinational usage of bioavailable Cd levels, the duration and frequency of the exposure to TAMs obtained by questionnaire data, as well as safety factor of TAMs. Our data showed that the percentage of bioavailability for Caco-2 cells in earthworms and leeches ranged from 3.29 to 14.17% and 4.32 to 12.61%, respectively. The percentage of bioavailability of MDCK cells in earthworms and leeches ranged from 4.83 to 15.74% and 6.53 to 15.04%, respectively. After adjusting by the bioavailability of Cd to target hazard quotient (THQ), excitingly, our findings manifested that the health risks induced by the ingestion of earthworms and leeches were acceptable in the clinic. Our key findings suggest that bioavailability characterization cannot be ruled out and health risks should be assessed on the basis of the bioavailable Cd levels rather than total levels. Our novel strategy provides insight into the bio-accumulation of Cd in organisms as well as a more realistic and accurate assessment of Cd-associated health risks in TAMs, with the main purpose of improving public health by scientifically using TAMs.

Description of a Human-Centered Design Approach to Develop Novel Virtual Reality Applications to Improve Functional Outcomes Following Brain Injury

<h3>Research Objectives</h3> Brain injury continues to be a leading cause of serious long-term disability. The objective of this project is to develop and implement innovative assessment and intervention strategies, via robotics and virtual reality systems, to improve balance and walking capabilities for persons with neurologic conditions. <h3>Design</h3> A Human-centered design (HCD) framework is being used to guide the development process and to ensure its end products are usable and useful. HCD develops solutions by involving the stakeholder perspective in all steps of the problem solving process - which includes brainstorming, conceptualizing, developing, and implementing the solution. <h3>Setting</h3> Remote collaboration between a Neuro-recovery Center and an independent virtual reality (VR) and robotics laboratory (Immersive Experience Laboratories LLC (IXL)). <h3>Participants</h3> Physical Therapists, Occupational Therapists, persons with traumatic brain injury, and VR application hardware and software developers. <h3>Interventions</h3> Not Applicable. <h3>Main Outcome Measures</h3> Description of the development process of a clinician-designed virtual reality (VR) application for deployment with individuals recovering from brain injury; time to initial uptake of technology; patient and clinician preferences; and clinical outcome scores. <h3>Results</h3> Focus group meetings with clinicians have revealed needs that have strong potential to be effectively addressed by immersive and interactive virtual-reality (VR) interventions, including Identification of Hazards, Care of Others, Garage Time, and Guided Walking. Development of functional platforms for immersive VR training and asset development of indoor household settings is principally complete. Development of treadmill-integrated gait-management platform is principally complete. We currently continue iterative development of content with biweekly feedback from subject-matter-expert clinicians. <h3>Conclusions</h3> The HCD framework could be readily adaptable to other rehabilitation centers interested in developing and implementing these kinds of technologies for the patients they serve. <h3>Author(s) Disclosures</h3> No conflicts of interest

Refocusing Sustainability Education: Using Students’ Reflections on Their Carbon Footprint to Reinforce the Importance of Considering CO2 Production in the Construction Industry

The construction industry is the most significant contributor to the UK’s CO2 emissions. It is responsible for an annual output of approximately 45% of the total. This figure highlights the role the industry must play in helping to achieve the UK Government’s CO2 reduction target. It is ergo incumbent on construction-related educators to emphasise this issue and explore ways in which it can be achieved. Unintentional desensitisation has resulted in the term ‘sustainability’, particularly CO2 production, being seen by students as just another concept to be studied from a theoretical perspective. Many students fail to grasp its broader implications and how it should affect strategic environmental decisions about construction processes, technologies, and products. In an attempt to address this problem, an innovative learning, teaching, and assessment strategy was used with final year undergraduate construction students to improve their level of sustainability literacy. The theory of threshold concepts in the context of transformative learning was used as the baseline philosophy to the study. The approach involved asking students to calculate their carbon footprint and to reflect upon and extrapolate their findings to the construction industry and its practice. Content analysis was performed on the reflective commentaries acquired from student portfolios collected over four academic years. The results showed how the students’ reflections on their carbon footprints proved to be an enlightening experience. Terms such as ‘shocked by my footprint’, ‘surprised at the findings’, and ‘change in attitude’ were among the contemplative comments. When students linked their findings to the construction industry, phrases such as ‘waste generation’, ‘technologies’, and ‘materials’ were some of the concepts considered. By using their personal experiences as a benchmark, students were able to gain a deeper level of understanding of the causes and consequences of CO2 production. They also found it more straightforward to relate these issues to the construction industry and its practice. Several novel recommendations are made to raise the level of sustainability literacy in the construction industry thereby facilitating a potential reduction in worldwide CO2 production.

Gut feeling: A grounded theory study to identify clinical educators' reasoning processes in putting students on a learning contract.

To develop a substantive theoretical explanation that makes sense of the decision-making process that clinical instructors use to place students on a learning contract. Clinical instructors are challenged with the task of objectively evaluating students using subjective tools such as anecdotal notes, diaries, unstructured observations and verbal feedback from other nurses. Clinical instructors' assessment decisions have a considerable impact on a variety of key stakeholders, not least of all students. Grounded theory method and its heuristic tools including the logic of constant comparison, continuous memoing and theoretical sampling to serve conceptualisation were used in the process of data collection and analysis. Seventeen individual semi-structured interviews with clinical instructors in one university in Western Canada were conducted between May 2016-May 2017. Data were analysed using open, axial and selective coding consistent with grounded theory methodology. The study was checked for the Standards for Reporting Qualitative Research (SRQR) criteria (See Appendix S1). Three subcategories, "brewing trouble," "unpacking thinking" and "benchmarking" led to the study's substantive theoretical explanation. "Gut feeling" demonstrates how clinical instructors reason in their decision-making process to place a student on a learning contract. Placing a student on a learning contract is impacted by personal, professional and institutional variables that together shift the process of evaluation towards subjectivity, thus influencing students' competency. A system-level approach, focusing on positive change through implementing innovative assessment strategies, such as using a smart phone application, is needed to provide some degree of consistency and objectivity. Making visible the objective assessments currently being done by clinical instructors has the potential to change organisational standards, which in turn impact patient and clinical outcomes.

Using Photo Elicitation to Introduce a Network Perspective on Attachment During Middle Childhood

In this article, we develop a child-centered network approach to attachment during middle childhood. Following monotropic ideas, current attachment research focuses on parental attachment figures despite the expansion of the children’s social environment during middle childhood, failing to generate a comprehensive and structured overview of all individuals who ensure the children’s feeling of safety. Relying on quantitative methods, these studies are also dominated by an adult perspective, limiting the children’s contributions. While there have been theoretical drafts of attachment networks during childhood, this article constitutes the first practical implementation. Using photo elicitation interviews and participant observations, we developed an innovative assessment strategy that allows children to exhaustively identify and characterize all their attachment figures on sociostructural and functional dimensions, thus positioning the children at the center of their comprehensive attachment networks that collectively contribute to their feeling of security. We combine qualitative and quantitative data to assess the children’s own understanding of their feeling of security and to locate the individual attachment figure on context-specific social dimensions, thus making the research setting, a clan in Cameroon, an inherent part of the methodological development. The data are translated into multidimensional network diagrams to visualize the children’s perception of their attachment environment and the emerging patterns of their selection. We present an exemplary network, supplementing it with observational data to discuss the ecological validity of our approach.

School of Medicine of Federal University of Rio Grande Do Norte: A traditional curriculum with innovative trends in medical education

Introduction: The Medical School of the Federal University of Rio Grande do Norte (UFRN) is one of the biggest public medical schools in Northeast Brazil. In the last decade, significant investment in faculty development, innovative learning methodologies and student engagement has been key milestones in educational improvement at this medical school, harnessed to recent political changes that strengthened community-based and emergency education. This study describes how curriculum changes in UFRN Medical School have been responsible for major improvements in medical education locally and which impacts such transformations may have on the educational community.Methods: A group of students and teachers revised the new curriculum and established the key changes over the past years that have been responsible for the local enhancement of medical education. This information was compared and contrasted to further educational evidences in order to define patterns that can be reproduced in other institutions.Results: Improvements in faculty development have been fairly observed in the institution, exemplified by the participation of a growing number of faculty members in programs for professional development and also by the creation of a local masters degree in health education. Alongside, strong student engagement in curriculum matters enhanced the teaching-learning process.Conclusions: Due to a deeper involvement of students and teachers in medical education, it has been possible to implement innovative teaching-learning and assessment strategies over the last ten years and place UFRN Medical School at a privileged position in relation to undergraduate training, educational research and professional development of faculty staff.

Linguistic and pedagogical considerations for teaching Portuguese to Spanish speakers

ABSTRACTThis research study examines the pedagogical considerations and linguistic challenges regarding teaching Portuguese to speakers of Spanish. That the Portuguese language is similar to Spanish presents a unique set of advantages and challenges to students and instructors. The study of this population of students has grown considerably since the 1980s, and most notably since the 2000s. Two unique and growing populations of Portuguese learners come into focus during this study—both native and heritage Spanish speakers. Their specific learning goals are analyzed in detail, particularly regarding what types of innovative linguistic methods and assessment strategies are relevant for these learners.

Development of a home-based training program for post-ward geriatric rehabilitation patients with cognitive impairment: study protocol of a randomized-controlled trail

BackgroundGeriatric patients with cognitive impairment (CI) show an increased risk for a negative rehabilitation outcome and reduced functional recovery following inpatient rehabilitation. Despite this obvious demand, evidence-based training programs at the transition from rehabilitation to the home environments are lacking. The aim of this study is to evaluate the efficacy of a feasible and cost-effective home-based training program to improve motor performance and to promote physical activity, specifically-tailored for post-ward geriatric patients with CI.MethodsA sample of 101 geriatric patients with mild to moderate stage CI following ward-based rehabilitation will be recruited for a blinded, randomized controlled trial with two arms. The intervention group will conduct a 12 week home-based training, consisting of (1) Exercises to improve strength/power, and postural control; (2) Individual walking trails to enhance physical activity; (3) Implementation of patient-specific motivational strategies to promote behavioral changes. The control group will conduct 12 weeks of unspecific flexibility exercise. Both groups will complete a baseline measurement before starting the program, at the end of the intervention, and after 24 weeks for follow-up. Sensor-based as well as questionnaire-based measures will be applied to comprehensively assess intervention effects. Primary outcomes document motor performance, assessed by the Short Physical Performance Battery, and level of physical activity (PA), as assessed by duration of active episodes (i.e., sum of standing and walking). Secondary outcomes include various medical, psycho-social, various PA and motor outcomes, including sensor-based assessment as well as cost effectiveness.DiscussionOur study is among the first to provide home-based training in geriatric patients with CI at the transition from a rehabilitation unit to the home environment. The program offers several unique approaches, e.g., a comprehensive and innovative assessment strategy and the integration of individually-tailored motivational strategies. We expect the program to be safe and feasible in geriatric patients with CI with the potential to enhance the sustainability of geriatric rehabilitation programs in patients with CI.Trial registrationInternational Standard Randomized Controlled Trial (#ISRCTN82378327). Registered: August 10, 2015.

Cost-effective conservation of amphibian ecology and evolution.

Habitat loss is the most important threat to species survival, and the efficient selection of priority areas is fundamental for good systematic conservation planning. Using amphibians as a conservation target, we designed an innovative assessment strategy, showing that prioritization models focused on functional, phylogenetic, and taxonomic diversity can include cost-effectiveness-based assessments of land values. We report new key conservation sites within the Brazilian Atlantic Forest hot spot, revealing a congruence of ecological and evolutionary patterns. We suggest payment for ecosystem services through environmental set-asides on private land, establishing potential trade-offs for ecological and evolutionary processes. Our findings introduce additional effective area-based conservation parameters that set new priorities for biodiversity assessment in the Atlantic Forest, validating the usefulness of a novel approach to cost-effectiveness-based assessments of conservation value for other species-rich regions.

Embedding Enterprise: A Business School Undergraduate Course with an Enterprise Focus

This study profiles, through a case study of an undergraduate business programme, how a business school has embedded the theme of enterprise in its core undergraduate programme. Key participants in the development of the strategy and programme tutors and students were interviewed, to provide information for an analysis of the principle objectives and the impact of the course. The findings confirm the need to adopt a top-down strategy to identify underpinning themes that support all programme activities: the programme curriculum should focus on key themes and use innovative assessment and teaching and learning strategies, as well as added-value activities that enhance its effectiveness. The authors offer recommendations for best practice, including specific modules of study, use of guest speakers, added-value activities for developing student skills and innovative assessment strategies. It is argued that the paper will help university policy makers to construct valid programmes in which themes of enterprise and self-employment are integrated in order to enhance experiential and added-value activities.

2973 – Early detection and treatment of depression: the role of clinical judgment

In medicine, prodromes can be identified with early signs and symptoms that differ from the acute clinical phase. Appraisal of prodromal symptoms has been of importance in clinical medicine for many progressive, dangerous and treatable diseases in which early detection and timely treatment (particularly of recurrences) are crucial. The prodromal phase of depression has a large inter-individual variability and lacks diagnostic specificity. However, within each patient there is striking consistency in the prodromal symptoms that precede each episode, even though the same initial symptoms of the acute disorder may occur and not be followed by another episode. Their interpretation requires clinical judgment. Research on clinical judgment has disclosed several innovative assessment strategies: the use of diagnostic transfer stations instead of diagnostic endpoints using repeated assessments, subtyping versus integration of different diagnostic categories, staging, macro-analysis, extension of clinical information beyond symptomatic features. Such strategies can be applied to clinical cases at risk for depressive relapse and may pave the way for more effective modalities of treatment and relapse prevention. Evidence-based medicine does not appear to provide an adequate scientific background for the challenges entailed by current treatment of depression and needs to be integrated with clinical judgment.

Toward an Evaluation Framework for Doctoral Education in Social Work: A 10-Year Retrospective of One PhD Program's Assessment Experiences

This article presents a framework for evaluation in social work doctoral education and details 10 years of successes and challenges in one PhD program's use of the framework, including planning and implementing specific assessment activities around student learning outcomes and larger program goals. The article argues that a range of innovative and traditional internal and external assessment strategies can be tailored to the needs and resources of other programs. Easily adaptable exemplars, rubrics and rating forms, measurement devices, and assessment reports are provided. The article ends by highlighting the ways used to build student, faculty, and alumni participation in evaluative activities and by proposing ideas for related research assessment.

Clinical judgment in psychiatry. Requiem or reveille?

Background: There is increasing awareness of a crisis in psychiatric research and practice. Psychopathology and clinical judgment are often discarded as non-scientific and obsolete methods. Yet, in their everyday practice, psychiatrists use observation, description and classification, test explanatory hypotheses, and formulate clinical decisions. Aim: The aim of this review was to examine the clinical judgment in psychiatry, with special reference to clinimetrics, a domain concerned with the measurement of clinical phenomena that do not find room in customary taxonomy. Methods: A MEDLINE search from inception to August 2011 using the keywords “clinical judgment” and “clinimetric” in relation to psychiatric illness for articles in English language was performed. It was supplemented by a manual search of the literature. Choice of items was based on their established or potential incremental increase in clinical information compared with use of standard diagnostic criteria. The most representative examples were selected. Results: Research on clinical judgment has disclosed several innovative assessment strategies: the use of diagnostic transfer stations instead of diagnostic endpoints using repeated assessments, subtyping versus integration of different diagnostic categories, staging, macro-analysis, extension of clinical information beyond symptomatic features. Evidence-based medicine does not appear to provide an adequate scientific background for challenges of clinical practice in psychiatry and needs to be integrated with clinical judgment. Conclusions. A renewed interest in clinical judgment may yield substantial advances in clinical assessment and treatment. A different clinical psychiatry is available and can be practiced now.

The Clinical Process in Psychiatry

The aim of this review was to examine the clinical process in psychiatry, with special reference to clinimetrics, a domain concerned with the measurement of clinical phenomena that do not find room in customary taxonomy. A MEDLINE search from inception to August 2010 was performed for English-language articles using the keywords clinical judgment, clinimetric, staging, comorbidity, sequential treatment, and subclinical symptoms in relation to psychiatric illness. It was supplemented by a manual search of the literature. Choice of assessment strategies was based on their established or potential incremental increase in clinical information compared to use of diagnostic criteria. Contributions were evaluated according to the principles of clinimetrics. Several innovative assessment strategies were identified: the use of diagnostic transfer stations with repeated assessments instead of diagnostic endpoints, subtyping versus integration of different diagnostic categories, staging methods, and broadening of clinical information through macroanalysis and microanalysis. The most representative examples were selected. Current assessment strategies in psychiatric research do not reflect the sophisticated thinking that underlies clinical decisions in practice. The clinimetric perspective provides an intellectual home for the reproduction and standardization of these clinical intuitions.

Using the patchwork text assessment as a vehicle for evaluating students’ perceptions of their clinical leadership development

A shift in universities world wide in providing theoretical post graduate programmes of study underpinned by traditional assessment strategies to work based learning programmes supported by innovative assessment strategies is required if Higher education institutions are to effectively educate contemporary healthcare leaders. Concurrently generating the evidence to evaluate the effectiveness of educational programmes is required by commissioners of healthcare education ( DH, 2010). This paper reports on the perceptions of twelve post graduate students attending a clinical leadership masters programme of their leadership development through analysis of the critical commentary provided by students as part of assessment strategy that utilised the Patchwork Text Assessment. Following a thematic content analysis six themes emerged: programme philosophy and its impact on the success of the Patchwork Text Assessment; leadership development targeted against leadership frameworks; application and applicability of learning to the students own healthcare organisation; integrating theory to practice through theoretical development and work based activities; the value of networking; and the importance of multi-professional reflective groups. This study has clearly demonstrated how the success of the Patchwork Text Assessment in promoting deep learning is determined by its integration into the overall philosophy of the programme. Concurrently systems needed to be in place to ensure that Patchwork text Assessment is operationalised effectively and embedded within the day to day management of the programme.

The process of creating a cross‐university network for formative and shared assessment in higher education in Spain and its potential applications

During the academic year 2005–2006 a Cross University Network for Formative and Shared Assessment in Higher Education was formed among 48 academics from 16 different universities within Spain and representing a range of academic areas. The Network was formed in response to the determination of a group of individuals who were dissatisfied with, or unconvinced by, the traditional forms of assessment used within the Spanish higher education environment. Its aim was to provide a framework within which these individuals could provide mutual support for their work on developing more innovative and effective assessment strategies within their own subject areas and at the same time provide a focus to stimulate a wider debate and more extensive research within the HE sector in Spain. The aim of this article is to explain the background and rationale for the formation of the Network, the framework it has provided, some of the initial findings and how this type of framework may be applicable to similar situations in other sectors or countries.