Self-assessment questions

PurposeResearch on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance.MethodIn this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014–2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students’ scores on the FM final examination.ResultsStudents scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students’ SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone.ConclusionsThis large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.

IntroductionFormative self‐assessments are frequently utilized in anatomy courses to gauge learning. Self‐assessment tools have been demonstrated to improve student learning and performance on exams. However, few studies have assessed how specific elements, such as the inclusion of visual components, Bloom's taxonomy level, and the degree of topic integration, influence the efficacy of self‐assessments in an anatomy course.AimThis study retrospectively examined the relationships between self‐assessment construction characteristics, exam construction characteristics, and student exam performance. Specifically, 1) What elements of exam question construction correlated with performance? and 2) What elements of self‐assessment questions correlated with performance on corresponding exam questions?MethodsSelf‐assessment and exam questions administered during the Clinical Anatomy Block at the University of Arizona College of Medicine ‐ Phoenix from 2014–2018 for the Thorax and Abdomen and Pelvis and Lower Extremity units were examined. Self‐assessment questions were distributed to students via email, while exams took place in the anatomy laboratory, with a majority of questions incorporating structures on the donors, images, or anatomical models. For all questions, the constituent learning objectives, use of visual components, Bloom's taxonomy level, and degree of topic integration were noted. Exam performance data were collected for each class as the percentage correct for each question. These variables were entered into linear multiple regression models to assess their relationship to student performance for individual learning objectives. Because no individually identifiable data were utilized in this study, IRB approval was waived by the University of Arizona Office of Research, Discovery & Innovation (Protocol #: 1810045386).ResultsRegression analysis indicates that having visual components in exam questions is negatively associated with performance (t = −5.53, p <0.00). Conversely, a higher average Bloom's level for targeted learning objectives in self‐assessments (t = 2.02, p = 0.04), and a greater number of image‐based questions (t = 2.33, p = 0.02) are positively correlated with performance on corresponding exam questions.ConclusionsThese findings provide evidence that elements of self‐assessment construction, such as a visual component and a higher cognitive level, influence student exam performance when the exam format also incorporates these elements.This abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

One of the key issues to business process control is the identification of measurable process attributes. For manufacturing processes these are typically physical parameters of the process (e.g. temperature, set points) or physical attributes of the manufactured product (e.g. dimension, functional performance). However, for business processes the metrics are more abstract. The challenge has been to develop metrics that capture the contributing subtle and hard to measure factors for business process control. This paper presents an analytical model that uses the weights-of-evidence concept to convert answers to audit or self-assessment questions into a single numerical process quality index. This index is used to forecast process success or failure and monitor its performance from start to end. The application of the approach is illustrated with an automotive industry product development sub-process where the process performance metric is the field warranty data, i.e. incidents per thousand vehicles (IPTV). The analytical model converts process self-assessment (failure mode and effect analysis) questions into a single numeric process quality index. The validity of the model is reflected in the strength of the correlation between the index and the IPTV results. Also, in this paper a measure is developed for identifying critical process quality assessment questions. This measure quantifies the deviation in the automotive business process that should have more focus. The significance of the analytical model proposed in this research is that the project managers or quality assurance auditors may be able to use the metric to predict product quality at any point in the product development process.

Screening tests for health problems can identify elderly people who should undergo the Comprehensive Geriatric Assessment, enabling the planning of actions to prevent disability. The aim of this study was to analyze the inter-rater reliability (IRR) of self-assessment questions (SAQ) and performance tests (PT) recommended in Brazil, in a sample of low-income elderly people, through an exploratory study performed with 165 elderly assessed by two professionals on different days. IRR was evaluated using the intraclass correlation coefficient (ICC) for continuous variables and the kappa statistic for categorical ones. The IRR for the PT (muscle strength, mobility body mass index, vision) was excellent and presented ICC values greater than 0.75. By contrast, the IRR for SAQ (urinary incontinence, self-perceived health and hearing impairment) was intermediate. Only the fall-related item presented a good IRR. In this study single SAQ had poor reliability when compared to PT, suggesting the necessity of revision of subjective self-assessment items with low reproducibility before implementation.

This section contains answers to the ‘Self-Assessment Quiz’ and ‘Self-Assessment Questions’ at the end of each chapter.

Chapter 14 - Self-assessment questions

Clinicians' self-assessment questions and answers in subtance abuse treatment

Ceylon Medical Journal (CMJ) is a peer-reviewed, open access journal published quarterly by the Sri Lanka Medical Association in the last week of March, June, September and December each year. The mission of the CMJ is to promote the science and art of medicine and betterment of public health. The Journal publishes original papers and commentaries which have relevance to medicine and allied sciences. The CMJ is committed to maintaining and conforming to the editorial and ethical standards recommended by the International Committee of Medical Journal Editors.

Many patients who are hospitalized cannot use inhalers correctly, yet education for their use is often not provided. To address the need for an effective intervention feasible for wide-scale implementation, a virtual teach-to-goal intervention was developed to provide tailored patient-directed education using adaptive learning technology. To assess whether the virtual teach-to-goal intervention is noninferior to an in-person teach-to-goal intervention for improving inhaler technique. An equivalence and noninferiority randomized clinical trial took place from January 13, 2016, through September 20, 2017, with analyses conducted between October 25, 2017, and September 23, 2019. Adult inpatients with asthma or chronic obstructive pulmonary disease (COPD) admitted to general inpatient wards were eligible. Enrolled participants were randomized to virtual (n = 61) or in-person (n = 60) educational interventions. Investigators and research assistants were masked to interventions. Initial enrollment, study assessments, and delivery of the educational intervention occurred in the hospital; participants returned at 30 days for a follow-up research visit. Virtual education was a module delivered via handheld tablet with self-assessment questions before demonstration, narrated video demonstration of the correct technique, and self-assessment questions after demonstration; up to 3 rounds were repeated as needed. In-person education participants received iterative rounds of inhaler technique assessment and education by trained staff. Noninferiority testing of whether virtual vs in-person education achieved an equal percentage with correct inhaler technique after education (>9 of 12 steps correct) against an a priori noninferiority limit of -10%; logistic regression models were used to adjust for differences in baseline technique and health literacy. Among 118 participants (59 in each group), most were black (114 [97%]) and female (76 [64%]), with a mean (SD) age of 54.5 (13.0) years. Correct technique increased similarly before vs after education in virtual (67%; range, 2%-69%) and in-person (66%; range, 17% to 83%) groups, although the difference after intervention exceeded the noninferiority limit (-14%; 95% CI lower bound, -26%). When adjusting for baseline inhaler technique, the difference was equivalent to the noninferiority limit (-10%; 95% CI lower bound, -22%). The findings suggest that patient-directed virtual education similarly improved the percentage of participants with correct technique compared with in-person education. Future work should confirm whether virtual teach-to-goal education is noninferior to in-person education and whether it is associated with long-term skills retention, medication adherence, and improved health outcomes. ClinicalTrials.gov identifier: NCT02611531.

Clinicians' self-assessment questions and answers in substance abuse treatment

Principles of electroanalytical methods : Wiley, London, 1987 (ISBN 0-471-91329-4). xx + 252 pp. Price £9.95 (softback), £28.00 (hardback)

Clinicians' self-assessment

The validity of single-item, self-assessment questions as measures of adult physical activity

Responsiveness is an important property of an outcomes questionnaire. It can be defined as the ability of an instrument to capture important changes in a patient's health status over time. The authors previously designed the Michigan Hand Outcomes Questionnaire (MHQ), a hand-specific outcomes instrument that contains six distinct scales: (1) overall hand function, (2) activities of daily living, (3) pain, (4) work performance, (5) aesthetics, and (6) patient satisfaction with hand function. In the first study, the authors demonstrated that the MHQ is a reliable and valid instrument for the hand. The purpose of this second study is to assess the responsiveness, or sensitivity, of the MHQ to clinical change in patient status. A total of 187 consecutive patients with chronic hand disorders completed a baseline MHQ prior to receiving treatment at a university plastic surgery clinic. Approximately 6 to 18 months after completing the first questionnaire, patients were sent a follow-up MHQ by mail. The second questionnaire was identical to the first, with the exception of one additional question added to each of the six MHQ scales. This additional question asked patients to rate the change in their hands since completing the last questionnaire using a seven-point response scale. Spearman's correlation coefficient was used to correlate the responses from patients' self-assessment questions with the actual score change (after score - before score). The response rate for the second administration was 49% (92 questionnaires returned)-a fairly good rate of return for mail surveys. There were no significant differences in gender, race, education, and income between responders and nonresponders. When patients' self-assessment of change was correlated with the change in the six scale scores over time, all six correlations were statistically significant, with p < 0.05. The correlations ranged from 0.25 for the aesthetics scale to 0.43 for the pain scale. The MHQ was responsive using patients' self-assessment of their clinical change. Future studies will evaluate the responsiveness of the MHQ compared with objective physiological measures such as grip strength, range of motion, and the Jebson-Taylor test. Additionally, research is underway to assess the responsiveness of the MHQ for specific procedures, including metacarpophalangeal arthroplasties for rheumatoid arthritis and microvascular toe-to-hand reconstructions.

Clinicians' self-assessment questions and answers in substance abuse treatment