Accuracy Of Self-assessments Research Articles

Awareness of Memory Ability and Change: (In)Accuracy of Memory Self-Assessments in Relation to Performance.

Little is known about subjective assessments of memory abilities and decline among middle-aged adults or their association with objective memory performance in the general population. In this study we examined self-ratings of memory ability and change in relation to episodic memory performance in two national samples of middle-aged and older adults from the Midlife in the United States study (MIDUS II in 2005-06) and the Health and Retirement Study (HRS; every two years from 2002 to 2012). MIDUS (Study 1) participants (N=3,581) rated their memory compared to others their age and to themselves five years ago; HRS (Study 2) participants (N=14,821) rated their current memory and their memory compared to two years ago, with up to six occasions of longitudinal data over ten years. In both studies, episodic memory performance was the total number of words recalled in immediate and delayed conditions. When controlling for demographic and health correlates, self-ratings of memory abilities, but not subjective change, were related to performance. We examined accuracy by comparing subjective and objective memory ability and change. More than one third of the participants across the studies had self-assessments that were inaccurate relative to their actual level of performance and change, and accuracy differed as a function of demographic and health factors. Further understanding of self-awareness of memory abilities and change beginning in midlife may be useful for identifying early warning signs of decline, with implications regarding policies and practice for early detection and treatment of cognitive impairment.

Evaluation of the validity of medical students' self-assessments of proficiency in clinical simulations

BackgroundThe accuracy of self-assessments has not been well supported in the literature. This study was undertaken to examine the validity of medical students' ratings of their proficiency during encounters with simulated patients and simulation devices. MethodsConfidential self-assessments for 10 skills were collected from 195 students during a formal clinical skills assessment related to 3 cases at the end of a surgery clerkship. The cases required students to gather data from simulated patients and perform procedures such as rectal examinations, nasogastric tube insertions, and suturing on bench simulation models. The patients were trained to assess student performance. ResultsThere were significant differences between student self-assessments and simulated patient scores for general clinical skills as opposed to procedural skills. Students' mean self-assessments in the data gathering and interpersonal skills were 2–6 % points higher than ratings of their proficiency by simulated patients. However, self-assessments on procedures were 5–8 points lower than patient ratings. The median correlation between self-assessments and patient ratings for general clinical skills such as data gathering and interpersonal skills was 0.08 (not significant), whereas the median correlation between student and patient ratings in procedures was 0.22 (P < 0.01). ConclusionsThird-year medical students' self-assessments for specific procedures are more valid than self-assessments of general clinical skills. Students are less confident in their procedural skills compared with general clinical skills. Although self-assessments should not be used as the sole measure of performance in clinical simulations, self-assessments for specific procedures can provide supplemental information on proficiency.

Verification of accurate technical insight: a prerequisite for self-directed surgical training.

Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments (SA-TA) were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between SA and TA was measured using weighted Cohen's kappa coefficients. Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5-28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between SA and TA improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were >0.81 after 18-21 task attempts. Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.

Misclassifications in financial risk tolerance

This paper analyses the empirical risk tolerance of individuals and the role of physiological measures of risk perception. By using a test that mimics the financial decision process in a laboratory setting (N = 445), we obtained an ex-post empirical measure of individual risk tolerance. Predictive classification models allow us to evaluate the accuracy of two alternative risk-tolerance forecasting methods: a self-report questionnaire and a psycho-physiological experiment. We find that accuracy of self-assessments is low and that misclassifications resulting from questionnaires vary from 36 to 65%: individuals asked to self-evaluate their risk tolerance reveal a high probability of failing their judgement, i.e. they behave as risk takers, even if, before the task, they define themselves as risk averse (and vice versa). Conversely, when the risk-tolerance forecast is obtained from individuals’ physiological arousal, observed via their somatic activation before risky choices, the rate of misclassification is considerably lower (~17%). Emotions are confirmed to influence the financial risk-taking process, enhancing the accuracy of the individual risk-tolerance forecasting activity. Self-report questionnaires, conversely, could lead to inadequate risk-tolerance assessments, with consequent unsuitable investment decisions. Bridging these results from the individual to the institutional level, our findings should enhance cautiousness, among regulators and financial institutions, on the (ab)use of risk tolerance questionnaires as tools for classifying individuals’ behaviour under risk.

Weight status in Chinese children: maternal perceptions and child self-assessments

Inaccurate parental perceptions of child weight status as well as children's own misperceptions can reduce motivation to adopt optimal nutritional and physical activity behaviors, thereby increasing overweight and obesity risk in child populations in the U.S. and elsewhere. Using population-based data from nine provinces of China, we analyzed the accuracy of maternal perceptions and children's self-assessments of weight status. The data were collected from 1265 children aged 6 to 18 years with self-reported weight status perceptions during the 2006 China Health and Nutrition Survey. Among these children, 863 had maternal estimation of child weight status. Descriptive and multiple regression analyses are conducted to find variations in the misperception of children's weight status. Among overweight children (n=176), 69% underestimated their weight status; 72% mothers of the overweight children (n=143) also underestimated their child's weight status. Less than one-quarter of overweight children and their mothers chose the correct classification of weight status. Multiple regression analyses showed that as children's body-mass-index (BMI)-for-age increased, the odds that mothers underestimated their weight status increased. Low maternal weight was significantly associated with maternal underestimation of child weight status but not with child's underassessment. Underestimation of childhood overweight is common among both mothers and children in China, particularly for children with the highest BMI. School-based BMI reporting may be beneficial in alerting parents and children to the problem and encouraging communication with health care providers.

Self- and peer-assessments of ambulance drivers' driving performance

The aim of the present study was to develop and examine the quality of the Ambulance Driver Self-assessment Questionnaire (ADSQ) and the Ambulance Driver Peer-assessment Questionnaire (ADPQ) measuring aspects of, driving performance, driving style and driving competence. In addition the ADSQ measures self-reflection and safety-attitudes. The aim of the study was also to examine ambulance drivers' self- and peer-assessments as well as to examine the accuracy of self-assessments by comparing self-assessed and peer-assessed driving performance, driving style and competence. 76 ambulance drivers employed at two ambulance stations in northern Sweden completed ADSQ and ADPQ. Item analyses were conducted to examine the psychometric properties of the items, and based on the results some revisions were made to improve the questionnaires. The revised questionnaires were functioning rather well, although some subscale demonstrated low internal consistency. Subscale inter-correlations provided support for construct validity. Self- and peer-assessments indicated safe driving performance and good driver competence, which is positive from a traffic safety perspective. A comparison of mean self- and peer-assessment ratings, controlling for age, gender and driving experience showed no significant differences, except for the subscale overtaking. This indicates that ambulance drivers' self-assessments are realistic in most areas.

Resident self-assessment versus faculty assessment of laparoscopic technical skills using a global rating scale

Accurate self assessment of laparoscopic technical skill development can accelerate a trainee's learning growth and guide lifelong development for surgeons. The aim of this study was to measure the accuracy of obstetrical and gynecology resident self assessments as compared to faculty assessments of laparoscopic technical skill performance using a technical skill global rating scale evaluation.

Relationships Between Self-Assessment Skills, Test Performance, and Demographic Variables in Psychiatry Residents

Some researchers have seen the capacity for self-assessment in trainees as a special skill, and some reports have concluded that this skill is positively and crucially correlated with academic competence. Thus, it is believed that those trainees who are most deficient in knowledge are least likely to be aware of their limitations. Other researchers have emphasized the impact of statistical regression and other technical considerations in the studies, which have led to these conclusions. Our study used a relative-ranking design to measure the accuracy of self-assessments of both strengths and weaknesses in psychiatry residents. We analyzed the relationships between indices of self-assessment accuracy and other resident characteristics, particularly current academic strength as measured by a standard test of psychiatric knowledge. A total of 56 residents in two general psychiatry programs evaluated their performance on the Psychiatry Resident in Training Examination by estimating the rank order of their scores in the 11 psychiatry subject areas. For each resident, actual examination results were then used to generate measures of the accuracy of the identification of strengths and weaknesses. Residents' identifications of their strengths and weaknesses were significantly more accurate than chance levels. Strengths and weaknesses were identified with roughly equal proficiency, and accuracy in these assessments was not correlated to any of the following variables: academic competence as measured by examination raw scores, postgraduate year, gender, international vs. American medical education, program membership, or age. Our results do not support the hypothesis that trainees who show the least academic mastery also make the most inaccurate self-assessments. In addition, we found no resident characteristics that accounted for variation in self-assessment accuracy.

Self-Assessment in the Health Professions: A Reformulation and Research Agenda

Many researchers and educators have identified self-assessment as a vital aspect of professional self-regulation.1,2,3 This rationale has been the expressed motivation for a large number of studies of self-assessment ability in medical education, health professional education, and professions educat

Racial differences in accuracy of self-assessment of sexual maturation among young black and white girls.

To examine the validity of maturation self-assessments and to investigate the association between race and validity of self-assessments among young black and white girls. Self-assessments and examiner-assessments of areolar and pubic hair development using line drawings were compared at three visits among a cohort of 11- to 14-year-old girls enrolled in the National Heart, Lung, and Blood Institute's Growth and Health Study. Accuracy rates and kappa coefficients were calculated to measure the agreement between girls and examiners. Logistic regression models were used to assess the racial differences in the accuracy of self-assessments while adjusting for possible confounders. Fair to moderate agreement was found between self- and examiner-assessments (areolar self-assessments: adjusted accuracy rates: 60.7-69.9%, kappa: 0.32-0.51; pubic hair self-assessments: adjusted accuracy rates: 57.9-70.7%, kappa: 0.36-0.55). While there were indications of racial differences in the ability to perform self-assessment with black girls tending to self-assess less accurately, most of the differences disappeared after adjusting for nurse-assessed stage. These findings suggest that self-assessment can substitute for examiner evaluations only when crude estimates of maturation are needed. However, when accurate maturation stage data are required, examiner-assessments are necessary. Because black girls are usually more pubertally advanced and tend to underestimate their development, the value of self-assessment is questionable for assessing populations with young black and white girls. Use of self-assessment might present biased estimates of maturation and confound research findings.

Students’ Self Assessment of Academic Attainment: accuracy and stability from 7 to 16 years and influence of domain and social comparison group1

Pupils’ appraisals of their own ability have been found to be important in their school progress. This paper examines the accuracy and stability of academic self assessments from 7 to 16 years, and the effect of two factors on developments over time: domain/school subject (mathematics or reading/English) and social comparison group (general to children of the same age or specific to own class). Results come from a major longitudinal study of pupils in London schools. An overall lowering in self assessments from 7 to 16 years was accompanied from 11 years by greater accuracy. But systematic differences were found between domains; overall ability ratings were higher in reading/English, and there was more stability in ratings from 11 to 16 years. There was a marked drop in self assessments from the general to the own class social comparison group. But again there were domain differences: there was a greater drop at 11 and 16 years in reading/English. 1An earlier version of the material presented was given in a paper to the American Education Research Association Annual Meeting, San Francisco, 1995.

The consistency of students' self-assessments in short-essay subject matter examinations.

This longitudinal study compares the accuracy of self-assessments of 22 students across four examinations during their first 2 years of medical school. The four examinations used a similar short-essay format and covered many of the same basic science disciplines at similar levels of difficulty. Immediately after answering an average of 20 questions on each examination, students predicted their performance on those questions. After assigned subject matter experts had scored the questions, the differences between students' predictions and the experts' scores were calculated for each question. The degree to which students had over- and underestimated their performance across all questions was determined by separately averaging all positive and negative differences between students' and experts' assessments on each examination. The results of the study indicated that accuracy in self-assessment improved from examination 1 to examination 3 (with less overestimation) and dropped on examination 4 (with more underestimation). The results revealed no relationship between self-assessment estimations and actual scores received. Furthermore, the self-assessment estimations tended to be statistically correlated between contiguous examinations (i.e., examinations 1 and 2, 2 and 3, etc.) but not between non-contiguous ones (i.e., examinations 1 and 3, etc.). The results of the study are interpreted to suggest that the students in the study have a self-assessment tendency towards over- or underestimation that is somewhat stable but that gradually evolves over time with experience, maturity and self-assessment practice. The most frequent direction of change is towards decreased overestimation and increased underestimation. These results are consistent with the findings of other recent longitudinal self-assessment studies.

A review of the validity and accuracy of self-assessments in health professions training

Valid self-assessment is fundamental to continuing professional competence but is seldom explicitly taught in health professions training. This review analyzed 18 scholarly articles published between January 1970 and February 1990 (14 articles regarding health professions trainees, and four concerning college students or graduate trainees) in which it was possible to compare performance as self-assessed by trainees with performance as assessed by experts or objective tests. The validity of self-assessed performance was found to be low to moderate and did not improve with time in conventional health professions training programs. Self-assessed performance seemed closely related to generalized self-attributions and was minimally influenced by external feedback in the form of test scores, grades, or faculty assessments. In five programs emphasizing explicit self-assessment goals and training strategies, moderate-to-high validity outcomes or improvements over time were demonstrated. Much of what passes for self-assessment in training seems the exercise of an underdeveloped skill, but effective training to improve validity and accuracy is available and feasible.

Effects of examples and embedded questions on the accuracy of comprehension self-assessments.

Des tests de prediction et des tests de comprehension sont pratiques aupres de 60 etudiants repartis selon quatre conditions de tâches (texte, texte+exemple, texte+questions, textes+questions+exemples), pour etudier l'influence des auto-evaluations suscitees par ces divers amenagements

Effects of examples and embedded questions on the accuracy of comprehension self-assessments.

Effects of examples and embedded questions on the accuracy of comprehension self-assessments.