Accuracy Of Self-evaluation Research Articles

Are Plastic Surgery Trainees Accurate Assessors of Their Own Microsurgical Skill?

Microsurgery is a technically demanding surgical discipline with a steep learning curve. Trainees have faced several difficulties due to less hands-on theater time and pandemic-related limits on access to technical training. To overcome this, trainees engaged in self-directed training, which requires an accurate self-assessment of skill. This study aimed to assess the ability of trainees to accurately self-assess their performance while performing a simulated microvascular anastomosis. Novice and specialist plastic surgery trainees performed a simulated microvascular anastomosis on a high-fidelity chicken femoral vessel model. Each participant objectively rated the quality of their anastomosis using the Anastomosis Lapse Index (ALI). Two expert microsurgeons subsequently blindly rated each anastomosis. To determine the accuracy of self-evaluation, self-scores and expert-scores were compared using a Wilcoxon signed-rank test. Twenty-seven surgical trainees completed the simulation, with a mean time to completion (TTC) of 40.3 minutes (range 14.2-106.0 minutes). For the entire cohort, the median ALI self-score was 4 (range 3-10), while the median ALI expert-score was 5.5 (range 2.5-9.5). There was a significant difference between the ALI self-score and the expert-score (p<0.001). When grouped by experience level, there was no significant difference between self-score and expert-score within the specialist group, while there was a significant difference within the novice group (p=0.001). These findings suggest that specialist trainees are accurate assessors of their own microsurgical skills, but novice trainees tend to overestimate their technical skills. While novice trainees can engage in independent self-directed microsurgical training, they should seek expert feedback to ensure targeted training.

The relationships between freshmen’s accuracy of self-evaluation and the likelihood of succeeding in chemistry and physics exams in two STEM undergraduate courses

ABSTRACT In this longitudinal study, we investigate how demographic, cognitive and meta-cognitive variables affect students’ success in chemistry and physics exams in two Science-Technology-Engineering-Mathematics undergraduate courses, biology and engineering. Independent variables include high school final examination performance, chemistry and physics marks, self-efficacy and accuracy of self-evaluation, namely the difference between the estimation of one’s own performance in a specific task measured with a confidence rating scale and the actual performance score. We involved 81 biology and 125 engineering freshmen students in the study. Biology students were attending a General Chemistry course, while Engineering students were attending a General Physics course. Accuracy of self-evaluation scores were calculated through Rasch analysis of responses to an instrument that included chemistry and physics items, and a confidence tier. We found that for both courses, the likelihood of passing the exams of chemistry and physics was significantly predicted solely by their accuracy of self-evaluation accuracy score. We found the overconfident students had lower likelihood of passing the exam. Our results suggest that the perception of one’s own ability is a relevant factor for predicting students’ success at undergraduate level and can be a better predictor of academic success than high school performance and self-efficacy.

Comparing Construction and Study of Concept Maps – An Intervention Study on Learning Outcome, Self-Evaluation and Enjoyment Through Training and Learning

Concept maps are graphical tools for organizing and representing knowledge. They are recommended for biology learning to support conceptual thinking. In this study, we compare concept map construction (CM-c, i.e., creating concept maps) and concept map study (CM-s, i.e., observing concept maps). Existing theories and indirect empirical evidence suggest distinct effects of both formats on cognitive, metacognitive and emotional aspects of learning. We developed a CM-c training, a CM-s training, and a brief introduction to concept maps (control training) for junior high school students. We investigated effects on learning performance, concept map quality, cognitive load (cognitive effects), accuracy of self-evaluation (metacognitive effects) and enjoyment (emotional effects) of these trainings in a subsequent learning phase (CM-c learning vs. CM-s learning) in a quasi-experimental two-factorial study with 3 × 2 groups (N = 167), involving the factors training type and learning type. Results reveal that CM-c training increased learning performance and concept map quality. Effects of CM-c training on learning performance transferred onto learning with CM-s. Self-evaluation was slightly more accurate after CM-c training than CM-s training. Students reported moderate, and highly varying enjoyment during CM-c and CM-s learning. The superiority of CM-c over CM-s in learning performance and concept map quality probably lies in its characteristic of being an active learning strategy. We recommend practitioners to favor CM-c training over CM-s training, and foster students’ active engagement and enjoyment.

Evaluation of absences and myoclonic seizures in adults with genetic (idiopathic) generalized epilepsy: a comparison between self-evaluation and objective evaluation based on home video-EEG telemetry.

People with focal epilepsies are known to under-document their seizures, but there is no data on self-documentation in adults with genetic (idiopathic) generalized epilepsy (GGE/IGE). We assessed the accuracy of self-evaluation of typical absences (TA) or myoclonic seizures (MS) in adults with IGE based on home video-EEG telemetry (HVET). Patients' own estimates were compared to the objective count of definite TA and MS, performed visually. We considered definite TA as generalized spike-wave discharges (GSWD) that met any of the following criteria: 1) coinciding with clear behavioural arrest on video, 2) followed after a few seconds by positive indication that an absence occurred, or 3) in the absence of video, consistently coinciding with clear motor arrest, as evidenced by interruption of continuous muscle activity. For each patient, we also classified probable TA as GSWDs that were longer than those corresponding to the shortest definite TA on HVET or based on the most recent sleep-deprived EEG (SDEEG). From the first 300 consecutive adults who had HVET, 24 had IGE with TA and / or MS; 23 were women. Only one patient had newly diagnosed IGE. Erroneous self-assessment of TA and MS was noted in 16/24 patients (66.7%). Seizures were overestimated in nine (37.5%) and underestimated in seven (29.2%). Only one patient (4.2%) documented all her TA and MS without false-positive estimates. Overestimation (but not underestimation) of TA on HVET could be predicted when patients reported daily or multiple weekly TA and a recent SDEEG was either normal or contained only subclinical GSWD (p=0.0095). Under- and over-self-documentation of TA and MS occurred in two thirds of adults with GGE/IGE, with substantial impact on their outpatient management and treatment. Diagnostic HVET is a useful tool for the detection of erroneous self-evaluation in these patients.

Accuracy of self-evaluation in a peer-learning environment: an analysis of a group learning model

There is a global competitive demand for graduates with soft skills, and higher education institutions are tasked to reduce the employee skill gap. Thus, we investigated the students' perceptions of peer assessment in facilitating engagement in soft-skill development through group work activities. Using group work to measure the effectiveness of students' feedback on their assessment, we posit that students perceive self-assessment in group work as a tool that represents fairness. By focusing on learning in a peer-assisted learning environment, the study is a two-period different observation on the effectiveness and validity of peer assessment practice. We applied a group learning model over two academic sessions to investigate if students can self-evaluate accurately in a peer-learning environment. The employed methods included both qualitative and quantitative analysis. The findings of the study differ from previous findings that students cannot self-assess accurately. Empirically, there was no significant difference between the peer marks and tutor marks. The study also found that peer learning improves students' quality of assessment as they reflect on their work better.

The Relationships Between the Accuracy of Self-Evaluation, Kanji Proficiency and the Learning Environment for Adolescent Japanese Heritage Language Learners

This paper focuses on Japanese heritage language (JHL) learners in an Australian context. The paper reports on a research project in a hoshuu-koo institution, a Japanese supplementary school, and explores the experiences of a group of Year 7 students. This study was initiated by identifying to what extent JHL learners can recognise their own skills, especially in proficiency in kanji, one of the Japanese scripts. It was predicted that several elements could relate to the accuracy of self-evaluation. By exploring levels of self-evaluation skills and the elements concerned in Japanese learning, the aim of the research was to help develop differentiated curriculum in the future. Data were based on student performance on kanji tests and answers to questionnaires, and the Excel Correl Function was used to calculate correlation coefficients. Graphs were also used to analyse the data. It was found that students who had relatively high kanji proficiency, especially in higher year levels, recognised their own skills but an overall overestimation was found amongst other students. Specific areas of kanji learning, such as okurigana and radicals, were identified as areas that need to be enhanced for appropriate self-evaluation for most of the students. Learning environment related to evaluation skills was also identified. Concluding comments centre on implications for further teaching approaches and research on the enhancement of kanji self-evaluation skills.

Development and implementation of a competency-based clinical evaluation tool for midwifery education

BackgroundThe learning goals and evaluation strategies of competency-based midwifery programs must be explicit and well-defined. In the US, didactic learning is evaluated through a standardized certification examination, but standardized clinical competence evaluation is lacking. Description of the toolThe Midwifery Competency Assessment Tool (MCAT) has been adapted from the International Confederation of Midwives’ (ICM) “Essential Competencies” and from the American College of Nurse-Midwives’ (ACNM) “Core Competencies”, with student self-evaluation based on Benner's Novice-to-Expert theory. The MCAT allows for the measurement and monitoring of competence development in all domains of full-scope practice over the course of the midwifery program. Application and evaluation of the toolStrengths of the MCAT are that it provides clear learning goals and performance evaluations for students, ensures and communicates content mapping across a curriculum, and highlights strengths and gaps in clinical opportunities at individual clinical sites and for entire programs. Challenges of the MCAT lie in balancing the number of competency items to be measured with the tedium of form completion, in ensuring the accuracy of student self-evaluation, and in determining “adequate” competence achievement when particular clinical opportunities are limited. ConclusionsUse of the MCAT with competency-based clinical education may facilitate a more standardized approach to clinical evaluation, as well as a more strategic approach to clinical site development and use.

Metacognitive monitoring and learning gain in foundation chemistry

The ability to make realistic judgements of one's performance is a demonstration of the possession of strong metacognitive skills. In this study we investigate the relationship between accuracy of self-evaluation as an expression of metacognitive skill, and learning gain in stoichiometry. The context is an academic development programme at a South African University offered for under-prepared students enrolled for science and engineering. These students generally exhibit unrealistically high levels of confidence in performance and this could potentially place them at risk by negatively affecting decisions regarding time management and self-regulation. We investigated whether overconfidence before instruction is corrected upon exposure to teaching. A three-tier stoichiometry test was used to collect qualitative and quantitative data before and after instruction. Findings indicate that the majority of the students were overconfident in the evaluation of their performance in both the pre- and posttests. Overconfidence was not a debilitating disposition when demonstrated in the pretest provided that it was corrected during teaching and learning. The most vulnerable students were those that judged their performance or lack thereof realistically in the pretest but became overconfident during the teaching and learning of stoichiometry. Our results suggest that under-prepared students are slow in developing accurate metacognitive monitoring skills within a classroom environment that did not include instruction focused on the development of such skills. We recommend a proactive and constructive response by educators which may reduce the incidence of failure and preserve the positive contribution of confidence, albeit excessively positive.

Self-assessment of functional ability in schizophrenia: Milestone achievement and its relationship to accuracy of self-evaluation

Between 50% and 80% of patients with schizophrenia do not believe they have any illness, and their self-assessment of cognitive impairments and functional abilities is also impaired compared to other information, including informant reports and scores on performance-based ability measures. The present article explores self-assessment accuracy in reference to real world functioning as measured by milestone achievement such as employment and independent living. Our sample included 195 people with schizophrenia examined with a performance-based assessment of neurocognitive abilities and functional capacity. We compared patient self-assessments across achievement of milestones, using patient performance on cognitive and functional capacity measures as a reference point. Performance on measures of functional capacity and cognition was better in people who had achieved employment and residential milestones. Patients with current employment and independence in residence rated themselves as more capable than those who were currently unemployed or not independent. However, individuals who had never had a job rated themselves at least as capable as those who had been previously employed. These data suggest that lifetime failure to achieve functional milestones is associated with overestimation of abilities. As many patients with schizophrenia never achieve milestones, their self-assessment may be overly optimistic as a result.

METACOGNITIVE ACCURACY AND LEARNING TO LEARN: A DEVELOPMENTAL PERSPECTIVE

Metacognition belongs to higher-order mental processes and enables us to control, plan and accordingly regulate our own learning and problem solving process. In the present study we researched developmental changes in different reasoning domains and in metacognitive accuracy, which is considered as part of successful metacognitive monitoring/ regulation, and as an essential element of self-regulated learning and learning to learn competence. The study involved 282 participants from four different age groups: 13-15-, 23-25-, 33-35- and 43-45- year olds. These participants solved tasks addressed to spatial, verbal-propositional and social reasoning, and evaluated their own performance on these tasks. To specify possible differences in metacognitive accuracy, the metacognitive accuracy index was computed. Results showed that metacognitive evaluations were accurate in spatial domain, less accurate in verbal-propositional and quite inaccurate in the social domain. The accuracy of self-evaluation increased with age and males were more accurate in their self-evaluations than females. Improvement of metacognitive accuracy with age is in tune with findings that metacognition becomes more effective with development and that people with age become more reflective and self-aware. Key words: reasoning, metacognition, metacognitive accuracy, self-regulated learning.

The association of positive and negative feedback with clinical performance, self-evaluation and practice contribution of nursing students

Providing consistent and high-quality feedback is a crucial component of clinical instruction. Such feedback can improve the students' ability to reflect themselves more accurately. However, giving feedback, especially negative is intricate. To evaluate the level of feedback provided to nursing students during clinical practice and investigate their association with related outcomes, such us clinical performance, self-evaluation of achievements and contribution of the practice to the professional skills. Cross-sectional study. 124 third-year nursing students during their "Emergency Nursing" (EN) clinical practice were instructed, criticized and graded by three teachers from the university staff. Following their clinical practice the students filled-out a questionnaire, in which they evaluated the feedback provided by their teachers, the contribution of the practice to their professional skills and their personal performance. Additionally, the teachers' grades of students' achievements were collected. Accuracy of students' self-evaluation was calculated as the arithmetical difference between the students' grades and the teachers' grades. The mean grades of positive and negative feedback were 74.5/100 and 70.7/100, respectively. Higher-quality positive feedback was associated with higher teachers' grade (p=0.027) and with "very high" evaluation of the contribution of the practice (p=0.022). Higher-quality positive feedback was associated with student's over-self-evaluation (p=0.02), whereas higher-quality negative feedback was associated with more accurate self-evaluation (p=0.015). High-quality positive feedback is associated with higher grades, higher contribution of the clinical practice to the student and over-self-evaluation whereas high-quality negative feedback is related to an accurate self evaluation of the students' performance. Teachers should pay more attention to administering high-quality positive and negative feedback.

Comparing the self-assessed and examiner-assessed driving skills of Japanese driving school students

Previous research about subjective driving skills has revealed that drivers, especially younger males, tend to rate their driving skills more highly than those of the average driver. This study examines the accuracy of Japanese novice drivers' self-evaluation of their driving skills by comparing their self-assessments with assessments made by a driving examiner. We also examined the effects of gender and age on the accuracy of driver self-evaluation of their skills.A sample of Japanese driving test candidates (n=2021) completed a self-assessment using a 5-point scale applied to 19 items. The candidates completed the assessment shortly after passing their practical driving test conducted at a driving school. Their performance was also assessed by an examiner who used the same scale. The comparison between self-assessment and examiner-assessment revealed that around 40% of Japanese driving school students made a realistic assessment of their skills. With regard to the gender differences, although males displayed higher levels of overconfidence than females did, the differences were not as large as earlier studies with questionnaires had suggested. Furthermore, the effect of age on the accuracy of novice drivers' skill assessment was found to be relatively small. Our findings, which are based on a comparison of subjective assessments of driving skills between examiners and novices, instead of a questionable method which relies on a comparison with a hypothetical average driver, suggest that the majority of candidates in fact do not overrate their own skills. These results were discussed from the viewpoint of the driver education system and compared to other European research using the same framework.

Inaccuracy of self-evaluation as additional variable for prediction of students at risk of failing first-year chemistry

Early identification of students at risk of failing first-year chemistry allows timely intervention. Cognitive factors alone are insufficient predictors for success; however, non-cognitive factors are usually difficult to measure. We have explored the use of demographic and performance variables, as well as the accuracy of self-evaluation as an indicator of metacognitive ability, as possible indicators for students at risk of failing the first semester course in General Chemistry (CMY 117) at the University of Pretoria. Variables with a strong correlation with performance in CMY 117 were used to develop a prediction model based on logistic regression. Three variables, i.e. prior performance in mathematics and in physical science, and the extent of overconfidence expressed as the ratio between expected and actual performance in a chemistry pre-test written at the start of the semester, were shown to be significant predictors for risk of failing. The highest overall accuracy of prediction (76%) was obtained for a subset of students with a C or D grade for their high school leaving examination in mathematics when high risk students were defined as those with a final mark for CMY 117 as 51% or lower. The prediction model, based on the model building data set, had a sensitivity of 92% and a specificity of 46%; whilst the sensitivity and specificity using the validation data set were 88% and 38% respectively.

Accuracy of Self-Evaluation in Adults with ADHD

Research on children with ADHD indicates an association with inaccuracy of self-appraisal. This study examines the accuracy of self-evaluations in clinic-referred adults diagnosed with ADHD. Self-assessments and performance measures of driving in naturalistic settings and on a virtual-reality driving simulator are used to assess accuracy of self-evaluations. The group diagnosed with ADHD (n= 44) has a higher rate of collisions, speeding tickets, and total driving citations in their driving history; report less use of safe driving behaviors in naturalistic settings; and use fewer safe driving behaviors in the simulator than the community comparison group (n= 44). Despite poorer performance, adults with ADHD provide similar driving self-assessments, thereby overestimating in naturalistic settings to a greater degree than the comparison group. These findings extend research in children with ADHD to an adult sample in an important domain of functioning and may relate to findings of executive deficits associated with ADHD.

To see ourselves as others see us: self-assessment in surgical residency

PurposeAlthough self-assessment is a widely used educational technique, the value of self-evaluation in surgical residency has not been clearly defined. This study was undertaken to assess the ability of residents to evaluate themselves using the same standards as the surgical faculty and to determine how this information may be used in a surgical training program. MethodsCategorical surgical residents were asked to grade themselves (scale, 1–10) in 12 performance characteristics by completing the same forms used by the faculty for 3 quarterly evaluation periods. Mean faculty grades ± standard deviation were computed for each resident and compared with resident self-evaluation grades for a global rating scale and for specific performance characteristics. Comparisons were made by Pearson correlation analysis, analysis of variance, and Bonferroni’s multiple comparisons test with significance accepted at p < 0.05. ResultsA significant correlation (r = 0.47; p < 0.0098) was identified between faculty and resident in the global score and in the specific performance characteristics of knowledge (r = 0.51; p = 0.0052), clinical judgement (r = 0.48; p = 0082), and technical ability (r = 0.52; p = 0.0036). For residents who accurately estimated their scores, mean faculty score was 7.3 ± 0.64, which was significantly higher than the mean score in overestimators (6.4 ± 0.89; p < 0.05), which in turn was significantly lower than the mean score in underestimators (8.2 ± 0.58; p < 0.001). No significant correlation was obtained for ethical standards and interpersonal relationships or between PGY level and accuracy of self-evaluation. ConclusionsWhereas resident self-evaluations correlated positively with faculty ratings both globally and in certain performance categories, the majority of residents over- or underestimated their abilities. Information from such self-evaluations may be useful in counseling residents as well as in monitoring and improving the evaluation process.

Self-evaluation on the ability to retrieve Kanji words: a comparison between an aphasiac and normal subjects

The present study investigated the accuracy of an aphasiac's self-evaluation on his ability to retrieve words. The aphasiac and six normal subjects were shown a list of words and that of drawings. The words were written in Kanji and the drawings could be translated into the Kanji words. For each word, the subjects rated the likelihood that they could write it, and for each drawing, they rated the likelihood that they could translate it into the Kanji word. A week later, they were asked to translate drawings into Kanji words, and the performance and self-ratings were compared. Although the aphasiac could write far less Kanji words than the normals, his self-rating was as high as the normals. It was suggested that metamnemonic judgments are based on the indirect inferential processes, not on the direct access to the memory-traces. In other words, the aphasiac might raise his rating scores when Kanji words or drawings seemed easy to understand.

Student self-evaluations: How useful? How valid?

Skill in self-evaluation is necessary for life-long learning, for competent performances—in short, for professional practice in the health sciences. In addition, self-evaluation skills can be learned and can be taught, although they are not now purposefully taught in most programs. There is research evidence that self-evaluation is more useful for learners as a formative rather than a summative tool. The purposes of this paper are (a) to provide a critical review of the literature on the accuracy of student self-evaluation and (b) to describe our experience and preliminary research findings at McMaster University regarding the most profitable approach to incorporating self-evaluation in baccalaureate nursing education.

Self-assessment and physician assessment of sexual maturation in Brazilian boys and girls: Concordance and reproducibility.

This study considered the concordance of self-assessment and physician assessment of sexual maturity status; breasts, and pubic hair in girls, and genitals and pubic hair in boys. The subjects were Brazilian children, youth and young adults, 174 females and 178 males, 6-26 years of age. Each subject evaluated his/her level of sexual maturation and then was examined by a physician. The sexual maturity ratings of the subjects and physicians were compared. Overall, the results showed moderate to high concordance (60.9-71.3%) for both assessments of secondary sex characteristics; better concordance was found for pubic hair (69.7-71.3%) than for the breasts or genitals (60%). Age did not appear to influence the accuracy of self-evaluation; rather, accuracy was more dependent on stage of maturation of the subject. Correlations between self- and physician assessments, and replicate self- and physician assessments were relatively high. © 1994 Wiley-Liss, Inc.

Influence of Self-Efficacy on Performance in a Cognitive Task

Abstract The present study investigated the influence of self-efficacy judgments on cognitive performance when subjects had equivalent knowledge and experience in the performance domain. High or low self-efficacy perceptions were experimentally induced in a sample of 64 Canadian college students. Results showed that differences in perceptions were related to measures of the number of problems completed, the efficiency of problem-solving strategies, and the accuracy of self-evaluation of responses. These results suggest that the perception of self-efficacy is a viable construct for comprehending performance, particularly on academic tasks requiring sustained self-monitoring.

Self-Evaluation to the Movement in Forehand-stroke of Tennis on Beginners

The purpose of this study was to clarify the accuracy of self-evaluation on the beginners in forehandstroke of tennis, further to ascertain the changes of the self-evaluation when the class proceeded in a half year.The subjects, who took a tennis class in general physical education of Tsukuba University, were 56 students, and all of them were beginners of tennis.The subjects performed forehand-stroke were recorded by video camera from two directions. After that, they evaluated the self-movement with questinnaires. The investigation were performed in May and December, 1986.Responses were obtained through a 3-point-scale-type selection (1. well done, 2. could not done, 3. not conscious). The accuracy of self-evaluation of subjects was checked by three tennis instructors using the video films.The results were as follows;(1) As the class proceeded, the subjects become more aware of their whole movement.(2) It appeared more difficult for the subjects to evaluate their movement during the impact and followthrough.(3) It seemed easier for the subjects to evaluate their movement while they were in still ready position.(4) It seemed that the subjects had difficulty in evaluating the racket position at the begining of the class, however the subjects learned to better evaluate the position as the class progressed.From above results, it could be suggested that instructors must place importance on the phases of impact and follow-through, and the sense of racket position while instructing beginner's tennis.