Self-reported Confidence Ratings Research Articles

A Psychometric Look at Principal Professional Development

This study evaluates aspects related to P12 principals’ professional development needs in South Carolina regarding the three domains of school leadership: management, instructional leadership, and program administration. A survey to rate principals’ current leadership knowledge, rank order their professional development needs, and provide a confidence rating regarding their abilities was given to over 1,100 principals and 85 superintendents. Through examining relationships with a psychometric model, results derived latent leadership ability scores and self-reported confidence ratings of principals as well as the superintendents’ leadership scores and confidence ratings of their principals. This study found a significant discrepancy between principals’ and superintendents’ confidence ratings and their corresponding leadership ability scores, respectively. A further analysis of the rank-ordered professional development needs highlighted instructional leadership to be the most needed topic for professional development. Finally, atypical response patterns regarding principal’s current leadership knowledge are also identified through person-fit analysis to provide additional information regarding P-12 principals’ professional development needs.

Promoting Evidence-Based Practice at a Primary Stroke Center: A Nurse Education Strategy.

Promoting a culture of evidence-based practice within a health care facility is a priority for health care leaders and nursing professionals; however, tangible methods to promote translation of evidence to bedside practice are lacking. The purpose of this quality improvement project was to design and implement a nursing education intervention demonstrating to the bedside nurse how current evidence-based guidelines are used when creating standardized stroke order sets at a primary stroke center, thereby increasing confidence in the use of standardized order sets at the point of care and supporting evidence-based culture within the health care facility. This educational intervention took place at a 286-bed community hospital certified by the Joint Commission as a primary stroke center. Bedside registered nurse (RN) staff from 4 units received a poster presentation linking the American Heart Association's and American Stroke Association's current evidence-based clinical practice guidelines to standardized stroke order sets and bedside nursing care. The 90-second oral poster presentation was delivered by a graduate nursing student during preshift huddle. The poster and supplemental materials remained in the unit break room for 1 week for RN viewing. After the pilot unit, a pdf of the poster was also delivered via an e-mail attachment to all RNs on the participating unit. A preintervention online survey measured nurses' self-perceived likelihood of performing an ordered intervention based on whether they were confident the order was evidence based. The preintervention survey also measured nurses' self-reported confidence in their ability to explain how the standardized order sets are derived from current evidence. The postintervention online survey again measured nurses' self-reported confidence level. However, the postintervention survey was modified midway through data collection, allowing for the final 20 survey respondents to retrospectively rate their confidence before and after the educational intervention. This modification ensured that the responses for each individual participant in this group were matched. Registered nurses reported a significant increase in perceived confidence in ability to explain how standardized stroke order sets reflect current evidence after the intervention (n = 20, P < .001). This sample was matched for each individual respondent. No significant change was shown in unmatched group mean self-reported confidence ratings overall after the intervention or separately by unit for the progressive care unit, critical care unit, or intensive care unit (n = 89 preintervention, n = 43 postintervention). However, the emergency department demonstrated a significant increase in group mean perceived confidence scores (n = 20 preintervention, n = 11 postintervention, P = .020). Registered nurses reported a significantly higher self-perceived likelihood of performing an ordered nursing intervention when they were confident that the order was evidence based compared with if they were unsure the order was evidence based (n = 88, P < .001). This nurse education strategy increased RNs' confidence in ability to explain the path from evidence to bedside nursing care by demonstrating how evidence-based clinical practice guidelines provide current evidence used to create standardized order sets. Although further evaluation of the intervention's effectiveness is needed, this educational intervention has the potential for generalization to different types of standardized order sets to increase nurse confidence in utilization of evidence-based practice.

Inquiry Guided Learning Projects for the Development of Critical Thinking in the College Classroom: a pilot study

This paper describes the inaugural success of implementing Inquiry Guided Learning projects in a college-level human anatomy and physiology course. In this context, scientific inquiry was used as a means of developing skills required for critical thinking amoung students. The projects were designed using the Information Search Process (Kuhlthau, 1991) as a framework with emphasis placed on three of the six stages: question selection, scientific research, and presentation of results. The projects were quantitatively assessed using self-reported confidence ratings from a 10-point Likert scale. The projects were also qualitatively assessed using informal student feedback focusing on student suggestions for project improvement. Moving forward, the Inquiry Guided Learning projects will continue to be a formal course component, with the next stage of implementation to include a thorough assessment of student learning outcomes.

Monitoring and meta-metacognition in the own-race bias

Although there is a great deal of research focused on identification issues related to own-versus other-race faces very few experiments have explored whether metacognitive monitoring contributes to the own-race bias. In the current experiment the typical own-race bias paradigm was modified so that type-2 signal detection measures (e.g. Higham & Arnold, 2007a,b) could be used to directly measure metacognitive monitoring at retrieval. A second goal of the experiment was to explore whether self-reported confidence ratings differed depending on whether they were directed at answer accuracy (e.g., judging a face as “studied”) versus at decisions about that answer (e.g., volunteering vs. withholding that answer). Overall the results demonstrated that monitoring does contribute to the own-race bias, in that participants were better at monitoring their memory for own-race faces. Further, there was a significant difference between the two confidence measures, and the pattern of this difference depended on whether responses had been volunteered or withheld.

Testing a Mnemonic on Response Skills during Simulated Codes

BackgroundBaccalaureate nursing students rarely have the opportunity to practice emergency response skills. Simulation gives students the ability to practice emergency procedures in an environment that is safe. MethodNursing students’ skills accuracy and confidence ratings were assessed before and after an instructional module that incorporated a mnemonic, named ABCD-COPIME, to improve emergency response techniques to a simulated code blue scenario. Seven nursing students completed 7 trials of responding to a code blue simulation. After each trial, skill accuracy was assessed with a skills checklist, and after Trials 1, 4, and 7, each student’s confidence was measured by self-report. A 20-minute intervention introducing the ABCD-COPIME mnemonic was inserted after Trial 3. ResultsBoth skills accuracy and self-reported confidence ratings significantly increased after introduction of the mnemonic (95% improvement in mean skills accuracy scores). Furthermore, analyses revealed that these changes were not due to practice effects. ConclusionThis study supports the accumulating body of evidence that suggests that simulated training exercises are important for improving skills and confidence in responding to emergency situations. Moreover, results suggest that the ABCD-COPIME mnemonic training technique substantially improves both the skills accuracy of students and their self-reported confidence when responding to a code blue simulation.

(P1-92) Safety Function Action: Current and Future Directions

IntroductionSAFETY FUNCTION ACTION for Disaster Responders (SFA) trains a framework for achieving and maintaining a high level of disaster health. Within SFA, disaster health is defined as, “maximal safety, optimal function, and effective action in preparedness for, and response to, emergencies, disasters, and extreme events.” A set of six strategies forms the backbone of the framework with two strategies each for SAFETY (safeguard and sustain), FUNCTION (comfort and connect), and ACTION (advise and activate).MethodsDuring 2008, a total of 2,553 participants were trained throughout the State of Florida. Participants were drawn from public health, healthcare, mental health, and professional/volunteer emergency responder workforces. During 2009, an additional 861 participants were trained as “SFA facilitators.” Facilitators were provided with guidance and training materials to return to their worksites and train peers on the SFA modules. Facilitators were in direct contact with a team of 5 DEEP Center “coaches” who supported their training efforts. To assess the training's effectiveness, pre/post-assessment data on the 2,533 SFA participants and 861 SFA facilitators were collected.ResultsLive-training evaluation data showed highly favorable quality ratings for the course, materials, presenters, and all individual course components. Pre/post comparisons of the data indicated consistent gains in self-reported confidence ratings for all 7 “facilitator skills” (recruiting, motivating, training colleagues; teaching SFA skills, working with coaches) and 15 SFA “strategies and response skills” (applying the six SFA strategies to responders (self, family, team) and disaster survivors). Consistent gains were evident for 12 scales asking facilitators to self-report their comfort in dealing with disaster survivors exhibiting distress or suffering trauma and loss.Future DirectionsSFA has been launched as an online training course and a cell phone “app” is in progress. SFA has recently been trained in South America and Europe. Experiential components of SFA are being refined and expanded.

HENRY: development, pilot and long‐term evaluation of a programme to help practitioners work more effectively with parents of babies and pre‐school children to prevent childhood obesity

Despite epidemic numbers of obese and overweight pre-school children, professionals report a lack of confidence and self-efficacy in working with parents around lifestyle change. HENRY--Health Exercise Nutrition for the Really Young--trains health and community practitioners to work more sensitively and effectively with parents of babies and pre-school children around obesity and lifestyle concerns. Underpinned by the Family Partnership Model, reflective practice and solution-focused techniques, it offers face-to-face training and e-learning. This paper describes the development, pilot and evaluation of HENRY Sure Start Children's Centres. Twelve Children's Centres in Oxfordshire took part in the pilot involving 137 staff. Questionnaires were administered at the end of training courses. Self-reported confidence ratings were obtained before and after training. Postal questionnaires were sent to Centre managers 2-6 months later to ascertain long-term effects. Nine managers participated in in-depth interviews. A further 535 learners completed the e-learning course and online feedback. One hundred and thirty-one staff (96%) completed the training course and valued it as a way of enhancing skills and knowledge. Mean (±SD) self-reported confidence ratings increased (4.1 ± 0.7 to 7.2 ± 0.7; P < 0.00001). An influence on personal as well as professional lives was apparent. Long-term follow-up indicated ongoing impact attributed to HENRY on both Centres and staff. All 535 e-learners successfully completed: 98% would recommend HENRY; 94% thought it enhanced their skills as well as knowledge. HENRY is an innovative approach that offers some promise in tackling obesity through training community and health practitioners to work more effectively with parents of very young children. It appears to have an effect on participants' personal lives as well as professional work. A large-scale long-term study would be required to ascertain if there is the desired impact on young children's lifestyles and risk of obesity.

Self-reported Confidence in Recall as a Predictor of Validity and Repeatability of Physical Activity Questionnaire Data

Self-reported confidence ratings have been used in other research disciplines as a tool to assess data quality, and may be useful in epidemiologic studies. We examined whether self-reported confidence in recall of physical activity was a predictor of the validity and retest reliability of physical activity measures from the European Prospective Investigation into Cancer and Nutrition (EPIC) past-year questionnaire and the International Physical Activity Questionnaire (IPAQ) last-7-day questionnaire.During 2005-2006 in Sydney, Australia, 97 men and 80 women completed both questionnaires at baseline and at 10 months and wore an accelerometer as an objective comparison measure for three 7-day periods during the same timeframe. Participants rated their confidence in recalling physical activity for each question using a 5-point scale and were dichotomized at the median confidence value. Participants in the high-confidence group had higher validity and repeatability coefficients than those in the low-confidence group for most comparisons. The differences were most apparent for validity of IPAQ moderate activity: Spearman correlation rho = 0.34 (95% confidence interval [CI] = 0.08 to 0.55) and 0.01 (-0.17 to 0.20) for high- and low-confidence groups, respectively; and repeatability of EPIC household activity: rho = 0.81 (0.72 to 0.87) and 0.63 (0.48 to 0.74), respectively, and IPAQ vigorous activity: rho = 0.58 (0.43 to 0.70) and 0.29 (0.07 to 0.49), respectively. Women were less likely than men to report high recall confidence of past-year activity (adjusted odds ratio = 0.38; 0.18 to 0.80). Confidence ratings could be useful as indicators of recall accuracy (ie, validity and repeatability) of physical activity measures, and possibly for detecting differential measurement error and identifying questionnaire items that require improvement.

Confidence and gender differences on the Mental Rotations Test

The present study examined the relation between self-reported confidence ratings, performance on the Mental Rotations Test (MRT), and guessing behavior on the MRT. Eighty undergraduate students (40 males, 40 females) completed the MRT while rating their confidence in the accuracy of their answers for each item. As expected, gender differences in favor of men were obtained. Results also indicated a positive correlation between confidence ratings and scores on the MRT, as well as negative correlations between confidence ratings and MRT outcomes presumed to reflect propensity to guess. More elaborate analyses using a measure of accuracy of predictions (the Brier score) indicated that men have a more accurate perception of their performance on the MRT than women do. Findings are discussed in terms of their implications for the interpretation of gender differences and guessing behavior on the MRT.

Predictors of Self-Reported Confidence Ratings for Adult Recall of Early Life Sun Exposure

Predictors of Self-Reported Confidence Ratings for Adult Recall of Early Life Sun Exposure

Predictors of Self-reported Confidence Ratings for Adult Recall of Early Life Sun Exposure

Use of self-reported confidence ratings may be an efficient method for assessing recall bias. In this exploratory application of the method, the authors examined the relation between case-control status and self-reported confidence ratings. In 2002 and 2003, melanoma cases (n = 141) and controls (n = 143) aged 20-44 years residing in Ontario, Canada, estimated the amounts of time they had spent outdoors in summer activities when they were 6-18 years of age and indicated their confidence in the accuracy of each estimate. The generalized estimating equations extension of logistic regression was used to examine dichotomized confidence ratings (more confident vs. less confident) for activities reported for ages 6-11 years and 12-18 years. Types of activity were associated with more confident reporting for both age strata; as the number of stable outdoor activity periods (total number of similar outdoor periods within each activity) reported by respondents increased, confidence decreased. Cumulative time spent outdoors was also associated with more confidence but reached statistical significance only for the age stratum 12-18 years. There was no statistically significant association between case-control status and self-reported confidence for either age stratum (6-11 years: odds ratio = 0.91; 12-18 years: odds ratio = 1.32), which suggests an absence of recall bias for reported time spent outdoors.

Evaluation of skin cancer seminar for general practitioners: Changes in knowledge, diagnostic and procedural skills, beliefs, and self-reported practices

A one-day seminar was conducted in six rural sites in New South Wales, Australia. Using a pre- and post-test design, we evaluated changes in general practitioners' knowledge, self-reported skills, self-reported practices in early detection and treatment of skin cancer, and their beliefs about the value of skin checks and screening. Of total participants, 93% completed pretests and 83% completed both pre- and post-tests. Significant improvements in knowledge, self-reported confidence ratings, and procedural skills were demonstrated, although not all participants attained desirable levels of knowledge and confidence at post-test. Further, the lack of scientific evidence to support skin checks, as taught during the seminar using a problem-based approach, was insufficient to alter most participants' beliefs in an intrinsic benefit from mass screening. Implications for medical educators and public health are discussed.

Intelligent Interface Design: An Empirical Assessment of Knowledge Presentation in Expert Systems

This research evaluates intelligent interface requirements for knowledge presentation in an expert system used for diagnostic problem solving. In a field study, interactions between employee expertise, knowledge presentation format (procedural vs. declarative), question type (requiring abstract vs. concrete knowledge organization), and task uncertainty are examined for employee problem-solving and decision-making performance (speed and accuracy). Also evaluated are confidence in system recommendations and lines-of-reasoning, as well as user satisfaction with the system interface. The study provides findings that are discussed within the context of intelligent interface requirements for organizational information systems. The results show that high-skill users perform significantly faster and more accurately when solving the problems and have self-reported confidence ratings that are higher than those of low-skill users. The expert system, however, has a greater impact on improving performance for low-skill users than for high-skill users. A relationship is found between skill level and task uncertainty indicating that different skill-level users require different presentation formats paralleling their conceptual representations of the problem. The interaction between skill level and knowledge organization is confirmed with results showing that low-skill users perform faster than high-skill users on questions requiring concrete knowledge organization; whereas high-skill users perform better when presented with questions requiring abstract knowledge organization.