Confidence Ratings Research Articles

Insights From the ISASS Webinar Series on Current and Emerging Techniques in Endoscopic Spine Surgery | Part 1: Polytomous Rasch Analysis of Surgeon Endorsement of Endoscopic Discectomy/Foraminotomy, Interbody Fusion, and Importance of Patient Feedback During Surgery.

The International Society for the Advancement of Spine Surgery hosted the first of a series of 4 webinars on endoscopic spine surgery techniques, focusing on end§oscopic discectomy, foraminotomy, instrumented endoscopic fusion, standalone lumbar interbody fusion with innovative materials, and the role of patient feedback in awake procedures. This series aims to share knowledge and discuss the complexities and clinical evidence of modern endoscopic spine surgery. To analyze the level of surgeon endorsement for the presented endoscopic spine surgery techniques before and after the webinar, utilizing polytomous Rasch analysis, and to evaluate the potential for these insights to inform clinical guideline recommendations. A survey was available to 1311 potential respondents during the Zoom webinar, collecting data on surgeon endorsements using a Likert scale. The polytomous Rasch model was employed to analyze responses, considering the complexity of decisions against surgeon expertise, developing a logarithmic measurement scale, allowing objective statistical analysis of categorical variables, highlighting incongruent or out-of-order items vs congruent and in-order items, and driving improvement in clinical guidelines. All 4 topics received higher confidence ratings demonstrated by descriptive statistics, highlighting the webinar's effective role in surgeon education and in identifying ongoing trends in spine surgery. The logarithmic transformation of these data during Rasch analysis showed noticeable shifts in surgeon confidence levels postwebinar, with increased endorsement for transforaminal full-endoscopic thoracolumbar interbody fusion for hard disc herniation and standalone endoscopic lumbar interbody fusion. The Wright plot and person-item map analyses demonstrated that the webinar effectively targeted areas of initial low confidence, significantly impacting surgeons' perceptions. Disordered endorsement thresholds remained in the topics of uniportal transforaminal discectomy/foraminotomy and patient feedback during endoscopic spine surgery, indicating issues in response category discrimination or confounding factors not captured by the survey. Ongoing controversies were highlighted by the influence of confounding factors, stemming from preconceived notions and limited familiarity with high-grade evidence. The first in the 4-part webinar series effectively shifted professional confidence and acceptance of innovative surgical approaches among spine surgeons. Observations indicated a high level of interest in applying the endoscopic surgery platform with other advanced technologies. The polytomous Rasch analysis provided nuanced insights into ongoing trends and areas in need of further clarification. Assessing surgeon confidence and acceptance of endoscopic spinal surgeries using polytomous Rasch analysis. Level 2 (inferential) and 3 (observational) evidence because Rasch analysis provides statistical validation of instruments rather than direct clinical outcomes.

Statewide Assessment of Confidence and Training Needs Among Educators Who Support Transition-Aged Students With Visual Impairments

Introduction: Students with visual impairments represent a low-incidence disability group with poor postschool outcomes. Educators who support these students at the transition-age level need access to high-quality training and resources. Method: I surveyed 87 special educators and teachers of students with visual impairments in a southeastern state to ask about their confidence in implementing 33 transition practices and their interest in training and resources on these same topics for youth with visual impairments. Results: Overall, teachers indicated moderate levels of confidence and moderate levels of interest in additional training across the 33 practices. There were some significant differences between teacher groups on ratings of confidence. Discussion: The findings of this study demonstrate the need for regular in-service opportunities for all teachers who support transition-aged youth with visual impairments to increase teacher confidence, as well as a stronger emphasis on transition in preservice programs. Findings of low interest in additional training suggest that more research is necessary to better understand the perceptions of teachers related to their roles in preparing youth in transition. Implications for Practitioners: This study should prompt educators to collaborate around transition for youth with visual impairments to ensure that knowledge that would improve student outcomes is shared, to encourage administrators to survey their staff members to understand which transition topics represent the most pressing need for ongoing training, and to prompt personnel preparation programs to place a stronger emphasis on transition education throughout their courses.

Temporal error monitoring: Does agency matter?

Error monitoring is the ability to report one's errors without relying on feedback. Although error monitoring is investigated mostly with choice tasks, recent studies have discovered that participants parametrically also keep track of the magnitude and direction of their temporal, spatial, and numerical judgment errors. We investigated whether temporal error monitoring relies on internal generative processes that lead to the to-be-judged first-order timing performance. We hypothesized that if the endogenous processes underlie temporal error monitoring, one can monitor timing errors in emitted but not observed timing behaviors. We conducted six experiments to test this hypothesis. The first two experiments showed that confidence ratings were negatively related to error magnitude only in emitted behaviors, but error directionality judgments of observed behaviors were more precise. Experiment 3 replicated these effects even after controlling for the motor aspects of first-order timing performance. The last three experiments demonstrated that belief of agency (i.e., believing that the error belongs to the self or someone else) was critical in accounting for the confidence rating effects observed in the first two experiments. The precision of error directionality judgments was higherin thenon-agency condition. These results show that confidence is sensitive to belief, and short-long judgment is sensitive to the actual agency of timing behavior (i.e., whether the behavior was emitted by the self or someone else).

A rate-them-all lineup procedure increases information but reduces discriminability.

Prior research has investigated ways to optimize identification performance, but an open question concerns exactly what variable should be optimized. One reasonable way to optimize performance is to maximize discriminability, which is achieved by increasing correct identifications of guilty suspects while simultaneously decreasing false identifications of innocent suspects. Another reasonable way to optimize performance is to maximize the information about the guilt or innocence of the suspect, which is best achieved by ensuring that a confidence rating is always made to the suspect. In a typical lineup, however, limited information about the suspect is obtained if the witness picks a filler or rejects the lineup. One proposed solution to that problem is to have the witness provide a confidence rating to every member of the lineup (a rate-them-all lineup). But what effect, if any, does a rate-them-all procedure have on discriminability? To answer that question, we compared a rate-them-all lineup procedure to standard simultaneous lineup and showup procedures using receiver operating characteristic analysis. In terms of discriminability, the rate-them-all procedure was diagnostically inferior to both. A reasonable goal for future research is to make use of theoretical models of eyewitness identification to simultaneously maximize both discriminability and information gain. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Speech Pathology and Occupational Therapy Students’ Perceptions and Experiences of an Interprofessional Simulation

Clinical placements and interprofessional practice for healthcare students are often met with anxiety, unpreparedness and a lack of confidence. Whilst interprofessional simulations may address these concerns, there are gaps in the literature. This study aimed to: (i) investigate the impact of interprofessional simulations on students’ perceived confidence, anxiety and preparedness for placements and their attitudes toward interprofessional practice and education, and, (ii) understand students’ experiences and satisfaction with the simulation. Pre-simulation and post-simulation survey results were collected from 25 speech pathology and 25 occupational therapy graduate-entry Masters degree students from one Australian university. Quantitative data was analysed using descriptive statistics and the Wilcoxon signed-rank test while qualitative data was reviewed through content analysis. Statistically significant (p<0.05) improvements across students’ confidence, anxiety and preparedness ratings were observed. Students’ attitudes towards interprofessional practice and education revealed statistically significant positive changes on 8/10 questions and they described the simulation to be a meaningful learning experience. Results indicate that following participation in an interprofessional simulation, students’ perceptions of their confidence, anxiety and preparedness for an adult inpatient hospital placement improved. Moreover, students’ attitudes towards interprofessional practice and education were more positive and the experience was reported to be beneficial. Future studies could explore timing-related considerations and simulation anxiety itself.

A comparison between numeric confidence ratings and verbal confidence statements.

Is confidence most diagnostic of accuracy when expressed in numbers or when expressed in words? This question bears immense importance in many real-world contexts especially within the confines of eyewitness identification. In an eyewitness identification task, we compared the diagnostic value of numeric confidence across rating scales that varied in grain size (3-point vs. 6-point vs. 21-point vs. 101-point rating scales). We also compared the diagnostic value of numeric confidence to verbal confidence statements using several machine-learning algorithms. We found that fine-grain ratings are more diagnostic of identification accuracy than coarse-grain ratings, which suggests that the former provides a closer correspondence to memory strength than the latter. Moreover, we found that verbal confidence statements capture diagnostic information about the likely accuracy of an identification that numeric confidence ratings do not capture. This suggests that verbal confidence statements and numeric confidence ratings reflect partially independent, nonoverlapping sources of information. These results shed light on the processes that provide diagnostic value to confidence. From an applied standpoint, these results suggest that verbal confidence statements and numeric confidence ratings ought to be collected from eyewitnesses after an identification decision. Collecting both captures more diagnostic information than either can capture in isolation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Psychometric Properties of the Saudi Version of the Mathematics Confidence Scale using the Modern Test Theory

The aim of the study was to analyze the psychometric properties of the Arabic version of the Math Confidence Rating Scale (MCS), using Item Response Theory (IRT) and Exploratory Factor Analysis (EFA). The participants were (315) from Imam Mouhammad Ibn Saud Islamic University; They are distributed into (92) female and (223) male, selected through a convenience sampling. The EFA models indicated that the one-dimensional structure better represents the data, and the pairs of items were considered locally independent, IRT findings indicate that all items had a very high discrimination parameter and all items had acceptable threshold parameters, And that MCS is more informative for mid-level ability of Math confidence. The MCS in Arabic has adequate psychometric properties to be used as a short measure of Math confidence.

MLA Research Training Institute (RTI) 2018 and 2019: participant research confidence and program effectiveness.

The article reports on an assessment of the effectiveness of the MLA Research Training Institute (RTI) for the years 2018 and 2019. The RTI is a year-long continuing education research methods training and support program for health sciences librarians. The study focuses on assessing RTI participants' research confidence after program completion and compares these results with their perceptions of workshop/program performance and learning outcomes. In addition, the authors discuss how the findings were applied to inform and improve the program. The study used a 26-item questionnaire, RTI Research Confidence Questionnaire, to gather information on participants' self-reported research confidence before the workshop, immediately after the workshop, and one year after the workshop to determine statistically significant differences. Differences in research confidence were identified by using three nonparametric statistical tests. Additional workshop and program surveys were used to corroborate the research confidence findings. Post-workshop and one-year-after-workshop research confidence ratings were significantly higher than pre-workshop levels for years 1 and 2. A comparison of median ratings between years 1 and 2 showed significant increases in research confidence for nine items in year 2. Participants' positive perceptions of workshop/program effectiveness and learning outcomes corroborated these findings. Overall assessment findings indicated that RTI training helped participants understand, use, and apply research skills to conduct research. Findings also revealed that participants' heightened research confidence persisted at least 12 months postintervention. The RTI Research Confidence Questionnaire proved effective for rigorously assessing and improving the RTI program. This study enhances the currently limited evidence on evidence-based approaches for assessing and improving research instruction for librarians.

Assessing Static Balance, Balance Confidence, and Fall Rate in Patients with Heart Failure and Preserved Ejection Fraction: A Comprehensive Analysis.

Chronic heart failure (CHF) is a complex clinical syndrome, associated with frailty, higher fall rates, and frequent hospitalizations. Heart Failure (HF) and preserved ejection fraction (HFpEF) is defined as a condition where a patient with HF have a diagnosis of left ventricular ejection fraction (LVEF) of ≥ 50%. The risk of HFpEF increases with age and is related to higher non-cardiovascular mortality. The aim of this study was to evaluate static balance and examine the effect of task difficulty on the discriminating power of balance control between patients with HFpEF (Patients with HFpEF) and their healthy controls. Moreover, the associations between static balance parameters, balance confidence, falls, lean muscle mass, and strength were assessed. Seventy two patients with HFpEF (mean age: 66.0 ± 11.6 years) and seventy two age- and gender-matched healthy individuals (mean age: 65.3 ± 9.5 years) participated in this study. Participants underwent a 30 s bilateral stance (BS) test and a 20 s Tandem-Romberg stance (TRS) on a force platform, evaluating the Range and Standard Deviation of Center of Pressure (COP) displacement parameters in both axes. Balance confidence was evaluated by the Activities-Specific Balance Confidence (ABC) Scale, and the number of falls during the last year was recorded. Lower limb strength was measured using an isokinetic dynamometer, isometric leg strength, and a Sit-to-Stand test. Bioelectrical impedance analysis was conducted to assess lean fat mass, lean fat mass index, and lean%. Patients with HFpEF presented with lower static balance in BS and TRS compared to healthy controls (p < 0.05), lower balance confidence by 21.5% (p < 0.05), and a higher incidence of falls by 72.9% (p < 0.05). BS was a better descriptor of the between-group difference. Furthermore, static balance, assessed in controlled lab conditions, was found to have little if no relationship to falls, strength, lean muscle mass, and balance confidence. Although no correlation was noted between the static balance parameters and falls, the fall rate was related to balance confidence, age, muscle strength, and lean fat.

Chronic stroke survivors underestimate their upper limb motor ability in a simple 2D motor task

BackgroundStroke survivors can exhibit a mismatch between the actual motor ability of their affected upper limb and how much they use it in daily life. The resulting non-use of the affected upper limb has a negative impact on participation in neurorehabilitation and functional independence. The factors leading to non-use of the affected upper limb are poorly understood. One possibility is that non-use comes about through inappropriately low confidence in their own upper limb motor abilities.ObjectiveWe asked whether chronic stroke survivors underestimate the motor ability of their affected upper limb.Methods20 chronic stroke survivors (Mean FM: 28.2 ± 10.5) completed a 2D reaching task using an exoskeleton robot. Target sizes were individually altered to ensure success rates were similar for both upper limbs. Prior to each reaching movement, participants rated their confidence about successfully hitting the target (estimated upper limb motor ability).ResultsConfidence ratings were significantly lower for the affected upper limb (estimated ability), even though it was equally successful in the reaching task in comparison to the less affected upper limb (actual ability). Furthermore, confidence ratings did not correlate with level of impairment.ConclusionsOur results demonstrate that chronic stroke survivors can underestimate the actual motor abilities of their affected upper limb, independent of impairment level. Low confidence in affected upper limb motor abilities should be considered as a therapeutic target to increase the incorporation of the affected upper limb into activities of daily living.

Prospective randomized evaluation of the sustained impact of assistive artificial intelligence on anesthetists’ ultrasound scanning for regional anesthesia

ObjectivesUltrasound-guided regional anesthesia (UGRA) relies on acquiring and interpreting an appropriate view of sonoanatomy. Artificial intelligence (AI) has the potential to aid this by applying a color overlay to key...

Observers Have Similar Improvements in Student Learning Outcomes as Participants in Athletic Training Simulation Education: A Pilot Study

Introduction: The use of standardized patient (SP) encounters in athletic training education programs have become more common; however, due to time and space constraints, it is possible that not all athletic training students will be able to participate and some must be observers. The objective of this study was to determine if observers experience similar improvements in student learning outcomes as participants for SP encounters. Methods: A pretest/posttest, non-randomized controlled trial was utilized. Nine athletic training students volunteered to participate in the research study. All participants completed a pretest survey including a knowledge assessment and confidence rating scale, followed by either a telehealth or in-person SP encounter of a lower extremity evaluation, and finished with a posttest survey. Results: Two separate repeated measures ANOVAs of the composite knowledge scores and confidence rating scale found significant differences between the pretest and posttest respectively (F = 14.117, p = 0.007, = 0.669; F = 88.439, p &lt; 0.001, = 0.927), with no significant difference between the participant or observer roles for either knowledge assessment (F = 0.213, p = 0.658, = 0.030) or confidence rating scale (F = 2.669, p = 0.146, = 0.276). Conclusion: Athletic training students demonstrated similar increases in knowledge and confidence as a result of telehealth and in-person SP encounters regardless of their role as an observer or participant. Athletic training educators should consider implementing SP encounters into their educational curriculums even if not all students are be able to be participants.

Addressing meniscal deficiency part 1: An umbrella review of systematic reviews and meta‐analyses on meniscal allograft transplantation

AbstractPurposeMeniscal injuries are common in the young and active population. There is increasing utilization of surgical interventions like meniscal allograft transplantation (MAT) to restore the protective function of menisci following injury leading to meniscal deficiency. Extensive research and publications exist on the management of meniscal injury and the sequalae of meniscal deficiency. However, a comprehensive synthesis of the existing evidence through an umbrella review is lacking. This study aims to fill this gap by providing a current examination of the literature on MAT.MethodsA comprehensive search was conducted in the MEDLINE, Embase and Scopus databases to identify relevant systematic reviews and meta‐analyses. Studies were screened based on predefined inclusion and exclusion criteria. The quality of the included studies was assessed using the AMSTAR‐2 tool.ResultsA total of 41 studies were included in the review, with most published within the last decade. The majority of studies (56.1%) received a ‘Critically Low’ confidence rating, 26.8% were rated as ‘Low’, and only 14.6% were rated as ‘High’ confidence. From the included studies, 51.2% reported on PROMs, with the Lysholm score being the most common. Transplant failure and reoperation rate were reported in 34.1% and 19.5% of studies respectively. Studies on MAT reported favourable short‐term outcomes in terms of patient‐reported outcome measures (PROMs) but were limited by the lack of randomized control trials and consistent comparison groups.ConclusionsThis umbrella review highlights an increase in interest in MAT but underscores the need for higher‐quality reviews with standardized reporting and rigorous methodologies. Future research should focus on long‐term outcomes, optimal surgical techniques, patient selection criteria and risk factors for transplant failure. There is also a need for more studies focusing on MAT in pediatric populations. Overall, this review provides a comprehensive assessment of the current state of research in MAT and identifies areas for improvement in future studies.Level of EvidenceLevel IV.

Training beginning therapists to respond to basic ethical situations in therapy: deliberate practice vs case discussion

ABSTRACT In this small, randomly controlled pilot study, we compared deliberate practice (DP) with case discussion (CD) for training beginning therapists to manage basic ethical situations. First-year graduate students were randomly assigned to a DP (n = 7) or CD (n = 8) workshop about ethics in therapy. After receiving didactic instructions, the DP group practiced responding to ethical scenarios with feedback and repetition while the CD group engaged in group discussions of illustrative cases. Outcomes included self-report skills, knowledge, and confidence and blind ethical performance ratings (EPRS) of pre- and post-video recorded performance on eight therapy simulations portraying basic yet challenging ethical situations. Both DP and CD improved self-report and performance-based outcomes for beginning trainees learning to handle ethical situations in therapy. However, the DP group performed better than the CD group in the multiple relationship scenarios. This small, pilot study provides initial evidence that DP may provide added benefits to traditional methods of ethical training with beginning therapists.

Toddlers' visual exploration during decisions predicts uncertainty monitoring 1 year later.

The longitudinal relation between toddlers' behaviors in uncertain situations (e.g., information seeking, hesitation) and preschoolers' uncertainty monitoring was investigated (between 2014 and 2019 in Northern California; Time 1: N = 183, M = 28.99 months, 53% female, 67.8% White; Time 2: N = 159, M = 41.64 months, 52.2% female). Eye movements and response latencies were recorded as children identified a target from two partially occluded (Time 1) or degraded (Time 2) images. Confidence ratings for identifications were collected at Time 2. At Time 1, gaze transitions between response options, but not response latencies and mental state language, predicted Time 2 uncertainty monitoring. Overall, these findings provide the first direct evidence of connections between toddlers' uncertainty behaviors and preschoolers' uncertainty monitoring.

Evaluation of the impact of artificial intelligence-assisted image interpretation on the diagnostic performance of clinicians in identifying pneumothoraces on plain chest X-ray: a multi-case multi-reader study

BackgroundArtificial intelligence (AI)-assisted image interpretation is a fast-developing area of clinical innovation. Most research to date has focused on the performance of AI-assisted algorithms in comparison with that of radiologists...

Do Confidence Ratings Reactively Modify Children's Academic Assessment Performance? Negative Answer from a Three-Year Longitudinal Study.

The reactivity effect of metacognitive judgments on first-order task performance has received increased research attention. Previous studies showed that soliciting retrospective confidence ratings (CRs) reactively enhances task performance itself, such as performance in decision making and reasoning tasks, especially for those with high self-confidence. It remains unknown whether CRs can improve students' academic assessment performance in real educational settings. The current study recruited 795 fourth-grade elementary school children to explore if making CRs reactively affects students' academic assessment performance in two main subjects (i.e., Chinese Language and Mathematics). The data were collected across six waves with half-year intervals. From Wave 2, children either provided (CR group) or did not provide CRs (no-CR group) when completing standardized academic assessments. The results showed Bayesian evidence supporting the claim that making CRs does not influence children's academic assessment performance (both the average performance across waves 2-6 and the performance in each wave) in both subjects. Furthermore, children's self-confidence did not moderate the reactive influence of CRs. The results from multilevel regression analyses re-confirmed the above conclusions. Possible explanations for the absence of the reactivity effect of CRs on children's academic assessment performance are discussed.

Reflector variables in augmented reality lineups: Assessing eyewitness identification reliability in children and adults with confidence, response time, and proximity to the lineup.

Attending to the behaviors of eyewitnesses at police lineups could help to determine whether an eyewitness identification is accurate or mistaken. Eyewitness identification decision processes were explored using augmented reality holograms. Children (n = 143; Mage = 10.79, SD = 1.12 years) and adults (n = 152; Mage = 22.12, SD = 7.47 years) viewed staged crime videos and made identification decisions from sequential lineups. The lineups were presented in augmented reality. Children were less accurate than adults on the lineup task. For adults, fast response times and high post-identification confidence ratings were both reflective of identification accuracy. Fast response times were also reflective of accuracy for children; however, children's confidence ratings did not reflect the likely accuracy of their identifications. A new additional measure, the witness' proximity to the augmented reality lineup, revealed that children who made mistaken identifications moved closer to the lineup than children who correctly identified the person from the crime video. Adults who moved any distance towards the lineup were less accurate than adults who did not move at all, but beyond that, adults' proximity to the lineup was not reflective of accuracy. The findings give further evidence that behavioral indicators of deliberation and information-seeking by eyewitnesses are signals of low lineup identification reliability. The findings also suggest that when assessing the reliability of children's lineup identifications, behavioral measures are more useful than metacognitive reports.

Deriving the functional form to fit confidence ratings in psychophysical experiments

Deriving the functional form to fit confidence ratings in psychophysical experiments

Lower confidence and increased error sensitivity in OCD patients while learning under volatility

A decoupling between confidence and action could relate to compulsive behaviour as seen in obsessive-compulsive disorder (OCD). The link between confidence and action in OCD has been investigated in clinical case-control studies and in the general population with discrepant findings. The generalizability of findings from highly-compulsive general population samples to clinical OCD samples has been questioned. Here, we investigate action-confidence coupling for 38 OCD patients compared to 37 healthy controls (HC), using a predictive inference task. We compared those results to a comparison between matched high and low compulsive individuals from the general population. Action-updating, confidence and their coupling were compared between the groups. Moreover, computational modeling was performed to compare groups on error sensitivity and environmental parameters. OCD patients showed lower confidence and higher learning rates in reaction to (small) prediction errors than HC, signaling hyperactive error signaling and lower confidence estimation. No evidence was found for differences in action-confidence coupling between groups. In contrast high the compulsive group showed higher confidence and stronger decoupling than the low compulsive group, both of which were related to symptoms. The underlying mechanisms of obsessive-compulsive behaviour might differ between clinical and highly-compulsive general population samples, resulting in different (meta)cognitive profiles.