Judgment Performance Research Articles

Across-task relations among monitoring judgments: Differential effects of item feedback on monitoring bias during reading

Using a practice testing paradigm, this study examined the effects of item feedback on learners' metacomprehension and monitoring judgments before, during, and after comprehension testing. Participants (N = 294) were randomly assigned to one of three conditions: a feedback, no feedback, and control condition. Participants completed an assessment of prior knowledge, read a 1000-word science text, and completed a measure of reading strategies. They also provided metacomprehension, item-level, and postdictive judgments of performance across comprehension testing. Measures of bias were calculated for each set of judgments. Results indicated that correctness feedback engendered less bias in learners' metacomprehension judgments but greater bias in item-level and postdictive judgments. Conditional and serial indirect effects were significant for those in the feedback condition. Findings provide evidence of positive but also differentiated effects of item feedback on students' monitoring bias. Unguided feedback may induce metacognitive bias during and after testing experiences in the context of text learning.

Performance, Momentary Judgments of Performance and Confidence on Metacognitive Wisconsin Card Sorting Test in Schizophrenia and Bipolar Disorder

Prior studies have reported that individuals with schizophrenia often lack awareness of their clinical symptoms and functional deficits. They also manifest response biases, with a trend towards overconfidence. It was previously found that overconfidence was associated with poorer performance on social cognitive and neurocognitive tests.

How creative am I?: Examining judgments and predictors of creative performance

Creativity is a highly valued construct in both educational and occupational contexts. Yet, little is known about the accuracy of individuals’ metacognitive judgments of their own creative performance. Moreover, the relative contribution of individual predictors of creative performance, such as gender, are still open for question. To some degree, this lack of understanding may stem from the challenge of collecting large data samples where creativity and metacognition can be measured and scored through rigorous approaches. The current study assessed college students’ (N = 350) creativity on the Similarities Test, the Remote Associates Test, and the Product Improvement Task from the Torrance Tests of Creative Thinking while having the students provide percentile rank judgments of their performance. In addition, data on personality and self-report grade point average were collected. Four major findings emerged. First, students were generally overconfident in their judgments of creative performance, however overconfidence was somewhat muted compared to findings from other domains. Second, response bias for judgments accounted for the majority of variance in predicting creative performance when simultaneously considering personality and grade point average, with greater performance associated with increased underconfidence. Third, females showed significantly higher performance for measures of fluency and originality on the Similarities Test and were less overconfident than their male counterparts on all three tests. Finally, openness emerged as the single personality variable that predicted creative performance, a finding that supported prior research. Implications related to the role of metacognitive judgments in creative performance is discussed.

A Validity Framework for Effective Analysis and Interpretation of Milestones Data.

A Validity Framework for Effective Analysis and Interpretation of Milestones Data.

Anaesthesiologists’ clinical judgment accuracy regarding preoperative full stomach: Diagnostic study in urgent surgical adult patients

BackgroundIn urgent situations, preoperative full stomach assessment mostly relies on clinical judgment. Our primary objective was to assess the diagnostic performance of clinical judgment for the preoperative assessment of full stomach in urgent patients compared to gastric point-of-care ultrasound (PoCUS). Our secondary objective was to identify risk factors associated with PoCUS full stomach in urgent patients. MethodsWe led a prospective observational study at our Hospital, between January and July 2016. Adult patients admitted for urgent surgery were eligible. Patients with altered gastric sonoanatomy, interventions reducing stomach content, impossible lateral decubitus were excluded. Clinical judgment and risk factors of full stomach were collected before gastric PoCUS measurements. Ultrasonographic full stomach was defined by solid contents or liquid volume ≥ 1.5 ml kg−1. Diagnostic performance was assessed through sensitivity, specificity, accuracy, positive and negative predictive value. ResultsThe prevalence of clinical and PoCUS full stomach in 196 included patients was 29% and 27%, respectively. Positive and negative predictive values were 42% (95% CI: 32.3–52.6%) and 79% (95% CI: 74.9–83.4%), respectively. Patients with PoCUS full stomach were clinically misdiagnosed in 55% of cases. PoCUS full stomach was associated with abdominal or gynaecological-obstetrical surgery (OR 3.6, 95% CI: 1.5–8.8, P < 0.01) but not with fasting durations. Positive solid intake after illness onset with respect to 6-h solid fasting rule was associated with PoCUS low-risk gastric content (OR 0.4, 95% CI: 0.2−0.9, P = 0.03). ConclusionsClinical judgment showed poor-to-moderate performance in urgent surgical patients for the diagnosis of full stomach. Gastric PoCUS should be used to assess risk of full stomach in this population.

Quantifying the potential value of antigen-detection rapid diagnostic tests for COVID-19: a modelling analysis

BackgroundTesting plays a critical role in treatment and prevention responses to the COVID-19 pandemic. Compared to nucleic acid tests (NATs), antigen-detection rapid diagnostic tests (Ag-RDTs) can be more accessible, but typically have lower sensitivity and specificity. By quantifying these trade-offs, we aimed to inform decisions about when an Ag-RDT would offer greater public health value than reliance on NAT.MethodsFollowing an expert consultation, we selected two use cases for analysis: rapid identification of people with COVID-19 amongst patients admitted with respiratory symptoms in a ‘hospital’ setting and early identification and isolation of people with mildly symptomatic COVID-19 in a ‘community’ setting. Using decision analysis, we evaluated the health system cost and health impact (deaths averted and infectious days isolated) of an Ag-RDT-led strategy, compared to a strategy based on NAT and clinical judgement. We adopted a broad range of values for ‘contextual’ parameters relevant to a range of settings, including the availability of NAT and the performance of clinical judgement. We performed a multivariate sensitivity analysis to all of these parameters.ResultsIn a hospital setting, an Ag-RDT-led strategy would avert more deaths than a NAT-based strategy, and at lower cost per death averted, when the sensitivity of clinical judgement is less than 90%, and when NAT results are available in time to inform clinical decision-making for less than 85% of patients. The use of an Ag-RDT is robustly supported in community settings, where it would avert more transmission at lower cost than relying on NAT alone, under a wide range of assumptions.ConclusionsDespite their imperfect sensitivity and specificity, Ag-RDTs have the potential to be simultaneously more impactful, and have a lower cost per death and infectious person-days averted, than current approaches to COVID-19 diagnostic testing.

Practices and Applications of Convolutional Neural Network-Based Computer Vision Systems in Animal Farming: A Review.

Convolutional neural network (CNN)-based computer vision systems have been increasingly applied in animal farming to improve animal management, but current knowledge, practices, limitations, and solutions of the applications remain to be expanded and explored. The objective of this study is to systematically review applications of CNN-based computer vision systems on animal farming in terms of the five deep learning computer vision tasks: image classification, object detection, semantic/instance segmentation, pose estimation, and tracking. Cattle, sheep/goats, pigs, and poultry were the major farm animal species of concern. In this research, preparations for system development, including camera settings, inclusion of variations for data recordings, choices of graphics processing units, image preprocessing, and data labeling were summarized. CNN architectures were reviewed based on the computer vision tasks in animal farming. Strategies of algorithm development included distribution of development data, data augmentation, hyperparameter tuning, and selection of evaluation metrics. Judgment of model performance and performance based on architectures were discussed. Besides practices in optimizing CNN-based computer vision systems, system applications were also organized based on year, country, animal species, and purposes. Finally, recommendations on future research were provided to develop and improve CNN-based computer vision systems for improved welfare, environment, engineering, genetics, and management of farm animals.

How Consumers Justify Their Unethical Behavior: The Role of Moral Recognition Strategies (Moral Rationalization, and Decoupling), Complemented by Culture, on the Purchase of Counterfeits in Pakistan

Moral recognition, is defined as a person’s description of a condition as ethical dilemma. However, people will not always interpret situations as ethical problems in the same way with the same force under all circumstances. The literature suggests that when two conditions of acceptance of ethics (moral rationalization and decoupling) are met, people will define the situation as a ethical problem differently. In the moral system, people use it to turn immoral acts into less immoral acts. Therefore, it allows them to violate ethical standards while maintaining a certain standard of conduct such as buying counterfeit goods because of their low prices. Here, consumers are more likely to make a profit by rearranging their actions for less ethical, which means seeking appropriate ethical reasons (including ethical justification, non-professional language, beneficial comparisons, migration of responsibilities, distribution of responsibility, distortion of results, prosecution; (Bandura et al., 1996), to coordinate adjustments and conditions, and to reach a judgment. Moral decoupling is defined as a psychological process chosen to prevent misconduct, in which one separates the judgment of performance from judgments of morality (Bhattacharjee et al., 2013). When people use this strategy, they focus on social benefits (e.g., image, use of status, etc.) as well as economic benefits (e.g., visual fashion content, physical appearance, performance, scarcity, etc. Chen et. al. (2018) linked these two strategies in an empirical assessment to explore the effect of dimensions of moral recognition (moral rationalization and moral decoupling) on counterfeit purchases mediated by moral judgment and perceived benefits respectively. We modified the Chen model with the complementary effect of materialistic culture in this relationship. We argue that moral recognition firstly affects counterfeit purchase (CP) through Moral rationalization, and then through moral judgment in a two-step mediation. Secondly, through moral decoupling and further through perceived benefits. Thirdly, directly affecting CP complemented by materialistic culture as a moderator. Empirical validity was established by conducting a survey employing a close-ended questionnaire. Data was collected from 230 consumers and analyzed using confirmatory factor analysis and structured equation modeling. The results suggested that moral recognition seems to limit purchase intention (PI) directly. Similarly, it neutralizes to Moral Rationalization (MR), and Decoupling (MD) behaviors. Moreover, MR tends to positively affect PI both directly, and well as indirectly through moral judgment (MJ). Similarly, MD also has a direct and positive effect on PI, as well as perceived benefits (PB), however, PB and PI relationship was not substantiated. Hence, MR seems to negatively affect PI through MD, as well as through MR and MJ as a first and second-order mediator. Lastly, materialism seems to promote the counterfeit purchase, at the same time positively complement the effect of MR on PI, in a way that MR would have a more pronounced effect on PI in case of the higher materialistic consumer.

Team member expectations of trainee communicator and collaborator competencies – so shines a good deed in a weary world?

Background Workplace-based assessment may be further optimized by drawing upon the perspectives of multiple assessors, including those outside the trainee’s discipline. Interprofessional competencies like communication and collaboration are often considered suitable for team input. Aim We sought to characterize multidisciplinary expectations of communicator and collaborator competency roles. Methods We adopted a constructivist grounded theory approach to explore perspectives of multidisciplinary team members on a clinical teaching unit. In semi-structured interviews, participants described expectations for competent collaboration and communication of trainees outside their own discipline. Data were analyzed to identify recurring themes, underlying concepts and their interactions using constant comparison. Results Three main underlying perspectives influenced interprofessional characterization of competent communication and collaboration: (1) general expectations of best practice; (2) specific expectations of supportive practice; and (3) perceived commitment to teaching practice. However, participants seemingly judged trainees outside their discipline according to how competencies were exercised to advance their own professional patient care decision-making, with minimal attention to the trainee’s specific skillset demonstrated. Conclusion While team members expressed commitment to supporting interprofessional competency development of trainees outside their discipline, service-oriented judgement of performance loomed large. The potential impact on the credibility of multidisciplinary sources for workplace-based assessment requires consideration.

Ecological momentary facial emotion recognition in psychotic disorders.

Cognitive tasks delivered during ecological momentary assessment (EMA) may elucidate the short-term dynamics and contextual influences on cognition and judgements of performance. This paper provides initial validation of a smartphone task of facial emotion recognition in serious mental illness. A total of 86 participants with psychotic disorders (non-affective and affective psychosis), aged 19-65, were administered in-lab 'gold standard' affect recognition, neurocognition, and symptom assessments. They subsequently completed 10 days of the mobile facial emotion recognition task, assessing both accuracy and self-assessed performance, along with concurrent EMA of psychotic symptoms and mood. Validation focused on task adherence and predictors of adherence, gold standard convergent validity, and symptom and diagnostic group variation. The mean rate of adherence to the task was 79%; no demographic or clinical variables predicted adherence. Convergent validity was observed with in-lab measures of facial emotion recognition, and no practice effects were observed on the mobile facial emotion recognition task. EMA reports of more severe voices, sadness, and paranoia were associated with worse performance, whereas mood more strongly associated with self-assessed performance. The mobile facial emotion recognition task was tolerated and demonstrated convergent validity with in-lab measures of the same construct. Social cognitive performance, and biased judgements previously shown to predict function, can be evaluated in real-time in naturalistic environments.

Demythologising A level Exam Standards

ABSTRACT There are two major myths concerning A level exam standards in England. First, the Ancient Myth, which insists that standards were norm-referenced until the 1980s, when they transitioned to being criterion-referenced. Second, the Modern Myth, which insists that standards transitioned again, during the 2010s, to being based upon the comparable outcomes principle. The present paper debunks these myths, arguing that: except for the occasional use of comparable outcomes to bridge qualification reforms, A level standards have always been attainment-referenced; and that this has always been operationalised using a combination of methods, including both examiner judgement of exam performances and statistical expectations of cohort attainment. The paper also argues that what has changed significantly is the degree of confidence that the exam industry has placed in examiner judgement relative to statistical expectations, which has waxed and waned over time. When statistical expectations have prevailed, pass rates have tended to plateau; somewhat implausibly. When examiner judgement has prevailed, pass rates have tended to rise; also somewhat implausibly. These trends have given a false impression of principled transitions, which the paper dispels.

Clinical judgment performance of undergraduate Nursing students

Objective:to evaluate the reported performance regarding clinical judgment by undergraduate Nursing students.Method:a cross-sectional study with the application of the Lasater Clinical Judgment Rubric-Brazilian Version in 166 undergraduate Nursing students from a Brazilian public university. The data were analyzed descriptively and analytically (by comparing the level of clinical judgment among students from the initial, intermediate, and concluding groups). The following tests were applied: Chi-square, Fisher’s Exact and Kruskal-Wallis, and a p-value of 0.05 was adopted. The reliability of the global instrument (Cronbach’s alpha) was 0.786.Results:of the 166 students, 65.7% evaluated themselves as proficient in relation to the reported performance on clinical judgment. Of the rubric’s 11 dimensions (focused observation, recognizing deviations from expected patterns, information seeking, prioritizing data, making sense of data, calm and confident manner, clear communication, well-planned intervention/flexibility, being skillful, evaluation/self-analysis, and commitment to improvement), only four groups did not present significant differences among them (p<0.05): focused observation, information seeking, prioritizing data, and calm and confident manner.Conclusion:the performance on clinical judgment reported as proficient was pointed out by 65.7% of the students and a significant statistical difference was verified in seven dimensions, among beginners, intermediate, and concluding students, compatible with the evolution of learning.

The Effect of Enforceability

In this paper, the author focuses on the effect of enforceability, in particular in relation to Austrian law. However, insights into German and European law are also provided. Enforceability is an effect of a judgment which is basically only granted to performance judgments. Declaratory and constitutive decisions (with the exception of the decision on costs) are not enforceable as such. As a result, the order for performance contained in the judgment can be enforced by state coercive measures. Enforceability occurs upon termination of the performance period. Enforceability is neither a consequence of, nor necessarily coincides with, res judicata. The introduction of the Brussels Ia Regulation has fundamentally changed the system of enforcement of foreign decisions. Decisions given in the EU Member State and enforceable in that State are now enforceable in another Member States without the need for a declaration of enforceability.

Combatting Linguistic Stereotyping and Prejudice by Evoking Stereotypes

AbstractThe article offers an account of two projects conducted at Örebro University and Umeå University, Sweden, which are aimed at raising awareness of issues related to linguistic stereotyping using matched-guise-inspired methods (Raising Awareness through Virtual Experiencing[RAVE] funded by the Swedish Research Council anda Cross-Cultural Perspective on Raising of Awareness through Virtual Experiencing(C-RAVE) funded by the Marcus and Amalia Wallenberg foundation). We provide an overview of the methods used in university courses, with the aim to raise awareness of how stereotyping can affect our perception. We also give a more detailed account of the findings from two case activities conducted in Sweden and the Seychelles. Here the response patterns indicate that the perceived gender of a voice as well as the accent (native vs non-native) do affect respondents’ judgements of performance. We were also able to show that discussions and reflections inspired by these response patterns led to raised self-awareness of matters related to language and stereotyping. The article then moves on to a critical query of our methods and also contextualizes our work in a broader discussion on methods and initiatives for how educational institutions actively can contribute to combatting (language) prejudice and discrimination in various ways.

Judging Older Adult Speakers: Only Young Adults Low in Ageism Notice the Nuances

Ageism negatively impacts hiring and electability success, as well as intergenerational relationships (Levy,2003;2009). The current study sought to examine whether personality cues influenced performance ratings of older adult (OA) speakers whose behavior had been modified by an embodiment intervention (i.e., “power posing”). One-hundred-and-three young adults (YA; Mage=19.6, SD=2.06; 49.5% women) rated the performance of 9 OA speakers performing 5-minute campaign speeches, and reported the cues that influenced their ratings. Two independent raters coded the cues (i.e., introversion and extroversion; coded by two independent raters, κ = .72 [moderate-to-substantial interrater reliability; Chen, 2019; McHugh, 2012]). Participants also completed the Refined-Aging Semantic Differential (Polizzi & Millikin, 2002) as a measure of ageism endorsement. Greater ageism was associated with lower performance ratings (F(1,101)=15.97, p<.001, R2=.14); performance was reduced by .12, 95%CIs[-.018,-.006] for each additional point of ageism endorsement. Next, we investigated whether personality cues would modify this relationship using Hayes PROCESS Model 1 (2018). A significant interaction emerged between ageism and introverted cues (b=.015, p=.05, 95%CIs[.006,.023]), suggesting that greater perceived introversion was negatively associated with performance ratings. Additionally, individuals lower in ageism were more likely to calibrate their performance judgments based on the pose condition of the speaker, with participants lower in ageism exhibiting greater variability across pose conditions (Ms=3.96-4.37), than individuals higher in ageism (Ms=4.21-4.31), suggesting that individuals lower in ageism were attending to nuances in the speech (e.g., pose influence), while higher ageist individuals relied more heavily on age-related judgments.

Metacognitive Training and Online Community College Students’ Learning Calibration and Performance

ABSTRACT Research suggested that community college students struggle to find time to commit to classes and often lack learning strategies to balance school amidst conflicting life needs. This study explored the use of metacognitive monitoring training in a Success and Study Skills class. Learners who participated in the training described the process of study skills selection by incorporating terms common to established self-regulation models. . Learning performance increased in both the control and experimental group. Still, relative and absolute calibration accuracy (measured as the difference between judgments of learning and actual performance) showed similar results in both the control and experimental group. These findings contradict prior research indicating that general training in metacognitive strategies, may lead to improvement in calibration.

Automatic Judgement of Online Video Watching: I Know Whether or Not You Watched

Videos have long been viewed through the free choice of customers, but in some cases currently, watching them is absolutely required, for example, in institutions, companies, and education, even if the viewers prefer otherwise. In such cases, the video provider wants to determine whether the viewer has honestly been watching, but the current video viewing judging system has many loopholes; thus, it is hard to distinguish between honest viewers and false viewers. Time interval different answer popup quiz (TIDAPQ) was developed to judge honest watching. In this study, TIDAPQ randomly inserts specially developed popup quizzes in the video. Viewers must solve time interval pass (RESULT 1) and individually different correct answers (RESULT 2) while they watch. Then, using these two factors, TIDAPQ immediately performs a comprehensive judgement on whether the viewer honestly watched the video. To measure the performance of TIDAPQ, 100 experimental subjects were recruited to participate in the model verification experiment. The judgement performance on normal watching was 93.31%, and the judgement performance on unusual watching was 85.71%. We hope this study will be useful in many areas where watching judgements are needed.

Preventing action slows down performance in perceptual judgment

Action and vision are known to be tightly coupled with each other. In a previous study, we found that repeatedly grasping an object without any visual feedback might result in a perceptual aftereffect when the object was visually presented in the context of a perceptual judgement task. In this study, we explored whether and how such an effect could be modulated by presenting the object behind a transparent barrier. Our conjecture was that if perceptual judgment relies, in part at least, on the same processes and representations as those involved in action, then one should expect to find a slowdown in judgment performance when the target object looks to be out of reach. And this was what we actually found. This indicates that not only acting upon an object but also being prevented from acting upon it can affect how the object is perceptually judged.

Tromboembolismo pulmonar agudo: adecuación del manejo de pacientes según las guías de práctica clínica en el servicio de urgencias

Background and aimsTo evaluate the frequency of acute pulmonary embolism, the use of clinical probability scores, and the appropriateness of the management of patients for whom computed tomography angiography (CTA) was requested from the emergency department for suspected acute pulmonary embolism. Materials and methodsThis was a retrospective observational study of CTA studies requested from the emergency department to rule out acute pulmonary embolism. We analyzed clinical variables and the explicit use of clinical probability scores. We determined the appropriateness of management according to the Wells Score and Geneva Score and the simplified versions of these two scores, calculated retrospectively. ResultsWe included 534 patients (52.8% women; mean age, 73 years). The frequency of acute pulmonary embolism was 23.0% and the Wells Score was explicitly used in 15.2%. The appropriateness of the management varied depending on the clinical probability score used to assess it (54.5%-75.8%) and on whether the standard D-dimer or age-adjusted D-dimer was used. ConclusionsThe failure to use the Wells Scores in all cases does not necessarily imply inappropriate management, and the performance of global clinical judgment can be similar to that of clinical probability scores; however, specific studies are necessary to confirm this hypothesis.

Escape Room Blueprint: Central Orientation Contagion Crisis

Background: Turnover rate for new graduate nurses at a midwestern community hospital was 30% for the first two years. Central clinical orientation for new hire registered nurses and care assistants was heavily executed via didactic format. Feedback from instructors suggested boredom and disengagement of learners during these sessions. Post central orientation evaluations resulted in 49% of attendees feeling not “very confident” to apply skills in clinical practice. Methods: Clinical educators created an escape room to improve learner satisfaction, interprofessional communication, and confidence in applying skills in practice. A convenience sample of twenty-three (23) registered nurses and care assistants participated in an escape room, Contagion Crisis, covering basic knowledge of infection control, emergency response, restraints, and drug calculation. Results: Average Likert scores on the post survey for satisfaction and communication were 4.67 and 4.0 respectively. Confidence in applying skills to practice averaged 4.72. Subjective feedback from participants included enjoyment of a fun, non-pressure environment. Conclusion: Multiple clinical disciplines can use this escape room, as it covers topics from central hospital orientation. Learners preferred the escape room format versus a high-fidelity simulation, due to it being less “high stakes” and not a judgement of individual performance.