Judgment Performance Research Articles

Context counts: Unveiling the impact of achievement level on teachers’ text assessment

BackgroundTeachers' assessments of student writing proficiency are essential for adaptive instruction and providing feedback. However, studies have revealed a lack of objectivity in text quality ratings of the same text in different contexts, although texts of identical quality should receive identical teacher judgments. AimsOur two studies – one experimental and one more ecologically valid – address whether the achievement level impacts teachers’ individual text assessments even when relevant explicit criteria are used. SamplesIn Study 1, participants were 102 student teachers from a university in Germany. In Study 2, participants were 136 student teachers and 91 experienced teachers from Germany. MethodsIn Study 1, participants were randomly assigned to two groups. Student teachers in the high-achievement level group assessed three high-quality texts and one standard text with average quality. Student teachers in the low-achievement level group assessed three low-quality texts and the same standard text with average quality. Participants in Study 2 assessed ten texts randomly drawn from a large corpus. ResultsThe comparison of the standard text in Study 1 showed that student teachers in the high-level achievement group assessed it more negatively on a holistic scale and three analytic scales compared to the low-level achievement group. Multi-level data analysis in Study 2 revealed a negative influence of the quality of the achievement level on the text assessment for experienced and for student teachers. ConclusionsWe discuss how the achievement level can bias teachers’ judgments of specific student performances when concrete assessment criteria should be used.

Exploring impression management through eye-tracking: A study on the influence of photographs in financial reporting

The objective of financial reporting is to provide decision-useful information for a wide range of existing and potential stakeholders. However, the lack of regulation regarding the use of visual imagery may enable companies to engage in impression management and subtly influence the reader’s judgments through careful selection of images and how they obfuscate or clarify information. This study utilizes eye-tracking technology to experimentally examine how non-explanatory photographs influence the performance judgments of non-professional investors comprising business students and auditors. Participants were exposed to an excerpt of the management summary of a fictitious company’s annual report. The findings suggest while a non-explanatory photograph attracts attention in a text-image combination, the information search process and performance-related judgments are free of influences from the photograph. This implies other photographic attributes, such as triggering emotional influences or providing explanatory information, may be more relevant for impression management in influencing judgments over attentional influences.

How does the Dunning-Kruger effect happen in creativity? The creative self-concept matters

The Dunning-Kruger effect refers to the tendency of unskilled individuals to overestimate their abilities while skilled individuals tend to underestimate themselves. The purpose of this study is to investigate the existence of the Dunning-Kruger effect in creativity and identify its key determinants. Using the cue-utilization framework, we explored how inaccurate metacognitive judgments result from creative self-concept, compared to those who can accurately evaluate themselves. Two hundred and ten university students completed an Alternative Uses Task (AUT) and evaluated their performance with self- and comparative judgments, then they reported the creative self-concept. The study conducted a two-step clustering analysis based on individuals' creative performance, judgments, and Bias Index of metacognitive judgments, and identified four distinct groups: 'skilled but underestimate' (SU group), 'unskilled but aware' (UA group), 'skilled and aware' (SA group), and 'unskilled and overestimate' (UO group). The SU and UO groups were typical examples of the Dunning-Kruger effect. Using binary logistic regression analysis, we found that high creative self-concept enabled skilled individuals to assess their creativity accurately, while it caused unskilled individuals to overestimate their creativity. This study identified the Dunning-Kruger effect in creativity while also offering new insights into the underlying mechanism of the Dunning-Kruger effect.

Athletic enhancement andhuman nature.

There is a well-established asymmetry in our judgments of performance enhancing drugs (PEDs) in sports and in other competitive activities. When an athlete is found using such drugs, it is a scandal that prompts public outrage, fan disappointment, and even loss of title. It seems that we judge enhanced results cannot be genuinely attributed to athletes. There is no similar reaction to use of PEDs in art, science, music, literature, business, and other human endeavors. The question I tackle in this paper is whether this disanalogy is justified: Is there some underlying difference in virtue of which PEDs should be thus stigmatized in sports but not elsewhere? I survey a couple of potential justifications that I find lacking. I then consider the difference in our judgments of the participation of superman-like characters in sports (which we censure) and in other activities (which we endorse). I argue that the fact that the athlete is human is relevant to the value of sports-and by extension, the status of the effort involved-while this fact plays no significant role with regard to the value of other activitiesand that this difference in the value of activities ultimately justifies our differing judgments here. I then return to my initial question and examine whether similar appeal to what is human can justify the varying judgments of the use of PEDs. I argue that it can but only under certain assumptions. I conclude by discussing wider implication of my suggestion.

The influence of performance measurement systems and supervisor trustworthiness on knowledge sharing

The objective of our study is to examine the influence of combined performance measurement systems (PMS), which include both subjective evaluation and objective measures, on the willingness of employees to share knowledge. Additionally, our study investigates another variable which may influence subjective judgments of employee performance, namely supervisor trustworthiness. We argue that incorporating subjective evaluations into a performance measurement system will have little effect on knowledge-sharing behavior unless the supervisor is considered trustworthy enough to evaluate such behavior. Using a 2x2 between-subjects experiment design with 73 undergraduate students who act as employees as subjects, the result of this study shows that including subjective evaluation in a PMS encourages the willingness to share knowledge more than only using objective measures. This study also shows that supervisor trustworthiness alone can encourage a willingness to share knowledge. Furthermore, the combination of PMS which include subjective and objective measures and high supervisor trustworthiness has a significant impact on increasing the willingness to share knowledge.

Rhythmic priming of syntactic processing in Jabberwocky: a short-lived effect

ABSTRACT Converging evidence points towards a link between musical rhythm and linguistic syntax processing. Several potentially shared cognitive mechanisms and overlapping brain regions have been proposed to account for these findings. The present study explores the hypothesis that a domain-general cognitive system responsible for hierarchical structure building constitutes one such component. In two experiments, French-speaking adults listened to rhythmically regular, irregular or silent primes before completing a grammaticality judgement task on Jabberwocky sentences. Both experiments showed a priming effect only in the first three sentences after priming. Experiment 1 (block design) showed a disadvantage of the irregular condition compared to both other conditions. Experiment 2 (mixed design) showed an advantage in the regular condition compared to the irregular condition. Across the two experiments, grammaticality judgement performance correlated with rhythm discrimination. These findings provide evidence for a domain-general cognitive network responsible for hierarchical structure building in rhythm and language processing.

Large-Language-Model-Based AI Agent for Organic Semiconductor Device Research.

Large language models (LLMs) have attracted widespread attention recently, however, their application in specialized scientific fields still requires deep adaptation. Here, an artificial intelligence (AI) agent for organic field-effect transistors (OFETs) is designed by integrating the generative pre-trained transformer 4 (GPT-4)model with well-trained machine learning (ML) algorithms. It can efficiently extract the experimental parameters of OFETs from scientific literature and reshape them into a structured database, achieving precision and recall rates both exceeding 92%. Combined with well-trained ML models, this AI agent can further provide targeted guidance and suggestions for device design. With prompt engineering and human-in-loop strategies, the agent extracts sufficient information of 709 OFETs from 277 research articles across different publishers and gathers them into a standardized database containing more than 10000 device parameters. Using this database, a ML model based on Extreme Gradient Boosting is trained for device performance judgment. Combined with the interpretation of the high-precision model, the agent has provided a feasible optimization scheme that has tripled the charge transport properties of 2,6-diphenyldithieno[3,2-b:2',3'-d]thiophene OFETs. This work is an effective practice of LLMs in the field of organic optoelectronic devices and expands the research paradigm of organic optoelectronic materials and devices.

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions.

Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students' performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

Fraud risk judgments performance: The role of audit technology, task structure, and auditor competence

This study aims to examine the factors affecting auditor performance in fraud risk judgment, specifically audit technology, task structure, and auditor competence. It explores the interaction effects of audit technology, task structure, and auditor competence on fraud risk judgments performance. Employing a survey method and partial least square analysis responses from 59 state auditors at the Supreme Audit Agency (BPK) Representative Office in Southeast Sulawesi Province. The results indicate that audit technology, task structure, and auditor competence each have a positive and significant impact on fraud risk judgment performance. However, the combined interaction effect of audit technology, task structure, and auditor competence does not significantly influence fraud risk judgments performance. Given these findings, further investigation is warranted to enhance our understanding of these tasks and to refine audit technology specifications for improved fraud risk evaluation.

When Firms Continue Doing Business in Aggressor Nations: A Refined Moral Decoupling Perspective on Consumer Response

Russia's invasion of Ukraine prompted many Western companies to pull out of the aggressor country. This study examines why and when consumers purchase from companies that choose to sustain operations in Russia. This study advances a refined moral decoupling model where effects of transgression relevance on purchase intention are channeled not only through judgments of performance and morality, but additionally through judgment of social responsibility. Results of an empirical study with four German companies and a sample of German consumers indicate that the relevance of a company's transgression influences consumer purchase intention through judgments of social responsibility and morality. This remains stable regardless of individual differences in moral decoupling. However, judgments of performance mediate a transgression's effects on purchase intention only with individuals who are less prone to decoupling. Effects of a company's perceived social responsibility on purchase intention are weaker with the politically more liberal than with the more conservative, whereas effects of morality are enhanced by a more liberal political orientation and the strength of a person's moral identity. These findings aid researchers and practitioners by advancing a refined moral decoupling framework that has greater relevance in the context of political conflict and war.

The Effect of Biographical Characteristics on the Performance of Production Employees PT. Candi Jaya Amerta

Age, the number of dependents, and years of service make up the classification of biographical features, which are one of the elements that affect employee performance judgments. The purpose of this study is to look at the biographical factors that affect employee performance. All of the personnel of PT. Candi Jaya Amerta's production department served as the study's population, and the sample was drawn using a saturated sampling technique. A census, which samples the entire population, is another name for a saturated sample. 46 personnel of the production division made up the sample that the researchers used. Multiple linear regression analysis is a quantitative method that was employed in this investigation. According to the study's findings, age has no bearing on employee performance, the number of dependents has an impact, and tenure has no bearing on the output of employees from PT. Candi Jaya Amerta.

Enhancing Teachers' Job Performance through Objective Performance Appraisal and Motivation: A Review

This review paper examines the nexus between performance appraisal, motivation, and teachers' job performance, synthesizing insights from eleven relevant articles. Effective performance management processes are pivotal in facilitating transparent communication of expectations and outcomes between employers and employees. The significance of teachers' motivation in driving productivity underscores the importance of establishing objective assessment criteria for evaluating their job performance. The complexity of work performance judgments necessitates a multifaceted approach to assessment aimed at enhancing the credibility of the evaluation process. Furthermore, feedback from both immediate supervisors and colleagues emerges as a valuable tool in fostering the professional development of teachers. This paper sheds light on the interplay between performance appraisal, motivation, and teachers' job performance, offering valuable insights for educators, administrators, and policymakers. Keywords: Performance, appraisal, motivation, teachers, job performance, assessment, feedback

High-Noise Grayscale Image Denoising Using an Improved Median Filter for the Adaptive Selection of a Threshold

Grayscale image processing is a key research area in the field of computer vision and image analysis, where image quality and visualization effects may be seriously damaged by high-density salt and pepper noise. A traditional median filter for noise removal may result in poor detail reservation performance under strong noise and the judgment performance of different noise characteristics has strong dependence and rather weak robustness. In order to reduce the effects of high-density salt and pepper noise on image quality when processing high-noise grayscale images, an improved two-dimensional maximum Shannon entropy median filter (TSETMF) is proposed for the adaptive selection of a threshold to enhance the filter performance while stably and effectively retaining the details of the images. The framework of the proposed improved TSETMF algorithm is designed in detail. The noise in images is filtered by means of automatically partitioning a window size, the threshold value of which is adaptively calculated using two-dimensional maximum Shannon entropy. The theoretical model is verified and analyzed through comparative experiments using three kinds of classical grayscale images. The experimental results demonstrate that the proposed improved TSETMF algorithm exhibits better processing performance than that of the traditional filter, with a higher suppression of high-density noise and denoising stability. This stronger ability while processing high-density noise is demonstrated by a higher peak signal-to-noise ratio (PSNR) of 24.97 dB with a 95% noise density located in the classical Lena grayscale image. The better denoising stability, with a noise density from 5% to 95%, is demonstrated by the minor decline in the PSNR of approximately 10.78% relative to a PSNR of 23.10 dB located in the classical Cameraman grayscale image. Furthermore, it can be advanced to promote higher noise filtering and stability for processing high-density salt and pepper noise in grayscale images.

Frontal P300 asymmetry and congruence judgment: Retroactive switching is impaired during school day mornings in female adolescents

Investigating frontal EEG asymmetry as a possible biomarker of cognitive control abilities is especially important in ecological contexts such as school and work. We used a novel approach combining judgment performance and hemispheric frontal event-related potential (ERP) P300 asymmetry (fP3As) to evaluate aspects of cognitive control (i.e., repetition and switching) in adolescent females over a two-week ordinary school period. While undergoing electroencephalographic recording, students performed a word-colour “Stroop-like” congruence judgment task during morning and afternoon sessions, on Mondays and Wednesdays. Proportion of incongruence and congruence trials was 75% and 25%, respectively. ERP analysis revealed larger “novelty” right hemispheric fP3As amplitude for infrequent congruence but equivalent or significantly smaller than left hemispheric fP3As amplitude for frequent incongruence. RTs increased with extent of right fP3As shift. Behaviorally, repeat trial pairs (i.e., congruent followed by congruent, incongruent followed by incongruent) generally did not differ by time or day and were associated with near-ceiling accuracy. In contrast, switch trial pairs (i.e., congruent followed by incongruent, incongruent followed by congruent) in the afternoon were slower and associated with lower accuracy at the expected 75% criterion rate (i.e., judging incongruence by default), dropping significantly below 75% in the mornings. Crucially, compared to afternoon, morning fP3As patterns did not change adaptively with switch trial pairs. Although retroactive switching during congruence judgment was affected at all testing times, we conclude it was most impaired in the mornings of both early and mid school weeks, supporting misalignment between adolescent circadian cycle and school start time. We discuss some implications for optimal learning of adolescents at school.

The Role of English Language and Literature in the Cultivation of Students’ Language Proficiency in the Context of Deep Learning

Abstract In this paper, the PSO-LSTM algorithm is used to construct a model for evaluating students’ language proficiency, and the improvement of the acceleration factor and inertia weights is used to adjust the influence of the individual optimal position and the global optimal position in the speed update. For model performance judgment after optimization of LSTM hyperparameters, the PSO fitness function is replaced by MAPE and MSE functions. Finally, the student’s English composition texts were used as experimental parameters and input into the language proficiency evaluation model to verify the model performance and analyze the influence of English language literature on language proficiency. The results show that English language literature is effective in predicting 18.55% of the variance in English proficiency level for self-efficacy, 8.17% of the variance in English proficiency level for language attitude, and 0.47% of the variance in English proficiency level for learning anxiety, which indicates that English language literature has a significant effect on students’ English proficiency. This paper investigates the role mechanism of the English language and literature in the cultivation of student’s English proficiency, which provides a theoretical reference for the reform of English teaching in colleges and universities.

Judgments of learning in bilinguals: Does studying in a L2 hinder learning monitoring?

Nowadays, use of a second language (L2) has taken a central role in daily activities. There are numerous contexts in which people have to process information, acquire new knowledge, or make decisions via a second language. For example, in academia and higher education, English is commonly used as the language of instruction and communication even though English might not be students' native or first language (L1) and they might not be proficient in it. Such students may face different challenges when studying and learning in L2 relative to contexts in which they study and learn in their L1, and this may affect their metamemory strategies. However, little is yet known about whether metamemory processes undergo significant changes when learning is carried out in L2. The aim of the present study was to investigate the possible consequences on learning derived from studying materials in L2 and, more specifically, to explore whether the interplay between monitoring and control (metamemory processes) changes as a function of the language involved. In three experiments, we explored whether font type (Experiment 1), concreteness (Experiment 2), and relatedness (Experiment 3) affected judgments of learning (JOLs) and memory performance in both L1 and L2. JOLs are considered the result of metacognitive strategies involved in the monitoring of learning and have been reported to vary with the difficulty of the material. The results of this study showed that people were able to monitor their learning in both L1 and L2, even though they judged L2 learning as more difficult than L1. Interestingly, self-perceived difficulty did not hinder learning, and people recognized L2 materials as well or better than L1 materials. We suggest that this might be an example of a desirable difficulty for memory.

Judgments of learning reflect the encoding of contexts, not items: evidence from a test of recognition exclusion

ABSTRACT Two sources of evidence seem to be shared by judgments of past recognition and judgments of future performance: item memory or familiarity (i.e., memory for the item independent of the context in which it was experienced) and context memory or recollection (i.e., memory for the context specific to a particular prior encounter). However, there are few studies investigating the link between these two putative memory processes and judgments of learning (JOLs). We tested memory and metamemory using a continuous exclusion procedure – a modified recognition memory task where study events for two classes of items are interleaved with test trials in which the subject must endorse items from one class and reject items from the other. This procedure allowed us to estimate the influences of memory for context and memory for item on JOLs and licenses conclusions about the relative role of item and context information in supporting JOLs. An analysis of forgetting revealed that JOLs reflect both the initial degree of learning and the rate of forgetting, but only of memory for context and not of memory for items. These findings suggest that JOLs are predictive of memory for context-bound episodes, rather than for the semantic content of those episodes.

Immediate post performance judgements about cognitive performance in schizophrenia and bipolar disorder: associations with test performance and subjective overall judgments regarding abilities

ABSTRACT Introduction: The study explored associations between the accuracy of post assessment judgements of cognitive performance with global self-assessments of psychosocial functioning compared to evaluations generated by observers in schizophrenia and bipolar disorder. Methods: An abbreviated cognitive assessment based on the MATRICS Consensus Cognitive Battery was administered to 122 individuals with schizophrenia and 113 with bipolar disorder. They provided self-estimates of their performance after each subtest. In addition, self-reports on cognition, social cognition, and everyday functioning were collected and compared to observer ratings. Results: Both groups overestimated their cognitive function, but in bipolar disorder, there was 30% shared variance between task performance and self-rated task performance (vs. 5% in schizophrenia). Significant correlations were found between self-reported everyday outcomes and both actual and self-assessed performance. In schizophrenia, immediate judgements were only related to self-rated functioning, not to observer rated functioning. In bipolar disorder, impairments in self-assessment of performance correlated with observer ratings of cognitive ability, which was not observed in schizophrenia. Conclusions: While both groups showed correlations between cognitive performance and introspective accuracy, individuals with bipolar disorder showed higher accuracy in assessing their cognitive performance and other outcomes. Notably, impairments in introspective accuracy were associated with observer-rated functioning exclusively in bipolar disorder.

Anger for good? Unethical‐behavior‐targeted leader anger expression and its consequences on team outcomes

SummaryAlthough leader anger expression targeted at employees' unethical behavior is pervasive in the workplace, we still know little about its theoretical meaning and consequences. To address this theoretical blind spot, we drew on fairness heuristic theory to investigate whether, how, and when unethical‐behavior‐targeted (UB‐targeted) leader anger expression affects team outcomes. Our findings from two time‐lagged field studies suggest that a punishment‐based distributive justice climate mediates the positive effects of UB‐targeted leader anger expression on team organizational citizenship behavior (OCB) and team viability. Moreover, leader moral decoupling weakens these indirect relationships. Specifically, the indirect relationships are weaker when the leader separated judgments of performance from those of ethics. These findings highlight the importance of a fairness perspective in understanding the consequences of leader anger expression targeted at unethical behaviors.

A window into your status: Environment-based social class’s effect on virtual leadership

In the wake of the COVID-19 pandemic, videoconferencing has become a prominent aspect of many daily work routines. Despite the benefits of this development, scholars have argued that research should consider how trends that emerged during the pandemic may exacerbate pre-existing inequalities. As such, we draw upon evolutionary signaling theory to examine how videoconferencing technology may inadvertently signal one’s social class position, affect judgments of informal leadership and follower performance, and become mechanisms of inequality in the workforce. Through an initial field study (Study 1), our results suggest that people’s home working environments feature various signals that correspond with their social class position. Our subsequent experimental studies (Studies 2 and 3) and meta-analytic summary show that once a window is opened into an employee's home environment, it can lead others to form inferences about their competence, which affects the likelihood that they emerge as a leader and followers’ task performance. Extending evolutionary signaling theory, results show that participants were influenced by the environment-based social class signals even when they knew these were fake, virtual backgrounds. Thus, these subtle environment-based social class signals reflect novel pathways to inequality that likely are relevant as virtual leadership and videoconferencing become part of the “new normal”.