Judgment Performance Research Articles

Prospective and Retrospective Metacognitive Judgments of Prospective Memory Performance Across the Lifespan

ABSTRACT Prospective Memory (PM), the ability to carry out future intentions at a specific time or event, develops in an inverted U-shaped function across the lifespan. Most studies have examined age-related changes in PM with regard to the role of executive functions. Although more recent studies have emphasized the importance of metacognition for successful prospective remembering, very little is known about the development of metacognitive abilities across the lifespan and their impact on PM performance. Therefore, the current study examined developmental differences in metacognitive monitoring and PM in five age groups in Germany (N = 151): children (9–10 years), adolescents (14–15 years), young adults (20–25 years), middle-aged adults (30–50 years) and old adults (60+ years). Metacognitive monitoring was assessed by asking participants to judge their performance before (prediction) and after (postdiction) working on a lab-based PM task (Virtual Week). PM performance increased from childhood into adolescence, remained stable until middle-adulthood and declined again in older adults. In terms of metacognitive monitoring, PM performance prediction was least accurate in children, and accuracy increased across the lifespan into old adulthood; interestingly, older adults’ more accurate metacognitive ability did not enable them to show better PM. Overall, individuals tended to overestimate their upcoming PM performance and to underestimate their past performance. Possible strategies to improve especially children and adolescents’ metacognitive skills and PM performance are discussed.

Exploring Self-Reported Quality of Life in Developmental Language Disorder

Purpose: Developmental language disorder (DLD) is a lifelong condition associated with poorer outcomes than neurotypical peers, yet relatively little is known about long-term quality of life in DLD. This preliminary study adopts a neurodiversity-informed approach by exploring self-reported quality of life in an adolescent and young adult DLD sample, as well as linguistic and risk factors contributing to quality of life. Method: Participants were five individuals with DLD aged 12–20 years ( M = 15.60, SD = 3.05). I administered two self-report quality-of-life scales, a language assessment, an experimental morphosyntax task, and measures of risk factors. Data were analyzed descriptively. Results: Participants generally reported positive views about their quality of life, although accessing accommodations and health services emerged as barriers. Relatively better grammaticality judgment performance appeared to be linked with poorer ratings of happiness and the ability to “be yourself.” Nonverbal ability represented a potential risk factor, although there may be a stronger cumulative role for risk factors. Conclusions: Participants with DLD reported relatively good quality of life. Exploratory findings suggest barriers to quality of life in some contexts, as well as roles for individual differences in language and risk factors. These descriptive findings should be examined in larger scale studies and may represent areas of consideration when clinicians address functional challenges that impact mental health and well-being in individuals with DLD. Supplemental Material: https://doi.org/10.23641/asha.27997529

The economic value of transport infrastructure in the UK: an input-output analysis

Transport infrastructure systems operate in, and are shaped by, the specific context in which they are expected to perform, contribute to the system-of-systems that support civilised life, and thy must strive to be sustainable and resilient. However, transport, one of the ‘economic infrastructures’, is viewed narrowly in political circles as a vehicle for economic prosperity: political focus falls on the economic pillar of sustainability. One perennial challenge, addressed herein, concerns the inclusion of social and environmental value into such performance judgements. A deep understanding of system interdependencies is essential when comparing input-output tables (a top-down approach) with cost-benefit analysis (CBA, a bottom-up approach). CBA works best when there is adequate information to calculate a monetary value for all economic, social and environmental outcomes, whereas input-output tables are most effective when environmental and social value are kept distinct from economic value, and where the beneficial economic impacts extend across multiple sectors (CBA is inadequate in this regard). This research uses the World Input-Output Database and principal component analysis (PCA) to develop a model to capture this complex accounting process. In recognising how past interdependencies inform future development, this study's model provides new insights into indirect economic value creation within infrastructure systems.

Verbal working memory and syntactic comprehension segregate into the dorsal and ventral streams, respectively.

Syntactic processing and verbal working memory are both essential components to sentence comprehension. Nonetheless, the separability of these systems in the brain remains unclear. To address this issue, we performed causal-inference analyses based on lesion and connectome network mapping using MRI and behavioural testing in two groups of individuals with chronic post-stroke aphasia. We employed a rhyme judgement task with heavy working memory load without articulatory confounds, controlling for the overall ability to match auditory words to pictures and to perform a metalinguistic rhyme judgement, isolating the effect of working memory load (103 individuals). We assessed non-canonical sentence comprehension, isolating syntactic processing by incorporating residual rhyme judgement performance as a covariate for working memory load (78 individuals). Voxel-based lesion analyses and structural connectome-based lesion symptom mapping controlling for total lesion volume were performed, with permutation testing to correct for multiple comparisons (4000 permutations). We observed that effects of working memory load localized to dorsal stream damage: posterior temporal-parietal lesions and frontal-parietal white matter disconnections. These effects were differentiated from syntactic comprehension deficits, which were primarily associated with ventral stream damage: lesions to temporal lobe and temporal-parietal white matter disconnections, particularly when incorporating the residual measure of working memory load as a covariate. Our results support the conclusion that working memory and syntactic processing are associated with distinct brain networks, largely loading onto dorsal and ventral streams, respectively.

Basis is also explanation: Interpretable Legal Judgment Reasoning prompted by multi-source knowledge

The task of Legal Judgment Prediction (LJP) aims to forecast case outcomes by analyzing fact descriptions, playing a pivotal role in enhancing judicial system efficiency and fairness. Existing LJP methods primarily focus on improving representations of fact descriptions to enhance judgment performance. However, these methods typically depend on the superficial case information and neglect the underlying legal basis, resulting in a lack of in-depth reasoning and interpretability in the judgment process of long-tail or confusing cases. Recognizing that the basis for judgments in real-world legal contexts encompasses both factual logic and related legal knowledge, we introduce the interpretable legal judgment reasoning framework with multi-source knowledge prompted. The essence of this framework is to transform the implicit factual logic of cases and external legal knowledge into explicit basis for judgment, aiming to enhance not only the accuracy of judgment predictions but also the interpretability of the reasoning process. Specifically, we design a chain prompt reasoning module that guides a large language model to elucidate factual logic basis through incremental reasoning, aligning the model prior knowledge with task-oriented knowledge in the process. To match the above fact-based information with legal knowledge basis, we propose a contrastive knowledge fusing module to inject external statutes knowledge into the fact description embedding. It pushes away the distance of similar knowledge in the semantic space during the encoding of external knowledge base without manual annotation, thus improving the judgment prediction performance of long-tail and confusing cases. Experimental results on two real datasets indicate that our framework significantly outperforms existing LJP baseline methods in accuracy and interpretability, achieving new state-of-the-art performance. In addition, tests on specially constructed long-tail and confusing case datasets demonstrate that the proposed framework possesses improved generalization abilities for predicting these complex cases.

Predicting and Explaining Assessment Center Judgments: A Cross‐Validated Behavioral Approach to Performance Judgments in Interpersonal Assessment Center Exercises

ABSTRACTAlthough Assessment Center (AC) role‐play assessments have received ample attention in past research, their reliance on actual behavioral information is still unclear. Uncovering the behavioral basis of AC role‐play assessments is, however, a prerequisite for the optimization of existing and the development of novel automated AC procedures. This work provides a first data‐driven benchmark for the behavioral prediction and explanation of AC performance judgments. We used machine learning models trained on behavioral cues (C = 36) to predict performance judgments in three interpersonal AC exercises from a real‐life high‐stakes AC (selection of medical students, N = 199). Three main findings emerged: First, behavioral prediction models showed substantial predictive performance and outperformed prediction models representing potential judgment biases. Comparisons with in‐sample results revealed overfitting of traditional approaches, highlighting the importance of out‐of‐sample evaluations. Second, we demonstrate that linear combinations of behavioral cues can be strong predictors of assessors' judgments. Third, we identified consistent exercise‐specific patterns of individual cues and cross‐exercise consistent behavioral patterns of behavioral dimensions and interpersonal strategies that were especially predictive of the assessors' judgments. We discuss implications for future research and practice.

Context counts: Unveiling the impact of achievement level on teachers’ text assessment

BackgroundTeachers' assessments of student writing proficiency are essential for adaptive instruction and providing feedback. However, studies have revealed a lack of objectivity in text quality ratings of the same text in different contexts, although texts of identical quality should receive identical teacher judgments. AimsOur two studies – one experimental and one more ecologically valid – address whether the achievement level impacts teachers’ individual text assessments even when relevant explicit criteria are used. SamplesIn Study 1, participants were 102 student teachers from a university in Germany. In Study 2, participants were 136 student teachers and 91 experienced teachers from Germany. MethodsIn Study 1, participants were randomly assigned to two groups. Student teachers in the high-achievement level group assessed three high-quality texts and one standard text with average quality. Student teachers in the low-achievement level group assessed three low-quality texts and the same standard text with average quality. Participants in Study 2 assessed ten texts randomly drawn from a large corpus. ResultsThe comparison of the standard text in Study 1 showed that student teachers in the high-level achievement group assessed it more negatively on a holistic scale and three analytic scales compared to the low-level achievement group. Multi-level data analysis in Study 2 revealed a negative influence of the quality of the achievement level on the text assessment for experienced and for student teachers. ConclusionsWe discuss how the achievement level can bias teachers’ judgments of specific student performances when concrete assessment criteria should be used.

Exploring impression management through eye-tracking: A study on the influence of photographs in financial reporting

The objective of financial reporting is to provide decision-useful information for a wide range of existing and potential stakeholders. However, the lack of regulation regarding the use of visual imagery may enable companies to engage in impression management and subtly influence the reader’s judgments through careful selection of images and how they obfuscate or clarify information. This study utilizes eye-tracking technology to experimentally examine how non-explanatory photographs influence the performance judgments of non-professional investors comprising business students and auditors. Participants were exposed to an excerpt of the management summary of a fictitious company’s annual report. The findings suggest while a non-explanatory photograph attracts attention in a text-image combination, the information search process and performance-related judgments are free of influences from the photograph. This implies other photographic attributes, such as triggering emotional influences or providing explanatory information, may be more relevant for impression management in influencing judgments over attentional influences.

How does the Dunning-Kruger effect happen in creativity? The creative self-concept matters

The Dunning-Kruger effect refers to the tendency of unskilled individuals to overestimate their abilities while skilled individuals tend to underestimate themselves. The purpose of this study is to investigate the existence of the Dunning-Kruger effect in creativity and identify its key determinants. Using the cue-utilization framework, we explored how inaccurate metacognitive judgments result from creative self-concept, compared to those who can accurately evaluate themselves. Two hundred and ten university students completed an Alternative Uses Task (AUT) and evaluated their performance with self- and comparative judgments, then they reported the creative self-concept. The study conducted a two-step clustering analysis based on individuals' creative performance, judgments, and Bias Index of metacognitive judgments, and identified four distinct groups: 'skilled but underestimate' (SU group), 'unskilled but aware' (UA group), 'skilled and aware' (SA group), and 'unskilled and overestimate' (UO group). The SU and UO groups were typical examples of the Dunning-Kruger effect. Using binary logistic regression analysis, we found that high creative self-concept enabled skilled individuals to assess their creativity accurately, while it caused unskilled individuals to overestimate their creativity. This study identified the Dunning-Kruger effect in creativity while also offering new insights into the underlying mechanism of the Dunning-Kruger effect.

Athletic enhancement andhuman nature.

There is a well-established asymmetry in our judgments of performance enhancing drugs (PEDs) in sports and in other competitive activities. When an athlete is found using such drugs, it is a scandal that prompts public outrage, fan disappointment, and even loss of title. It seems that we judge enhanced results cannot be genuinely attributed to athletes. There is no similar reaction to use of PEDs in art, science, music, literature, business, and other human endeavors. The question I tackle in this paper is whether this disanalogy is justified: Is there some underlying difference in virtue of which PEDs should be thus stigmatized in sports but not elsewhere? I survey a couple of potential justifications that I find lacking. I then consider the difference in our judgments of the participation of superman-like characters in sports (which we censure) and in other activities (which we endorse). I argue that the fact that the athlete is human is relevant to the value of sports-and by extension, the status of the effort involved-while this fact plays no significant role with regard to the value of other activitiesand that this difference in the value of activities ultimately justifies our differing judgments here. I then return to my initial question and examine whether similar appeal to what is human can justify the varying judgments of the use of PEDs. I argue that it can but only under certain assumptions. I conclude by discussing wider implication of my suggestion.

The influence of performance measurement systems and supervisor trustworthiness on knowledge sharing

The objective of our study is to examine the influence of combined performance measurement systems (PMS), which include both subjective evaluation and objective measures, on the willingness of employees to share knowledge. Additionally, our study investigates another variable which may influence subjective judgments of employee performance, namely supervisor trustworthiness. We argue that incorporating subjective evaluations into a performance measurement system will have little effect on knowledge-sharing behavior unless the supervisor is considered trustworthy enough to evaluate such behavior. Using a 2x2 between-subjects experiment design with 73 undergraduate students who act as employees as subjects, the result of this study shows that including subjective evaluation in a PMS encourages the willingness to share knowledge more than only using objective measures. This study also shows that supervisor trustworthiness alone can encourage a willingness to share knowledge. Furthermore, the combination of PMS which include subjective and objective measures and high supervisor trustworthiness has a significant impact on increasing the willingness to share knowledge.

Rhythmic priming of syntactic processing in Jabberwocky: a short-lived effect

ABSTRACT Converging evidence points towards a link between musical rhythm and linguistic syntax processing. Several potentially shared cognitive mechanisms and overlapping brain regions have been proposed to account for these findings. The present study explores the hypothesis that a domain-general cognitive system responsible for hierarchical structure building constitutes one such component. In two experiments, French-speaking adults listened to rhythmically regular, irregular or silent primes before completing a grammaticality judgement task on Jabberwocky sentences. Both experiments showed a priming effect only in the first three sentences after priming. Experiment 1 (block design) showed a disadvantage of the irregular condition compared to both other conditions. Experiment 2 (mixed design) showed an advantage in the regular condition compared to the irregular condition. Across the two experiments, grammaticality judgement performance correlated with rhythm discrimination. These findings provide evidence for a domain-general cognitive network responsible for hierarchical structure building in rhythm and language processing.

Large-Language-Model-Based AI Agent for Organic Semiconductor Device Research.

Large language models (LLMs) have attracted widespread attention recently, however, their application in specialized scientific fields still requires deep adaptation. Here, an artificial intelligence (AI) agent for organic field-effect transistors (OFETs) is designed by integrating the generative pre-trained transformer 4 (GPT-4)model with well-trained machine learning (ML) algorithms. It can efficiently extract the experimental parameters of OFETs from scientific literature and reshape them into a structured database, achieving precision and recall rates both exceeding 92%. Combined with well-trained ML models, this AI agent can further provide targeted guidance and suggestions for device design. With prompt engineering and human-in-loop strategies, the agent extracts sufficient information of 709 OFETs from 277 research articles across different publishers and gathers them into a standardized database containing more than 10000 device parameters. Using this database, a ML model based on Extreme Gradient Boosting is trained for device performance judgment. Combined with the interpretation of the high-precision model, the agent has provided a feasible optimization scheme that has tripled the charge transport properties of 2,6-diphenyldithieno[3,2-b:2',3'-d]thiophene OFETs. This work is an effective practice of LLMs in the field of organic optoelectronic devices and expands the research paradigm of organic optoelectronic materials and devices.

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions

IntroductionResearch in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts.MethodsWe address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students’ performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics.ResultsWe found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs.DiscussionInconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

Fraud risk judgments performance: The role of audit technology, task structure, and auditor competence

This study aims to examine the factors affecting auditor performance in fraud risk judgment, specifically audit technology, task structure, and auditor competence. It explores the interaction effects of audit technology, task structure, and auditor competence on fraud risk judgments performance. Employing a survey method and partial least square analysis responses from 59 state auditors at the Supreme Audit Agency (BPK) Representative Office in Southeast Sulawesi Province. The results indicate that audit technology, task structure, and auditor competence each have a positive and significant impact on fraud risk judgment performance. However, the combined interaction effect of audit technology, task structure, and auditor competence does not significantly influence fraud risk judgments performance. Given these findings, further investigation is warranted to enhance our understanding of these tasks and to refine audit technology specifications for improved fraud risk evaluation.

When Firms Continue Doing Business in Aggressor Nations: A Refined Moral Decoupling Perspective on Consumer Response

Russia's invasion of Ukraine prompted many Western companies to pull out of the aggressor country. This study examines why and when consumers purchase from companies that choose to sustain operations in Russia. This study advances a refined moral decoupling model where effects of transgression relevance on purchase intention are channeled not only through judgments of performance and morality, but additionally through judgment of social responsibility. Results of an empirical study with four German companies and a sample of German consumers indicate that the relevance of a company's transgression influences consumer purchase intention through judgments of social responsibility and morality. This remains stable regardless of individual differences in moral decoupling. However, judgments of performance mediate a transgression's effects on purchase intention only with individuals who are less prone to decoupling. Effects of a company's perceived social responsibility on purchase intention are weaker with the politically more liberal than with the more conservative, whereas effects of morality are enhanced by a more liberal political orientation and the strength of a person's moral identity. These findings aid researchers and practitioners by advancing a refined moral decoupling framework that has greater relevance in the context of political conflict and war.

The Effect of Biographical Characteristics on the Performance of Production Employees PT. Candi Jaya Amerta

Age, the number of dependents, and years of service make up the classification of biographical features, which are one of the elements that affect employee performance judgments. The purpose of this study is to look at the biographical factors that affect employee performance. All of the personnel of PT. Candi Jaya Amerta's production department served as the study's population, and the sample was drawn using a saturated sampling technique. A census, which samples the entire population, is another name for a saturated sample. 46 personnel of the production division made up the sample that the researchers used. Multiple linear regression analysis is a quantitative method that was employed in this investigation. According to the study's findings, age has no bearing on employee performance, the number of dependents has an impact, and tenure has no bearing on the output of employees from PT. Candi Jaya Amerta.

Enhancing Teachers' Job Performance through Objective Performance Appraisal and Motivation: A Review

This review paper examines the nexus between performance appraisal, motivation, and teachers' job performance, synthesizing insights from eleven relevant articles. Effective performance management processes are pivotal in facilitating transparent communication of expectations and outcomes between employers and employees. The significance of teachers' motivation in driving productivity underscores the importance of establishing objective assessment criteria for evaluating their job performance. The complexity of work performance judgments necessitates a multifaceted approach to assessment aimed at enhancing the credibility of the evaluation process. Furthermore, feedback from both immediate supervisors and colleagues emerges as a valuable tool in fostering the professional development of teachers. This paper sheds light on the interplay between performance appraisal, motivation, and teachers' job performance, offering valuable insights for educators, administrators, and policymakers. Keywords: Performance, appraisal, motivation, teachers, job performance, assessment, feedback

High-Noise Grayscale Image Denoising Using an Improved Median Filter for the Adaptive Selection of a Threshold

Grayscale image processing is a key research area in the field of computer vision and image analysis, where image quality and visualization effects may be seriously damaged by high-density salt and pepper noise. A traditional median filter for noise removal may result in poor detail reservation performance under strong noise and the judgment performance of different noise characteristics has strong dependence and rather weak robustness. In order to reduce the effects of high-density salt and pepper noise on image quality when processing high-noise grayscale images, an improved two-dimensional maximum Shannon entropy median filter (TSETMF) is proposed for the adaptive selection of a threshold to enhance the filter performance while stably and effectively retaining the details of the images. The framework of the proposed improved TSETMF algorithm is designed in detail. The noise in images is filtered by means of automatically partitioning a window size, the threshold value of which is adaptively calculated using two-dimensional maximum Shannon entropy. The theoretical model is verified and analyzed through comparative experiments using three kinds of classical grayscale images. The experimental results demonstrate that the proposed improved TSETMF algorithm exhibits better processing performance than that of the traditional filter, with a higher suppression of high-density noise and denoising stability. This stronger ability while processing high-density noise is demonstrated by a higher peak signal-to-noise ratio (PSNR) of 24.97 dB with a 95% noise density located in the classical Lena grayscale image. The better denoising stability, with a noise density from 5% to 95%, is demonstrated by the minor decline in the PSNR of approximately 10.78% relative to a PSNR of 23.10 dB located in the classical Cameraman grayscale image. Furthermore, it can be advanced to promote higher noise filtering and stability for processing high-density salt and pepper noise in grayscale images.

Frontal P300 asymmetry and congruence judgment: Retroactive switching is impaired during school day mornings in female adolescents

Investigating frontal EEG asymmetry as a possible biomarker of cognitive control abilities is especially important in ecological contexts such as school and work. We used a novel approach combining judgment performance and hemispheric frontal event-related potential (ERP) P300 asymmetry (fP3As) to evaluate aspects of cognitive control (i.e., repetition and switching) in adolescent females over a two-week ordinary school period. While undergoing electroencephalographic recording, students performed a word-colour “Stroop-like” congruence judgment task during morning and afternoon sessions, on Mondays and Wednesdays. Proportion of incongruence and congruence trials was 75% and 25%, respectively. ERP analysis revealed larger “novelty” right hemispheric fP3As amplitude for infrequent congruence but equivalent or significantly smaller than left hemispheric fP3As amplitude for frequent incongruence. RTs increased with extent of right fP3As shift. Behaviorally, repeat trial pairs (i.e., congruent followed by congruent, incongruent followed by incongruent) generally did not differ by time or day and were associated with near-ceiling accuracy. In contrast, switch trial pairs (i.e., congruent followed by incongruent, incongruent followed by congruent) in the afternoon were slower and associated with lower accuracy at the expected 75% criterion rate (i.e., judging incongruence by default), dropping significantly below 75% in the mornings. Crucially, compared to afternoon, morning fP3As patterns did not change adaptively with switch trial pairs. Although retroactive switching during congruence judgment was affected at all testing times, we conclude it was most impaired in the mornings of both early and mid school weeks, supporting misalignment between adolescent circadian cycle and school start time. We discuss some implications for optimal learning of adolescents at school.