Comparative Judgments Research Articles

A thermal energy analysis of binary (Go-Co/H2O) and ternary (Go-Co-Zro2/H2O) nanofluids based on characterization and thermal performance

Hybrid or binary nanofluids have superior mechanical and thermal characteristics but the tri-hybrid nanofluids comprise of more embellished thermal properties, better physical strength, and enhanced stability. The present work characterizes the thermal and physical aspects of the hybrid and tri-hybrid nanofluids. The nano-composition of graphene oxide ( Go) and cobalt ( Co) is used in the amalgamation of hybrid nanofluid Go-Co/H2O, whereas the addition of zirconium oxide ( ZrO2) in this mixture gives rise to the ternary Go-Co-ZrO2/H2O hybrid nanofluid. The activation energy and viscous dissipation terms are also amended in the governing equations. The mathematical framework consists of a complex natured dynamical system. However, a numerical algorithm based on finite-difference discretization is developed which can solve the system numerically via the MATLAB software. A comparison with the existing literature is provided in order to validate the numerical procedure. From the outcomes, it is noticed that the temperature of hybrid as well as tri-hybrid nanofuid increases rapidly with change in concentration of zirconium oxide and cobalt. Temperature increases up to 20% by taking 0.1 volume fraction of both zirconium oxide and cobalt. Porous medium and activation energy resist the flow and concentration respectively. A comparative judgment evidently reveals that tri-hybrid Go-Co-ZrO2/H2O nanofluid has a substantial effect on temperature as equated to hybrid or pure nanofluid.

A new method to prioritize the QFDs’ engineering characteristics inspired by the Law of Comparative Judgment

The primary module of Quality Function Deployment (QFD) is the House of Quality (HoQ), which supports the design of new products and services by translating customer requirements (CRs) into engineering characteristics (ECs). Within the HoQ framework, the traditional technique for prioritizing ECs is the independent scoring method (ISM), which aggregates the weights of the CRs and the relationships between CRs and ECs (i.e., null, weak, medium, and high) through a weighted sum. However, ISM incorporates two questionable operations: (i) an arbitrary numerical conversion of the relationships between CRs and ECs, and (ii) the “promotion” of these relationships from ordinal to cardinal scale. To address these conceptual shortcomings, this paper introduces a novel procedure for prioritizing ECs, inspired by the Thurstone’s Law of Comparative Judgment (LCJ). This procedure offers a solution that is conceptually sound and practical, overcoming the conceptual shortcomings of ISM, while maintaining its simplicity, flexibility, and ease of implementation. The proposed approach is supported by a realistic application example illustrating its potential.

Factors that determine design similarity

AbstractIn recent years, the importance of design has been pointed out as a source of competitive advantage. However, creating a great design also increases the risk of copy products being created. In this study, we used an approach based on visual information and conducted an international comparative judgment survey of elements considered to be similar within products.

Enterprise Credit Risk Decision: Application Based on Improved AHP

The credit risk of Shouguang vegetable enterprises in China is the biggest obstacle to corporate loans. Building a credit risk assessment model for Shouguang vegetable enterprises and accurately rating the risk of loan enterprises is the key to successful loans. This article aims to construct an AHP evaluation model for the credit risk of Shouguang vegetable enterprises. The data is sourced from managers, bank credit personnel, university researchers, and enterprise related customers who are familiar with the enterprise, while considering four risk levels: impact degree(I), occurrence probability(P), risk manageability(M), and government support(S). This article uses AHP and risk index scores to evaluate the credit risk of Shouguang vegetable enterprises. This model calculates the risk index score based on survey data from 41 corporate credit risk professionals, constructs a pairwise comparison judgment matrix, and conducts consistency testing. It calculates the risk level membership vectors of impact degree, occurrence probability, risk manageability, and government support level at four risk levels, and then calculates the comprehensive evaluation membership vector of credit risk for Shouguang vegetable enterprise. The calculation results show that the comprehensive credit risk assessment level of Shouguang vegetable enterprise belongs to the general risk level, with a membership value of 0.5836. The results still show that the credit rating of Shouguang vegetable enterprises in the four risk levels of impact degree, occurrence probability, risk manageability, and government support are all average risk levels, but there are differences in membership values. The maximum membership value under the impact degree level is 0.6163, and the minimum membership value under the risk manageability level is 0.5572. This study provides a feasible and practical model for enterprise credit risk assessment and conducts a detailed evaluation of the credit risk of Shouguang vegetable enterprise, providing valuable reference for enterprise managers, bank credit personnel, and related researchers.

Testing the Relationship of Linguistic Complexity to Second Language Learners’ Comparative Judgment on Text Difficulty

AbstractThis study examined the relationship of linguistic complexity, captured using a set of lexical richness, syntactic complexity, and discoursal complexity indices, to second language (L2) learners’ perception of text difficulty, captured using L2 raters’ comparative judgment on text comprehensibility and reading speed. Testing materials were 180 texts abridged from college English coursebooks, and raters were 90 advanced Chinese learners of L2 English. Forty‐five raters read paired texts and determined which text was harder to understand in each pair, and another 45 raters read paired texts and determined which text they read faster in each pair. Two stepwise linear regression models containing lexical, syntactic, and discoursal features explained 48.1% and 54.6% of the variance in L2 learners’ estimates of text comprehensibility and reading speed, respectively, outperforming four commonly used language readability models. These findings contribute useful insights into the relationship between linguistic complexity and L2 learners’ perception of text difficulty.

Order effects in stimulus discrimination challenge established models of comparative judgement: A meta-analytic review of the Type B effect.

This paper provides a comprehensive review of the TypeB effect (TBE), a phenomenon reflected in the observation that discrimination sensitivity varies with the order of stimuli in comparative judgment tasks, such as the two-alternative forced-choice (2AFC) paradigm. Specifically, when the difference threshold is lower (higher) with the constant standard preceding rather than following the variable comparison, one speaks of a negative (positive) TBE. Importantly, prominent psychophysical difference models such as signal detection theory (Green & Swets, 1966) cannot easily account for the TBE, and are hence challenged by it. The present meta-analysis provides substantial evidence for the TBE across various stimulus attributes, suggesting that the TBE is a general feature of discrimination experiments when standard and comparison are presented successively. Thus, inconsistent with psychophysical difference models, subjective differences between stimuli are not merely a function of their physical differences but rather also depend on their temporal order. From the literature, we identify four classes of potential candidate theories explaining the origin of the TBE, namely (1) differential weighting of the stimulus magnitudes at the two positions (e.g., Hellström, Psychological Research, 39, 345-388 1977), (2) internal reference formation (e.g., Dyjas, Bausenhart, & Ulrich, Attention, Perception, & Psychophysics, 74, 1819-1841 2012), (3) Bayesian updating (e.g., de Jong, Akyürek, & van Rijn, Psychonomic Bulletin and Review, 28, 1183-1190 2021), and (4) biased threshold estimation (García-Pérez & Alcalá-Quintana, Attention, Perception & Psychophysics, 72, 1155-1178 2010). As these models, to some extent, make differential predictions about the direction of the TBE, investigating the respective boundary conditions of positive and negative TBEs might be a valuable perspective for diagnostic future research.

A multilevel investigation of factors related to achievement in Ireland and Spain using PISA data

The Program for International Assessment (PISA) is a methodology for making comparative judgments about the quality of education systems. Celebrated by proponents as a transparent process that allows policy makers produce data informed judgments about relative quality of their national education system PISA – and through it the OECD – has become a key vehicle for informing and explaining educational policy development. This paper explores the Irish and Spanish outcomes of the 2018 round of PISA. It examines the contextual factors that are associated with performance at student and school level while at the same time developing a multi-level statistical model to explain divergent school performance profiles It finds that issues associated with the socio-economic level of the students, the repetition rate, and student age are common across all domains in both countries. It suggests that the socio-economic status of Spanish students at school level is not significant, that the shortage of teachers in Ireland affects student performance, and that immigrant status does not disadvantage Spanish student performance. It concludes by suggesting that studies involving a wider application of the model be undertaken to assess possible social, economic, and cultural causes that may explain the differences in variable significance in each country.

Moderation of non-exam assessments: a novel approach using comparative judgement

ABSTRACT In England, some secondary-level qualifications comprise non-exam assessments which need to undergo moderation before grading. Currently, moderation is conducted at centre (school) level. This raises challenges for maintaining the standard across centres. Recent technological advances enable novel moderation methods that are no longer bound by centre. This study used simulation to investigate the feasibility of using comparative judgement (CJ) for moderating non-exam assessments. Our study explored the effects of CJ design parameters on the CJ estimates of script quality and how to assign moderator marks after the CJ procedure. The findings showed that certain design parameters had substantial effects on reliability and suggested minimum values for CJ protocols. The method used for assigning moderator marks maintained the rank order of scripts within centres and calibrated the centres to a common standard. Using CJ for moderation could transform current assessment practices, taking advantage of technological developments and ensuring reliability and fairness.

Who’s better? Adaptive comparative judgment of dance performances

Introduction
 Adaptive Comparative Judgment (ACJ) is a promising digital assessment method that allows measurement of performance or competencies by repeated comparisons of two items. Whereas ACJ is becoming a popular assessment method in educational measurement there are no such studies or published use cases in the context of sport or physical education-related teacher education (Bartholomew & Jones, 2022). To address this research gap, an explorative and comparative study was conducted to investigate whether ACJ offers an advantage over the traditional form of criteria-oriented scoring (TA) in the evaluation of students' dance performances.
 Methods
 In four face-to-face examinations the dance performances of 61 student teachers (82% female) were assessed by two lecturers in each case (n = 5; Age M = 50, 3 women, 2 men). Each lecturer scored independently on an 18-point scale on five different evaluation categories (e.g., technical quality). In addition, the dance performances were videotaped, and the same five lecturers assessed the dance performances again using the ACJ tool Comproved. To analyze interrater agreement and reliability, intraclass correlation (ICC) was calculated for the traditional assessment (Sato, 2022). The reliability of the ACJ was analyzed by calculating scale separation reliability (SSR; Verhavert et al., 2019). A Spearman’s rank correlation analysis was conducted to analyze whether there is a correlation between the ranked results of the two assessment methods. To assess the validity of the assessment methods, a focus group interview was conducted with the lecturers involved in the study.
 Results and Discussion
 Both assessment methods are characterized by very high and high reliability values (TA: ICC = 0.974, 95% confidence interval (CI): 0.955-0.985 p < .001; ACJ: SSR: 0.83, Ability [-5.97, 4.99], Misfit [-1.68, 1.34]). In particular, the ICC of the TA is higher than comparable results in dance research (Sato, 2022). There are doubts as to whether the lecturers really scored independently of each other at the face-to-face examinations. The ranked results of both methods correlate with a very strong effect (Spearman’s-Rho: rs -.818, p < 0.001). However, detailed analyses show some differences. The answer to the question of who delivered the best dance performance differs depending on the assessment method. In addition, in the traditional assessment, many scores fall on a value at which the dance examination is just passed (10 points). The results of the focus group interview are still being analyzed and will be presented at the conference.
 References
 Bartholomew, S. R., & Jones, M. D. (2022). A systematized review of research with adaptive comparative judgment (ACJ) in higher education. International Journal of Technology and Design Education, 32(2), 1159-1190. https://doi.org/10.1007/s10798-020-09642-6
 Sato, N. (2022). Improving reliability and validity in hip-hop dance assessment: Judging standards that elevate the sport and competition. Frontiers in Psychology, 13, Article 934158. https://doi.org/10.3389/fpsyg.2022.934158
 Verhavert, S., Bouwer, R., Donche, V., & De Maeyer, S. (2019). A meta-analysis on the reliability of comparative judgement. Assessment in Education: Principles, Policy & Practice, 26(5), 541-562. https://doi.org/10.1080/0969594X.2019.1602027

Exploring quantitative measures in metacognition of emotion

Metacognition of emotion (meta-emotion) refers to the ability to evaluate and identify one’s emotional feelings. No previous study has defined and measured this construct through objective and quantitative procedures. We established a reliable method to measure meta-emotion. With a two-interval forced-choice procedure, participants selected which of two pictures elicited stronger positive emotion; via the Law of Comparative Judgment, their responses were used to compute individual psychological distances for the emotional responses triggered by the pictures. Then, participants were asked to judge whether a pre-exposed picture induced a stronger positive emotion than the median of that elicited by the whole picture set, followed by a confidence rating. By utilizing each individual’s psychological distance, the correctness of a participant’s emotional experience was quantified by dʹ, and meta-emotion was quantified using meta-dʹ, M-ratio, and M-diff as indices of metacognitive sensitivity and efficiency based on Signal-Detection Theory. Test–retest reliabilities, validated by Spearman correlation, were observed in meta-dʹ, M-ratio, and marginally with M-diff, suggesting the stability of meta-emotion in the current design. This study unveils a validated procedure to quantify meta-emotion, extendable for assessing metacognition of other subjective feelings. Nevertheless, caution is warranted in interpretation, as the measured processes may be influenced by non-metacognitive factors.

Comparative Sensing and Judgment Control System for Temperature Maintenance for Optimal Treatment in Hyperthermic Intraperitoneal Chemotherapy Surgery.

For tumors wherein cancer cells remain in the tissue after colorectal cancer surgery, a hyperthermic anticancer agent is injected into the abdominal cavity to necrotize the remaining cancer cells with heat using a hyperthermic intraperitoneal chemotherapy system. However, during circulation, the processing temperature is out of range and the processing result is deteriorated. This paper proposes a look-up table (LUT) module design method that can stably maintain the processing temperature range during circulation via feedback. If the temperature decreases or increases, the LUT transmits a command signal to the heat exchanger to reduce or increase heat input, thereby maintaining the treatment temperature range. The command signal for increasing and decreasing heat input is Tp and Ta, respectively. The command signal for the treatment temperature range is Ts. If drug temperatures below 41 and above 43 °C are input to the LUT, it sends a Tp or Ta signal to the heat exchanger to increase or decrease the input heat, respectively. If the drug's temperature is 41-43 °C, the LUT generates a Ts signal and proceeds with the treatment. The proposed system can automatically control drug temperature using temperature feedback to ensure rapid, accurate, and safe treatment.

Do comparative judgements affect the perceived relevance of mobile phone road safety campaigns?

Objective Given the prevalence of illegal mobile phone use while driving and reliance upon messaging campaigns in deterring the behavior, there is a need to explore to what extent comparative judgements may influence desired outcomes of such campaigns. This exploratory study investigated (a) the perceived relevance and influence of different mobile phone road safety campaigns within a sample of Queensland motorists who reported using their mobile phone while driving and (b) if these varied depending on whether participants believed that their crash risk whilst using a phone was higher (comparative pessimism), lower (comparative optimism) or similar to the same-age and sex driver. Methods A total of 350 licensed drivers residing within Queensland (Australia) were included in this study, of which 200 reported using their hand-held phone on some occasion. Participation involved completing a 20-25 min online anonymous survey, which included viewing three mobile phone road safety campaigns (injury-based, sanction-based and humor) and responding to questions about the perceived relevance and impact of each campaign. Results A total of 64 (32%) participants displayed comparative optimism, 50 displayed similar judgements (25%) and 86 (43%) exhibited comparative pessimism. First, it was found that the injury-based campaign was perceived to be significantly more relevant than the humor campaign. Second, whilst the relevance of each campaign did not vary as a function of group membership, the campaigns were significantly less relevant to those displaying comparative optimism relative to those with similar judgements and comparative pessimism. Finally, the injury-based campaign was perceived to be significantly more influential than the other campaigns. However, overall, participants displaying comparative optimism believed that they would be less influenced by the campaigns compared to those with comparative pessimism. Conclusions Although preliminary, these findings suggest that low perceptions of risk may dilute or extinguish the desired behavioral outcomes of mobile phone road safety campaigns. Nonetheless, experimental research is needed to examine these effects directly.

Criação de Escalas Quantitativas para Estudos de Eventos Mentais: Uma Revisão da Lei do Julgamento Comparativo

Louis Leon Thurstone (1887-1955) was a pioneer in developing scales for non-metric stimuli, creating an experimental model centered on the subject without the need for physical units of comparison. This model, known as the Law of Comparative Judgment, relies on probabilistic measures rather than discrete ones, making it a powerful tool for quantifying psychological attributes through interval scales. Thurstone proposed that sensory organs operate probabilistically, infl uenced by an inherent noise within the organism, which can be either internally or externally induced. This noise impacts the hypothetical psychological continuum, causing the observer’s response to be not absolute but based on information received at the moment. His model functions analogously to a statistical sample. The Law of Comparative Judgment is structured around three fundamental premises: (i) each stimulus initiates a discriminative process represented on a psychological continuum; (ii) momentary fl uctuations, both internal and among diff erent observers, can cause a stimulus to be perceived variably, characterized by a normal distribution; (iii) the mean (μ) and standard deviation (σ) of this distribution represent the scale value and its discriminative dispersion, respectively. One of the most widely adopted experimental methods derived from Thurstone’s model is the pairwise comparison protocol. In this protocol, the observer’s task is to discriminate between two stimuli presented simultaneously. In each trial, the observer must judge which of the two stimuli is superior in a specifi ed dimension. The process involves comparing each possible pair of stimuli, with each pair being repeated a specifi c number of times. To organize and analyze the results, a matrix is constructed to capture the diff erences between all stimuli in the pairwise comparison. The fi nal value assigned to each stimulus is determined by the average diff erence between that stimulus and all others in the set. The comparative judgment equation provides a method for transforming these diff erences from all pairwise comparisons into scalar values, whether metric or non-metric. This protocol not only off ers a systematic and rigorous approach for measuring psychological attributes but also allows for a detailed and quantitative analysis of the observer’s perceptions and preferences. Faced with the challenge of conducting pairwise comparisons under various experimental conditions, Thurstone developed a statistical approach to transform data from a simple ranking task into a binary matrix of comparisons. This method allows for the application of the comparative judgment equation, overcoming the challenges of traditional psychophysical methods related to the duration and complexity of experimental sessions. The ranking method, by simplifying data collection, has proven highly eff ective in obtaining interval scales in various psychological and social measurements, particularly in contexts of limited experimental control. This effi ciency suggests that the method could be adapted for large-scale studies and outside laboratory environments, enhancing research in applied fi elds of psychology and related areas.Keywords: Law of Comparative Judgment, ranking order procedure, interval scale; quantitative measure, psychophysics.Creación de Escalas Cuantitativas para Estudios de Eventos Mentales: Una Revisión de la Ley del Juicio ComparadoResumenLouis Leon Thurstone (1887-1955), interesado en cuantifi car actitudes, creencias, preferencias y opiniones, creó una ecuación conocida como la Ley del Juicio Comparativo. El modelo de escala

Perceptual Constancy in the Speed Dependence of Friction During Active Tactile Exploration.

Fingertip friction is a key component of tactile perception. In active tactile exploration, friction forces depend on the applied normal force and on the sliding speed chosen. We have investigated whether humans perceive the speed dependence of friction for textured surfaces of materials, which show either increase or decrease of the friction coefficient with speed. Participants perceived the decrease or increase when the relative difference in friction coefficient between fast and slow sliding speed was more than 20%. The fraction of comparison judgments which were in agreement with the measured difference in friction coefficient did not depend on variations in the applied normal force. The results indicate a perceptual constancy for fingertip friction with respect to self-generated variations of sliding speed and applied normal force.

Interactively iterative group decision-making method with interval-valued intuitionistic fuzzy preference relations based on a new additively consistent concept

Interval-valued intuitionistic fuzzy preference relation (IVIFPR) is a powerful instrument for describing uncertain judgement of expert in preference group decision making (PGDM). Nevertheless, the extant additive consistency definitions for IVIFPRs violate the invariance regarding arrangement of objectives. In addition, the modification for comparative judgments in the consensus reaching process (CRP) of existing PGDMs with IVIFPRs do not involve the interaction or communication with experts. Therefore, we propose an PGDM to avoid these situations. First, a new additive consistency definition for IVIFPRs is given satisfying the invariance regarding arrangement of objectives. Then, an interactively iterative algorithm considering minimum information loss is developed to enhance the consistency of IVIFPRs. Next, as to group decision making (GDM) with IVIFPRs, a bi-objective optimization model is constructed to determine the experts’ weights, and an interactively iterative algorithm considering minimum information loss is developed to enhance the consensus level among experts. Furthermore, a fuzzy optimization model is constructed to derive the interval priority weights of objectives from an IVIFPR. Finally, the feasibility and superiority of the proposed GDM method are verified by application examples and analyses. The proposed GDM method has obtained an average of 0.0432 for information deviations and 22/72 for modified judgments on two datasets, respectively, which is one of the best results compared to state-of-art.

Assessing perceived assembly complexity in human-robot collaboration processes: a proposal based on Thurstone’s law of comparative judgement

Due to the growing demand for customised products, companies have faced increasing product and process complexity levels. To address this issue, manufacturing processes should become more flexible. One of the most promising technologies to achieve this goal is collaborative robotics (or ‘cobots’). In collaborative assembly processes, human and robot combine their skills. However, the co-existence of humans and cobots in the same workspace may influence the operators’ perception of assembly complexity. The analysis and control of assembly complexity are crucial to achieving better performances in terms of process quality and operators’ well-being. Many qualitative methods have been proposed in the literature to provide a holistic assessment of assembly complexity. This paper proposes a novel method to define a quantitative scale of perceived assembly complexity, based on Thurstone Law of Comparative Judgements. This method was applied to an experimental case-study concerning the assembly of three different products in two modalities (i.e. manual and collaborative). Regression analysis showed that the perceived complexity may be related to the occurrence of process failures and to the perceived workload. The method also proved capable of identifying assembly processes where cobot assistance was helpful, providing process designers with a supporting tool to minimise perceived complexity.

Analisa Pengambilan Keputusan Pemilihan Venodr Seragam Dengen Merode Analytical Hierarchy Proces (AHP) ( Studi Kasus General Affairs PT Penjallindo Nusantara)

Decision makers need analytical tools that allow them to solve problems. Decisions to be taken are complex with major risks such as policy formulation, decision makers often need tools in scientific, logical, and structured forms. To determine the priority stages of criteria in order to obtain the most optimal results in returning uniform vendor selection decisions at PT Penjalindo Nusantara (Metaform) using the AHP (Analytical Hierarchy Process) method. In essence, AHP is a comprehensive decision-making model by taking into account qualitative and quantitative matters. There are three main principles in problem solving in AHP, namely: Decomposition, Comparative Judgment, and Logical Concistency. Bycomparing in pairs to determine the priority weight on each criterion and alternative. One of the paired comparisons is carried out in pairwise comparisons between criteria. Vendor selection criteria include quality, price, flexibility, delivery and service criteria. . Compiling alternatives that have been determined in the form of matirx in order to obtain the assessment weight of each variable, a table of pair comparison rating scales is made. Based on the results of the AHP calculation, each weight for each venodr is obtained PT Intan of 0.62860 and the weight value of the Super Decision is 0.638683. PT Parahita as a vendor with a weight value of 0.20475 and a weight value in Super Decision of 0.203985. PT Marcapada with a weight value of 0.16665 and a weight value in Super Decision of 0.159189. So the main uniform supplier is PT Intan with the largest weight.

Designing a Website-Based Analytical Hierarchy Process (AHP) in Decision Support Systems (DSS) for Parental Control of Children's Media

The massive amount of children's television programming poses a challenge for parents seeking to mediate suitable viewing habits. This research puts forward a website-based Decision Support System (DSS) leveraging Analytic Hierarchy Process to recommend appropriate shows per personalized criteria. The DSS allows parents to set priorities across factors like educational value, empathy, and social norms reflected in shows. Quantitative AHP methodology ranks options by synthesizing comparative judgments. A prototype is implemented on PHP/MySQL with an intuitive interface explaining decision logic. Testing with parent focus groups indicates significantly increased perceived utility over traditional controls. Recommendation accuracy metrics averaged 0.81 against test cases. Outcomes demonstrate feasibility and benefits of applying multi-criteria DSS techniques to aid value-based media selection for impressionable minds. The system aims to aid busy caregivers overwhelmed by the exponential growth in media choices, promoting discerning consumption. Further enhancements incorporating explanatory facilities and predictive analytics promises wider applicability inside content distribution ecosystems. This research highlights the importance of DSS-AHP integration for child development in the modern information age.

Why fractions are difficult? Modeling optimal and sub-optimal integration strategies of numerators and denominators by educated adults

Many children and educated adults experience difficulties in understanding and manipulating fractions. In this study, we argue that a major cause of this challenge is rooted in the need to integrate information from two separate informational sources (i.e., denominator and numerator) according to a normative arithmetic rule (i.e., division). We contend that in some tasks, the correct arithmetic rule is replaced by an inadequate (sub-optimal) operation (e.g., multiplication), which leads to inaccurate representation of fractions. We tested this conjecture by applying two rigorous models of information integration : (a) functional measurement (Experiments 1-3) and (b) conjoint measurement (Experiment 4-5) to data from number-to-line and comparative judgment tasks. These allowed us to compare participants’ integration strategies with that of an ideal-observer model. Functional measurement analyses on data from the number-to-line task, revealed that participants could represent the global magnitude of proper and improper fractions quite accurately and combine the fractions’ components according to an ideal-observer model. However, conjoint measurement analyses on data from the comparative judgment task, showed that most participants combined these fractions’ components according to a sub-optimal (saturated) observer model, that is inconsistent with an ideal-observer (additive) model. These results support the view that educated adults are capable of extracting multiple types of representations of fractions depending on the task at-hand. These representations can be either accurate and conform with normative arithmetic or approximated and inconsistent with normative arithmetic. The latter may lead to the observed difficulties people experience with fractions.

What is a holistic judgement, anyway?

ABSTRACT Holistic judgement is an appealing approach for many assessment contexts due to its perceived simplicity and efficiency. However, it has been somewhat under-explored conceptually. Drawing on examples from assessment contexts including vocational performance, comparative judgement and the use of holistic judgement in teacher assessment for high stakes grading, we explicate a three-part definition of what holistic judgement constitutes. Holistic judgements result in singular outputs, derive from a comprehensive consideration of relevant constructs and acknowledge that the elements considered within them interconnect. However, holistic judgements may be made using considerably different processes by different judges without contravening this definition, and the ways in which different elements are weighted may vary. We then explore some factors, specific to assessment contexts, that might make holistic judgements more challenging, including materials being very different from one another, non-uniform candidate performance across materials and the presence of construct-irrelevant material. We set this assessment-specific discussion in the context of literatures on decision-making in psychology, medicine and other contexts. We conclude with some recommendations for when holistic judgement should and should not be used, and how holistic judgements can be made less cognitively challenging through the use of appropriate guidance and feedback to judges.