Law Of Comparative Judgment Research Articles

The Effects of Speaker Head Posture on Auditory Perception of Vocal Masculinity

ObjectiveThe purpose of this study was to investigate the impact of head position on listeners’ perception of vocal masculinity. MethodsTwelve cisgender women were recorded reciting two voiced sentences with varying head positions: baseline, flexed, and extended. Voice samples were cropped and fundamental frequency (fo) was resynthesized to control for any changes in fo across conditions. Twelve cisgender adults were recruited as listeners. Listeners were presented with 144 paired comparisons of speaker samples and were prompted to select the sample that sounded more masculine in each presented pairing. Ratings of masculinity were analyzed using Thurstone’s law of comparative judgment. A repeated measures analysis of variance (ANOVA) assessed the effects of head positioning, followed by Dunnett’s post hoc tests. ResultsThe ANOVA showed a statistically significant effect of head position on listener perceptions of masculinity: speech in the flexed position was perceived as statistically more masculine than that in the baseline condition. ConclusionsThe results of this study support the use of head posture manipulation to achieve increased vocal masculinity, which adds to the limited research related to voice masculinization strategies for those seeking gender-affirming voice care.

Algebraic analysis of pairwise comparisons with non-reciprocal property

The law of comparative judgement usually implies the reciprocal property of pairwise comparisons. But time error and position preference could yield some judgements that contradict this identity assumption. This paper proposes an algebraic approach to dealing with the challenge of non-reciprocal property in pairwise comparisons. The concept of non-reciprocal pairwise comparison matrix (NrPCM) is used to characterize the situation of non-reciprocal property. Algebraic analysis of NrPCMs is carried out, where a linear space is constructed by proposing a set of basis. Then the linear space is decomposed into the direct sum of some subspaces. The properties of the space are analyzed such as inner product, decomposition and coordinate. Finally, we apply the algebraic method to investigate the Condorcet paradox under the impact of indifferent voters. A novel transformation formula of non-reciprocal pairwise comparisons is proposed, where the non-reciprocal property holds. It is revealed that the occurrence of voting paradox can be understood clearly according to the transitivity property of NrPCMs.

A new method to prioritize the QFDs’ engineering characteristics inspired by the Law of Comparative Judgment

The primary module of Quality Function Deployment (QFD) is the House of Quality (HoQ), which supports the design of new products and services by translating customer requirements (CRs) into engineering characteristics (ECs). Within the HoQ framework, the traditional technique for prioritizing ECs is the independent scoring method (ISM), which aggregates the weights of the CRs and the relationships between CRs and ECs (i.e., null, weak, medium, and high) through a weighted sum. However, ISM incorporates two questionable operations: (i) an arbitrary numerical conversion of the relationships between CRs and ECs, and (ii) the “promotion” of these relationships from ordinal to cardinal scale. To address these conceptual shortcomings, this paper introduces a novel procedure for prioritizing ECs, inspired by the Thurstone’s Law of Comparative Judgment (LCJ). This procedure offers a solution that is conceptually sound and practical, overcoming the conceptual shortcomings of ISM, while maintaining its simplicity, flexibility, and ease of implementation. The proposed approach is supported by a realistic application example illustrating its potential.

Exploring quantitative measures in metacognition of emotion

Metacognition of emotion (meta-emotion) refers to the ability to evaluate and identify one’s emotional feelings. No previous study has defined and measured this construct through objective and quantitative procedures. We established a reliable method to measure meta-emotion. With a two-interval forced-choice procedure, participants selected which of two pictures elicited stronger positive emotion; via the Law of Comparative Judgment, their responses were used to compute individual psychological distances for the emotional responses triggered by the pictures. Then, participants were asked to judge whether a pre-exposed picture induced a stronger positive emotion than the median of that elicited by the whole picture set, followed by a confidence rating. By utilizing each individual’s psychological distance, the correctness of a participant’s emotional experience was quantified by dʹ, and meta-emotion was quantified using meta-dʹ, M-ratio, and M-diff as indices of metacognitive sensitivity and efficiency based on Signal-Detection Theory. Test–retest reliabilities, validated by Spearman correlation, were observed in meta-dʹ, M-ratio, and marginally with M-diff, suggesting the stability of meta-emotion in the current design. This study unveils a validated procedure to quantify meta-emotion, extendable for assessing metacognition of other subjective feelings. Nevertheless, caution is warranted in interpretation, as the measured processes may be influenced by non-metacognitive factors.

Criação de Escalas Quantitativas para Estudos de Eventos Mentais: Uma Revisão da Lei do Julgamento Comparativo

Louis Leon Thurstone (1887-1955) was a pioneer in developing scales for non-metric stimuli, creating an experimental model centered on the subject without the need for physical units of comparison. This model, known as the Law of Comparative Judgment, relies on probabilistic measures rather than discrete ones, making it a powerful tool for quantifying psychological attributes through interval scales. Thurstone proposed that sensory organs operate probabilistically, infl uenced by an inherent noise within the organism, which can be either internally or externally induced. This noise impacts the hypothetical psychological continuum, causing the observer’s response to be not absolute but based on information received at the moment. His model functions analogously to a statistical sample. The Law of Comparative Judgment is structured around three fundamental premises: (i) each stimulus initiates a discriminative process represented on a psychological continuum; (ii) momentary fl uctuations, both internal and among diff erent observers, can cause a stimulus to be perceived variably, characterized by a normal distribution; (iii) the mean (μ) and standard deviation (σ) of this distribution represent the scale value and its discriminative dispersion, respectively. One of the most widely adopted experimental methods derived from Thurstone’s model is the pairwise comparison protocol. In this protocol, the observer’s task is to discriminate between two stimuli presented simultaneously. In each trial, the observer must judge which of the two stimuli is superior in a specifi ed dimension. The process involves comparing each possible pair of stimuli, with each pair being repeated a specifi c number of times. To organize and analyze the results, a matrix is constructed to capture the diff erences between all stimuli in the pairwise comparison. The fi nal value assigned to each stimulus is determined by the average diff erence between that stimulus and all others in the set. The comparative judgment equation provides a method for transforming these diff erences from all pairwise comparisons into scalar values, whether metric or non-metric. This protocol not only off ers a systematic and rigorous approach for measuring psychological attributes but also allows for a detailed and quantitative analysis of the observer’s perceptions and preferences. Faced with the challenge of conducting pairwise comparisons under various experimental conditions, Thurstone developed a statistical approach to transform data from a simple ranking task into a binary matrix of comparisons. This method allows for the application of the comparative judgment equation, overcoming the challenges of traditional psychophysical methods related to the duration and complexity of experimental sessions. The ranking method, by simplifying data collection, has proven highly eff ective in obtaining interval scales in various psychological and social measurements, particularly in contexts of limited experimental control. This effi ciency suggests that the method could be adapted for large-scale studies and outside laboratory environments, enhancing research in applied fi elds of psychology and related areas.Keywords: Law of Comparative Judgment, ranking order procedure, interval scale; quantitative measure, psychophysics.Creación de Escalas Cuantitativas para Estudios de Eventos Mentales: Una Revisión de la Ley del Juicio ComparadoResumenLouis Leon Thurstone (1887-1955), interesado en cuantifi car actitudes, creencias, preferencias y opiniones, creó una ecuación conocida como la Ley del Juicio Comparativo. El modelo de escala

Determinants of First College Major Choice among Yarmouk University Students and Estimating their Scaling Values Using Three Scaling Methods

The study aimed at determining the determinants of first college major choice for Yarmouk University students, and estimating their scaling values using three different scaling methods, and determining the extent of agreement among these methods. To achieve the purpose of the study, (165) male and female students from the university colleges volunteered to determine the determinants primarily, and two equivalent samples with regard to gender and college type were used to collect the scaling data. The first consisted of (120) male and female students studying a major which is their first choice, the second consisted of (120) male and female students studying a major differs from their first choice. The scaling values on importance scale were estimated using the Law of Comparative Judgments, Direct Estimation and the Law of Catogerical Judgments. The results showed that the most important major choice determinants are: opportunities to obtain work, secondary school exam scores mean, personal desire, and admission opportunities. Also, it was found that there was a consistency among scaling methods in ranking the determinants on the importance scale.

Psychophysical scaling method for measurement of colors concept in children and adults

Our study aimed to present a new method to generate a scale for color concept distances using the Law of Comparative Judgment and derivate an interval scale for the amount of color concept salience. A hundred thirty-eight participants (mean age ​= ​22.7 ​yrs; SD ​= ​1.9) with normal visual acuity and color vision were evaluated. Additional 225 children were also evaluated (mean age ​= ​8,9yrs; SD ​= ​2.3). The task consisted of writing a list of the one-term word for colors and data were analyzed based on Thurstone's rank order scaling method. For adults, the most salient concept was blue and the less was violet with a distance for concepts of 3.51 units between them, whereas for children the most salient concept was blue and the less was violet with a distance for concepts of 3.67 units between them. Significant differences between adults and children occurred for white, brown, pink, orange, gray, and violet. The internal consistency of our measurement was lower than 0.1% for almost all colors which suggests high reliability for both age groups, except for brown and anil in children. Thus, our innovative approach produces quantitative interval class data rather than categorical or ordinal data allowing quantitative measurement of the amount of the concept salience on a one-dimensional scale, feasible to be used for adults and children. • We evolved the color concept measure by representing concepts on an interval scale. • The method allows the use in children and adults with a simple task adaptation. • Our study eliminates the use of physical stimuli and, therefore, is of more cognitive nature. • We can thus study the salience that a given concept has in a continuum, without restrict them in a discrete ordinal scale.

P-006. The pathophysiological role of calreticulin in maternal circulation for preeclampsia

Our study aimed to present a new method to generate a scale for color concept distances using the Law of Comparative Judgment and derivate an interval scale for the amount of color concept salience. A hundred thirty-eight participants (mean age ​= ​22.7 ​yrs; SD ​= ​1.9) with normal visual acuity and color vision were evaluated. Additional 225 children were also evaluated (mean age ​= ​8,9yrs; SD ​= ​2.3). The task consisted of writing a list of the one-term word for colors and data were analyzed based on Thurstone's rank order scaling method. For adults, the most salient concept was blue and the less was violet with a distance for concepts of 3.51 units between them, whereas for children the most salient concept was blue and the less was violet with a distance for concepts of 3.67 units between them. Significant differences between adults and children occurred for white, brown, pink, orange, gray, and violet. The internal consistency of our measurement was lower than 0.1% for almost all colors which suggests high reliability for both age groups, except for brown and anil in children. Thus, our innovative approach produces quantitative interval class data rather than categorical or ordinal data allowing quantitative measurement of the amount of the concept salience on a one-dimensional scale, feasible to be used for adults and children.

Investigation of hue effects in tooth whiteness perception.

To identify the hue angles that provide the optimal perception of tooth whitening. Thirty-three male/female adult observers with normal color vision were recruited to assess teeth images on a color calibrated display. Images of teeth were generated which simulated color changes in each of eight hue directions in the CIE a*b* plane, each equi-distant from a baseline tooth color. Using a paired comparison study design, observers were asked to indicate which of two images had whiter teeth. The data were converted into an interval scale using Thurstone's Law of Comparative Judgment. The relationship between the hue angles and the scores was modeled by curve fitting. The optimal hues were found in a region between green and blue in the CIE a*b chart. When the tooth color travels in an optimal hue direction, the same amount of color change will result in a greater change in whiteness perception than when traveling in other hue directions. The optimal hues for tooth whitening were identified from the visual perception study. The most effective tooth whitening direction is to change tooth color towards the greenish-blue hue direction. This study gives clinicians and researchers a better understanding of tooth color hue effects in tooth whiteness perception.

The giant freshwater prawnMacrobrachium rosenbergiialters background colour preference after metamorphosis from larvae to postlarvae: In association with nature of phototaxis

The giant freshwater prawn Macrobrachium rosenbergii larvae have apposition eyes and are positively phototactic, whereas the postlarvae (PL) have superposition eyes and are negatively phototactic. M. rosenbergii has colour vision as early as larval stage. We discovered that M. rosenbergii alters background colour preference after metamorphosis from larvae to PL in association with nature of phototaxis. The test circular glass aquaria covered with a pair of two-colour papers contained with a group of 100 larvae or 20 Pl, and the number of individuals in each colour background was recorded five times for each colour pair. The background colours tested were light blue, green, yellow, red, white and black. The numbers of larvae or PL in each colour background of different pairs were analysed by the Thurstone's law of comparative judgment. In the larvae, significant bias towards yellow was evident. In the PL, of the four pairings of black with other colours, all biased to black. The mean z-scores were highest for yellow in the larvae, and for black in the PL. To determine the possible background brightness preference of the larvae and PL, six different colour backgrounds were presented in pairs. The larvae significantly preferred light blue over dark blue, white over yellow and white over black. The PL exhibited reversed preference. The relationship between z-scores and light reflectance levels of five colour papers was significantly positive in the larvae and negative in the PL. The observed background colour preference was probably due to relative brightness rather than chromaticity difference.

Scaling of secondary school students' evaluation preferences according to learning approaches

The aim of this study is to determine the evaluation preferences of secondary school students adopting deep and superficial learning approach by using pairwise comparison method. The study was carried out with 444 students chosen from 9th, 10th, 11th and 12th grade students in four different types of secondary education in Mersin. The case V and case III models of the scaling method based on Thurstone's (1927) Law of Comparative Judgment, which can be applied in cases where the items can be given to participants in pairs, are used to answer the research questions. As a result of the research, it was determined that students adopting both deep and superficial learning approaches prefer the questions measuring the skills at remembering level with higher chance success such as matching, true or false and multiple choice in the first place, performance tasks and long answer questions in the last place while determining their academic success. When this result was examined according to the overall and grade levels of secondary students, it was observed that it did not change except for minor differences.

Evaluating the Graininess Attribute by Visual Scaling for Coatings with Special-Effect Pigments

In our society, objects’ visual appearance is an essential factor because it allows us to recognize and differentiate one object from another. In different industrial sectors like cosmetics, textiles and automotive, special-effect pigments are largely used to achieve attractive visual features. These pigments provide a color change with viewing and illumination direction, and visually provide texture. Depending on a finish’s properties, and also on the viewing and illumination conditions, coatings exhibit sparkle or a graininess-like texture. Currently, not many scientific works on the visual perception of these texture effects can be found in the literature. In addition, choice of experimental method can influence the measurement scale obtained from visual data. For this reason, the purpose of this work was to analyze graininess visual scaling constructed by two different psychophysical methods. The experimental design was based on the rank-order and paired-comparison methods. The data analysis was conducted by following the law of comparative judgments to obtain a visual scale of the graininess attribute to compare it to instrumental data. A good correlation appeared between both magnitudes with a correlation coefficient close to 0.9. Both methods provided useful results with a reasonable correspondence between them, which ensures that data can be considered reliable, while the visual obtained scale can act as a good graininess scale perceived by the human visual system.

DP-FMEA: An innovative Failure Mode and Effects Analysis for distributed manufacturing processes

The Failure Mode and Effects Analysis (FMEA) is a powerful tool to design and maintain reliable systems (products, services or manufacturing processes), investigating their potential failure modes from the threefold perspective of severity, occurrence and detection. The Process FMEA, or more briefly P-FMEA, is a declination of the FMEA for manufacturing processes (or parts of them). Being progressively characterized by decentralized networks of flexible manufacturing facilities, the current scenario significantly hampers the implementation of the traditional P-FMEA, which requires the joint work of a group of experts formulating collective judgments. This paper revises the traditional P-FMEA approach and integrates it with the ZMII-technique – i.e. a recent aggregation technique based on the combination of the Thurstone’s Law of Comparative Judgment and the Generalized Least Squares method – allowing experts distributed through organizations to formulate their judgments individually. The revised approach – referred to as “distributed-Process FMEA” or more briefly dP-FMEA – allows to manage a number of experts, without requiring them to physically meet and formulate collective decisions, thus overcoming a relevant limitation of the traditional P-FMEA. The dP-FMEA approach also includes a relatively versatile response mode and overcomes several other limitations of the traditional approach, including but not limited to: (i) arbitrary formulation and aggregation of expert judgments, (ii) lack of consideration of the dispersion of these judgments, and (iii) lack of estimation of the uncertainty of results. The description is supported by a real-life application example concerning a plastic injection-molding process.

Using Gaussian Spectra to Derive a Hue-linear Color Space

. A new color space, IGPGTG, was developed. IGPGTG uses the same structure as IPT, an established hue-uniform color space utilized in gamut mapping applications. While IPT was fit to visual data on the perceived hue, IGPGTG was optimized based on evidence linking the peak wavelength of Gaussian-shaped light spectra to their perceived hues. The performance of IGPGTG on perceived hue data was compared to the performance of other established color spaces. Additionally, an experiment was run to directly compare the hue linearity of IGPGTG with those of other color spaces by using Case V of Thurstone's law of comparative judgment to generate hue-linearity scales. IGPGTG performed well in this experiment but poorly on extant visual data. The mixed results indicate that it is possible to derive a moderately hue-linear color space without visual data.

Shelter colour preference in the purple mud crab Scylla tranquebarica (Fabricius)

Cannibalism a serious problem in the grow-out of mud crabs in ponds. Studies have proven that presence of shelters decreases aggressive behaviour of Scylla spp. in ponds and tanks. In this study, the shelter colour preference was determined in the wild captured purple mud crab Scylla tranquebarica (Fabricius) in an outdoor round black tank. Eight out of 48 PVC pipes were painted either blue, green, red, black or white. Other eight were not painted and were provided as grey shelters. The shelters were presented in different two-colour pairs to a group of 35 crabs; eight shelters of one colour and eight of other colour were arranged alternately on the tank bottom, and the number of occupants was recorded. After emptying the shelters, the shelters rearranged for the next observation. Each colour combination was repeated five times, and total of 15 combinations were tested. The number of occupants against each colour of the different pairs were analyzed by the χ2-test and Thurstone’s law of comparative judgment. The data were converted into mean z-scores and significant biases from zero were noted. Underwater visibility of each colour shelter was visually measured through a waterproof digital CCD camera. The crabs showed a strong bias for blue shelter and white shelter chosen the least under the holding condition. The underwater visibility was highest in blue and white colour shelters among the six colour shelters. The observed colour preference was probably due to colour per se and may be an innate ability, rather than due to relative underwater visibility. During the observations, the shelter occupants were tolerant of intruders; there was no aggressive competition for shelter and two to four crabs shared one shelter. It was hypothesized that the touch stimulus of shelter suppresses the aggressiveness and results in low cannibalism of S. tranquebarica.

Fusing incomplete preference rankings in design for manufacturing applications through the ZM II -technique

The authors recently presented a technique (denominated “ZM”) to fuse multiple (subjective) preference rankings of some objects of interest—in manufacturing applications—into a common unidimensional ratio scale Franceschini and Maisano J Intell Manuf, 2019. Although this technique can be applied to a variety of decision-making problems in the manufacturing field, it is limited by a response mode requiring the formulation of complete preference rankings, i.e., rankings that include all objects. Unfortunately, this model is unsuitable for some practical contexts—such as decision-making problems characterized by a relatively large number of objects and field surveys—where respondents can barely identify the more/less preferred objects, without realistically being able to construct complete preference rankings. The purpose of this paper is to develop a new technique (denominated “ZMII”) which also “tolerates” incomplete preference rankings, e.g., rankings with the more/less preferred objects only. This technique borrows the underlying postulates from the Thurstone’s Law of Comparative Judgment and uses the Generalized Least Squares method to obtain a ratio scaling of the objects of interest, with a relevant uncertainty estimation. Preliminary results show the effectiveness of the new technique even for relatively incomplete preference rankings. Description is supported by an application example concerning the design of a coach-bus seat.

Prioritizing customer requirements in a product-service system (PSS) context

PurposeProduct-service systems (PSSs)’s popularity has expanded significantly throughout recent years. The purpose of this paper is to integrate products and services to achieve functional results that augment the offering’s value. Nevertheless, the intangibility of services hinders the diffusion of PSSs: services are characterized by imprecisions and ambiguities that render the assessment and prioritization of customer requirements problematic. An inadequate evaluation of the latter leads to an inconsistent PSS design that results in the customer dissatisfaction.Design/methodology/approachTo address these concerns, the paper proposes an approach integrating the quality function deployment for product-service systems (QFDforPSS) method with Thurstone’s Law of Comparative Judgments. This approach was tested at a manufacturer in the medical sector seeking to improve his market stance through a PSS model.FindingsAlthough the case study is based on a limited sample, the results achieved highlight the importance of the flow of information between the PSS provider and the customers (i.e. the PSS receivers) in the healthcare sector. The proposed approach can facilitate the company in collecting information even in the case of incomplete answers to surveys and questionnaires providing a practical method to handle the uncertainty due to incomplete data.Originality/valueThe study represents one of the first applications of the PSS approach in the healthcare sector, introducing a novel integration of easy-to-use management tools to augment the understanding of customer needs and expectations.

Leading the field: Fortune favors the bold in Thurstonian choice models

Schools with the highest average student performance are often the smallest schools; localities with the highest rates of some cancers are frequently small; and the effects observed in clinical trials are likely to be largest for the smallest numbers of subjects. Informal explanations of this “small-schools phenomenon” point to the fact that the sample means of smaller samples have higher variances. But this cannot be a complete explanation: If we draw two samples from a diffuse distribution that is symmetric about some point, then the chance that the smaller sample has larger mean is 50%. A particular consequence of results proved below is that if one draws three or more samples of different sizes from the same normal distribution, then the sample mean of the smallest sample is most likely to be highest, the sample mean of the second smallest sample is second most likely to be highest, and so on; this is true even though for any pair of samples, each one of the pair is equally likely to have the larger sample mean. The same effect explains why heteroscedasticity can result in misleadingly small nominal $p$-values in nonparametric tests of association. Our conclusions are relevant to certain stochastic choice models, including the following generalization of Thurstone’s Law of Comparative Judgment. There are $n$ items. Item $i$ is preferred to item $j$ if $Z_{i}<Z_{j}$, where $Z$ is a random $n$-vector of preference scores. Suppose $\mathbb{P}\{Z_{i}=Z_{j}\}=0$ for $i\ne j$, so there are no ties. Item $k$ is the favorite if $Z_{k}<\min_{i\ne k}Z_{i}$. Let $p_{i}$ denote the chance that item $i$ is the favorite. We characterize a large class of distributions for $Z$ for which $p_{1}>p_{2}>\cdots>p_{n}$. Our results are most surprising when $\mathbb{P}\{Z_{i}<Z_{j}\}=\mathbb{P}\{Z_{i}>Z_{j}\}=\frac{1}{2}$ for $i\ne j$, so neither of any two items is likely to be preferred over the other in a pairwise comparison. Then, under suitable assumptions, $p_{1}>p_{2}>\cdots>p_{n}$ when the variability of $Z_{i}$ decreases with $i$ in an appropriate sense. Our conclusions echo the proverb “Fortune favors the bold.”

A new proposal to improve the customer competitive benchmarking in QFD

ABSTRACTQuality Function Deployment (QFD) is a structured tool that supports the design of new products/services, translating customer requirements into technical and process characteristics. The so-called Customer Competitive Benchmarking is a module of the QFD's House of Quality, in which a sample of (potential) customers express their perceptions on a set of competing products/services; this information is then elaborated by a cross-functional team of experts and used to define improvement and strategic goals. Despite the importance of this kind of benchmarking for the whole QFD process, the scientific literature reveals limited research. This article critically analyzes the traditional procedure of customer-competitive benchmarking, highlighting its major weaknesses and problematic aspects. Additionally, it proposes an alternative procedure to overcome (at least partly) those weaknesses, without undermining the simplicity in data collection and processing of the traditional procedure. The alternative procedure utilizes the Thurstone's Law of Comparative Judgment, which allows to transform subjective judgments by multiple respondents into a collective cardinal scaling. The description is supported by several pedagogical and real-life examples.

Scale Separation Reliability: What Does It Mean in the Context of Comparative Judgment?

Comparative judgment (CJ) is an alternative method for assessing competences based on Thurstone's law of comparative judgment. Assessors are asked to compare pairs of students work (representations) and judge which one is better on a certain competence. These judgments are analyzed using the Bradly-Terry-Luce model resulting in logit estimates for the representations. In this context, the Scale Separation Reliability (SSR), coming from Rasch modeling, is typically used as reliability measure. But, to the knowledge of the authors, it has never been systematically investigated if the meaning of the SSR can be transferred from Rasch to CJ. As the meaning of the reliability is an important question for both assessment theory and practice, the current study looks into this. A meta-analysis is performed on 26 CJ assessments. For every assessment, split-halves are performed based on assessor. The rank orders of the whole assessment and the halves are correlated and compared with SSR values using Bland-Altman plots. The correlation between the halves of an assessment was compared with the SSR of the whole assessment showing that the SSR is a good measure for split-half reliability. Comparing the SSR of one of the halves with the correlation between the two respective halves showed that the SSR can also be interpreted as an interrater correlation. Regarding SSR as expressing a correlation with the truth, the results are mixed.