Experienced Judges Research Articles

SENSORY AND MECHANICAL ATTRIBUTES OF GEL TEXTURE

Eighteen experienced judges evaluated the texture of gels varying in gelatin concentration (22‐45 g/L) in terms of firmness by oral and manual shear and compression, cohesiveness, and extent of breakdown in the mouth. Manual compression and biting with the front teeth discriminated well across gel concentrations. All sensory measures except extent of breakdown increased with gelatin concentration. Instron (IUTM) measurements showed that increasing gelatin concentration resulted in an increase in maximum force and force/deformation, but had little effect on deformation at yield and rupture or in elasticity and cohesiveness. Results from mechanical measurements varied with the type of force applied (compression, shear or puncture), the loading rate (50 or 200 mm/min), and the extent of deformation attained (40–90%). The highest discrimination across gel concentrations was achieved with shear force at a rate of 200 mm/min and at greater deformations. Sensory responses correlated most highly with the following IUTM measurements: (1) Compression forces at yield and at deformations of 70 and 85% at the higher crosshead speed; (2) Compression forces below the yield point at the lower crosshead speed; and (3) Shear forces measured at maximum deformation (90%) at 200 mm/min.

A note on the ability of three portable ultrasonic probes to predict backfat thickness in cattle

ABSTRACTThree portable ultrasonic probes (Scanoprobe, Meritronics and Krautkramer), each used by the one experienced operator, were found to be superior to the visual and tactile assessments of two experienced judges in predicting carcass backfat depth (range 4 to 20 mm) in 132 live steers. Residual standard deviations for the three probe/operator combinations were 2·01, 2·16 and 2·77 mm respectively; those for the judges were 2·96 and 3·09 mm. Combining ultrasonic estimates by multiple regression improved slightly the accuracy of prediction (residual s.d. = 1·87 mm), the level of which was comparable to that previously reported for scanning ultrasonic instruments used on pigs.

Measurements used in Experiments to Evaluate the Performance of Watch Officers

This paper reviews the measures commonly used in experiments to evaluate collision avoidance systems. The use of Closest Point of Approach, Time to Closest Point of Approach, and Manoeuvre Magnitude was criticized particularly for multi-ship encounters. However, the data do suggest that two measures, Entropy and the use of experienced judges to give rank scores of performance, seem to be good measures even in multi-ship encounters. The use of a composite measure is not experimentally evaluated in this paper, but it is suggested that if one were used, it would have to be very complicated. Mr Cook is with the Department of Applied Psychology at UWIST, Dr Crawshaw in the Department of Psychology at the University of Hull.

Screening Police Applicants: A Study of Reliability with the MMPI

The problem of selecting candidates for law enforcement agencies is difficult and poorly understood. Behavioral scientists are becoming more involved as consultants to police departments and are being asked to validate current selection procedures. The MMPI is being used as a screening device. This study explored the interrater reliability of two experienced judges using the MMPI “clinically” in a simulated selection procedure. The judges used markedly different selection strategies and disagreed on nominal placement into two classes on about one-third of the cases. They reached the near maximum non-chance agreement possible, given that they used the nominal classes at different rates. Neither judge was more accurate in his selections than the other despite different selection strategies. They could not improve their over-all individual performances when collaborating. Criterion variables require improved definition before the search for valid predictors can be objectified.

Talker effects on word-discrimination scores of adults with sensorineural hearing impairment.

Speech discrimination testing was completed on 30 adults with varying degrees of sensorineural hearing impairment. Tape recordings of four talkers' utterances of CID W-22 word lists served as the stimulus materials. Listeners' responses were scored independently by three experienced judges. For 26 of the 30 subjects, the difference between the lowest and highest word-discrimination score was 8% or greater. Variations in scores could not be attributed to a single talker but were spread across all talkers. A moderate negative correlation was found between each listener's average word-discrimination score and amount of variability among his or her individual scores. The statistical analysis indicated that talker differences were responsible for only a small portion of the variability in scores and suggested that a factor of greater importance is the talker-listener interaction.

Some effects of delayed auditory feedback on dyspraxia of speech

This investigation was designed to determine if delayed auditory feedback (DAF) might have a facilitating effect on dyspraxic speech. Five subjects with dyspraxia of speech and minimal aphasic involvement were used. Two reading tasks were administered under conditionsof DAF and no DAF. Reading tasks were constructed of 24 monosyllabic words with initial consonant clusters and 10 polysyllabic words. Three experienced judges scored word production, consonant cluster production, and 12 types of production errors. Temporal duration of word productions was measured in both conditions. No significant differences between the conditions were found in any of the measured parameters. There was no demonstrable improvement of dyspraxia of speech errors under DAF. Contrary to earlier research, distortions rather than phonemic substitutions were found to be the most frequently occuring articulatory errors.

Effect of Potassium Sorbate on Toxinogenesis by Clostridium botulinum in Bacon

In an effort to reduce the initial levels of nitrite used to cure bacon and still supply the botulinal inhibition expected in cured meats, bacon was produced at nitrite levels of 0 and 40 ppm NaNO2 with and without 0.13 and 0.26% potassium sorbate. This bacon was inoculated with 1100 spores per g of a mixture of five Type A and five Type B strains of Clostridium botulinum. The time for occurrence of the first swollen package and number of toxic swells were recorded over 110 days of incubation at 27 C. The above variables were compared to bacon containing 80 and 120 ppm NaNO2 as well as a commercial sample. Presence of potassium sorbate in the cure significantly reduced the number of toxic swollen packages occurring during incubation and lengthened the time before a toxic swollen package was observed. The presence or absence of 40 ppm NaNO2 appeared to have no significant effect on the sorbate inhibition of C. botulinum in bacon in this study. Microbial growth of uninoculated samples was also retarded by addition of potassium sorbate to the brine. Flavor panel evaluations indicated that potassium sorbate decreased preference slightly using experienced judges. Also, reduced occurrence of nitrosopyrrolidine with reduced nitrite was observed.

Pre-school physique and late-adolescent somatotype.

SummaryPhysique photographs of 319 pre-school children were evaluated on somatotype-like variables by three methods: ratings by inexperienced judges, ratings by experienced judges, and statistical prediction of the variables that now determine adult somatotype—mature height, a height-weight ratio, and a trunk index (TI). Inter-rater agreement and one- to two-year stability were moderate for both groups of judges' ratings of endomorphy and ectomorphy, low for their ratings of mesomorphy. Experienced and inexperienced judges showed higher correlation between each others' ratings (0·8 for endomorphy and ectomorphy, 0·6 for mesomorphy) than with the statistical somatotype forecasts; inexperienced judges' ratings had little relation to the somatotype forecasts for mesomorphy and ectomorphy.Of this sample, 100 boys and 50 girls followed into adolescence were somatotyped by Sheldon. Pre-school ratings by inexperienced judges showed significant but low correlations with adolescent somatotype (0·3–0·5 for different components), those by experienced judges somewhat higher (0·4–0·6 for composite scores); the measurement-based method showed the highest predictive correlations (0·6–0·7).TI, a ratio of thoracic to abdominal trunk area, showed pronounced sex differences at the pre-school as well as the adolescent ages. Boys' means were identical and girls' very close at the two ages. Correlation of early and later TI was 0·72 for the boys, 0·63 for the girls.Factor analysis showed differences in the models of the four methods. The judges, expert or inexperienced, produced two factors in their ratings: relative massiveness and muscularity. The somatotype forecasts and somatotypes, in line with Sheldon's current methods, produced three: relative massiveness, muscularity, and size.

The effect of task variables on speech during oral anesthesia

A normal, adult female performed a variety of speech tasks before and after a series of nerve-block injections that anesthetized the oral cavity. The tasks included diadokokinesis, imitation of unfamiliar Swedish phonemes, production of one-, two-, three-, and four-syllable words, and of two prose passages. Four experienced judges scored misarticulations. Intelligibility was determined in a series of listening studies involving college students. The highest percentage of misarticulations occurred on the single-syllable word list and the most complex prose passage. There were virtually no misarticulations on the two- or three-syllable words. Intelligibility tended to covary with articulation, but the specific nature of the relationship depended on the particular speech task. A misarticulation was especially likely to result in unintelligibility on the most difficult passage, although not all unintelligible words on that passage were attributable to phonemic errors.

Illusory correlation and the MMPI.

The relationships between 10 behavioral descriptions and 10 Minnesota Multiphasic Personality Inventory (MMPI) scale names were judged by 28 naive and 26 experienced student subjects. Both groups of judges produced large amounts of illusory correlation, with the experienced judges producing more. Estimates of strength of semantic association between the behavioral descriptions and scale names provided by 45 naive and 18 experienced male and female subjects tended to support the notion that semantic association is an important factor in producing illusory correlations, but there were some interesting exceptions to this tendency. The testing model was discussed as an analogue to the clinical situation.

Changes of sensory value by interaction of alkanals with amino acids and proteins.

Alkanals as carriers of rancid flavour of fat-containing foods were stored in mixtures with nonlipidic substances. The intensity of odour due to alkanals decreased with increasing time of storage, the changes being more pronounced in case of casein than in that of cellulose. Results of sensory tests did not depend on the molecular mass of aldehyde. The water content affected the sensory evaluation only slightly. The formation of aldolization products or polymers and of gluey flavour compounds modified the character of flavour and confused less experienced judges. Both the binding of alkanals and the aldolization reactions were enhanced by presence of primary amino groups.

Effect of subthreshold concentrations of limonin, naringin and sweeteners on bitterness perception

AbstractA sensory panel of experienced judges was used to determine taste thresholds of mixtures of limonin and naringin and various sweeteners. Data from these experiments showed that within the 95% confidence interval, subthreshold interactions between limonin and naringin were additive, indicating that less than threshold amounts of these compounds contributed to bitterness of a mixture. Mixtures of hesperetin dihydrochalcone glucoside (HDG), neohesperidin dihydrochalcone (NHD), naringin dihydrochalcone (ND), sucrose and naringin were neither additive nor synergistic at subthreshold concentrations. At suprathreshold levels, sucrose, (HDG) and (NHD) were effective in suppressing naringin bitterness, but (ND) was not.

Effect of Prior Experience, Access to Information and Level of Performance on Individual and Group Performance Ratings

The effects of feedback, prior judging experience and the level of the performance being judged on individual and group ratings of the performances were determined. 36 experienced and 36 inexperienced gymnastic judges viewed a film containing 20 gymnastic performances of low, moderate and high caliber under conditions of complete, partial, and no feedback of information as to the ratings of the other judges in their respective groups. On the basis of conformity theory it was hypothesized that groups rating performances in an open system are more reliable and accurate in their ratings than are groups operating in partial or no-feedback conditions. Further, this tendency was hypothesized to become manifest more if performances are of low caliber as opposed to moderate or high caliber and if the observers are experienced at rating performances as opposed to inexperienced. While the data supported these hypotheses, only partial support was achieved for the hypothesis that intraindividual variability will be greatest among no-feedback and inexperienced groups. It was found that inexperienced judges were individually more variable in their ratings than experienced judges and that more intraindividual variability was displayed among ratings of low-level performances. Judges' ratings were compared to absolute criterion scores derived by a group of experts through various film analysis techniques.

A Defence of Mill's Qualitative Hedonism

In his well known proposition that pleasures differ qualitatively, Mill seems to be arguing three principal points. (1) ‘Mental’ pleasures as a kind are intrinsically ‘more desirable and more valuable’ than ‘bodily pleasures’ (p. 12). (2) This estimation of pleasure, Mill says, is such as to rule out the claim that it ‘should be supposed to depend on quantity alone.’ Indeed, he continued, the ‘superiority in quality’ might be ‘so far outweighing quantity as to render it, in comparison, of small account’ (p. 12). (3) The ‘test of quality and the rule for measuring it against quantity,’ Mill says, is ‘the preference’ of experienced judges (p. 16). ‘[T]he judgment of those who are qualified by knowledge of both, or, if they differ, that of the majority among them, must be admitted as final’ (p. 15).

The Visual Description of Carcasses

SummaryA traditional type score card for ten carcass characteristics was tested against an experimental reorganization of this card on a simplified 7 point scaling system. Results showed that both experienced judges and novices were more consistently discriminating when using the revised system. This was most evident when the concept of an optimum was eliminated from the judgements. It was also evident when complex judgements which do not involve an optimum were simplified. The scale of marks and the diversity in maximum marks for different characteristics appeared to have less influence upon consistency and discrimination. However, it is probable that, with traditional cards, the maximum marks are often too high and that the diversity in total possible marks adds a further difficulty for judges.

Scaling of numerical dimensions important to clinical and educational psychologists.

Expert judges see intervals of intelligence from different parts of the scale as unequal in an individually consistent manner and there is also some consistency among judges. There are considerable differences among judges as a function of amount of experience in dealing with the mentally retarded. Highly experienced judges are not influenced in comparing intervals by the traditional categories of intelligence, e.g., Normal, Dull Normal. They also judge intervals from the lower end of the scale as being the largest, whereas other judges see intervals from the upper end of the scale as largest. The implications of the findings for the clinical assessment of change are discussed.

Interscorer reliability and the KTSA.

Abstract : Interpretive use of the Kahn Test of Symbol Arrangement (KTSA) requires that the scorer first make a series of decisions regarding the levels and quantities of abstraction associated with the verbal responses of a subject to the test objects. The assigned scoring categories were the subject of this study of interscorer reliability. The protocols of 181 patients were scored independently by eight experienced judges. Interscorer reliability yielded median phi coefficients for the nine scoring categories between .53 and .92. These coefficients were interpreted as corroborating the adequate interscorer reliability of the test. (Author)

Kinaesthesis and Mental Practic

Abstract Twenty-one male, teachers' college students were given six daily periods, each lasting 5 min., in which they mentally practiced a gross motor skill of which they were previously naive. On the day after the final practice their performance at the skill was rated by three experienced judges. The subjects also completed the Wiebe Test of Kinaesthesis and the scores obtained on this test were found to correlate +.161 with a criterion score derived from the T-scaled form of the mean of the judges ratings.

Visual judgements of quality in meat II. A simplification of the assessment of the proportion of lean to fat in bacon

1. The visual judgement of bacon quality called the ‘proportion of lean to fat’ has been simplified by breaking down the overall assessment of the surface seen on the cut bacon side at the last rib into four simpler scores—of the ‘eye’-muscle size and shape, the amount and distribution of fat on the back rasher, the streak thickness and the proportion of lean to fat in the streak, each judged on a 10-point scale. An experiment is described in which eight experienced judges attempted to score these new characteristics in conjunction with the overall assessment.2. The judging standard adopted in the overall assessment of the proportion of lean to fat varied less from batch to batch in this experiment (in which the seven batches were arrayed in fixed positions and assessed in one day) than in an earlier experiment (when the examples seen in each batch could be shuffled and the eleven batches were seen several days apart). The ranking of judges according to their discrimination and repeatability for the overall assessment was also changed in the second experiment.

Visual judgements of quality in meat I. Assessments of the proportion of lean to fat in bacon from photographs of the cut side

1. Eight experienced judges used a 20-point scoring scale in different ways when assessing the ‘proportion of lean to fat’ from the same photographs of cut bacon sides presented in the same arrangements. The average score for the 220 judgements made during the experiment by a single judge, which represents his over-all level of scoring or ‘standard of judging’, ranged from 13·7 to 12·0. The standard deviation, a measure of the degree of discrimination attempted by the judge, ranged from 3·26 to 4·74.2. The judges varied in their consistency of assessment or ‘repeatability’. The standard error for a single judge, measuring the extent of the variation of his scores within photographs, ranged from 0·89 to 1·99. The judges who attempted to discriminate most between photographs tended to be the least consistent in their judgements, although one judge was a notable exception to this trend.3. The consistency of judgement tended to decrease from the good (high-scoring) rashers to the poor (low-scoring) rashers, but this effect was more marked for some judges than for others.4. Some of the variation in the scores awarded to each photograph was due to alterations in the standard of judging from batch to batch. When this was allowed for, the standard errors were all reduced and ranged from 0·61 to 1·31. The judges tended to adjust their standards according to the average quality of the batch being assessed. This led to the variation among the average scores for the batches being less than it would have been had their standards remained constant.5. Alterations in the standard of judging during the experiment affected the scores awarded to good (high-scoring) rashers rather less than those awarded to poor (low-scoring) ones.6. The correlations between the individual judge's mean scores for the forty-four photographs and the over-all mean scores were very high. For seven judges, they ranged from 0·962 to 0·984, whereas for the eighth judge the correlation was 0·918. The lower correlation for this judge was due to two rashers with very thin fat being heavily marked down.7. The correlations between the over-all mean scores and two different combinations of three objective measurements were both about 0·92; these measurements were therefore slightly less closely related to the over-all scores than were an individual judge's mean scores.8. The possibility of making the experimental technique more realistic and of improving the precision of such visual judgements by providing photographic scales of reference are discussed.