Naive Assessors Research Articles

Taste assessment for paediatric drug Development: A comparison of bitterness taste aversion in children versus Naïve and expert young adult assessors

Medicines for children often taste bitter, presenting a significant challenge to treatment compliance. However, most studies on paediatric drug development rely on adult volunteers for sensory research, and the level of expertise required from these assessors is unclear. This study aimed to address this gap by investigating perceived bitterness aversion to taste strips impregnated with different concentrations of quinine hydrochloride in 439 school-aged children. Expert (n = 26) and naïve (n = 65) young adult assessors evaluated quinine solutions as well as taste strips, for methodological bridging purposes. All assessors differentiated the aversiveness of the taste strips in a dose dependent manner. Younger children aged 4–8 years had difficulty discriminating higher bitter concentrations, whereas pre-adolescents 9–11 years and naive adults showed better discrimination at the top of the scale. Naive assessors showed similar bitter perception as children. However, the results were slightly different between strips and solution in adults. These findings highlight the key role that adult panels can play in paediatric formulation development. Taste strips show promise as a safe and pragmatic tool for sensory pharmaceutical evaluations, though further studies are warranted to establish the relationship between age and hedonic taste perception using compounds with diverse physicochemical and sensory qualities.

Sensory analysis and Principal Component Analysis: a sustainable approach for quality control of stretch denim fabrics

This study proposes a simplified approach to optimise the portfolio of denim manufacturers by reducing the cost of unattractive or undifferentiated assortments for future consumers based on their ability to perceive and discriminate sensory comfort. A key factor in a successful textile value chain is end consumers, who can be considered naive evaluators when it comes to sensory analysis, as they tend to touch apparel fabrics to perceive the sensory comfort they feel when wearing them. In this context, 16 naive assessors were recruited to quantitatively characterise six bipolar sensory attributes as hand descriptors of an assortment of five washed stretch denim fabrics based on tactile properties. Statistical analysis of the extensive data was performed using the multivariate technique PCA. Two principal components that explained 70.46% to 76.67% of the total observed variance for the five washed denim fabrics provided an adequate summary of the sensory data reported, so relationships between sensory attributes as hand descriptors and ratings given by the 16 untrained assessors were examined. Statistical tests showed that all six bipolar attributes were important for sensory analysis and that inter-rater agreement was low. However, considering the sensory perception of the untrained evaluators (i.e., consumers), it was concluded that four of the five washed stretch denim fabrics could be an option for product portfolio diversification. Thus, from a strategic and sustainability perspective, PCA for sensory analysis may be helpful in the quality control of washed stretch denim

Tactile Perception in the Sensory Comfort of Fabric Samples

This investigation aims to present the process of development of the attributes that will form the textile lexicon of the northeast region of Brazil for the assessment of textile tactile comfort. For this purpose, the following were adapted: ISO 11035:1994 Sensory analysis – Identification and selection of descriptors for establishing a sensory profile by a multidimensional approach; and ISO 8586:2012 Sensory analysis – General guidelines for the selection, training and monitoring of selected assessors and expert sensory assessor’s standards concerning the areas of food and cosmetics. Three panels of naive assessors from three different cities in northeastern Brazil were invited to touch 8 samples of fabrics with different textures and compositions. Initially, they generated 322 terms. Afterwards, in a second phase, they qualitatively eliminated terms with the same meaning. In the third phase, they grouped the terms through similarity analysis, which resulted in 23 terms. Then, the terms that were most cited by the assessors were analyzed and, as a result, 4 terms were eliminated. Finally, similarities were evidenced in the haptic perceptions that were perceived by the different panels, due to their cultural proximity and the fact that they come from the same region of a country with continental dimensions, as well as the transformation of human subjectivity into objective parameters provided by the textile lexicon.

Non-technical Skills for Medical Students: Validating the Tools of the Trade.

The Medical Students’ Non-Technical Skills (Medi-StuNTS) is a behavioural marker scheme (BMS) designed to assess non-technical skills (NTS) in medical students in emergency simulations. This study aimed to assess the evidence for validity and usability of Medi-StuNTS by naive, near-peer educators. Nine doctors assessed four students in simulations of common medical emergencies. The scores were used to assess inter-rater reliability, inter-class correlation, and observability. Students and assessors completed questionnaires that assessed the tool’s usability and consequence.Inter-rater agreement across all skill elements was “high” with rWG scores >0.8. An inter-class correlation was “good” with ICC3K kappa scores of 0.86 and 0.89 overall, when measured per simulation and per skills element respectively. Overall skill observability was high (>80%) except for coping with stress. Assessors found the tool “difficult to use” but “useful for feeding back in a constructive way”. Students appreciated the comprehensiveness of the feedback as well as knowing what to expect during debriefs.This study has shown that the Medi-StuNTS BMS has good usability and evidence of validity in naive assessors and near-peer educators. It shows the particularly good internal structure and overall beneficial consequences. Further study will be necessary to understand how best to deploy it in formative and summative contexts.

Hedonic and descriptive sensory evaluation of instant and fresh coffee products

Sensory acceptance testing of instant and filtered coffees using naive assessors (n = 40) and rapid descriptive profiling using trained assessors (n = 10) was used to determine the sensory hedonic and descriptive drivers of 13 (8 filtered, 5 instant) of the most popular coffee brands on the Irish and UK market. Training was undertaken using commercial coffees to highlight sensory attributes, which is common in industry. ANOVA-Partial Least Squares Regression (APLSR) was used to process the data generated by the assessors. Sensory results indicated that training was effective and that filter coffees were more positively correlated with liking of flavor, aroma and overall acceptability. In contrast, instant coffees were negatively correlated with hedonic attributes (with the exception of one instant coffee, I2). This indicates large quality and sensory variation between instant and fresh filter coffees currently on the market. It also shows a large variation between the instant coffees currently on the market. It is clear that instant coffee lacks the desirable aromas and flavors associated with fresh filter coffee as determined from the descriptive analysis. Thus, we can conclude that there is market potential for a good quality instant coffee that has similar attributes to a fresh filter coffee.

Psoriasis and Psoriatic Arthritis Video Project: An Update from the 2012 GRAPPA Annual Meeting

The Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) has developed online videos intended to provide training on the most commonly used physical examination measures for psoriasis and psoriatic arthritis (PsA). At the 2012 GRAPPA annual meeting, attendees were updated on the development, availability, use, and validation of these video modules. To date, 1300 users from 45 different countries have used the Psoriasis Area and Severity Index (PASI) module at least once. Results were presented from a recently completed study of pre- and post-video scoring of the PASI by experienced and naive physicians and patient assessors. Future modifications of the video collection were also discussed.

Psoriasis and Psoriatic Arthritis Video Project: An Update from the GRAPPA 2011 Annual Meeting

Numerous physical examination instruments are used to assess and measure severity of psoriasis and psoriatic arthritis (PsA) in practice and in clinical trials. The Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) has developed several online training modules used by GRAPPA members and investigators participating in psoriasis and PsA research. At the 2011 GRAPPA meeting, attendees were updated on the ongoing development of the training modules. Several Internet-based multimedia presentations for psoriasis and PsA assessments have been completed. Available psoriasis modules include the Psoriasis Area and Severity Index (PASI) and Body Surface Area, one 5-point and two 6-point Physician Global Assessments, the original and modified Nail Psoriasis Severity Index, the Palmar-Plantar Pustular Psoriasis Area and Severity Index, the Psoriasis Scalp Severity Index, and the Total Plaque Severity Score. Rheumatology modules that demonstrate evaluation of swollen and tender joints, enthesitis, and dactylitis are now available, and an axial disease evaluation module is near completion. Each video includes the background and rationale for each measure, demonstration videos of select examinations, diagrams, and photographs to emphasize teaching points, and for most dermatology modules, an optional test to assess competence. Preliminary data generated by a pilot study of pre- and post-education PASI scoring by experienced and naive physicians and patient assessors were presented, revealing improved accuracy of scoring after viewing the PASI video. Attendees agreed that additional patient examples with more diverse skin types and psoriasis phenotypes, translation to languages other than English, and further validation studies are needed.

Peer assessment of oral presentations: effects of student gender, university affiliation and participation in the development of assessment criteria

Peer assessment provides a useful mechanism to develop many positive qualities in students studying in higher education (HE). Potential influences on peer‐awarded marks include student qualities such as gender, HE background (e.g. university affiliation) and participation in the development of the assessment criteria. Many studies that have investigated peer assessment have placed great emphasis on marks from a single tutor, or very few tutors, from a single university. This study examined grades awarded by 11 tutors (affiliated with four universities) to oral presentations delivered on a residential field course by second‐year undergraduate students from two universities studying environmental or biological disciplines. Student assessors awarded marks of fairly high precision (correlating strongly with tutor grades) but averaged 5% higher than their tutors (i.e. of only moderate accuracy). Marginally higher marks (circa 1.6%) were awarded by student assessors to speakers studying at the same university. Gender influences were detected as males tended to grade other male speakers very slightly more highly than female speakers. Marks from females were unaffected by speaker gender. Students who participated in the development of the assessment criteria did not achieve higher grades for their presentations. However, when these ‘participants’ were assessing, they awarded lower marks than their peers (i.e. closer to, but not as low as, those awarded by tutors). Lower marks were also awarded during the middle of sessions, possibly resulting from factors associated with motivation and attention of speakers and markers. Overall, the potential biases in marking by naive assessors examined in this study may influence the validity of marks generated by peer assessment schemes, but the experience of this type of assessment had positive effects on those involved.

THE RELIABILITY OF NAÏVE ASSESSORS IN SENSORY EVALUATION VISUALIZED BY PRAGMATICAL MULTIVARIATE ANALYSIS

AbstractThe first part of this paper demonstrates a simple graphical way to visualize estimated variances, in terms of a plot of the total initial variance (“SIGNAL”) versus residual variance (“NOISE”), as a pragmatic alternative to tables of F‐tests. The recently developed Procrustes rotation in the bilinear “jack‐knifing” form is then presented as a method for simplifying the comparison of PLS Regression models from different data sets. These methods are applied to sensory data in order to study if naïve (untrained) sensory panelists can produce reliable descriptions of systematic differences between various test meals. The results confirm that three panels of 15 naïve assessors each could give repeatable intersubjective description of the most dominant sensory variation dimensions.

Application of Finger Span Cross Modality Matching Method (FSCM) by Naive Assessors for Olfactometric Discrimination of Gewürztraminer Wines

A gas-chromatography-olfactometric detection (GCO) using a direct estimation of odor intensities with the finger span (FSCM) was used to establish aroma profiles of three wines of gewürztraminer from different ‘terroirs’ of the Alsace region. The GCO-FSCM analyses of their hydroalcoholic wine extracts were performed using a group of untrained assessors. Forty-four olfactory signals were perceived by at least four assessors out of six and their intensities were scored in triplicate in each of the three wine extracts, amongst which ten of them were found discriminant between wines. The reliability of each assessor was studied on the basis of the finger span estimation repeatability, from the number of discriminant variables and the mean finger span values found for the assessor. These untrained assessors appear to be able to achieve self-calibration from odor intensities without using any internal of external quantitative reference. The results suggest that GCO-FSCM may be performed by naive assessors to analyse and compare odor profiles of complex extracts without time-consuming training periods.

EFFECT OF ASSESSORS EXPERTISE LEVEL ON EFFICIENCY OF WARM-UP FOR TRIANGLE TESTS

The effect of warm-up on performance for repeated triangle tests is studied according to assessors' expertise level for both triangle test strategy and the pair of products to compare. Three experiments performed with orange flavored soft drinks show that the effect of warm-up depends on the assessors' expertise: (1) naive assessors do not increase their performance with warm-up; (2) assessors with a moderate practice of both triangle tests and the pair of products improve their performance with warm-up; (3) assessors with a moderate practice of triangle tests, but not familiar with the pair of products, improve their performance with warm-up too; and (4) assessors highly experienced for both triangle tests and products do not improve their performance with warm-up. These results support the idea that the positive effect of warm-up is due to an attentional process: Warm-up seems to help assessors focusing their attention on the dimension on which the two products are actually different and ignoring the other dimensions. Thus, results show that assessors are able to learn the difference between the two products and to learn to focus their attention on this difference at any time. They also suggest that prior knowledge of the triangle test scheme is required to benefit from warm-up.

COMPARATIVE TRAINING PROCEDURES TO LEARN ODOR DESCRIPTORS: EFFECTS ON PROFILING PERFORMANCE1

Three groups of ten naive assessors were recruited to perform an odor profiling of 10 orange juices using 8 odor descriptors. These panels differed on the way they learned each descriptor. Group 1R learned to match one descriptor with one external standard; group 3R learned to associate one descriptor with 3 external standards; group OR defined by itself the 8 descriptors from a set of orange juices. Results showed that the use of one standard per descriptor seemed to be only efficient when the standard was typical of the odor perception in the orange juices. Learning one odor concept with 3 standards led to redundant use of discriminant descriptors and failed on the agreement among assessors. Finally, group OR was a discriminant as well a homogenous panel.

Prototypes of Facial Attributes Developed Through Image Averaging Techniques

Image capture and quantification has proven useful in a variety of scientific applications, for example, biology, medicine, geology, meteorology and forensics. The objective of this research was to utilize this technology to quantify clinical- and consumer-perceivable changes in facial attributes. A panel of expert assessors was trained, and, in a large consumer study, consumer facial attributes were identified and grading scales for each attribute were established. These experts then rated over 240 subjects on a total of 19 different facial attributes. Based on methodology developed by Perrett et al., facial averages or prototypes were computed from panelists rated high or low for each attribute. Prototypes were developed in a 3 step process: 1) selection of 224 predefined feature points; 2) calculation of average face shape; and 3) 'morphing' individual faces into that shape and blending the images together. Naive assessors could readily appreciate the differences in facial appearance of the prototypes. In addition, expert graders were able to identify the general class of attribute affected. This method provides a powerful tool for assessing the effects of skin care technologies.

Intuitive Perceptions of Decision‐making Strategy: Naive Assessors' Concepts of Integrative Complexity

Although there are measurable differences in integrative complexity among solutions that individuals generate in dealing with problems, it is uncertain to what extent people comprehend, recognize, and have preferences among different levels of complexity. Integrative complexity is a function of differentiation (the perception of several attributes within, or perspectives about, a topic) and integration (combining the differentiated characteristics in an interactive or synthesizing solution). The current paper reports two experiments dealing with how university students perceive, interpret, and choose among solutions differing in complexity. Experiment 1 showed that subjects accurately rated the complexity of described solutions differing along the continuum, but that their assessment of their own responses differed from the results of expert scoring. Their self‐estimated complexity was highly correlated with their preferences, and preferred complexity was reliably higher than either expert‐ or self‐assessed complexity of subject‐generated solutions. Subjects were able to hypothesize quite accurately about environmental and endogenous factors likely to affect complexity. Experiment 2 found that in response to problem scenarios, solutions selected as being potentially most effective were consistently more complex than solutions that participants considered themselves most likely to use. The idea of complexity seems to be intuitively recognizable and understandable by untrained subjects: They can and do distinguish among problem solutions (self‐generated or presented) that vary on that dimension, and are able to assess accurately the effects of relevant variables. Such subjects also share the bias shown by experts in favour of the superiority of more complex approaches.