Individual Assessor Research Articles

Video-based peer assessment of collaborative teamwork in a large-scale interprofessional learning activity

BackgroundThe assessment of team performance within large-scale Interprofessional Learning (IPL) initiatives is an important but underexplored area. It is essential for demonstrating the effectiveness of collaborative learning outcomes in preparing students for professional practice. Using Kane’s validity framework, we investigated whether peer assessment of student-produced videos depicting collaborative teamwork in an IPL activity was sufficiently valid for decision-making about team performance, and where the sources of error might lie to optimise future iterations of the assessment.MethodsA large cohort of health professional students (n = 1218) of 8 differing professions was divided into teams containing 5–6 students. Each team collaborated on producing a short video to evidence their management of one of 12 complex patient cases. Students from two other teams, who had worked on the same case, individually rated each video using a previously developed assessment scale. A generalisability study quantified sources of error that impacted the reliability of peer assessment of collaborative teamwork. A decision study modeled the impact of differing numbers of raters. A modified Angoff determined the pass/fail mark.ResultsWithin a large-scale learning activity, peer assessment of collaborative teamwork was reliable (G = 0.71) based on scoring by students from two teams (n = 10–12) for each video. The main sources of variation were the stringency and subjectivity of fellow student assessors. Whilst professions marked with differing stringency, and individual student assessors had different views of the quality of a particular video, none of that individual assessor variance was attributable to the assessors’ profession. Teams performed similarly across the 12 cases overall, and no particular professions marked differently on any particular case.ConclusionA peer assessment of a student-produced video depicting interprofessional collaborative teamwork around the management of complex patient cases can be valid for decision-making about student team performance. Further refining marking rubrics and student assessor training could potentially modify assessor subjectivity. The impact of professions on assessing individual peers and the case-specificity of team performances in IPL settings need further exploration. This innovative approach to assessment offers a promising avenue for enhancing the measurement of collaborative learning outcomes in large-scale Interprofessional learning initiatives.

Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury – a comparison of an image extraction and classification system to expert scoring

BackgroundWalking impairment is a common disability post acquired brain injury (ABI), with visually evident arm movement abnormality identified as negatively impacting a multitude of psychological factors. The International Classification of Functioning, Disability and Health (ICF) qualifiers scale has been used to subjectively assess arm movement abnormality, showing strong intra-rater and test-retest reliability, however, only moderate inter-rater reliability. This impacts clinical utility, limiting its use as a measurement tool. To both automate the analysis and overcome these errors, the primary aim of this study was to evaluate the ability of a novel two-level machine learning model to assess arm movement abnormality during walking in people with ABI.MethodsFrontal plane gait videos were used to train four networks with 50%, 75%, 90%, and 100% of participants (ABI: n = 42, healthy controls: n = 34) to automatically identify anatomical landmarks using DeepLabCut™ and calculate two-dimensional kinematic joint angles. Assessment scores from three experienced neurorehabilitation clinicians were used with these joint angles to train random forest networks with nested cross-validation to predict assessor scores for all videos. Agreement between unseen participant (i.e. test group participants that were not used to train the model) predictions and each individual assessor’s scores were compared using quadratic weighted kappa. One sample t-tests (to determine over/underprediction against clinician ratings) and one-way ANOVA (to determine differences between networks) were applied to the four networks.ResultsThe machine learning predictions have similar agreement to experienced human assessors, with no statistically significant (p < 0.05) difference for any match contingency. There was no statistically significant difference between the predictions from the four networks (F = 0.119; p = 0.949). The four networks did however under-predict scores with small effect sizes (p range = 0.007 to 0.040; Cohen’s d range = 0.156 to 0.217).ConclusionsThis study demonstrated that machine learning can perform similarly to experienced clinicians when subjectively assessing arm movement abnormality in people with ABI. The relatively small sample size may have resulted in under-prediction of some scores, albeit with small effect sizes. Studies with larger sample sizes that objectively and automatically assess dynamic movement in both local and telerehabilitation assessments, for example using smartphones and edge-based machine learning, to reduce measurement error and healthcare access inequality are needed.

Development of a reliable surgical quality assurance tool for gastrectomy in oncological trials

BackgroundDespite its recognized importance, there is currently no reliable tool for surgical quality assurance (SQA) of gastrectomy in surgical oncology. The aim of this study was to develop an SQA tool for gastrectomy and to apply this tool within the ADDICT Trial in order to assess the extent and completeness of lymphadenectomy.MethodsThe operative steps for D1+ and D2 gastrectomy have been previously described in the literature and ADDICT trial manual. Two researchers also performed fieldwork in the UK and Japan to document key operative steps through photographs and semi-structured interviews with expert surgeons. This provided the steps that were used as the framework for the SQA tool. Sixty-two photographic cases from the ADDICT Trial were rated by three independent surgeons. Generalizability (G) theory determined inter-rater reliability. D-studies examined the effect of varying the number of assessors and photographic series they rated. Chi-square assessed intra-rater reliability, comparing how the individual assessor’s responses corresponded to their global rating for extent of lymphadenectomy.ResultsThe tool comprised 20 items, including 19 anatomical landmarks and a global rating score. Overall reliability had G-coefficient of 0.557. Internal consistency was measured with a Cronbach’s alpha score of 0.869 and Chi-square confirmed intra-rater reliability for each assessor as < 0.05.ConclusionsA photographic surgical quality assurance tool is presented for gastrectomy. Using this tool, the assessor can reliably determine not only the quality but also the extent of the lymphadenectomy performed based on remaining anatomy rather than the excised specimen.

Odor emission measurements: The role of n-butanol as a reference material and origins of large inter-laboratory variability

Malodors negatively impact the quality of life of people who are exposed to them. In order to enforce legal emission limit values that aim to reduce these adverse effects, reliable methods for the quantification of odor emissions are needed. The standard EN 13725 describes a method to quantify odor concentration, which is based on the use of human assessors as sensors. Previously published work that studied the performance characteristics of this method indicates that the variability of results between different laboratories is higher than expected and that the uncertainty associated with this method may be underestimated. We analyze here a unique data set, which contains the response of 368 assessors that participated in 21 odor proficiency tests performed in 2016–2022. By investigating the response of the smallest unit of an odor measurement by EN 13725, namely the individual assessor, as opposed to the panel, we gain unprecedented insight into the sources of inter-laboratory variations in odor measurements. Our results indicate that the assessor sensitivity to n-butanol is not transferable to other odorants and odor mixtures, which is contrary to the assumption made in EN 13725. By Monte-Carlo simulation coupled with our data set, we investigate how the responses of individual assessors transfer to panel responses, focusing on the influence of panel size, panel selection strategy and number of rounds. We find that for the laboratories investigated, larger panels only modestly reduce the inter-laboratory variability and we attribute this effect to non-random selection of assessors for the panels of these laboratories. The present work provides new insight into the sources of inter-laboratory variability in odor emission monitoring and is expected to be useful in the further development of reliable methods for odor concentration measurements and odor abatement technology.

Fairness in Assessment: Identifying a Complex Adaptive System.

Assessment design in health professions education is continuously evolving. There is an increasing desire to better embrace human judgement in assessment. Thus, it is essential to understand what makes this judgement fair. This study builds upon existing literature by studying how assessment leaders conceptualise the characteristics of fair judgement. Sixteen assessment leaders from 15 medical schools in Australia and New Zealand participated in online focus groups. Data collection and analysis occurred concurrently and iteratively. We used the constant comparison method to identify themes and build on an existing conceptual model of fair judgement in assessment. Fairness is a multi-dimensional construct with components at environment, system and individual levels. Components influencing fairness include articulated and agreed learning outcomes relating to the needs of society, a culture which allows for learner support, stakeholder agency and learning (environmental level), collection, interpretation and combination of evidence, procedural strategies (system level) and appropriate individual assessments and assessor expertise and agility (individual level). We observed that within the data at fractal, that is an infinite pattern repeating at different scales, could be seen suggesting fair judgement should be considered a complex adaptive system. Within complex adaptive systems, it is primarily the interaction between the entities which influences the outcome it produces, not simply the components themselves. Viewing fairness in assessment through a lens of complexity rather than as a linear, causal model has significant implications for how we design assessment programs and seek to utilise human judgement in assessment.

Novice assessors demonstrate good intra-rater agreement and reliability when determining pressure pain thresholds; a cross-sectional study.

Experienced assessors show good intra-rater reproducibility (within-session and between-session agreement and reliability) when using an algometer to determine pressure pain thresholds (PPT). However, it is unknown whether novice assessors perform equally well. This study aimed to determine within and between-session agreement and reliability of PPT measurements performed by novice assessors and explored whether these parameters differed per assessor and algometer type. Ten novice assessors measured PPTs over four test locations (tibialis anterior muscle, rectus femoris muscle, extensor carpi radialis brevis muscle and paraspinal muscles C5-C6) in 178 healthy participants, using either a Somedic Type II digital algometer (10 raters; 88 participants) or a Wagner Force Ten FDX 25 digital algometer (nine raters; 90 participants). Prior to the experiment, the novice assessors practiced PPTs for 3 h per algometer. Each assessor measured a different subsample of ~9 participants. For both the individual assessor and for all assessors combined (i.e., the group representing novice assessors), the standard error of measurement (SEM) and coefficient of variation (CV) were calculated to reflect within and between-session agreement. Reliability was assessed using intraclass correlation coefficients (ICC1,1). Within-session agreement expressed as SEM ranged from 42 to 74 kPa, depending on the test location and device. Between-session agreement, expressed as SEM, ranged from 36 to 76 kPa and the CV ranged from 9-16% per body location. Individual assessors differed from the mean group results, ranging from -55 to +32 kPa or from -9.5 to +6.6 percentage points. Reliability was good to excellent (ICC1,1: 0.87 to 0.95). Results were similar for both types of algometers. Following 3 h of algometer practice, there were slight differences between assessors, but reproducibility in determining PPTs was overall good.

A prospective, randomized assessment of a novel, local antibiotic releasing platform for the prevention of superficial and deep surgical site infections

BackgroundDespite significant advances in infection control guidelines and practices, surgical site infections (SSIs) remain a substantial cause of morbidity, prolonged hospitalization, and mortality among patients having both elective and emergent surgeries. D-PLEX100 is a novel, antibiotic-eluting polymer–lipid matrix that supplies a high, local concentration of doxycycline for the prevention of superficial and deep SSIs. The aim of our study was to evaluate the safety and efficacy of D-PLEX in addition to standard of care (SOC) in preventing superficial and deep surgical site infections for patients undergoing elective colorectal surgery.MethodsFrom October 10, 2018 to October 6, 2019, as part of a Phase 2 clinical trial, we randomly assigned 202 patients who had scheduled elective colorectal surgery to receive either standard of care SSI prophylaxis or D-PLEX100 in addition to standard of care. The primary objective was to assess the efficacy of D-PLEX100 in superficial and deep SSI reduction, as measured by the incidence of SSIs within 30 days, as adjudicated by both an individual assessor and a three-person endpoint adjudication committee, all of whom were blinded to study-group assignments. Safety was assessed by the stratification and incidence of treatment-emergent adverse events.ResultsOne hundred and seventy-nine patients were evaluated in the per protocol population, 88 in the intervention arm [51 males, 37 females, median age (64.0 range: 19–92) years] and 91 in the control arm [57 males, 34 females, median age 64.5 (range: 21–88) years]. The SSI rate within 30 day post-index surgery revealed a 64% relative risk reduction in SSI rate in the D-PLEX100 plus standard of care (SOC) group [n = 7/88 (8%)] vs SOC alone [n = 20/91 (22%)]; p = 0.0115. There was no significant difference in treatment-emergent adverse events.ConclusionsD-PLEX100 application leads to a statistically significant reduction in superficial and deep surgical site infections in this colorectal clinical model without any associated increase in adverse events.

An updated survey of beliefs and practices related to faking in individual assessments

AbstractThe present study is an updated survey examining individual assessor beliefs and practices related to faking in the individual assessment context. The responses from a mix of quantitative and qualitative survey questions were compared across individual assessors from the original 2005 sample (n = 77) and an updated 2020 sample (n = 78). Results suggest that single stimulus personality assessments are still the predominant form of personality assessment in use, but many individual assessors employ other types of personality assessments such as forced‐choice. In 2020, individual assessors do not appear to be heavily concerned about the effects of faking on their recommendations, do not believe that a large number of candidates fake, and believe that even fewer candidates successfully fake.

HRCT characteristics of severe emphysema patients: Interobserver variability among expert readers and comparison with quantitative software

PurposeFor a successful bronchoscopic lung volume reduction coil treatment it is important to place the coils in the most emphysematous lobes. Therefore assessment of the lobe with greatest destruction is essential. Our aims were to investigate the level of agreement among expert reviewers of HRCT-scans in emphysema patients and the comparison with QCT (quantitative computed tomography) software. MethodFive experienced CT-assessors, conducted a visual assessment of the baseline HRCT-scans of emphysema patients who participated in the RENEW bronchoscopic lung volume reduction coil study. On the same HRCT-scans, a QCT analysis was performed. ResultsIn total 134 HRCT-scans were rated by all 5 experts. All 5 CT-assessors agreed on which was the most destructed lobe in 61 % of the left lungs (ƙ:0.459) and 60 % of the right lungs (ƙ:0.370). The consensus of the 5 assessors matched the QCT in the left lung for 77 % of the patients (ƙ:0.425) and in the right lung for 82 % (ƙ:0.524). ConclusionsOur results show that the interobserver agreement between five expert CT-assessors was only fair to moderate when evaluating the most destructed lobe. CT-assessor consensus improved matching with QCT determination of lobar destruction compared to individual assessor determinations. Because some CT-features are associated with treatment outcomes and important for optimal patient selection of bronchoscopic lung volume reduction treatment, we recommend including more than one CT-reviewer and supported by QCT measurements.

Managing extremes of assessor judgment within the OSCE

Context: There is a growing body of research investigating assessor judgments in complex performance environments such as OSCE examinations. Post hoc analysis can be employed to identify some elements of “unwanted” assessor variance. However, the impact of individual, apparently “extreme” assessors on OSCE quality, assessment outcomes and pass/fail decisions has not been previously explored. This paper uses a range of “case studies” as examples to illustrate the impact that “extreme” examiners can have in OSCEs, and gives pragmatic suggestions to successfully alleviating problems.Method and results: We used real OSCE assessment data from a number of examinations where at station level, a single examiner assesses student performance using a global grade and a key features checklist. Three exemplar case studies where initial post hoc analysis has indicated problematic individual assessor behavior are considered and discussed in detail, highlighting both the impact of individual examiner behavior and station design on subsequent judgments.Conclusions: In complex assessment environments, institutions have a duty to maximize the defensibility, quality and validity of the assessment process. A key element of this involves critical analysis, through a range of approaches, of assessor judgments. However, care must be taken when assuming that apparent aberrant examiner behavior is automatically just that.

The relationship between basic audio quality and overall listening experience.

Basic audio quality (BAQ) is a well-known perceptual attribute, which is rated in various listening test methods to measure the performance of audio systems. Unfortunately, when it comes to purchasing audio systems, BAQ might not have a significant influence on the customers' buying decisions since other factors, like brand loyalty, might be more important. In contrast to BAQ, overall listening experience (OLE) is an affective attribute which incorporates all aspects that are important to an individual assessor, including his or her preference for music genre and audio quality. In this work, the relationship between BAQ and OLE is investigated in more detail. To this end, an experiment was carried out, in which participants rated the BAQ and the OLE of music excerpts with different timbral and spatial degradations. In a between-group-design procedure, participants were assigned into two groups, in each of which a different set of stimuli was rated. The results indicate that rating of both attributes, BAQ and OLE, leads to similar rankings, even if a different set of stimuli is rated. In contrast to the BAQ ratings, which were more influenced by timbral than spatial degradations, the OLE ratings were almost equally influenced by timbral and spatial degradations.

P167. Patient satisfaction following immediate breast reconstruction using 2 stage implants

S S73 Conclusion: There is a growing body of literature regarding 3D-SI. However, evidence of its superiority over current methods of clinical decision making, surgical planning, communication and evaluation of outcome is required before it can be accepted into mainstream practice. http://dx.doi.org/10.1016/j.ejso.2015.03.204 P167. Patient satisfaction following immediate breast reconstruction using 2 stage implants Claire Quinnell, Eifion Vaughan-Williams 1 Cardiff University, Cardiff, UK Breast Unit, Royal Glamorgan Hospital e Cwm Taf NHS Trust, Llantrisant, UK Introduction: Studies have shown that breast reconstruction is an important determinant of long-term health, and is recommended for all patients without contraindications following mastectomy. The aim of this study was to ascertain the level of patient satisfaction with the cosmetic outcomes following immediate breast reconstruction (IBR). Methods: Data was collected for 127 patients, treated by IBR under a single surgeon between January 2009 and May 2014. Nineteen patients were excluded (10 failed IBRs, 9 expanders not exchanged), the remaining 108 were sent the BREAST-Q questionnaire; 74 were returned, 5 were unanswered. Results: Sixty nine questionnaires were included (64%) with a mean patient age 49 (30e73) years. Forty four (64%) reconstructed breasts matched or superseded the patients’ natural cup size; the largest implant used was 795cc e which gave its recipients at the most, a DD cup. Of the patients whose reconstructions did not match, 68% had at least a D cup naturally. Following IBR, thirty three (48%) women had further elective surgery to the contralateral breast. With regards to shape, size and alignment, 57 (83%), 50 (72%) and 37 (54%) of patients reported that they are at least somewhat happy with each of these elements respectively, and no patients reported very unsatisfactory results with their appearance when clothed. Despite encouraging results only 75% reported their reconstructions matched their expectations Conclusion: This studyhas shown that IBRoffers cosmetically acceptable results, especially if the natural breast was small. However there is clearly an opportunity for progress with regards to meeting patient expectations. http://dx.doi.org/10.1016/j.ejso.2015.03.205 P168. The ability of human observers to discriminate cosmetic outcomes of DIEP breast reconstruction following mastectomy Andrew Clayton Lee, Justin Chatterjee, Andrew Devine 1 Changi General Hospital, Singapore NHS Tayside, Dundee, UK Department of Plastic Surgery, University of British Columbia, British Columbia, Canada Dundee University School of Medicine, Dundee, UK Aim: Most systems for assessing cosmetic outcomes of breast reconstructions use humans as assessors. The validity of such systems was dependent on the ability of individuals to discriminate between good and bad cosmetic outcomes consistently within individuals and between individuals, which had never been studied. The aim of this study was to test the hypothesis that individuals were capable to discriminate cosmetic outcomes with strong intra-assessor and inter-assessor reliabilities. Methods: Clinical photographs of 20 women who underwent free DIEP-flap breast reconstruction were selected from a series of 96 to ensure a full range of aesthetic outcomes. 24 assessors ranked the clinical photographs in order according to the following parameters on 2 separate occasions: (A) Overall appearance, differences in size, shape and position between the reconstructed and natural breasts, and (B) Overall appearance, shape, deformity and scarring of the reconstructed breast. Each photo received a score fromeach ranking activity, whichwas equivalent to the respective rank position. The primary endpoints were (a) median Spearman’s rank correlation coefficients (rs) between the 1st and 2nd rankings of each individual assessor and (b) Kendall’s coefficient of concordance for ranks (W) for inter-observer concordance, for all of the above parameters. For the hypothesis to be accepted, median rs and W had to be greater than 0.70 and 0.50 respectively for each of the cosmetic parameters. Results: Median rs andW for each parameter were recorded in Table 1.

Aspects of rating communicative interaction

Communication partner training can be effective in improving communication in aphasia. However, further research is needed on how to measure the outcome of such interventions. In this paper we discuss the phenomenon of reliability in assessments in relation to the results of analyses using a rating scale designed to measure the ability to support a person with aphasia in natural conversational interaction. The scale was used by four assessors to rate 45 video recordings. Calculations of reliability and agreement produced varying results but were mostly satisfactory. However, the results highlight how interaction between factors such as complexity of assessments; design of the rating scale; factors inherent in the individual assessor; and the statistical measures used to analyse the outcome may result in a conflict between aspects of validity and reliability. Interpretations of outcome obtained with rating scales thus need to be based on knowledge about factors influencing the results.

Evaluating Objective Structured Clinical Assessment (OSCA) in undergraduate nursing

The aims of this mixed methods study were to gain insight into how individual assessors determine an Objective Structured Clinical Assessment (OSCA) result for undergraduate nursing students and identify whether individual assessor perceptions and professional characteristics have an impact on students' results. Results from 25 participants showed that although less than half (44%) of the participants were teaching in the course that they were assessing, the participants were highly experienced clinicians and nearly three-quarters (72%) had completed formal teaching qualifications. There were wide variations in pass rates (16.7–90%) between assessors. The widest disparity was observed between assessors with and those without critical care experience (66% versus 39%), as well as assessors who were teaching the course and those who were not (68% versus 49%). Qualitative analysis revealed three dominant themes within participants' transcripts. The themes focused on determining student safety, and the use of personal perceptions and clinical experience to determine competency. Findings indicate that assessors' individual perceptions and clinical experience have the potential to influence and determine undergraduate nursing students OSCA results. Development of criteria standards and objective assessment may be enhanced by greater involvement of assessors and thorough education and training within the context of student assessments.

Crowdsourcing to Assess MOOCs: A Position Paper

The argument against assessment in MOOCs is that traditional means are impossible due to the volume of students. Crowdsourcing provides quality assessment through its diversity and in its patterns of open discourse. Crowdsourced responses help the assessment activity to serve as a knowledge building tool for both the student and the participating community. The authors advance the argument that two indices of learning are a student's ability to integrate learning elements and the subsequent discourse initiated by the student's integration. Based on this premise, student performance in MOOCs can be quantitatively assessed by a means that takes advantage of the volume of students, as opposed to being hindered by it. In the suggested method provided in the article, an instructor generates a collection of learning elements, and students are prompted to generate integration rationales that link these elements and then post the rationales on a discussion board. Data is then collected on the number of integration rationales the student creates, the number of integration rationales that receive a response from other MOOC participants, and the volume of responses individual rationales receive. The authors note that the individual assessor makes the determination of how the data is interpreted and applied.

Caffeine Perception, Effects of Matrix Complexity, and Individual Sensitivity

Background: Caffeine is included as a “flavoring” in over 60% of soft drinks consumed in the United States and its inclusion has come under scrutiny in recent years. Recent trends have seen an increase in products containing elevated caffeine levels, marketed as energy drinks, but few data are available regarding the impact of higher caffeine levels on sensory perception of beverages. Method: This study examined the ability of assessors to discriminate between caffeinated and noncaffeinated solutions of increasing complexity. Individual assessor's caffeine threshold levels were determined and a forced-choice triangle test methodology (BS ISO 4120:2004) was used to compare caffeinated (0.12 and 0.48 g/L; comparable to both standard and energy drink levels) and noncaffeinated model beverages. The model beverages increased in complexity from simple water solutions to glucose-sweetened solutions; citrus-flavored solutions; and flavored, sweetened, carbonated solutions. Results: Results indicated that caffeine...

Prediction of the handle characteristics of lightweight next-to-skin knitted fabrics using a fabric extraction technique

By testing fabric extraction properties, the PhabrOmeter Fabric Evaluation System offers the potential for routine objective evaluation of fabric handle. In this paper, a series of simple linear models has been developed using fabric extraction curve data for a series of lightweight next-to-skin knitted wool fabrics. These models predict eight fabric handle characteristics: overall handle, rough–smooth, hard–soft, loose–tight, heavy–light, hairy–clean, warm–cool and greasy–dry. The models were validated using an independent set of fabrics. The prediction precision of the models is significantly better than the precision of an experienced individual assessor.

Evaluating the assessment of essay type questions in the basic medical sciences

Background: Amongst the contemporary methods of assessment of examinations in the Basic Medical Sciences, the essay type method is the most subjective as it relies mainly on the judgment of an individual assessor and question sampling. Examiners therefore resort to the close system of marking of theory papers in professional examinations to bring most of the grades to the middle of the score line, thereby reducing the impact of an individual assessor and segment on the overall outcome. Aim: To look at the merits and demerits of the 'Open' and 'Closed' Systems of marking of essay type questions in its present form and recommend modalities for an appropriate marking scheme for the Faculty of Basic Medical Sciences. Methodology: We examined the merits and demerits of the closed and open systems of assessment of essay type questions and viva voce in professional exams in the Basic Medical Sciences together with the challenges of present day Medical Education. Result: The result showed that the closed system of marking in its current form is not discriminatory enough to allow for academic excellence and influence the weeding process. Conclusion: We recommend a semi-closed or modified closed system of marking which ensures a moderate influence of the different components of a Professional examination on the final outcome while maintaining some discriminatory power. A marking scheme for this system is also proposed. Keywords: Assessor, Subjective, Grades, Professional, Scheme, Discriminatory

Measuring fabric handle to define luxury: an overview of handle specification in next-to-skin knitted fabrics from Merino wool

An examination is presented of the relevance of luxury to the wool textile and garment supply chain. This examination leads to a review of the concept and importance of fabric handle as a means of defining important aspects of fabric quality. Examples are given for woven fabrics of the general relationships between subjectively assessed fabric handle attributes such as fabric softness and smoothness and measured low stress, generally high deformation, fabric properties such as fabric bending rigidity and extensibility. A brief overview is presented of the development of a system for predicting a set of subjectively assessed handle attributes for next-to-skin knitted fabrics from measurable fabric properties. Seven handle attributes selected by experienced assessors as being important for defining tactile sensations associated with next-to-skin knitted fabrics were: fabric smoothness, hairiness, softness, tightness, dryness, warmth and weight. Subjective assessments on a 1–10 scale of these seven attributes, plus an assessment of overall handle, were conducted by 12 experienced assessors on 74 next-to-skin knitted fabrics. The precision of the mean assessment of the 12 assessors ranged between 0.8 and 1.1, indicating that there was sufficient consensus on key fabric handle assessments to justify development of a method for predicting them from measurements of the physical properties of fabrics. All fabrics were tested using the PhabrOmeter fabric evaluation system, which records the force exerted during insertion of a fabric into and through an orifice. Geometric parameters were derived to describe the PhabrOmeter force-displacement curve results, and statistical models were developed to predict the average handle assessments of the 12 assessors. The precision of the models in predicting the handle intensities of eight fabric attributes on an independent validation set of 22 fabrics was significantly better than the precision of an individual assessor (confidence limits = 1.4–2.6 and 2.5–3.8, for predicted and assessed ratings, respectively). A case is made that this technology has the potential to assist in the growth of new markets for Merino wool products.

Monte Carlo based test for inferring about the unidimensionality of a Brazilian coffee sensory panel

Sensory science uses the human senses as instruments of measurement. Sensory analysis makes it possible to study organoleptic properties of products using a panel of assessors. Kermit and Lengard (2006) say that a good sensory panel should provide results that are accurate, discriminating and precise. Thus, in a successful analysis, it is key to have a set of robust tools for monitoring individual assessor’s performances as well as the performance of the panel as a whole.The success of using a sensory panel depends on its performance, i.e., its ability to identify small differences between products in certain attributes with statistical significance. A good panel performance is achieved when each panelist discriminates between products (large product variability), repeats the assessments (small within-assessor variability) and agrees with all other panelists on the sensory sensation that is described by a particular attribute with certain strength (small between-assessor variability) (Derndorfer, Baierl, Nimmervoll, & Sinkovits, 2005).Evaluating unidimensionality of panel is essential for the efficient use of the sensory analysis. For such purpose, the Monte Carlo based test revealed to be powerful for large panel and sample sizes. It can be adopted as a tool for checking the efficiency of the training process of a panel.