Groups Of Raters Research Articles

Role of canine vertical position and gingival zenith level on smile aesthetics and qualitative evaluation of perception by orthodontists, general dentists and laypersons: A descriptive cross-sectional study

: Our objectives were to verify the impact of alterations in the vertical position of the maxillary canines in smile aesthetic perceptions and to determine whether exposure of the gingival margins directly affects laypersons' and orthodontists' perceptions.: To evaluate the impact of vertical position and gingival zenith level of maxillary canine on smile esthetics as judged by orthodontists, general dentist and laypersons: This is a qualitative, descriptive cross-sectional study aimed at studying the role of vertical position and gingival zenith level on the smile esthetic perception. A smiling photograph of a male subject with and without exposing the gingival zeniths was selected, and vertical positions and gingival zenith levels of canine were symmetrically modified in increments of 0.5 mm, creating two sets of 4 new modified images varying from 1.0 mm of intrusion to 1.0 mm of extrusion and 1.0 mm of increasing and decreasing the height with gingival exposure. The total of 15 images were evaluated by 40 orthodontists, 40 general dentists and 40 laypersons, who determined the level of attractiveness of each smile on a visual analog scale.: Both smile arc ( <0.05) and gingival display amount ( <0.001) had statistically significant influences on the perception of smile attractiveness. Smile attractiveness scores with reference to gingival display amount showed a statistically significant ( <0.001) difference between the rater groups. A significant ( <0.05) interaction between smile arc and gingival display amount was observed. No significant difference were noted with standard smiles and intrusion smiles. Also, with 0.5mm extrusion and intrusion images in low smile. The intra-evaluator groups showed no significant difference between intrusion and extrusion high as well as low smile images, (value<0.05 for all). significant difference was seen in extrusion and intrusion high smile at 0.5mm and low smile images at 1mm level, extrusion and intrusion low smile images at 0.5mm level, (value>0.05 for all), significantly higher intrusion and extrusion high smile images at 0.5mm level, (-value<0.05 for all). extrusion and intrusion high as well as low smile images at 1mm and 0.5mm levels, (-value<0.05 for all).: Orthodontists were more critical in their assessments. There were no differences in the esthetic evaluations of smiles with and without gingival margin exposure for both groups of evaluators. For all the three evaluator groups, the most attractive ones were the standard smile and smile with the intrusion of 0.5 in high smile, standard and extrusion of 0.5 mm in the low smile group 0.5 mm of increased gingival zenith level (GZL) as perceived by orthodontists and general dentists, while for laypeople it was the standard smile. The unattractive smiles have extrusion and intrusion of 1mm for all the three evaluator groups 1 mm followed of decreased GZL of canine, for orthodontists and general dentist with 1mm increased. Laypeople, orthodontists scored all the images more critically by giving lesser scores compared to the general dentists and laypeople.

Rating pome fruit quality traits using deep learning and image processing.

Quality assessment of pome fruits (i.e. apples and pears) is used not only for determining the optimal harvest time but also for the progression of fruit-quality attributes during storage. Therefore, it is typical to repeatedly evaluate fruits during the course of a postharvest experiment. This evaluation often includes careful visual assessments of fruit for apparent defects and physiological symptoms. A general best practice for quality assessment is to rate fruit using the same individual rater or group of individual raters to reduce bias. However, such consistency across labs, facilities, and experiments is often not feasible or attainable. Moreover, while these visual assessments are critical empirical data, they are often coarse-grained and lack consistent objective criteria. Granny, is a tool designed for rating fruit using machine-learning and image-processing to address rater bias and improve resolution. Additionally, Granny supports backward compatibility by providing ratings compatible with long-established standards and references, promoting research program continuity. Current Granny ratings include starch content assessment, rating levels of peel defects, and peel color analyses. Integrative analyses enhanced by Granny's improved resolution and reduced bias, such as linking fruit outcomes to global scale -omics data, environmental changes, and other quantitative fruit quality metrics like soluble solids content and flesh firmness, will further enrich our understanding of fruit quality dynamics. Lastly, Granny is open-source and freely available.

Perceived warmth and competence predict callback rates in meta-analyzed North American labor market experiments.

Extensive literature probes labor market discrimination through correspondence studies in which researchers send pairs of resumes to employers, which are closely matched except for social signals such as gender or ethnicity. Upon perceiving these signals, individuals quickly activate associated stereotypes. The Stereotype Content Model (SCM; Fiske 2002) categorizes these stereotypes into two dimensions: warmth and competence. Our research integrates findings from correspondence studies with theories of social psychology, asking: Can discrimination between social groups, measured through employer callback disparities, be predicted by warmth and competence perceptions of social signals? We collect callback rates from 21 published correspondence studies, varying for 592 social signals. On those social signals, we collected warmth and competence perceptions from an independent group of online raters. We found that social perception predicts callback disparities for studies varying race and gender, which are indirectly signaled by names on these resumes. Yet, for studies adjusting other categories like sexuality and disability, the influence of social perception on callbacks is inconsistent. For instance, a more favorable perception of signals like parenthood does not consistently lead to increased callbacks, underscoring the necessity for further research. Our research offers pivotal strategies to address labor market discrimination in practice. Leveraging the warmth and competence framework allows for the predictive identification of bias against specific groups without extensive correspondence studies. By distilling hiring discrimination into these two dimensions, we not only facilitate the development of decision support systems for hiring managers but also equip computer scientists with a foundational framework for debiasing Large Language Models and other methods that are increasingly employed in hiring processes.

Standard area diagrams for severity assessment of eucalypt bacterial leaf blight caused by Xanthomonas spp.

AbstractBacterial leaf blight (BLB) caused by Xanthomonas spp. is an important threat for eucalypt production. Planting of resistant genotypes is the best strategy for eucalypt BLB control. However, selection of resistant clones is challenging since the disease is caused by at least three Xanthomonas species that can be sympatric and cause a range of foliar symptoms. So far, standard area diagrams (SADs) to assess eucalypt BLB severity caused by different Xanthomonas species have not been reported. Here, SADs for eucalypt BLB severity assessment were developed. Plants of five eucalypt genotypes were individually inoculated with different Xanthomonas species and leaves exhibiting lesions varying in colour, shape, and size were digitally scanned for disease assessment. Among 150 scanned leaves, the actual leaf affected area ranged from 0.01% to 55%. Based on these minimum and maximum values and the frequency of severity distribution, the 10 leaves that best represented the distribution of affected leaf area were selected to assemble the SADs. Groups of experienced and inexperienced raters estimated BLB severity on a subset of 50 out of the 150 original leaves without and then with the use of the SADs. Correlation concordance analyses revealed that SADs provided a significant improvement in accuracy, precision and reliability of disease estimates by both rater groups, with higher gains observed for the inexperienced group. The SADs developed in this study will be useful for selection of eucalypt genotypes resistant to BLB under controlled conditions when knowledge on the Xanthomonas species composition is lacking or when local populations are comprised of several species.

QRATER: a collaborative and centralized imaging quality control web-based application

Quality control (QC) is an important part of all scientific analyses, including neuroscience. With manual curation considered the gold standard, there remains a lack of available tools that make manual neuroimaging QC accessible, fast, and easy. In this article we present Qrater, a containerized web-based Python application that enables viewing and rating any type of image for QC purposes. Qrater functionalities allow collaboration between various raters on the same dataset which can facilitate completing large QC tasks. Qrater was used to evaluate QC rater performance on three different magnetic resonance (MR) image QC tasks by a group of raters having different amounts of experience. The tasks included QC of raw MR images (10,196 images), QC of linear registration to a standard template (10,196 images), and QC of skull segmentation (6,968 images). We measured the proportion of failed images, average rating time per image, intra- and inter-rater agreement, as well as the comparison against QC using a conventional method. The median time spent rating per image differed significantly between raters (depending on rater experience) in each of the three QC tasks. Evaluating raw MR images was slightly faster using Qrater than an image viewer (expert: 99 vs. 90 images in 63 min; trainee 99 vs 79 images in 98 min). Reviewing the linear registration using Qrater was twice faster for the expert (99 vs. 43 images in 36 min) and three times faster for the trainee (99 vs. 30 images in 37 min). The greatest difference in rating speed resulted from the skull segmentation task where the expert took a full minute to inspect the volume on a slice-by-slice basis compared to just 3 s using Qrater. Rating agreement also depended on the experience of the raters and the task at hand: trained raters’ inter-rater agreements with the expert’s gold standard were moderate for both raw images (Fleiss’ Kappa = 0.44) and linear registration (Fleiss’ Kappa = 0.56); the experts’ inter-rater agreement of the skull segmentation task was excellent (Cohen’s Kappa = 0.83). These results demonstrate that Qrater is a useful asset for QC tasks that rely on manual evaluation of QC images.

A Robust Rating Aggregation Method based on Rater Group Trustworthiness for Collusive Disturbance

A Robust Rating Aggregation Method based on Rater Group Trustworthiness for Collusive Disturbance

Communal factors in rater severity and consistency over time in high-stakes oral assessment

This article focuses on rater severity and consistency and their relation to major changes in the rating system in a high-stakes testing context. The study is based on longitudinal data collected from 2009 to 2019 from the second language (L2) Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. We investigated whether rater severity and consistency changed over that period and whether the changes could be explained by major changes in the rating system, such as the change of lead examiner, the modus of rating and training (on-site or remote), and the composition of the rater group. The data consisted of 45 rating sessions with 104 raters and 59,899 examinees and were analysed using the many-facet Rasch model and generalized linear mixed models. The analyses indicated that raters as a group became somewhat more lenient over time. In addition, the results showed that the rater community and its practices, the lead examiners, and the modus of rating and training can influence rating behaviour. Finally, we elaborate on implications for both research and practice.

Anxiety in Turner syndrome: Engaging community to address barriers and facilitators to diagnosis and care.

Turner syndrome (TS), caused by complete or partial loss of the second sex chromosome, is associated with complex medical manifestations. The TS community identifies anxiety as a major contributor to reduced quality of life. The study aimed to improve understanding of anxiety symptomatology, diagnosis, and care in individuals with TS. A mixed methods design integrated community engagement, including community leaders as co-investigators and a community advisory board, an online survey (N = 135), and in-depth interviews (N = 10). The majority of respondents reported that anxiety symptoms occur two or more days per week, with self-advocates reporting more frequent symptoms than caregivers (p = 0.03). Self-advocates reported feeling anxious more often at school/work; both rater groups reported anxiety-related behaviors were most likely to be expressed at home. Insomnia was the most common symptom of anxiety endorsed across age and rater groups (>70%). Anxiety symptoms and triggers changed with age and often were undiagnosed or untreated during childhood. Therapy and medication were reported as helpful by most respondents who had tried these strategies. Qualitative themes included: 'Triggers for anxiety are related to TS', 'Anxiety impacts the whole family', and 'Opportunities for early identification and intervention'.

Detecting Randomness Effect among Raters in Physics Essay Items using Many-Facet Rasch Measurement

This study sought to detect randomness effects among raters in physics essay items using Many-Facet Rasch Measurement. The research design adopted for this study is descriptive research design based on survey method. The population of the study comprised eighty-eight (88) public schools in all the local government areas with a physics student population of 3,642 students and ninety-four (94) physics teachers in all the Senior Secondary Schools in Uyo Senatorial District for the 2022/2023 academic session. Three hundred and sixty-four (364) SSS3 physics students and 37 physics teachers from the 31 selected secondary schools in Uyo Senatorial District were sampled using multistage sampling technique for effective selection. The multi-stage sampling technique was adopted for the study. The instrument used for data collection was Physics Achievement Test (PAT) obtained from WAEC and NECO 2020 Physics Essay items with reliability coefficients of 0.91 and 0.90 respectively. The finding revealed that most of the individual raters commit randomness effect when rating physics essay Items. It also revealed there is a significant difference at the rater's group level exhibiting randomness effect when rating physics essay items, which implies that there is no group-level randomness effect present among raters when rating physics essay items. We concluded that rater effects are sources of variance in performance ratings that are associated with the raters' behaviour and not the actual performance of the ratee. It was recommended that raters should follow the rating guidelines to reduce the impact of randomness in their ratings and provide more accurate and objective evaluations.

Any slice is predictive? On the consistency of impressions from the beginning, middle, and end of assessment center exercises and their relation to performance

AbstractThis study generates new insights on the role of initial impressions in assessment centers. Drawing from the “thin slices” of behavior paradigm in personality and social psychology, we investigate to what extent initial impressions of assessees—based on different slices of assessment center exercises (i.e., two minutes at the beginning, middle, and end of AC exercises)—are consistent across and within AC exercises, and are relevant for predicting assessment center performance and job performance. Employed individuals (N = 223) participated in three interactive assessment center exercises, while being observed and evaluated by trained assessors. Based upon video-recordings of all assessment center exercises, a different, untrained group of raters subsequently provided ratings of their general initial impressions of assessees for the beginning, middle, and end of each exercise. As criterion measure, supervisors rated assessees’ job performance. Results show that initial impressions in assessment centers are (a) relatively stable, (b) consistently predict assessment center performance across different slices of behavior (i.e., across the three time points and exercises), and (c) mostly relate to job performance.

Raters’ scoring process in assessment of interpreting: an empirical study based on eye tracking and retrospective verbalisation

ABSTRACT Human raters’ assessment of interpreting is a complex process. Previous researchers have mainly relied on verbal reports to examine this process. To advance our understanding, we conducted an empirical study, collecting raters’ eye-movement and retrospection data in a computerised interpreting assessment in which three groups of raters (n = 35) used an analytic rubric to assess 12 English-to-Chinese consecutive interpretations. We examined how the raters interacted with the source text, the rating scale, and the audio player displayed on the computer screen when they were assessing. We found that a) the source text and the rating scale were competing for the raters’ visual attention, with the former attracting more attention than the latter across the rater groups; b) when the raters were consulting the rating scale, they fixated less frequently on the sub-scale of target language quality than the other two sub-scales; c) the rater groups did not seem to exhibit substantially discrepant gazing behaviours overall, although there emerged different eye-movement patterns concerning certain sub-scales; and d) the raters utilised an array of strategies and shortcuts to facilitate their assessment. We discuss these findings in relation to rater training and validation of score meaning for interpreting assessment.

A Descriptive Study on the Impacts of Hyperbaric Oxygen Therapy on Autistic Individuals Using Parent Testimonies.

Introduction Hyperbaric oxygen therapy (HBOT) has been influential in treating many physical and psychological ailments, including the symptoms of autism. This current study aims to evaluate HBOT parents' goals and exit interviews describing the positive, negative, or no impacts experienced from the HBOT dives, asking the question, "Are your child's symptoms improving?" Methods Between January 2020 and July 2023, a Class B monoplace hyperbaric chamber (Sechrist 3300H, Sechrist Industries, Inc., Anaheim, California, United States) was used to administer HBOT sessions to patients with autism. Medical-grade oxygen was pressurized to 1.5-2.0 atmospheres absolute at a rate of 1-2 psi/min, with an average oxygen percentage of 100%, for up to five sessions per week. Retrospective descriptive data and patient information through parent testimonials were collected through a chart review of 30 children and one adult with autism who experienced HBOT sessions. Data were presented through exit interviews describing how parents felt about their child's progress toward goals. Four raters rated parent testimonies on a 5-point Likert scale (1 = Much worse, 2 = Somewhat worse, 3 = Stayed the same, 4 = Somewhat improved, and 5 = Much improved), and an inter-rater reliability estimate using interclass correlation (2) (r = 0.831) was derived, indicating excellent agreement between raters. Results Parents/caregivers provided testimony in an exit interview with a registered nurse after the individual with autism received an entire course of HBOT dives. Descriptive statistics resulted in Rater #1 (M = 4.19, median = 4, SD = 0.654): 87.1% of Rater #1 ratings were Somewhat improved and Much improved; Rater #2 (M = 4.23, median = 4, SD = 0.717): 83.9% of Rater #2 ratings were Somewhat improved and Much improved; Rater #3 (M = 4.23, median = 4, SD = 0.560): 93.5% of Rater #3 ratings were Somewhat improved and Much improved; and Rater #4 (M = 4.26, median = 4, SD = 0.631): 90.3% of Rater #4 ratings were Somewhat improved and Much improved. One-way ANOVA resulted in F (3,123) = 0.052, p = 0.984, which indicated a nonstatistically significant mean difference between rater groups. Conclusions The current study assessed HBOT parents'/caregivers' goals and exit interviews, describing the effects experienced from the complete course of HBOT dives on their children/individuals. A majority of parents/caregivers declared that their condition had "Much improved" or "Somewhat improved," based on the 5-point Likert scale. Based on parents'/caregivers' testimonies, HBOT was demonstrated as a safe and effective intervention, and side effects were primarily mild and did not lead to treatment discontinuation. As a result of this analysis, we recommend continued use of HBOT for treatment.

Interrater reliability of photographic assessment of thyroid eye disease using the VISA classification

PurposeTo determine the interrater reliability (IRR) of thyroid eye disease (TED) photographic assessment using the VISA classification. To assess whether a VISA grading atlas improves ophthalmology trainees’ performance in photographic assessment of TED.MethodsA prospective, partially randomized, international study conducted from September 2021 to May 2022. Online study invitation was emailed to a volunteer sample group of 68 ophthalmology college accredited consultants and trainees, and 6 were excluded from the study. Participants were asked to score 10 patient photographs of TED using only the inflammation and motility restriction components of the VISA classification. IRR was compared between groups of practitioners by their level of experience. A clinical activity grading atlas was randomized to 50% of the ophthalmology trainees.ResultsOverall rater ICC was 0.96 for inflammation and 0.99 for motility restriction. No statistically significant difference in IRR between rater groups was identified. Trainees with a grading atlas had the highest IRR for inflammation (ICC = 0.95). Each subcomponent of the inflammation and motility restriction components of VISA classification had an ICC considered good to excellent. The mean overall rater score was 4.6/9 for inflammation and 3.5/12 for motility restriction. For motility restriction there was a reduced mean score variance among all raters when scoring photographs with more severe motility restriction.ConclusionIRR using the inflammation and motility restriction components of the VISA classification was excellent. A VISA grading atlas improved trainee performance in grading inflammation.

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model.

Reliability of measurement instruments providing quantitative outcomes is usually assessed by an intraclass correlation coefficient. When participants are repeatedly measured by a single rater or device, or, are each rated by a different group of raters, the intraclass correlation coefficient is based on a one-way analysis of variance model. When planning a reliability study, it is essential to determine the number of participants and measurements per participant (i.e. number of raters or number of repeated measurements). Three different sample size determination approaches under the one-way analysis of variance model were identified in the literature, all based on a confidence interval for the intraclass correlation coefficient. Although eight different confidence interval methods can be identified, Wald confidence interval with Fisher's large sample variance approximation remains most commonly used despite its well-known poor statistical properties. Therefore, a first objective of this work is comparing the statistical properties of all identified confidence interval methods-including those overlooked in previous studies. A second objective is developing a general procedure to determine the sample size using all approaches since a closed-form formula is not always available. This procedure is implemented in an R Shiny app. Finally, we provide advice for choosing an appropriate sample size determination method when planning a reliability study.

Perceived Effects of Orthognathic Surgery versus Orthodontic Camouflage Treatment of Convex Facial Profile Patients.

Increased facial profile convexity has a common occurrence in the population and is a primary reason for seeking orthodontic treatment. The present study aimed to compare the perceived changes in facial profile appearance between patients treated with combined orthognathic/orthodontic treatment versus only orthodontic camouflage treatment. For this reason, 18 pairs of before- and after-treatment facial profile photos per treatment group (n = 36 patients) were presented to four types of assessors (surgeons, orthodontists, patients, laypeople). Ratings were recorded on 100 mm visual analogue scales depicted in previously validated questionnaires. All rater groups identified minor positive changes in the facial profile appearance after exclusively orthodontic treatment, in contrast to substantial positive changes (14% to 18%) following combined orthodontic and orthognathic surgery. The differences between the two treatment approaches were slightly larger in the lower face and the chin than in the lips. The combined orthodontic and orthognathic surgery interventions were efficient in improving the facial appearance of patients with convex profile, whereas orthodontic treatment alone was not. Given the significant influence of facial aesthetics on various life aspects and its pivotal role in treatment demand and patient satisfaction, healthcare providers should take these findings into account when consulting adult patients with a convex facial profile.

Assessing frailty in elderly patients with hip fractures: A retrospective review comparing geriatrician and orthopedic trainee assessments.

To assess the correlation of orthopedic surgery residents compared with expert geriatricians in the assessment of frailty stage using the Clinical Frailty Scale (CFS) in patients with hip fractures. A retrospective chart review was performed from January 1, 2015 to December 31, 2019. Patients admitted with a diagnosis of hip fracture were identified. Those patients with a CFS score completed by orthopedic residents with subsequent CFS score completed by a geriatrician during their admission were extracted. Six hundred and forty-eight patients over age 60 (mean 80.5 years, 73.5% female) were admitted during the study period. Orthopaedic residents completed 286 assessments in 44% of admissions. Geriatric medicine consultation was available for 215 patients such that 93 patients were assessed by both teams. Paired CFS data were extracted from the charts and tested for agreement between the 2 groups of raters. CFS assessments by orthopedic residents and geriatrician experts were significantly different at P < .05; orthopedic residents typically assessed patients to be one CFS grade less frail than geriatricians. Despite this, the CFS assessments showed good agreement between residents and geriatricians. Orthopaedic surgery residents are reliable assessors of frailty but tend to underestimate frailty level compared with specialist geriatricians. Given the evidence to support models such as orthogeriatrics to improve outcomes for frail patients, our findings suggest that orthopedic residents may be well positioned to identify patients who could benefit from such early interventions. Our findings also support recent evidence that frailty assessments by orthopedic surgeons may have predictive validity. Low rates of initial frailty assessment by orthopedic residents suggests that further work is required to integrate more global comprehensive care.

The Reading Everyday Emotion Database (REED): a set of audio-visual recordings of emotions in music and language

AbstractMost audio-visual (AV) emotion databases consist of clips that do not reflect real-life emotion processing (e.g., professional actors in bright studio-like environment), contain only spoken clips, and none have sung clips that express complex emotions. Here, we introduce a new AV database, the Reading Everyday Emotion Database (REED), which directly addresses those gaps. We recorded the faces of everyday adults with a diverse range of acting experience expressing 13 emotions—neutral, the six basic emotions (angry, disgusted, fearful, happy, sad, surprised), and six complex emotions (embarrassed, hopeful, jealous, proud, sarcastic, stressed)—in two auditory domains (spoken and sung) using everyday recording devices (e.g., laptops, mobile phones, etc.). The recordings were validated by an independent group of raters. We found that: intensity ratings of the recordings were positively associated with recognition accuracy; and the basic emotions, as well as the Neutral and Sarcastic emotions, were recognised more accurately than the other complex emotions. Emotion recognition accuracy also differed by utterance. Exploratory analysis revealed that recordings of those with drama experience were better recognised than those without. Overall, this database will benefit those who need AV clips with natural variations in both emotion expressions and recording environment.

Investigating The Central Tendency Effect Among Raters in Complex Problem-Solving Skills in Physics Essay Items: An Application of Many-Facet Rasch Measurement

The study sought to investigate the central tendency effect among raters in complex problem-solving skills in physics essay items using Many-Facet Rasch Measurement. This became necessary because the central tendency effect can affect students' scores. The research design adopted for this study is a descriptive research design based on the survey method. The population of the study comprised eighty-eight (88) public schools in all the local government areas with a physics student population of 3642 students and ninety-four (94) physics teachers in all the Senior Secondary Schools in Uyo Senatorial District for the 2022/2023 academic session. 364 SS3 physics students and 37 physics teachers from the 31 selected secondary schools in Uyo Senatorial District were sampled using a multistage sampling technique for effective selection. The multi-stage sampling technique was adopted for the study. The instrument used for data collection was the Physics Achievement Test (PAT) obtained from WAEC and NECO 2020 Physics Essay items. However, the reliability coefficient of 0.91 and 0.90 respectively. The finding revealed that there was no central tendency effect error committed by individual raters when rating complex problem-solving skills in WAEC and NECO 2020 physics Essay Items. Also, reveals there is a significant difference at the rater's group level exhibiting central tendency effect when rating complex problem-solving skills between WAEC and NECO 2020 physics Essay Items which implies that there is no group-level central tendency effect present among raters when rating complex problem-solving skills between WAEC and NECO 2020 physics Essay Items. The researchers concluded that rater effects are sources of variance in performance ratings that are associated with the rater's behaviour and not the actual performance of the ratee. It was recommended that to prevent the central tendency effect, it is important for superior raters to regularly observe their subordinates throughout the evaluation period and keep notes for future reference.

Perceptual Evaluation of the Tracheoesophageal Voice: Italian Validation of the Sunderland Tracheoesophageal Perceptual Scale

This study aimed to validate the Sunderland Tracheoesophageal Perceptual Scale (SToPS) in the Italian language by checking the inter- and intra-rater reliability. Scale validation METHODS: The validation of the tool involved the following steps: (1) translation and adaptation of the SToPS into Italian language; (2) recruitment of participants (60 laryngectomized patients with a voice prosthesis and 12 health professionals-six speech and language therapists (SLTs) and six ear, nose, and throat surgeons-classified into experienced (Exp) or not at assessing voice; (3) recording of patients' speech samples; (4) perceptual evaluation of recorded speech samples (test and retest) performed by the 12 health professionals; and (5) statistical analysis (quadratic weighted Cohen kappa and weighted kappa of Light coefficients). When all raters were considered as one group, an agreement ≥0.50 was reached for all parameters. The "ExpSLTs" group rated more reliably than the other groups, achieving a "good" intrarater agreement for 9/10 parameters. Despite the interrater coefficients were lower than the intrarater ones for all rater groups, "ExpSLTs" obtained the best levels of interrater agreement, achieving a level of agreement ≥0.50 for all parameters. Finally, considering intrarater+interrater agreement ("good+good" or "good+moderate"), the "ExpSLTs" group showed the greatest agreement, attaining all parameters that met the criteria for "good" or "moderate" agreement levels. The Italian version of SToPS can be considered a reliable tool. As in the original version, expert SLTs are the better judges for tracheoesophageal voice assessment.

Morphologically-Aware Consensus Computation via Heuristics-based IterATive Optimization (MACCHIatO)

The extraction of consensus segmentations from several binary or probabilistic masks is important to solve various tasks such as the analysis of inter-rater variability or the fusion of several neural network outputs. One of the most widely used methods to obtain such a consensus segmentation is the STAPLE algorithm. In this paper, we first demonstrate that the output of that algorithm is heavily impacted by the background size of images and the choice of the prior. We then propose a new method to construct a binary or a probabilistic consensus segmentation based on the Fréchet means of carefully chosen distances which makes it totally independent of the image background size. We provide a heuristic approach to optimize this criterion such that a voxel’s class is fully determined by its voxel-wise distance to the different masks, the connected component it belongs to and the group of raters who segmented it. We compared extensively our method on several datasets with the STAPLE method and the naive segmentation averaging method, showing that it leads to binary consensus masks of intermediate size between Majority Voting and STAPLE and to different posterior probabilities than Mask Averaging and STAPLE methods. Our code is available at &lt;a href='https://gitlab.inria.fr/dhamzaou/jaccardmap'&gt;https://gitlab.inria.fr/dhamzaou/jaccardmap&lt;/a&gt;