Experienced Judges Research Articles

A primer on performers’ rights—for the sixth time

Richard Arnold, currently serving at the Court of Appeal, also known as one of the most experienced judges of IP cases in UK, published the first edition of his seminal monograph on performers’ rights in 1990. Since then, the system of copyright law and related/neighbouring rights has gone through enormous changes. On the one hand, this is due to the rapid advancement of technologies, especially the internet (which coincidentally emerged around the time the first edition of Arnold’s book came out). On the other hand, this field of law went through significant harmonization both at the regional level (especially in the European Union) and at the international forum (mainly via the Trade-Related Aspects of Intellectual Property Rights Agreement and the World Intellectual Property Organization [WIPO] Internet Treaties). Well, the original WIPO Internet Treaties (namely, the Copyright Treaty and Performances and Phonograms Treaty) did not provide a complete coverage of performers’ interests though. Until the conclusion of the Beijing Treaty, performers’ rights were lagging behind copyright holders’ and related rightholders’ (other than performers’) rights. Needless to say, they are still several steps behind those stronger rightholders, but more on this later.

Improving reliability and validity in hip-hop dance assessment: Judging standards that elevate the sport and competition.

This study examined the reliability and validity of judging system scores of past hip-hop dance competitions in Japan. The analysis focused on the scores for each assessment category separately. Judges’ scores were obtained from national dance competitions held annually in Japan between 2014 and 2019. In these competitions, five experienced judges evaluated the dancers’ performances. The judges scored on a 10-point scale in five categories as follows: creativity, expression and interpretation, impression, technical quality, and synchronisation. This study found that the technical quality category demonstrated good reliability, whilst the impression showed poor reliability. Systematic bias was significant for all categories. There are no levels of difficulty defined for technique, no criteria set for correct movement and no explanation provided for each scoring level, which suggests that each judge may have interpreted the criteria for evaluating hip-hop dance differently. Developing these definitions and identifying the biases that affect evaluation would ensure a reliable evaluation system.

The Complexity of Stuttering Behavior in Adults and Adolescents: Relationship to Age, Severity, Mental Health, Impact of Stuttering, and Behavioral Treatment Outcome

This study investigated the complexity of stuttering behavior. It described and classified the complexity of stuttering behavior in relation to age, behavioral treatment outcomes, stuttering severity, anxiety-related mental health, impact of stuttering, and gender. For this study, a taxonomy was developed-LBDL-C7-which was based on the Lidcombe Behavioral Data Language of stuttering. It was used by five experienced judges to analyze the complexity of stuttering behavior for 84 adults and adolescents before and after speech restructuring treatment. Data were 3,100 stuttering moments, which were analyzed with nominal logistic regression. The complexity of stuttering behavior appears not to change as a result of treatment, but it does appear to change with advancing age. Complexity of stuttering behavior was found to be independently associated with clinician stuttering severity scores but not with percentage of syllables stuttered or self-reported stuttering severity. Complexity of stuttering behavior was not associated with gender, anxiety, or impact of stuttering. Clinical and research applications of these findings are discussed.

The Accuracy and Validity of the Simplified Pairs Method of Comparative Judgement in Highly Structured Papers

Comparative judgement (CJ) is often said to be more suitable for judging exam questions inviting extended responses, as it is easier for judges to make holistic judgements on a small number of large, extended tasks than a large number of smaller tasks. On the other hand, there is evidence it may also be appropriate for judging responses to papers made up of many smaller structured tasks. We report on two CJ exercises on mathematics and science exam papers, which are constructed mainly of highly structured items. This is to explore whether judgements processed by the simplified pairs version of CJ can approximate the empirical difference in difficulty of pairs of papers. This can then be used to maintain standards between exam papers. This use of CJ, not its other use as an alternative to marking, is the focus of this paper. Within the exercises discussed, panels of experienced judges looked at pairs of scripts, from different sessions of the same test, and their judgements were processed via the simplified pairs CJ method. This produces a single figure for the estimated difference in difficulty between versions. We compared this figure to the difference obtained from traditional equating, used as a benchmark. In the mathematics study the difference derived from judgement via simplified pairs closely approximated the empirical equating difference. However, in science, the CJ outcome did not closely align with the empirical difference in difficulty. Reasons for the discrepancy may include the differences in the content of the exams or the specific judges. However, clearly, comparative judgement need not lead to an accurate impression of the relative difficulty of different exams. We discuss self-reported judge views on how they judged, including what questions they focused on, and the implications of these for the validity of CJ. Processes used when judging papers made up of highly structured tasks were varied, but judges were generally consistent enough. Some potential challenges to the validity of comparative judgement are present with judges sometimes using re-marking strategies, and sometimes focusing attention on subsets of the paper, and we explore these. A greater understanding of what judges are doing when they judge comparatively brings to the fore questions of judgement validity that remain implicit in marking and non-comparative judgement contexts.

What is the Value of Judicial Experience? Exploring Judge Trajectories Using Longitudinal Data

Judicial experience is considered essential for the proper functioning of the sentencing system. We investigate how it influences judicial decisions and its role in reducing sentencing disparity. To do so, we analyze all Czech criminal decisions imposed in 2007–2017 using data that includes judge identifiers. This unique feature of our data enables us to measure judges’ experience directly, as the number of criminal cases processed, and to assess patterns in between-judge disparities longitudinally over the course of judges’ careers. We find that experienced judges impose more prison sentences, decide fewer cases via shortened procedure and find fewer defendants guilty. In addition, as judges become more experienced, between-judge disparities reduce across all the outcomes considered. Experience is thus an instrumental factor affecting judicial decisions throughout the criminal process, and one that contributes to greater consistency.

Development and Validation of the Japanese Version of the Consensus Auditory-Perceptual Evaluation of Voice.

Auditory-perceptual evaluation is essential for the assessment of voice quality. The Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) provides a standardized protocol and assessment form for clinicians to analyze the voice quality and has been adapted into several different languages. The aims of this study were to develop the Japanese version of the CAPE-V and to investigate its reliability and validity. The Japanese CAPE-V consisted of the same three speech contexts (vowels, sentences, and conversation) as developed in the original English version. The sentences were designed according to the concepts of the original version and reviewed by Japanese phoneticians. To validate the usefulness of the Japanese CAPE-V, voices of 173 Japanese-speaking subjects (76 subjects with dysphonia and 97 without voice complaints) were evaluated by five experienced judges, according to the Japanese CAPE-V as well as the GRBAS (Grade, Roughness, Breathiness, Asthenia, Strain) scale. The Japanese CAPE-V provided a high interrater reliability (intraclass correlation coefficients [ICCs] > .85 for all the parameters) as well as a high intrarater reliability (ICCs > .85 for all the parameters). In addition, overall severity, roughness, and breathiness in the Japanese CAPE-V were highly correlated with the corresponding dimensions in the GRBAS scale, having Spearman correlation coefficients greater than .8. This study demonstrated the reliability and validity of the newly developed Japanese CAPE-V as an auditory-perceptual evaluation instrument.

Reliability in the evaluation of international and national judges in an artistic swimming routine

Abstract Artistic swimming (AS) is a sport evaluated by fifteen judges in routine sessions. The athletes' goal is to achieve proficient motor patterns according to pre-established criteria. The present studied analyzed whether there is difference between the two groups of AS judges with different levels of experience. The group of the International Swimming Federation - FINA (IG) evaluates AS at national and international level and the non-FINA group (NG) evaluates AS only at national level. Twenty experienced judges were divided into groups, 10 IG judges and 10 NG judges. Thus, judges evaluated the item execution of three routines with five required elements. Cronbach's alpha coefficient showed high internal consistency in IG (a= 0.85 in T1 and 0.83 in T2). In NG, internal consistency was observed in T1 and low consistency in T2 (a= 0.82 in T1 and 0.39 in T2). Evaluation analysis between IG and NG was significant (p?0.0330) and reliability analysis (bias: -0.1266 95% agreement limit: -1.642 to 1.388) showed consistency and high degree of confidence in results. The findings suggest that the item execution of required elements showed high objectivity regarding judges with different levels of experience, IG and NG, regardless of categorization and time of practice. FINA has changed the number of judges and the number of items evaluated in routine sessions. It is suggested that the reduction of items has contributed in a positive way so that judges can focus more on evaluation itself.

Escala de Estilo de Gestão para Produtores Rurais

A produção rural nacional tem respondido por grande parte do PIB brasileiro e é considerando um dos principais players de exportação de commodities. A gestão das propriedades rurais tem sido assunto de interesse dos produtores, visto que as questões tecnológicas têm sido atendidas. Para tanto, o objetivo desta pesquisa foi validar uma escala para avaliar o estilo de gestão dos produtores rurais brasileiros. Tratou-se estilo de gestão como a manifestação das características de se administrar a interação entre as pessoas e entre pessoas e a estrutura, de um ou coletivo de gestores de uma determinada organização. Foram considerados os fatores liderança, decisão, comunicação, controle, planejamento e motivação como seus componentes. Os dados foram obtidos por meio da técnica survey. Para validação semântica, as sentenças foram submetidas ao teste piloto de dez juízes experientes em gestão de organizações agropecuaristas, sendo oito modificadas. A validação empírica foi realizada com a aplicação do instrumento de avaliação a 101 gestores de propriedades rurais, posteriormente submetidas aos testes de análise fatorial confirmatória e análise fatorial exploratória unidimensional. A solução final para o instrumento ficou composta por 22 sentenças, distribuídas em seis dimensões. As limitações da pesquisa são pertinentes à amostragem por conveniência. O instrumento final é indicado para aplicação na gestão das organizações rurais, permitindo a realização de diagnóstico organizacional, como também pode ser utilizado para investigações científicas.

The Behavioral Analysis of Dance Sports Judge's Visual Searching Strategies: Focusing on the Skating System

Objective: The purpose of this study was to investigate the differences between visual search, eye fixation positions, and eye fixation position on whole body regions while judging final matches of dance sports. Method: Ten experienced judges (male 6, female 4) and six dance sports couples belonging to the top Korean leagues in the Korea Dance sports Federation were participated. Wearable eye trackers were used for data collection. The independent t-test was conducted to examine the effects of gender, and the two-way ANOVA was conducted to examine the effects of judges'' gender, dance event, and ranking. Results: It was found that the movement of male contestants was preferred, and the gaze fixation was high for torso movements in the judges'' evaluations. Moreover, attention was focused on the upper body rather than the lower body; this was maintained for about seven seconds to evaluate each couple. Conclusion: This find was the relative comparison between the two couples was the determining factor in winning or losing.

Juvenile and Family Court Judges’ Knowledge and Attitudes About Sex Trafficking of Minors: Associations with Gender, Race, and U.S. Region

AbstractJuvenile and family court judges are a professional group that have a significant amount of decision‐making power in cases of sex trafficking of minors. The purpose of this project is to examine the association of juvenile and family court judges’ gender, race, and U.S. region with their attitudes and knowledge about sex trafficking of minors. Drawing from a survey of 55 juvenile and family court judges in the U.S., this study used standardized scales to measure attitudes and knowledge about child sex trafficking. Results indicate some differences by gender and geography in a sample of experienced judges across the U.S. The consistency of these findings are discussed in the context of other research and implications for targeted training.

QUALITATIVE ASSESSMENT OF A SEQUENCE OF “ALLEGRO” IN CLASSICAL BALLET DANCERS WITH DIFFERENT TECHNICAL LEVEL

Since training of professional classical ballet dancers is primarily focused on the aesthetic component of performance there is a necessity to assess its qualitative aspects. This study aimed to investigate whether the Performance Competence Evaluation Measure (PCEM) adapted version for classical ballet can be a valid and useful tool: 1) to assess the performance of an “Allegro” sequence; 2) to discriminate between different levels of dancers’ experience. Twenty-four female classical ballet dancers divided in two subgroups [Ballet students (n = 12) and Pre-professional dancers (n = 12)] were recruited to perform a 70 seconds sequence of “Allegro” specifically choreographed. Each performance was assessed by three very experienced judges, which scores were based on a Likert scale (ranging from 1 to 5) to evaluate the following four parameters: 1) Full Body Involvement (FBI); 2) Body Integration and Connectedness (BIC); 3) Articulation of Body Segments (ABS); 4) Movement Skills (MS). The inter-rater reliability values were ranging from good (between 0.6 and 0.8) and very good (over 0.8) for all parameters, while the test-retest reliability showed a very good agreement for all parameters. The Allegro is a dance sequence with greater dynamism that better highlights differences of execution between dancers with different technical levels and the Mann-Whitney test showed significant differences for all parameters (P=0.001) with lower values for the ballet students. In conclusion, the PCEM adapted version for classical ballet showed to be a valid tool: 1) to assess the performance of Allegro; 2) to discriminate between different dancers’ level of experience.

Judges’ perceptions of expert reports: The effect of neuroscience evidence

This article explores the impact of neuroscience evidence on how expert reports are perceived and their effects on the decisions made by trial judges. Experimental psychology has demonstrated a number of cognitive effects arising from exposure to neuroimaging data which may bias judgments and lead to (mis)interpretations that can affect decisions.We conducted a study on a sample of 62 Swiss and French judges in order to determine whether their perceptions of the credibility, quality and scientific basis of a psychiatric evaluation of a criminal defendant vary according to whether or not the evaluation includes neuroscientific data. Quantitative analyses were conducted in order to evaluate significant differences between the two conditions (one-way analyses of variance) and moderation and conditional analyses to examine whether the participants' sex and length of professional experience moderated the effect of the conditions. Terminological and thematic analyses were carried out on open questions.Quantitative and qualitative results suggest that the presence of neuroscience data in an expert report affects judges' perceptions of the quality, credibility, and scientificity (reliability, objectivity, scientific basis) of the report, and the persuasiveness of the evidence it provided. Moreover, this phenomenon was stronger in more experienced judges than in less experienced judges.

The Effect of the Speech Task Characteristics on Perceptual Judgment of Mild to Moderate Dysphonia: A Methodological Study

Objective: To study the differences in perceptual ratings of mild and moderate dysphonia related to the speech task, and their impact on intrarater and interrater reliabilities. Patients and Methods: Voice recordings of 15 outpatients with mild or moderate dysphonia related to laryngopharyngeal reflux were presented to 6 female experienced judges blinded to the clinical state of the patients. From these, the GRBASI (Grade, Roughness, Breathiness, Asthenia, Strain, and Instability) evaluations were performed on connected speech and sustained vowel of the pretreatment voice recordings and absolute agreement, and both intrarater and interrater reliabilities were assessed. Results: The average GRBASI scores were significantly worse when performed on sustained vowel. Intrarater reliability substantially varied according to the judge and the task. Good interrater reliability was broadly found for the evaluations of all GRBASI components irrespective of the speech task. Concerning agreement, we only found absolute agreement between judges for G and R items assessed on text. Conclusion: Average grade of perceptual voice impairment, intrarater reliability, and agreement vary according to the speech task.

Auditory-Perceptual Evaluation of Voice Quality of Cochlear-implanted and Normal-hearing Individuals: A Reliability Study

This study aimed to present an experience in rating voices of adults with normal hearing and adults with cochlear implants and critically examine the outcomes, discussing pros and cons of the methodology used. This is a cross-sectional, prospective study. One hundred and fifty voice samples, consisting of 50 sustained vowels, 50 samples of connected speech, and 50 samples of conversational speech, belonging to 25 adults with hearing impairment with cochlear implants and 25 adults with normal hearing, were perceptually analyzed for inter-rater agreement and intra-rater reliability. Three experienced judges rated the voice samples using visual analog scales of parameters considered relevant for cochlear-implanted population such as articulation, intonation, and resonance. The raters participated in three training sessions for calibration and had 1 month to complete the ratings individually. Twenty percent of the samples were repeated randomly to verify intra-rater reliability. The levels of agreement and reliability were verified using the interclass correlation coefficient. The inter-rater agreement varied widely across the parameters and speech tasks, from poor to excellent agreement. The only parameter for which the raters maintained consistently good or excellent agreement for all groups and emissions was the pitch. For intra-rater reliability, two of the raters presented excellent reliability for most parameters across all of the speech tasks, whereas one rater presented more inconsistencies. In this reliability study, factors such as extensive deadline for the auditory perceptual evaluation, lack of periodic recalibration, speech tasks, and familiarity with the population studied were identified as factors that contributed to inconsistent reliability results.

Large Market Declines and Securities Litigation: Implications for Disclosing Adverse Earnings News

This study finds that large marketwide declines in stock prices are associated with higher litigation incidence and settlements even though marketwide events are legally irrelevant. The probability of litigation nearly doubles (from 0.29% to 0.55%) and the amount of settlements also doubles (from $5.0 million to $10.1 million) when earnings disclosures occur during a large market decline, even after controlling for the firm’s market-adjusted return. Furthermore, judges with (without) specialized experience in securities litigation are more (less) likely to dismiss cases triggered by disclosures during large market declines. This pattern is consistent with experienced judges recognizing and dismissing weaker cases. Finally, managers are less likely to disclose adverse news at the end of trading days with large market declines. Although we cannot definitively identify the motive behind this pattern, it is consistent with managers recognizing increased litigation risk during large market declines. This paper was accepted by Mary Barth, accounting.

Julgamento perceptivo-auditivo das oclusivas velares associadas à fissura labiopalatina por juízes com e sem experiência

RESUMO: Objetivo: verificar: a) o grau de concordância de juízes no julgamento perceptivo-auditivo da produção de oclusivas velares, antes e depois da fonoterapia; b) a possível influência da composição fonética das amostras de fala nesta concordância e c) se os julgamentos obtidos por juízes com experiência diferem daqueles obtidos por juízes sem experiência, nas condições investigadas. Métodos: 60 amostras de fala de uma criança com fissura labiopalatina (30 previamente e 30 posteriormente a fonoterapia) e 30 amostras de fala de uma criança com fala típica (normal) foram julgadas por um grupo de 9 juízes. Três fonoaudiólogos estabeleceram os julgamentos consensuais "padrão ouro" para o estudo. Seis outros juízes julgaram as amostras: três considerados com experiência (fonoaudiólogos) e três sem experiência (alunos de graduação). As amostras de fala incluíram palavras constituídas pelas oclusivas velares /k/ e /g/ combinada com as vogais /a/, /i/ e /u/. Os juízes foram instruídos a julgar a presença, a ausência das oclusivas velares ou a presença de AC nestas amostras. Resultados: verificou-se diferenças no grau de concordância de juízes com experiência (Kappa moderada) e sem experiência (Kappa baixo) para os julgamentos realizados na condição pré-fonoterapia. O contexto fonético das amostras de fala influenciaram os julgamentos nas condições pré e pós-fonoterapia. Houve maior percentual de acerto para os juízes experientes condição pré-fonoterapia (p-valor <0,001). Conclusão: a experiência dos juízes e composição fonética das amostras de fala influenciam os julgamentos perceptivo-auditivos das AC.

Judging the judges' performance in rhythmic gymnastics.

Rhythmic gymnastics (RG) is an aesthetic event balancing between art and sport that also has a performance rating system (Code of Points) given by the International Gymnastics Federation. It is one of the sports in which competition results greatly depend on the judges' evaluation. In the current study, we explored the judges' performance in a five-gymnast ensemble routine. An expert-novice paradigm (10 international-level, 10 national-level, and 10 novice-level judges) was implemented under a fully simulated procedure of judgment in a five-gymnast ensemble routine of RG using two videos of routines performed by the Greek national team of RG. Simultaneous recordings of two-dimensional eye movements were taken during the judgment procedure to assess the percentage of time spent by each judge viewing the videos and fixation performance of each judge when an error in gymnast performance had occurred. All judge level groups had very modest performance of error recognition on gymnasts' routines, and the best international judges reported approximately 40% of true errors. Novice judges spent significantly more time viewing the videos compared with national and international judges and spent significantly more time fixating detected errors than the other two groups. National judges were the only group that made efficient use of fixation to detect errors. The fact that international-level judges outperformed both other groups, while not relying on visual fixation to detect errors, suggests that these experienced judges probably make use of other cognitive strategies, increasing their overall error detection efficiency, which was, however, still far below optimum.

Nasalância na presença e ausência da fricativa faríngea

RESUMO Objetivo: comparar os valores de nasalância em amostras de fala com e sem o uso de fricativa faríngea e, também, com e sem hipernasalidade. Métodos: um total de 840 amostras de fala foi analisado neste estudo. As amostras foram julgadas por três juízas experientes por consenso quanto aos aspectos hipernasalidade e fricativa faríngea. Os julgamentos foram distribuídos em quatro grupos: G1: 255 amostras de fala julgadas como representativas de hipernasalidade; G2: 130 amostras julgadas como representativas do uso de fricativa faríngea e hipernasalidade; G3: 280 amostras julgadas como representativas de fala normal em falantes com história de fissura labiopalatina; G4: 175 amostras julgadas como representativas de fala normal em falantes sem história de fissura labiopalatina. Para análise dos dados foi utilizando o teste Kruskal-Wallis e quando houve diferença estatisticamente significante foi aplicado o teste Dunn's para comparar os grupos aos pares. Resultados: os julgamentos aferidos por consenso pelas três juízas permitiram a identificação de amostras representativas do uso de fricativa faríngea e da presença e ausência de hipernasalidade. Foram estabelecidos valores de nasalância (média e desvio padrão) para cada grupo e observou-se que houve diferença estatisticamente significante entre os grupos com alteração de fala (G1 e G2) e aqueles sem alteração (G3 e G4). A diferença entre o grupo com hipernasalidade (G1) e o grupo com FF (G2) não foi significante. Conclusão: o uso de FF não influenciou significantemente os valores de nasalância para a amostra estudada.

Absolute and Relative Reliability of Percentage of Syllables Stuttered and Severity Rating Scales

Percentage of syllables stuttered (%SS) and severity rating (SR) scales are measures in common use to quantify stuttering severity and its changes during basic and clinical research conditions. However, their reliability has not been assessed with indices measuring both relative and absolute reliability. This study was designed to provide such information. Relative reliability deals with the rank order of participants in a sample, whereas absolute reliability measures the closeness of scores to one other and to a hypothetical true score. Eighty-seven adult participants who stutter received a 10-min unscheduled telephone call. Three experienced judges measured %SS and also used a 9-point SR scale to measure stuttering severity from recordings of the telephone calls. Relative intrajudge and interjudge reliability were satisfactory for both scales. However, absolute intrajudge and interjudge reliability were not satisfactory. Results showed that paired-judge SR and %SS procedures improved absolute reliability compared with single-judge measures. Additionally, the paired-judge procedure improved relative reliability from high to very high levels. Measurement of group changes of stuttering severity can be done in research contexts using either %SS or SR. However, for detecting changes within individuals using such measures, a paired-judge procedure is a more reliable method.

Event- and interval-based measurement of stuttering: a review.

Event- and interval-based measurements are two different ways of computing frequency of stuttering. Interval-based methodology emerged as an alternative measure to overcome problems associated with reproducibility in the event-based methodology. No review has been made to study the effect of methodological factors in interval-based absolute reliability data or to compute the agreement between the two methodologies in terms of inter-judge, intra-judge and accuracy (i.e., correspondence between raters' scores and an established criterion). To provide a review related to reproducibility of event-based and time-interval measurement, and to verify the effect of methodological factors (training, experience, interval duration, sample presentation order and judgment conditions) on agreement of time-interval measurement; in addition, to determine if it is possible to quantify the agreement between the two methodologies The first two authors searched for articles on ERIC, MEDLINE, PubMed, B-on, CENTRAL and Dissertation Abstracts during January-February 2013 and retrieved 495 articles. Forty-eight articles were selected for review. Content tables were constructed with the main findings. Articles related to event-based measurements revealed values of inter- and intra-judge greater than 0.70 and agreement percentages beyond 80%. The articles related to time-interval measures revealed that, in general, judges with more experience with stuttering presented significantly higher levels of intra- and inter-judge agreement. Inter- and intra-judge values were beyond the references for high reproducibility values for both methodologies. Accuracy (regarding the closeness of raters' judgements with an established criterion), intra- and inter-judge agreement were higher for trained groups when compared with non-trained groups. Sample presentation order and audio/video conditions did not result in differences in inter- or intra-judge results. A duration of 5 s for an interval appears to be an acceptable agreement. Explanation for high reproducibility values as well as parameter choice to report those data are discussed. Both interval- and event-based methodologies used trained or experienced judges for inter- and intra-judge determination and data were beyond the references for good reproducibility values. Inter- and intra-judge values were reported in different metric scales among event- and interval-based methods studies, making it unfeasible to quantify the agreement between the two methods.