External Judges Research Articles

Introducing the brief reverse correlation: An improved tool to assess visual representations

AbstractThe reverse correlation is an innovative method to capture visual representations (i.e., classification images, CIs) of social targets. However, this method necessitates many trials to compute high‐quality CIs, which poses important practical and economic challenges. We introduce a new version of the reverse correlation method, namely, the Brief‐RC. By increasing the number of stimuli (i.e., noisy faces) presented at each trial, the Brief‐RC improves the quality of individual (and average) CIs and lowers the overall task length. In two experiments, assessments by external judges confirmed that the new method delivers equally good (Experiment 1) or higher‐quality (Experiment 2) outcomes than the traditional method for the same number of trials, time length and number of stimuli. The informational values of CIs were also compared using a more objective metric (infoVal). Because the Brief‐RC facilitates the production of higher‐quality individual CIs, social psychology researchers may more easily address a series of relevant research questions.

Validation of the Italian version of a patient-reported outcome measure for Hereditary Spastic Paraplegia.

Patient-reported outcome measures (PROMs) are recognized as valuable measures in the clinical setting. In 2018 we developed the Italian version of the "Hereditary Spastic Paraplegia-Self Notion and Perception Questionnaire" (HSP-SNAP), a disease-specific questionnaire that collects personal perception on motor symptoms related to HSP such as stiffness, weakness, imbalance, reduced endurance, fatigue and pain. In this study our primary aim was to assess the questionnaire validity and reliability. Our secondary aim was to characterize the symptoms "perceived" by patients with HSP and compare them with those "perceived" by age-matched healthy subjects. The 12-item HSP-SNAP questionnaire was submitted to 20 external judges for comprehensibility and to 15 external judges for content validity assessment. We recruited 40 subjects with HSP and asked them to fill the questionnaire twice for test-retest procedure. They also completed the Medical Outcome Survey Short Form (SF-36) and were evaluated by the Spastic Paraplegia Rating Scale and the Six-Minute Walk Test. We also recruited 44 healthy subjects who completed the HSP-SNAP once to test score variability. The HSP-SNAP content validity index was high (0.8±0.1) and the test-retest analysis showed high reliability (ICC = 0.94). The mean HSP-SNAP score (score range 0-48) of the HSP group was 22.2±7.8, which was significantly lower than healthy subjects (43.1±6.3). The most commonly perceived symptom was stiffness, followed by weakness and imbalance. Although HSP-SNAP does not investigate non-motor symptoms and we validated only its Italian version, it showed good validity and reliability and it could be used in combination with other objective outcome measures for clinical purposes or as endpoints for future clinical rehabilitation studies. Trial Registration: ClinicalTrial.gov, NCT04256681. Registered 3 February 2020.

Dreaming of being chased reflects waking-life experiences related to negative relationships with others metaphorically.

It has long been argued that there are dream metaphors which express waking-life experiences indirectly. Most of empirical evidence concerning this topic was in a qualitative way, while few studies explored the topic in a quantitative way. Under this background, we investigated whether dreaming of the typical theme 'being chased or pursued' was a metaphorical expression for waking-life experiences related to 'negative relationships with others'. One hundred and sixty participants reported their waking-life experiences and dreams for a single day. Following this, two external judges rated whether there were any elements related to 'negative relationships with others' in both waking-life experiences and dreams. In addition, the judges assessed if there was any content related to 'being chased or pursued' in both waking-life experiences and dreams. The frequency of dreaming of 'negative relationships with others' was higher than the frequency of the same topic in waking-life experiences, which in turn was higher than the frequency of typical theme dreaming of 'being chased or pursued'. In addition, 'negative relationships with others' in waking-life experiences were correlated with both dreaming of 'being chased or pursued', and 'negative relationships with others' in dreams. These results suggested that the typical theme 'being chased or pursued' in dreams may represent some waking-life experiences metaphorically. In addition, the results support the threat simulation theory of dreaming, which suggests that threatening events in waking life increase the possibility of threatening events in dreams.

Construction and validation of a scale of sexual self-concept for the elderly Chilean population.

Sexual self-concept has a central role in the life of the elderly population. Indeed, their sexual self-concept has significant and positive relationships with their satisfaction with life, pleasure, and willingness to interact with others. However, social-cultural prejudice means that the elderly are considered asexual individuals, harming their sexual self-concept. This prejudice is prominent in Chile, where the elderly do not have access to clear information about their sexuality. However, research on the Chilean elderly population is still in its infancy and requires more attention. Hence, this research aims to construct and validate a scale of sexual self-concept for the elderly Chilean population to cover this identified gap in the literature. Sixty items were integrated into the first version of the scale. Ten external judges were asked to assess the content validity. Twenty-eight items were maintained. Subsequently, an instrumental and cross-sectional design was implemented with a non-probabilistic sampling (N = 188). Items were refined with corrected homogeneity indices and conditional estimates of Cronbach's alpha and Omega coefficient. A final scale of nine items equally distributed in three dimensions was obtained: Sexual self-efficacy (ω = 0.867), Sexual assertiveness (ω = 0.764), and Sexual self-esteem (ω = 0.803). The confirmatory factor analysis reflects that the theoretical model has an adequate fit (CFI = .989; TLI = .984; RMSEA = .086). The data analyses confirmed that the scale has adequate psychometric properties. This scale can be used for multidimensional measurements of sexual self-concept in the elderly in Chile. Further research can confirm its psychometric properties in different settings within the Spanish language population.

Presleep focusing on positive spontaneous thoughts enhanced the possibility of dreaming of them.

Dreaming is the subjective experience during sleep. A spontaneous thought is a thought that comes to one's mind involuntarily. This study investigated whether presleep focusing on a positive spontaneous thought enhanced the possibility of dreaming of the thought. Ninety-seven participants were quasi-randomly assigned to an expression condition (focus on an spontaneous thought for 5-Min before sleeping; N = 45) and a control condition (think about anything for 5-Min before sleeping; N = 45). Participants completed a dream diary upon waking. Then, both participants themselves (the selfrating method) and external judges (the external-rating method) rated the correlation between the positive spontaneous thought and the dream. The result of the external-rating method indicated that presleep focusing on positive spontaneous thoughts enhanced the possibility of dreaming of the thoughts. In addition, the external-rating method found that presleep focusing on positive spontaneous thoughts enhanced the possibility of dreaming of thoughts that were related to the positive spontaneous thoughts but not the positive spontaneous thoughts themselves. These results supported the current concern theory which suggests that one's current concerns increase responses to cues related to the concerns implicitly. In addition, these results supported the continuity hypothesis which states that dreaming is in continuous with waking life, and thus the intensity of a daily concern may be related to the possibility of dreaming of the daily concern.

Functional brain activation patterns of creative metacognitive monitoring

Creative metacognitive monitoring represents the ability to accurately evaluate the quality of own ideas during idea generation. To the best of our knowledge, this study presents the first EEG investigation of creative metacognitive monitoring in the brain, using data, of 100 participants, who generated single, original uses of common objects (alternate uses task). After each response, participants subjectively rated the creative quality of their idea. Additionally, five independent external judges rated the creative quality of all ideas. The correspondence between the subjective and the external performance ratings served as a measure of monitoring accuracy. We applied a generalized linear mixed effects model to investigate effects of creative metacognitive monitoring and creative potential on EEG activity in the alpha band at idea and person level. Participants with both higher monitoring skills and higher creative potential showed stronger alpha power decreases at parietal/occipital sites during creative idea generation and evaluation. Interestingly, only more creative people with lower metacognitive monitoring skills showed the expected alpha power increases at parietal/occipital sites during both phases. Furthermore, metacognitive monitoring skills were associated with lower frontal and temporal/central alpha power during idea evaluation (compared to generation) at the person level. This pattern of findings seems to suggest that less internal attention, less memory load, and increased sensory processing are associated with more effective and accurate monitoring of the creative process. This study sheds first light on the brain mechanisms underlying the interplay of creative metacognitive monitoring skills and creative potential.

Construction and validation of a scale of sexual self-concept for the elderly Chilean population

Background: Sexual self-concept has a central role in the life of the elderly population. Indeed, their sexual self-concept has significant and positive relationships with their satisfaction with life, pleasure, and willingness to interact with others. However, social-cultural prejudice means that the elderly are considered asexual individuals, harming their sexual self-concept. This prejudice is prominent in Chile, where the elderly do not have access to clear information about their sexuality. However, research on the Chilean elderly population is still in its infancy and requires more attention. Hence, this research aims to construct and validate a scale of sexual self-concept for the elderly Chilean population to cover this identified gap in the literature. Methods: Sixty items were integrated into the first version of the scale. Ten external judges were asked to assess the content validity. Twenty-eight items were maintained. Subsequently, an instrumental and cross-sectional design was implemented with a non-probabilistic sampling (N = 188). Items were refined with corrected homogeneity indices and conditional estimates of Cronbach's alpha and Omega coefficient. Results: A final scale of nine items equally distributed in three dimensions was obtained: Sexual self-efficacy (ω = 0.867), Sexual assertiveness (ω = 0.764), and Sexual self-esteem (ω = 0.803). The confirmatory factor analysis reflects that the theoretical model has an adequate fit (CFI = .989; TLI = .984; RMSEA = .086). Conclusions: The data analyses confirmed that the scale has adequate psychometric properties. This scale can be used for multidimensional measurements of sexual self-concept in the elderly in Chile. Further research can confirm its psychometric properties in different settings within the Spanish language population.

Excellence bias related to rating scales with summative jury assessment

Juries are a high-stake practice in higher education to assess complex competencies. However common, research remains behind in detailing the psychometric qualities of juries, especially when using rubrics or rating scales as an assessment tool. In this study, I analyze a case of a jury assessment (N = 191) of product development where both internal teaching staff and external judges assess and fill in an analytic rating scale. Using polytomous item response theory (IRT) analysis developed for the analysis of heterogeneous juries (i.e. jury response theory or JRT), this study attempts to provide insight into the validity and reliability of the used assessment tool. The results indicate that JRT helps detect unreliable response patterns that indicate an excellence bias, i.e. a tendency not to score in the highest response category. This article concludes with a discussion on how to counter such bias when using rating scales or rubrics for summative assessment.

JIC: Scientific Initiation Day to Promote University Research From Early Stages

To build “research culture” from early phases in student training, the Universidad Tecnologica de Panama implemented the Scientific Initiation Day (JIC). This problem-based learning model consists of presenting research projects as complementary activities to those developed in classrooms, within the scope of a particular subject. At the beginning of the academic period, questions - problems are formulated – and through the use of the scientific method, the teams, of up to 4 students, try to find a solution to the problem posed; being that the key to success lies in the professor, who serves as a research advisor. At the end, the projects are presented to a panel of external judges and are evaluated according to the written scientific article, the oral presentation and the poster delivered. The best evaluated projects participate in the National JIC that includes all the accredited universities in Panama, who have also adopted this educational model.

Ejercicios de tracto vocal semi-ocluido en la rehabilitación de parálisis recurrenciales: Un estudio de caso

El presente trabajo tiene como propósito mostrar la efectividad de una secuencia de ejercicios de tracto vocal semi-ocluido en la rehabilitación de un caso clínico vocal diagnosticado con parálisis recurrencial unilateral. Los ejercicios de tracto vocal semi-ocluido corresponden a una serie de posturas que involucran alargamientos artificiales o estrechamientos del tracto vocal con la finalidad de aumentar la impedancia y generar cambios favorables en la vibración de los pliegues vocales. Dichos ejercicios están enmarcados dentro de la tendencia de rehabilitación fisiológica la cual busca reestablecer un balance de los tres subsistemas involucrados en la producción de la voz (Respiración-fonación-resonancia) de manera simultánea, contraponiéndose a la tendencia tradicional de abordaje sintomatológico. Por otro lado, la tendencia fisiológica se basa en los principios del aprendizaje sensoriomotor el cual busca reales cambios en el patrón de producción de la voz, y que estos permanezcan en el tiempo.
 Se realizó terapia vocal durante 15 sesiones con ejercicios de tracto vocal semi-ocluido. Para valorar los cambios en la voz, se realizó una evaluación inicial y final a través de nasofibrolaringoscopía, análisis acústico mediante medidas de perturbación y análisis espectral, estudio perceptual mediante la escala GRBASI valorada por una comisión de jueces externos, tiempos máximos fonatorios y valoración del propio paciente. Posterior al tratamiento se produjeron cambios favorables en la voz y se concluye que los ejercicios de tracto vocal semi-ocluido pueden constituir una herramienta efectiva para el abordaje de este tipo de patologías.

Relationship Between Personality Types in MBTI and Dream Structure Variables.

This study aimed to explore relationships between personality type variables and dream structure variables. In the questionnaire experiment (N = 410), we investigated associations between different personality variables in the Myers-Briggs Type Indicator questionnaire (MBTI) and various aspects of dreams in the Mannheim Dream questionnaire (MADRE). The MBTI has four dimensions. In the Extroversion/Introversion (E/I) dimension, I types dreamt more of emotional intensity and passive emotions than E types. In addition, I types may become more distressed in nightmares than E types. E types more frequently shared their dreams with others. In the Sensation/Intuition (S/N) dimension, N types had a more positive attitude toward dreams and can get more novel ideas and help from their dreams than S types. In the dream diary experiment (N = 47), we investigated whether the S/N dimension may influence waking events’ incorporation into dreams. External judges decoded paired waking events and dream reports. N types had more metaphorical incorporation than S types. More specifically, N types had more metaphorical expressions in their dreams than S types. This result may be due to the different characteristics between S types and N types. It may provide support for the dream continuity hypothesis.

The general factor of personality and creativity: Diverging effects on intrapersonal and interpersonal idea evaluation

The general factor of personality (GFP) has been found to promote social adaptiveness in a variety of contexts. However, GFP may exhibit both positive and negative effects on creative processes. In the present experiment, 178 undergraduates were asked to critique their own ideas or a fictional peer's ideas. The quality of these critiques was appraised by external judges based on five markers of idea evaluation—number, range, depth, specificity, and usefulness. In line with research on self-enhancement bias, results showed that participants who evaluated their own ideas produced lower quality critiques compared with those who evaluated peer ideas. However, this effect was moderated by GFP, such that individuals higher in GFP provided lower quality critiques when evaluating their own ideas. In contrast, participants higher in GFP tended to produce higher quality critiques of peer ideas. Thus, GFP appears to facilitate interpersonal idea evaluation while hindering intrapersonal idea evaluation.

Gratuidad en la educación superior chilena: análisis de prensa basado en minería de datos

El presente artículo investiga sobre los titulares de los medios de comunicación chilenos referentes a gratuidad en la educación superior, que fue una de las temáticas más contingentes tanto en la agenda política como en la noticiosa, desde que emergió el llamado movimiento estudiantil en 2011 a la fecha. Dichos titulares se obtuvieron de las cuentas en la red Twitter de 290 medios de comunicación chilenos. En total se trabajó con 2.137 titulares, recolectados entre el 1° de marzo y el 31 de julio de 2016. Con base en este material se descubrió cuáles fueron las voces que informaron sobre el tema, los tipos de medios y la calificación de los titulares, lo que realizó un trío de jueces externos.
 Entre los resultados podemos destacar la gran relevancia de actores oficialistas (entendidos como los representantes del gobierno), y la preeminencia de notas calificadas como negativas, que, aunque tuvieron una leve superioridad frente a las positivas, primaron entre las noticias más repetidas.

Psychodynamic Profile in an Early Dropout Case: Comparing Therapist’s and External Judges’ perspectives

Psychodynamic Profile in an Early Dropout Case: Comparing Therapist’s and External Judges’ perspectives

TA Treatment of Depression: A Simplified Hermeneutic Single-Case Efficacy Design - The Case of “Margherita”

This study is the seventh of a series of seven and belongs to the second Italian systematic replication of findings from previous series that investigatedthe effectiveness of a manualized transactional analysistreatment for depressionthrough Hermeneutic Single-Case Efficacy Design. We address problems and difficulties emerged in previous case series, such as: spending time in training a group of people to conduct the hermeneutic analysis, organizing the involvement of external judges to give the final adjudication, and dealing with inconsistencies between quantitative and qualitative data. For these reasons, this study suggests a simplified method to conduct the hermeneutic analysis that require one person only, maintaining its validity. Therefore, we integrated hermeneutic design with the pragmatic case evaluation methodology in order to follow pre-defined criteria in analysing qualitative material. Furthermore, we present a way to use the Script System to detect changes in depressive symptomatology and depressive personality. We tested this approach to HSCED in the case of ‘Margherita’, a 56-years old white Italian woman who attended sixteen session of transactional analysis psychotherapy with a white Italian woman therapist with 5 years of clinical experience. The patient satisfied DSM-5 criteria for moderately severe major depressive disorder with anxious distress, and SWAP 200 criteria for traits of depressive, dependent, avoidant and hostile personality typeswith a high level of functioning.

TA Treatment of Depression: A Simplified Hermeneutic Single-Case Efficacy Design - The Case of “Giovanni”

This study is inspired by previous case series replications of Hermeneutic Single-Case Efficacy Design which aim was to evaluate the effectiveness of a manualized transactional analysis treatment for depressive disorders and depressive personality. We address problems and difficulties emerged in previous case series, such as: spending time in training a group of people to conduct the hermeneutic analysis, organizing the involvement of external judges to give the final adjudication, and dealing with inconsistencies between quantitative and qualitative data. For these reasons, this study suggests a simplified method to conduct the hermeneutic analysis that require one person only, maintaining its validity. Therefore, we integrated hermeneutic design with the pragmatic case evaluation methodology in order to follow pre-defined criteria in analysing qualitative material. Furthermore, we present a way to use the Script System to detect changes in depressive symptomatology and depressive personality. We tested this approach to HSCED in the case of ‘Giovanni, a 17-years old white Italian boy who attended sixteen session of transactional analysis psychotherapy with a white Italian woman specializing in psychotherapy with 2 years of clinical experience. The patient satisfied DSM-5 criteria for moderate major depressive disorder and generalized anxiety disorder.This is the second investigation which has evaluated the effectiveness of Transactional Analysis psychotherapy for depressed adolescents.

Why Do Great Creative Ideas Get Rejected?

ABSTRACT One of the most contentious issues in advertising is the evaluation and selection of creative ideas. This selection process is central to the success of agencies and clients, yet it appears difficult for external observers to see the merits of highly creative ideas. To understand this issue, the authors compared self-reports of creative ideas from industry professionals with those of external judges. Results suggest that the difficulty in the external evaluation of creative ideas is not recognizing originality but recognizing appropriateness when the idea is also highly original. Creative ideas do not speak for themselves; presentation is crucial.

Alignment of the Performance Assessment for School Leaders to the Professional Standards for Educational Leaders: A Distance‐Based Approach

AbstractWe conducted a distance‐based alignment study for the Performance Assessment for School Leaders (PASL). PASL is a multiple‐task portfolio assessment that requires candidates to submit contextual information, written responses, and artifacts (e.g., student work) to provide evidence of the completion of the task requirements. This alignment study supports the content validity of the PASL by providing evidence from external judges indicating that the assessment measures the knowledge and skills outlined in the Professional Standards for Educational Leaders (PSEL). This research report also documents the distance‐based methods that were used to complete the alignment study. An expert panel of eight educators provided judgments indicating the alignment between the PASL and the PSEL. Panelists were provided with preparatory information about the assessment and the standards before attending a webinar that provided more detailed information about the PASL. Training was provided via recorded presentations, and panelists used an online survey to complete their alignment judgments independently. Panelists then completed final evaluations using an online survey. The panel judged that the PASL measures all 10 PSEL standards and 61% of the supporting elements.

Accretion theory of ideation: evaluation regimes for ideation stages

This paper presents an exercise on theory building to characterise design ideation. It starts by examining how early ideas are defined and evaluated in the literature. Anessentialistview is identified that explains the creativity of a final design solution by the creative qualities of early ideas attributed by external judges. Criteria for a theory of ideation that does not rely on the primacy of essence are enumerated. Advanced professional practice is examined to understand evaluation of early ideas ‘in the wild’. Accretion is then introduced as an analogical model to imaginatively drive definitions and conjectures about idea formation in the co-evolution of problem and design spaces. Vignettes from ideation episodes are used to illustrate an accretion theory of ideation. An accretion theory supports new ways to think about ideation as a complex formation process where creative solutions emerge from the synthesis of a multitude of fragmentary and partial ideas – or ‘ideasimals’. An accretion theory of ideation helps to explain the creative value of a final design solution without relying on early ideas having a creative essence, because the creativity of a solution is viewed as emergent rather than present in early versions. An accretion lens is used to suggest new ideation metrics to study the qualities of idea fragments and the process of idea formation. Definitions and relevant assessment regimes for different stages of ideation are discussed. The paper concludes with a discussion on entailments of an accretion theory and next steps for this theory building enterprise.

Acculturation strategy and innovation in higher education: a study of individuals and dyads

This novel quasi-experimental study examines the relation between two salient constructs: acculturation strategy and innovation in higher education. A sample of 54 undergraduates, comprising equal numbers of assimilated and bicultural students, participated in the first study session where they individually assembled comic books and stationery into new product prototypes to cheer stressed classmates. In the second study session, the students were randomly grouped into 13 homogeneous and 13 heterogeneous acculturated pairs to assemble the comic books and stationery into novel prototypes to help classmates reconcile over a disagreement. Two external innovation judges blindly scored the degree of innovation of each prototype. Findings exhibit that acculturation strategy significantly predicts the original dimension of individual innovation, but dyad acculturation type does not significantly predict dyad level innovation. The study suggests that pairing may not need to be solely based on acculturation strategy for enhancing pair innovation in higher education and organisations.