International Test Commission Research Articles

Evaluating Construct Validity and Reliability of Work Family Strength (WFS) : Indonesia Women Leader Context

The strategic role of leadership on organizational performance is the reason why this topic continues to be researched in research and practical studies. Female leaders in Indonesia are still faced with the problem of a narrow symbolic idealization perspective such as feeling it is impossible to achieve success in both the domestic and public fields, and also feeling guilty for violating. The nature of women Therefore, awareness of the power of work and family is important for female leaders because by creating positive feelings, individuals will have a broad perspective in influencing other roles and aspects of their lives. Work family strength (WFS) is a positive experience brought from work roles as well as roles in the family. For that we need a measurement related to work family strength (WFS). So far, research related to female leaders is more related to work family conflict, for this reason the research provides updates related to positive things, namely work family strength, especially in measuring tools. The absence of adaptation in other languages is deemed necessary to adapt the work family strength scale in Indonesian. Adaptation is carried out based on the guidelines for adapting measuring instruments from the International Test Commission (ITC), which includes a forward - backward translation process, followed by expert review to test language and conceptual equivalence. The validation process is carried out by testing content validity, through the Content Validity Index (CVI), and construct validity, using Confirmatory Factor Analysis (CFA). The trial was conducted on 300 female leaders who work in for-profit companies, who work more than 40 hours/week and have families (husbands or children). Based on this trial process, it is known that each indicator, dimension, and construct meets all the criteria for Goodness of Fit (RMSEA < 0.08; CFI > 0.9). In addition, the value of Construct Reliability (CR) > 0.70 and Average Variance Extracted (AVE) > 0.50 were also obtained. Thus, the work family strength scale can be used as a valid and reliable measuring tool. Â Keywords : Work Family Strengths, Women Leader, Confirmatory Factor Analysis

Adaptation and Validation of Cyber Incivility Scale in Indonesian Language and Culture with Confirmatory Factor Analysis

This research, divided into two studies, aimed to modify and adapt the cyber incivility scale compiled by Lim and Teo (2009) in the Indonesian language and culture. Study 1, namely the process of translating the items and testing their suitability, involved six translators, three expert reviewers, and 10 participants for cognitive debriefing purposes. Meanwhile, Study 2 involved 232 workers in Indonesia who occupied structural leadership positions. At this stage, a trial of the cyber incivility scale was carried out. The scale was adapted with confirmatory factor analysis using AMOS version 22. The entire adaptation process referred to the procedures of the 2016 International Test Commission (ITC) Guidelines for Test Adaptation. The CFA results showed that the cyber incivility scale measurement model fitted with Lim and Teo's (2009) cyber incivility construct. Overall, this research showed that the Indonesian version of the cyber incivility scale is valid and reliable, thus applicable to measure the cyber incivility in the context of the perpetrator in Indonesia.

Indonesian Adaptation and Validation of The Child-Adolescent Perfectionism Scale – Short Form (CAPS-SF)

This study aimed to adapt and examine the psychometric properties of The Child-Adolescent Perfectionism Scale – Short Form (CAPS – SF) in an Indonesian sample. In Indonesia, research on perfectionism is being developed, but there is no measurement for adolescent perfectionism that can be used. Therefore, a valid and reliable measurement is needed. The Child-Adolescent Perfectionism Scale – Short Form (CAPS – SF) developed by Bento et al. is an instrument consisting of 2 dimensions of Socially Prescribed Perfectionism and Self-Oriented Perfectionism with a total of 9 items that can measure perfectionism in children and adolescents. The adaptation process followed the guidelines established by the International Test Commission such as translation, cognitive interview, and test of the pre-final Indonesian version which involved 322 adolescents aged 13-15 years who lived in Indonesia. Reliability test using Cronbach's Alpha and CFA, which was used to measure the construct validity, showed that the nine items in adapted The Child-Adolescent Perfectionism Scale – Short Form (CAPS – SF) could be used to measure the perfectionism of Indonesian adolescents.

Adaptasi dan validasi Skala Stres Peran (Role Stressor Scale) dalam budaya Indonesia: Eksplorasi ekuivalensi konstruk dan analisis faktor konfirmatori

This study aims to conduct instrument adaptation of role stressor scale developed by Peterson et al. (1995) in the Indonesian language and culture. The research was performed in two studies. In study 1, the translation process and a construct equivalence test was carried out involving 6 translators, 3 expert reviewers, and 10 laypeople for cognitive debriefing. Study 2 involved 232 subjects of workers in Indonesia and examined the validity using confirmatory factor analysis. The entire adaptation process refers to the procedures of the 2016 International Test Commission (ITC) Guidelines for Test Adaptation. The CFA results showed that the measurement models of the role stressor instrument fitted the 3 factors model. Overall, this study showed that the Indonesian version of the role stressor instrument is valid and reliable, therefore, it can be utilized to measure the role stressors in Indonesia

Adaptasi dan validasi Skala Stres Peran (Role Stressor Scale) dalam budaya Indonesia: Eksplorasi ekuivalensi konstruk dan Confirmatory Factor Analysis

This study aims to conduct instrument adaptation of role stressor scale developed by Peterson et al. (1995) in the Indonesian language and culture. The research was performed in two studies. In study 1, the translation process and a construct equivalence test was carried out involving 6 translators, 3 expert reviewers, and 10 laypeople for cognitive debriefing. Study 2 involved 232 subjects of workers in Indonesia and examined the validity using confirmatory factor analysis. The entire adaptation process refers to the procedures of the 2016 International Test Commission (ITC) Guidelines for Test Adaptation. The CFA results showed that the measurement models of the role stressor instrument fitted the 3 factors model. Overall, this study showed that the Indonesian version of the role stressor instrument is valid and reliable, therefore, it can be utilized to measure the role stressors in Indonesia

Adaptasi alat ukur kesuksesan karier subjektif

Subjective career success is an attitude regarding an individual’s achievement and satisfaction with their career resulting from a personal evaluation of their career experience. Subjective career success has eight dimensions: recognition, quality work, meaningful work, influence, authenticity, personal life, growth and development, and satisfaction. Subjective Career Success Inventory – SCSI (Subjective Career Success Questionnaire) has 24 items. The use of instruments at different cultural backgrounds requires an adaptation process so that the measurement results are valid and reliable, but until now, there has been no research on the adaptation of subjective career success measurement tools in Indonesia. The purpose of this study is to obtain a standardized subjective career success measurement tool. The adaptation process was carried out using the International Test Commission reference in2016. Based on the results of the CFA analysis of the LISREL 8.8 program, it can be concluded that according to theory, subjective career success models have a good fit model. This means that the zero hypotheses that state the subjective career success model of the goodness of fit are acceptable. This explains that the resulting model can describe actual conditions and can be used to measure subjective career success.

Adapting the Trait Emotional Intelligence Questionnaire Short Form (TEIQUE-SF) into Indonesian Language and Culture Using Confirmatory Factor Analysis

This study aimed to adapt the TEIQue-SF 1.50 measuring instrument to the Indonesian culture. This research was conducted in two studies. In Study 1, the translation process and the equivalence testing were carried out, involving six translators, three expert reviewers, and ten laypeople for cognitive debriefing. Study 2 involved 200 subjects who were workers in Indonesia and tested the TEIQue-SF 1.50 measuring instrument with confirmatory factor analysis using AMOS version 22. The entire adaptation process referred to the International Test Commission (ITC) Guidelines for Test Adaptation 2016. The CFA results showed that the TEIQUE-SF 1.50 measurement model was fit to the one-factor model. Overall, this study showed that the Indonesian version of TEIQue-SF 1.50 was valid and reliable so that it can be used to measure the Trait Emotional Intelligence in Indonesia

Emotional Development Questionnaire for Primary Education (CDE_9–13)

This study presents the construction of the Emotional Development Questionnaire (CDE_9–13) and examines its psychometric properties. This questionnaire measures the emotional competence and its five dimensions—emotional awareness, emotional regulation, emotional autonomy, social competence, and life and well-being competence—of boys and girls from 9 to 13 years of age. Its construction followed the guidelines of the International Test Commission. The final version consists of 41 items. The total sample is 1905 boys and girls between the ages of 9 and 13, although partial samples have been used for specific analyses. Various studies have been carried out to demonstrate the reliability and validity of the instrument: the calculation of the reliability coefficient, a confirmatory factor analysis (CFA), and the correlational comparison of the CDE_9–13 with recognized measures of emotional intelligence, personality, adjustment difficulties, and self-esteem. Likewise, a regression study has been carried out to confirm the incremental validity. The CDE_9–13 is a theoretically well-founded questionnaire with appropriate psychometric characteristics. Therefore, it is considered an optimal tool to assess emotional competence in interventions aimed at promoting mental health and well-being.

Psychometric Properties of Indonesian Version Co-Worker Support Scale

This study aimed to validate the Indonesian Version of the Co-worker Support Scale. A scale used to measure co-support in organization. The International Test Commission (ITC) Guidelines for Translating and Adapting Test Second Edition procedure used to adapt this scale. A total of 318 employees (M = 44%, F = 56%; SD Age = 8.3) from various organizations were involved in this study. Based on the Rasch model analysis, it indicated that the Co-worker Support Scale has good internal consistency. The result showed that the scale’s factor structure was unidimensional, RV= 60.8%. It means that the items can be effectively able to measure co-worker support. The significant chi-square value indicates that the data is fit to the model, χ2 (4118) = 8017.19, p < .01. The answer choice category option with 5 answer choices can be preserved. Moreover, no differential item functioning was found based on gender, education level, and type of organization. Overall, the results indicated that this 14-item scale, including its psychometric properties, was satisfactorily reliable and valid to measure co-worker support in the organizational context in Indonesia.

Adaptación transcultural al español y validez de contenido de 3 escalas de riesgo nutricional

IntroductionThere are various scales designed to determine the risk of malnutrition at hospital admission in children. However, most of these instruments are developed and published in English. Their cross-cultural adaptation and validation being mandatory in order to be used in our country. ObjectivesCross-culturally adapt three scales designed to determine the risk of malnutrition linked to the disease and determine the validity of their content. Material and methodsCross-cultural adaptation using the translation-back-translation method in accordance with the recommendations of the International Test Commission Guidelines for Translating and Adapting Tests. Content validity was measured by a panel of experts (under seven basic selection criteria adapted from the Fehring model) who evaluated each item of the scales by measuring 4 criteria: ambiguity, simplicity, clarity and relevance. With the extracted score, Aiken's V statistic was obtained for each item and for the complete scales. ResultsStarting from three independent translations per scale, 3 definitive versions in Spanish of the PNRS, STRONGkids and STAMP scales were obtained semantically equivalent to their original versions. The PNRS and STRONGkids scales presented an Aiken's V greater than 0.75 in all their items, while the STAMP scale presented a value less than 0.75 for the item “weight and height”. ConclusionThis study provides the transculturally adapted Spanish versions of the PNRS, STRONGkids and STAMP scales. The PNRS and STRONGkids scales present valid content to be applied in the state hospital context. STAMP requires the adaptation of its item “weight and height” to consider its use in a Spanish child population adequate.

Composition Factor Analysis and Factor Invariance of the Physical Appearance State and Trait Anxiety Scale (PASTAS) in Sports and Non-Sports Practitioner Mexican Adolescents.

The purpose of the present study was to adapt and validate the Physical Appearance State and Trait Anxiety Scale (PASTAS) for Mexican adolescents, verifying the factor invariance by sports and non-sports practitioners. A sample of 930 Mexican adolescents (46.0% females), aged 11-15 years old, voluntarily participated in the study. A total of 415 participants regularly played sports in a club and/or regularly participated in sports competitions and 515 were non-sports practitioners. The adolescents filled out the trait version of the PASTAS questionnaire, which was previously translated and adapted for Mexican-speaking adolescents following the International Test Commission guidelines. The results of the confirmatory factor analyses showed an adequate measurement model for the original two-factor structure (e.g., GFI = 0.913; RMSEA = 0.078; CFI = 0.943). The internal consistency of the two dimensions was excellent (α and Ω = 0.92-0.93). Additionally, the results of the factorial invariance analyses showed an appropriate fit of the two-structure model (e.g., GFI = 0.96; CFI = 0.98; RMSEA = 0.04) among both sports and non-sports practitioners. The proposed trait version of the PASTAS questionnaire adapted to a Mexican-speaking population shows adequate psychometric properties among Mexican adolescents. The Mexican version of the PASTAS questionnaire supports the original two-factor structure (i.e., factor related to the body weight and factor not related to body weight) among adolescents. Additionally, the factorial invariance analyses support the equivalence of the two-factor structure among both sports and non-sports practitioners.

THE SOROGAN VERSION OF ACADEMIC MOTIVATION SCALE (AMS-VS) FOR ISLAMIC BOARDING SCHOOL STUDENTS

There are many instruments that have been developed by researchers to measure motivation. However, none of the instruments has been specifically designed to measure sorogan motivation on Islamic boarding school students (santri). Therefore, this article was written to discuss the processes of adaptation, testing the validity, and reliability of academic motivation scales to measure sorogan motivation of santri. The authors use Theory of Self Determination (Deci & Ryan, 1995), Academic Motivation Scale (Vallerand et al., 1992), and AMS Bahasa Indonesia (Natalya & Purwanto, 2018) as guidelines in compiling indicators, aspects, and items of Academic Motivation Scale Sorogan Version (AMS-VS). Test adaptation guidelines issued by the International Test Commission (2018) are used in adapting the AMS. The data collected from 102 respondents were analysed using Confirmatory Factor Analysis (CFA). The results showed that AMS-VS is a proper and strong measurement to determine the level sorogan motivation of santri.

Adaptation transculturelle de tests et échelles de mesure psychologiques : guide pratique basé sur les Recommandations de la Commission Internationale des Tests et les Standards de pratique du testing de l’APA

Thirty years ago, Vallerand (1989) wrote a pioneering article entitled “Toward a methodology for the transcultural validation of psychological questionnaires: Implications for research in the French language”, whose success has never failed, as well as its usefulness. However, considerable progress has been made in the field of test development and test adaptation since the publication of this article, notably with the publications (and their updates) of the ITC Guidelines for Translating and Adapting Tests (2017) by the International Test Commission, and of the Standards for Educational and Psychological Testing (2014) by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME). Based on these two key references, the present article aimed to update the procedure proposed by Vallerand. Without claiming to replace these major references, on the contrary, our objective here is to propose a 10-step procedure for adapting tests and psychological measures originally developed for other cultures and/or languages, a sort of framework guiding in step-by-step manner the adaptation of tests and the validation of their scores.

Психометрические стандарты оценивания на основе методологии «Performancebased Assessment»

The relevance of the study is due to the fact that there has been a request for a comprehensive assessment of knowledge and skills based on the methodology of competency-based assignments, which consists in a performance-based assessment. In the article, the author presents the psychometric standards and recommendations existing in foreign practice that are necessary for a fair, reliable and objective assessment of the learning outcomes of company employees. The purpose of the work is to study existing standards and recommendations for assessing learning outcomes in the KPI system. The empirical basis of the study was the data of the International Test Commission; British Psychological Society – British Psychological Society, BPS and others. The author analyzes the empirical base of the research and systematizes the existing standards and recommendations for assessing testing. The practical significance of the research results lies in the fact that, given the challenges and demands of the modern world, most organizations will allow them to form a systemic development of the measurement process and quickly respond to new challenges that arise before psychometrics.

Design and Development of a Virtual Reality-Based Mobility Training Game for People With Parkinson's Disease.

People with Parkinson's disease (PD) commonly have gait impairments that reduce their ability to walk safely in the community. These impairments are characterized, in part, by a compromised ability to turn and negotiate both predictable and unpredictable environments. Here, we describe the development and usability assessment of a virtual reality training application, Wordplay VR, that allows people with PD to practice skills such as turning, obstacle avoidance, and problem-solving during over-ground walking in a game-based setting. Nine people with PD completed three sessions with Wordplay VR, and each session was directed by their personal physical therapist. Our outcome measures included perceived sense of presence measured using the International Test Commission–Sense of Presence Inventory (ITC-SOPI), levels of motivation using the Intrinsic Motivation Inventory (IMI), overall system usability using the System Usability Scale (SUS), and setup time by the physical therapists. Both the people with PD and the physical therapists rated their sense of presence in the training system positively. The system received high ratings on the interest and value subscales of the IMI, and the system was also rated highly on usability, from the perspective of both the patient during gameplay and the therapist while controlling the experience. These preliminary results suggest that the application and task design yielded an experience that was motivating and user-friendly for both groups. Lastly, with repeated practice over multiple sessions, therapists were able to reduce the time required to help their patients don the headset and sensors and begin the training experience.

Adaptation and Validation of Conners-3 Teacher and Parent Rating Scales on Lebanese Children

Conners-3 rating scale is primarily used in the assessment of ADHD and it has been found to be instrumental in variety of areas such as screening, assessment, and treatment monitoring. Most, if not all, rating scales are developed on western standards and available in different languages especially, English. For this reason, we adapted the Conners-3 teacher and parent rating scales since we are ethically responsible to have a rating scale that assess ADHD and takes into consideration the linguistic and cultural differences among the Lebanese population. Conners-3 aims to measure ADHD and most common comorbid problems in children and adolescents. The purpose of this study was to adapt and validate the Conners-3 teacher and parent rating scales to the Lebanese context. The International Test Commission (ITC) guidelines were used for translating the rating scales, and then they were reviewed by professionals, piloted, revised, and then administered. The sample consisted of students enrolled in grades 1 through 12 from nine schools. Their parents (455) and 108 teachers of (509) students filled the respective Conners-3 rating scales. The reliability and validity of the scales was investigated by examining the internal consistency and test-retest reliabilities, construct validity in terms of gender and age differences, discriminant validity between ADHD and non-ADHD groups, and Exploratory Factor Analysis. The reliability coefficients and discriminant validity were moderate to high. Age and gender differences were not well defined, and the obtained factor structure partially resembled that of the original Conners-3. Norms in terms of T scores and percentiles were reported. Results were discussed and limitations and recommendations for the future studies were proposed.

An interpretive phenomenological analysis of formative feedback in anesthesia training: the residents\u2019 perspective

BackgroundConsistent formative feedback is cornerstone to competency-by-design programs and evidence-based approaches to teaching and learning processes. There has been no published research investigating feedback from residents’ perspectives. We explored the value residents place on feedback in routine operating room settings, their experiences, and understanding of the role of feedback in their training and developing professional identity.MethodsInterpretive phenomenological analysis of residents’ experiences with feedback received in clinical settings involved two focus groups with 14 anesthesia residents at two time points. Analysis was completed in the context of a teaching hospital adapting to new practices to align with nationally mandated clinical competencies. Focus group conversations were transcribed and interpreted through the lens of a social constructivist approach to learning as a dynamic inter- and intra-personal process, and evidence-based assessment standards set by the International Test Commission (ITC).ResultsResidents described high quality feedback as consistent, effortful, understanding of residents’ thought processes, and containing actionable advice for improvement. These qualities of effective evaluation were equally imperative for informal and formal evaluations. Residents commented that highest quality feedback was received informally, and formal evaluations often lacked what they needed for their professional development.ConclusionResidents have a deep sense of what promotes their learning. Structured feedback tools were seen positively, although the most important determinants of their impact were faculty feedback- and broader evaluation-skills and motivations for both formal and informal feedback loops.

Development of two instruments for assessing maternity health needs: protocol of a clinimetric study

BackgroundThere is an unquestionable need to adapt health care to the needs of each woman, to foster her self-confidence and provide her with the autonomy to manage her own maternity. This involves empowering her to choose and face her model of childbirth and childcare responsibly. The range of self-management health needs tests offered by the scientific community at this stage of life is practically non-existent. In this project, we intend to develop and evaluate the validity, reliability and ease of use of two self-administered analysis instruments for: 1.- Needs of women preparing for childbirth and 2.- Identification of alarm symptoms in the puerperium.MethodsThis is a descriptive study of the clinimetric characteristics and usability of two developed self-applied digital instruments for measuring needs in childbirth and postpartum based on the recommendations made in the consensus-based standards for the selection of health measurement instruments (COSMIN) and by the International Test Commission (ITC). The study consists of two phases: 1 - Evaluation of the clinimetric properties of the two instruments, which were developed and then altered, based on their comprehensibility and global usability estimated from a pilot study and 2 - Pre-implementation study.DiscussionThe final product will be two valid, reliable, usable instruments for self-assessment of health needs that are highly acceptable to young couples and the professionals who serve them. They will be a valuable resource for meeting the needs of the population more efficiently and guiding decision-making, and they will contribute to the greater sustainability of the health system.

Analisis Faktor Adaptasi Instrumen Resiliensi Akademik Versi Indonesia: Pendekatan Eksploratori dan Konfirmatori

Academic resilience is an individual response to overcome the academic difficulties they face. Academic resilience has three dimensions, namely perseverance, which reflects behavioral responses, reflecting and adaptive help-seeking, which reflects cognitive responses and negative affect and emotional responses that reflect the affective dimension. The Academic Resilience Scale (ARS-30) is the latest instrument that measures academic resilience. To be used in Indonesia, an adaptation process is needed to suit the context and culture in Indonesia. The purpose of this study is to validate the adaptation of ARS-30 to produce a valid and reliable measurement model. Adaptation is carried out following guidelines issued by the International Test Commission. Based on the results of reliability testing, EFA and CFA using JASP note that ARS-Indonesia is valid and reliable in measuring academic resilience. The final ARS-Indonesia fit model can be seen in the results section.

International Test Commission guidelines for test adaptation: A criterion checklist.

To improve the quality of test translation and adaptation, and hence the comparability of scores across cultures, the International Test Commission (ITC) proposed a number of guidelines for the adaptation process. Although these guidelines are well-known, they are not implemented as often as they should be. One possible reason for this is the broad scope of the guidelines, which makes them difficult to apply in practice. The goal of this study was therefore to draw up an evaluative criterion checklist that would help test adapters to implement the ITC recommendations and which would serve as a model for assessing the quality of test adaptations. Each ITC guideline was operationalized through a number of criteria. For each criterion, acceptable and excellent levels of accomplishment were proposed. The initial checklist was then reviewed by a panel of 12 experts in testing and test adaptation. The resulting checklist was applied to two different tests by two pairs of independent reviewers. The final evaluative checklist consisted of 29 criteria covering all phases of test adaptation: planning, development, confirmation, administration, score interpretation, and documentation. We believe that the proposed evaluative checklist will help to improve the quality of test adaptation.