Rasch Model Analysis Research Articles

Application of the Rasch Model to the Evaluation of Biological Problems

Evaluation of student abilities in Biology subjects often faces challenges in ensuring that the instruments used actually measure what they are supposed to measure. The data analysis used is the Rasch model, which is part of item response theory, offering a more accurate approach in analyzing evaluation data. This model allows the identification of items that are inappropriate or too difficult, as well as providing more in-depth information about individual student abilities. Through analysis of instruments and question items from 30 multiple choice questions that emphasize critical thinking skills, the aim of this research is to assess the quality of Biology questions for class X SMA. Testing was carried out using WinStep software and Rasch modeling analysis on 166 students from five high schools/vocational schools. The research method used is quantitative. The results of the instrument investigation show that the issue has very good dependability and the unidimensionality of the instrument is indicated by unexplained changes in the first differentiation. These results show that the Rasch test can help educators in selecting questions that meet the evaluation criteria and contribute to improving the quality of national education. The implications of this research are By applying the Rasch Model, educators can develop more accurate and fair evaluation tools, which are able to identify individual student difficulties and strengths.

Survey of Rasch Model Analysis of Mathematical Literacy Abilities of Senior High School Students

Mathematical literacy skills are needed in everyday life, but the mathematical literacy skills of students in schools show that Indonesian students are very low in the field of mathematical literacy. Therefore, this study aims to determine the Rasch model analysis survey of mathematical literacy skills of senior high school students. This study uses quantitative research methods with a survey approach and uses the Rasch model as a measurement. The subjects studied were 100 students consisting of grades X and XI of high schools in Bekasi city. Furthermore, the results of this study indicate that there is 1 out of 8 items that are misfit and for the provisional calculation of reliability is 0.42, so it is declared not reliable and for the DIF test results there is only one item that is biased.

Development of a Sustainability Awareness Tool for High School Biology Students

In the context of classroom-based learning, the majority of students tend to acquire cognitive knowledge. It is essential for students to not only acquire cognitive knowledge but also develop awareness of sustainability as a result of teaching and learning activities. The purpose of this study was to develop an instrument to measure sustainability awareness among high school students. Developing an instrument of sustainability awareness for students is important to measure and encourage their understanding and actions in preserving the environment sustainably. The instrument developed consists of three subdimensions: economy, society, and environment. This instrument assesses awareness of the concept of sustainable development and practices, attitudes, and moral values ​​for sustainability at the individual level. This study used the Rasch modeling analysis technique. The instrument is a closed questionnaire containing 21 items using a 4-point Likert scale. The results of the Rasch modeling showed that the overall Cronbach's alpha reliability coefficient was in the "good" category, while the item reliability coefficient was in the "very good" category. However, the student reliability coefficient was in the "weak" category. The results of the instrument validity test showed four invalid items. These items showed ZSTD and Pt Mean Corr values ​​that did not meet the established criteria. However, the existence of invalid instrument items and weak student reliability coefficients did not affect the overall quality of the instrument. Therefore, it can be concluded that the developed instrument is considered appropriate for measuring the sustainability awareness profile of students.

Analysis of biology midterm exam items using a comparison of the classical theory test and the Rasch model

In biology learning, test instruments are essential for assessing students' understanding of complex concepts. A test instrument is a crucial factor in learning evaluation; however, its implementation remains minimal. This descriptive quantitative study aims to analyze the quality of test items using the classical approach in terms of validity, reliability, difficulty index, discrimination power, distractor effectiveness, and the Rasch model analysis. The data consists of 30 multiple-choice questions from a biology midterm exam administered to 40 students. Classical test data analysis uses Microsoft Excel, while Rasch model analysis uses Winsteps software. The validity test results from both approaches show 14 valid questions and 16 invalid ones. The reliability scores are 0.619 (adequate) for the classical approach's Cronbach's Alpha, 0.85 (good) for the Rasch model, and 0.65 (weak) for personal reliability. The classical test theory and the Rasch model categorize item difficulty into four levels. The classical approach produces five categories for item discrimination, while the Rasch model identifies three groups based on the item separation index (H=3.45) and two groups based on respondent ability (H=1.96). Distractor effectiveness shows 93.3% functional distractors in the classical test and 80% in the Rasch model. The Rasch model offers greater precision in measuring student ability and detecting bias. Both models should be integrated for comprehensive item analysis. Future tests should focus on improving invalid items and the quality of distractors.

Relationship between maltreatment and mental health in adolescents: A school-based study in Indonesia.

Maltreatment affects emotional development in adolescents and inhibits social adjustment. This study aimed to analyze the relationship between maltreatment and mental health among adolescents. A cross-sectional study was conducted on adolescents in the first and second grades of middle school (12-14 years old) and high school (15-17 years old) in eight cities and municipalities in the province, selected through several stages of simple random sampling (N = 1837). The International Society for the Prevention of Child Abuse and Neglect (ISPCAN) Child Abuse Screening Tool for Children (ICAST-C) questionnaire for detecting maltreatment was translated, simplified, and validated by an expert based on a theoretical framework that involved pediatricians, public health, and medicolegal perspectives. The Strengths and Difficulties Questionnaire (SDQ) was used to assess emotional states. ICAST-C and SDQ scores were transformed to logit values using Rasch model analysis. Distribution frequency and linear regression were used for data analysis. The results indicated that 85.6% of adolescents aged 12-14 and 83% of those aged 15-17 experienced physical maltreatment, while 89.4% of the 12-14 age group and 82.9% of the 15-17 age group experienced psychological maltreatment. The emotional states of the two groups were 52.8% and 59.2%, respectively. There was a significant correlation between the experience of physical maltreatment and emotions among 12-14 (r1 = 0.148 (0.190-0.257)) and 15-17 years old (r1 = 0.047 (0.084-0.156)). There was a significant correlation between the experience of psychological maltreatment and emotions among 12-14 years old '(r2 = 0.191 (0.270-0.350)) and 15 to 17 years old (r2 = 0.097 (0.167-0.252)). In conclusion, physical and psychological maltreatment were correlated with mental health states among adolescent students in West Java, Indonesia.

Pilot Study of Digital Competency Mapping of Indonesian Preservice Teachers: Rasch Model Analysis

Typically, biology students pursue natural sciences. As the world becomes increasingly interconnected and reliant on digital technologies and platforms, it is essential to evaluate their technological preparedness. This study aims to validate a mapping instrument of the digital competence of pre-service Biology teachers in a leading private university in Indonesia. The analysis used the Rasch model to measure the validity and reliability of questionnaire instrument utilizing the Logit Value of Item (LVI). The Digital competency (Digcom) questionnaire contains 36 statements in the Indonesian language (Bahasa). This questionnaire aims to record respondents' answers to digital skills for preservice teachers. The statement is adapted from the digital competence area for EU citizens combined with the Digital Literacy guidelines of the Indonesian Ministry of Communication and Information. The result of the retranslation has been validated to remove the ambiguity of each item of the statement by English language experts. The pilot research was conducted on preservice teachers from different backgrounds (gender, department, and year of study) in a private University (n=261). The findings indicated that all the items were deemed acceptable as they met the criteria for the PTMEA-CORR range, MNSQ outfit, and ZSTD outfit. This study is among the few that have explored the digital competency framework of preservice teacher students in Indonesia, specifically in relation to the difficulty level of each competency area.

Development of Essential Thinking Test Instrument using the Presseisen Taxonomy on Ecosystem Material for Grade X High School Students

The test instrument is a measuring tool in the assessment process used to obtain information in the form of data on student learning outcomes, both cognitive aspects and thinking skills. The use of test instrument to measure students' thinking skills is still rarely used. This study aims to develop an essential thinking test instrument using the Presseisen taxonomy on valid and reliable ecosystem material. This research uses the Research Development (RD) method with the 3D modified development model combined with the test development stages consisting of 6 stages. Data collection in this research was carried with interviews, tests, and test instrument validation sheets. The analysis included content validity analysis, interrater reliability, and Rasch model analysis. The results showed that all items were valid based on content validity. Interrater reliability has a value of 0.783 with a good category. All items are declared fit, but item number 7 contains bias so it needs to be discarded. Cronbach's alpha has a value of 0.68 in the enough category, person reliability has a value of 0.50 in the weak category, and item reliability has a value of 0.99 in the excellent category. The item difficulty level group consists of 2 very difficult questions, 4 difficult questions, 2 easy questions, and 2 very easy questions. Based on the result of the research, it can be concluded that there are 9 items that are declared valid and reliable for use.

Validation of the Indonesian version of the self-compassion scale for youth using Rasch model analysis

Adolescence is a phase marked by stress, emotional instability, and low self-esteem, often leading to self-criticism and social withdrawal. To cope with these challenges, adolescents need self-compassion to maintain self-kindness and achieve better psychological well-being. This study aims to measure adolescent self-compassion using the Self-Compassion Scale for Youth (SCSY), a 17-item scale developed to suit the competencies of adolescents, as the original Self-Compassion Scale (SCS) is less applicable for this age group. A quantitative survey design was employed, with data from 105 high school students (aged 16-17, predominantly Sundanese, residing in rural areas) analyzed using JASP and Winstep for Rasch Model measurement. Results indicated that the SCSY demonstrated acceptable validity and reliability, confirming its suitability for adolescent self-compassion measurement. Findings underscore the importance of self-compassion in fostering emotional stability and resilience, suggesting that school-based guidance and counseling could benefit from incorporating self-compassion training to enhance mental health support for adolescents. The SCSY’s tailored approach facilitates early detection of self-compassion levels, enabling counselors to provide targeted interventions that align with adolescents’ developmental needs.

The Children's Somatic Symptoms Inventory-8: Psychometric Properties of a Brief Measure of Somatic Distress.

Children often present to primary and specialty care clinics with multiple somatic symptoms of nonspecific origin that can be highly distressing and prompt significant health service use. We evaluated the psychometric properties of the eight-item Children's Somatic Symptoms Inventory (CSSI-8) as a brief measure of somatic distress that could be easily integrated into clinical systems. Eight items from the 24-item CSSI were selected based on their representation of multiple bodily systems, association with high base rates, and ability to maximize the separation of the items' Rasch measure scores. The psychometric quality of the eight-item scale was evaluated in 876 pediatric patients with chronic abdominal pain and a nonclinical sample of 954 school children using methods from three psychometric models (the classical test theory, Rasch modeling, and confirmatory factor analysis). The CSSI-8 showed good measurement properties on an extensive array of psychometric criteria, had adequate Rasch person separation reliability for a brief instrument (rsep = 0.74-0.75), and distinguished between clinical and nonclinical youth. Girls in both groups had significantly higher CSSI-8 scores than boys. Norms for the clinical sample are presented. The CSSI-8 is a psychometrically sound measure suitable for use as a brief dimensional assessment of pediatric somatic distress.

Exploring the influence of EFL teachers’ informal digital learning on creative classroom instruction: a cross-sectional study

This study addresses the limited exploration in the literature regarding the influence of EFL teachers’ informal digital learning of English (IDLE) on their creativity in Indonesian EFL classroom instructions. Using a cross-sectional survey design, data were collected from 244 Indonesian EFL teachers, encompassing both pre-service and in-service teachers. Through multiple regression analysis and Rasch modelling analysis, the study evaluated the association between teachers’ IDLE participation, demographics, and their teaching creativity. Findings of the present study revealed that Indonesian EFL pre-service and in-service teachers actively participate in Informal Digital Learning of English (IDLE), which impacts their teaching creativity. Nearly half (49.18%) engage strongly in activities like listening to English songs and watching YouTube videos, while 36.88% moderately engage in online educational content and social media conversations. A smaller group (13.93%) shows lower engagement in IDLE. The regression analysis revealed that while IDLE significantly predicted teaching creativity (p < 0.001), demographic factors did not indicate a substantial influence on creativity in teaching practices. Additionally, a Rasch-Wright map analysis unveiled that over half of the participants displayed strong creativity in teaching, with a notable inclination among younger teachers towards innovative and student-centered pedagogical strategies. The findings of the current study highlight the need for further exploration into the dynamics between informal digital learning, demographics, and creative teaching approaches, emphasizing the potential for fostering innovative pedagogical practices in language education.

Validity and reliability of the Yonsei lifestyle profile-active, balance, connection, diversity.

Lifestyle, defined as a way of living resulting from an individual's deliberate choices, is a crucial factor in improving and maintaining health. Consequently, the measurement and analysis of lifestyle are of significant importance. This study aims to validate the validity and reliability of the Yonsei Lifestyle Profile-Active, Balance, Connection, Diversity (YLP-ABCD) in measuring health-related lifestyles. Data were collected from 300 participants aged 55 and older using the YLP-ABCD. To analyse the validity and reliability of the YLP-ABCD, analysed using frequency, descriptive statistics, confirmatory factor analysis, and Rasch model analysis. The results indicate that the YLP-ABCD, which consists of 35 items, demonstrates unidimensionality and local independence. The Rasch model confirmed the suitability of the 5-point Likert scale for the factors, excluding subfactors Balanced and Unbalanced. The item fit criteria (0.5 < MNSQ < 1.5) were met for all items. The distribution of the respondents' abilities suggests the need for additional items to measure their ability levels. Both the item and respondent separation indices and their reliabilities were satisfactory. This study confirmed the utility of the YLP-ABCD as a valuable tool for measuring and understanding the multifaceted diversity of lifestyles. Therefore, by utilising YLP-ABCD to quantitatively measure health lifestyles, we anticipate contributing to improvements in human health and quality of life.

Evaluation of the psychometric properties of the homophobia scale in religious-based university students in Indonesia

This study aims to evaluate the psychometric properties of the homophobia scale in students attending religion-based universities in Indonesia. This research is important as homosexuality is a controversial issue in the country and is still a topic of debate. The Homophobia Scale is a tool that assesses attitudes towards homosexuality through 17 items measuring positive affirmation, negative cognition, and the perceived threat of homosexual behavior. The scale was adapted for the Indonesian context, which is predominantly religious, based on The Heterosexual Attitudes Towards Homosexuality (HATH) Scale and Items, originally translated by bilingual experts. The translated scale was then reviewed for content by psychologists and communication experts, and field-tested for reliability and validity. Data from 327 students aged 18–35 from both state and private religion-based universities were analyzed using Rasch model analyses, including principal component analysis (PCA), reliability analysis, and differential item functioning (DIF) assessment. The study found that the homophobia scale accounted for 42.4 % of the raw variance, indicating its unidimensionality. The scale demonstrated an acceptable level of personal reliability and excellent reliability for individual items. Results revealed significant demographic effects, with age and study program showing more differential item functioning (DIF). Male students were more tolerant towards homosexuals than females. Additionally, students at state universities tended to be more tolerant but held negative views of homosexuality when associated with AIDS. In conclusion, the homophobia scale assessed in this study exhibits promising construct validity and sufficient psychometric properties. The findings indicate that negative stigma towards homosexuals and homophobia still persist among students at religion-based universities in Indonesia, despite limited interaction with homosexuals.

Adapting the science teaching efficacy beliefs instrument to assess engineering teaching efficacy beliefs of pre-service elementary teachers: Rasch model and confirmatory factor analysis

&lt;b&gt;Background&lt;/b&gt;: The next generation science standards (NGSS) (NGSS Lead States, 2013) place a particular emphasis on the integration of engineering into the science curriculum. Consequently, the NGSS calls on teachers to engage students in engineering practices to facilitate their experience with the engineering design process similar to engineers and develop a more contemporary view of engineering as a discipline. Since engineering education research in K-12 is still in its infancy, there is limited empirical evidence related to how teachers integrate engineering concepts into their classrooms. To assess the quality of teachers’ engineering teaching practices, teaching self-efficacy can be used as an indicator of their instructional practices because teaching efficacy beliefs are often associated with greater use of student-oriented instructional practices, increased teacher effort, and other positive instructional behaviors.&lt;br /&gt; &lt;b&gt;Purpose:&lt;/b&gt; The main aim of this study was to validate an engineering teaching efficacy beliefs instrument (ETEBI) to measure pre-service elementary teachers’ engineering teaching efficacy beliefs.&lt;br /&gt; &lt;b&gt;Design/method&lt;/b&gt;: The science teaching efficacy beliefs instrument version B was modified to develop the ETEBI. The instrument was administered to 561 pre-service elementary teachers. A Rasch model analysis and confirmatory factor analysis (CFA) were conducted on the data obtained from 561 pre-service elementary teachers to provide evidence supporting the validity of the instrument.&lt;br /&gt; &lt;b&gt;Results:&lt;/b&gt; The Rasch model and CFA suggested a two-factor solution: personal engineering teaching efficacy and engineering teaching outcome expectancy. Also, Cronbach’s coefficient alphas for each subscale were measured to assess the internal consistency of the subscales. Based on the analyses, the study provided evidence supporting the reliability and validity of the ETEBI to assess pre-service elementary teachers’ engineering teaching efficacy.&lt;br /&gt; &lt;b&gt;Conclusions:&lt;/b&gt; The ETEBI can be confidently utilized to assess pre-service elementary teachers’ engineering teaching efficacy beliefs. It is effective in gauging the current status of their beliefs and/or determining changes in their beliefs as a result of any teacher training and professional development effort.

Reliability and separation index analysis of mathematics questions integrated with the cultural architecture framework using the Rasch model

This research uses Rasch model analysis to identify the reliability and separation index of an integrated mathematics test instrument with a cultural architecture structure in measuring students' mathematical thinking abilities. The study involved 357 students from six eighth-grade public junior high schools in Bima. The selection of schools was based on average school exam scores and considered the effectiveness of the learning process that used cultural settings to explore mathematical content. Data analysis was conducted using Microsoft Excel to calculate the content validity of Aiken's index with four experts and the jMetrik software to measure reliability and the separation index. The research results indicate that the mathematics test instrument passed validation by mathematics experts and measurements with a valid content validity level. Rasch model calibration shows a very high level of instrument reliability. Separation analysis on the logit scale indicates the instrument's ability to differentiate students with different ability levels with good homogeneity in the distribution of test items and individual abilities. Scale quality statistics show good item response variability, low error rates and a high separation index. This study has limitations because it focuses solely on multiple-choice questions. Similar research must be conducted using other types of questions (such as those used in PISA, namely open- constructed and closed-constructed questions) and integrating other mathematical materials within relevant cultural architectural structures.

Do source use features impact raters’ judgment of argumentation? An experimental study

This experimental study explores how source use features impact raters’ judgment of argumentation in a second language (L2) integrated writing test. One hundred four experienced and novice raters were recruited to complete a rating task that simulated the scoring assignment of a local English Placement Test (EPT). Sixty written responses were adapted from essays written by EPT test-takers. These responses were crafted to reflect different conditions of source use features, namely source use quantity and quality. Rater scores were analyzed using the many-facet Rasch model and mixed two-way analyses of variance (ANOVAs) to examine how they are affected by source use features and rater experience. Results show that source use features impacted the argumentation scores assigned by raters. Paragraphs with more source text ideas that are better incorporated received the highest argumentation scores, and vice versa for those with limited, poorly integrated source information. Rater experience impacted scores but did not influence rater performance meaningfully. The findings of this study connect specific source use features with raters’ evaluation of argumentation, helping to further disentangle the relationships among examinee performance, rater decision, and task features of integrated argumentative writing tests. They also provide meaningful implications for writing assessment research and practices.

Evaluating Students' Academic Resilience in Chemistry Learning: Insights from a Rasch Model Analysis

Teaching chemistry is a complex subject that requires a certain level of knowledge and skills to understand. Students must overcome this challenge because teaching chemistry involves abstract ideas (atoms, molecules, and electrons) and principles, laws, reaction equations, and mathematical operations. Increasing academic resilience is important in enhancing students' understanding and well-being in learning. This research aimed to test the validity and reliability of students' chemistry academic resilience tests using the Rasch model. Data collection was conducted using Google Forms. Data analysis utilized the Rasch model, assisted by the Winstep application, to reveal various aspects of the assessment. Based on the research findings, a Cronbach's alpha coefficient of 0.78 indicates strong internal consistency. In addition, item reliability reached a significant value of 0.99, while person reliability of 0.78 confirmed the consistency of respondents in providing accurate answers in the assessed categories. Furthermore, the OUTFIT and INFIT MNSQ (persons) have average values of +1.02 and +1.01, respectively (the closer to 1, the better). In contrast, the INFIT and OUTFIT ZSTD values are 0.1 and 0.3, respectively (the closer to 0, the better). The most difficult question is coded ra11 with a logit score of 0.61, and the easiest is coded ra4 with a logit score of -0.05. Therefore, the student academic resilience instrument is an effective measuring tool. In the future, chemistry educators and researchers can benefit from the potential impact of this research on Indonesian education.

The conceptualisation implies the statistical model: implications for measuring domains of teaching quality

ABSTRACT Classroom observation rubrics are a widely adopted tool for measuring the quality of teaching and provide stable conceptualisations of teaching quality that facilitate empirical research. Here, we present four statistical approaches for analysing data from classroom observations: Factor analysis, Rasch modelling, latent class or profile analysis, and formative measurement models. Each statistical model conceptualises the latent variable differently, which may or may not align with the observation rubric’s conceptualisation of teaching quality. We discuss the differences across these models, focusing on the alignment between the rubric’s conceptualisation of teaching quality and the model’s modelling of the latent variable. We discuss the need to align model selection with observation rubric so that the measured teaching quality reflects the theoretically conceptualised teaching quality.

Truth on ProQOL and Burnout Assessment—Development and Intercultural Validation of the Brief ProQOL-12 from the Professional Quality of Life, Version 5

Objective The prevalence of burnout is becoming a public health issue across the world. This study developed and validated Brief ProQOL-12, formed from the existing, but yet unvalidated 30-item ProQOL-5. Methods Study 1—Eight intercultural samples of helping professionals from four continents (n = 4,129). Validation included step-wise appraisal and multi-group invariance testing. Study 2—Ethnically and occupationally-balanced sample (n = 453). Rasch modeling, factor analysis, analysis of measure correlates, and scale refinements. Results ProQOL-5 (30-item) did not fit among any continential or national samples, including North America. Study 1—Provisional ProQOL-12 showed good internal structure and measurement invariance. Study 2—Brief ProQOL-12 had an excellent fit in the ethnically and occupationally diverse sample. Included ProQOL 5 items were: 9, 10, 12-14, 18, 19, 21, 24-26, and 30. The reliability and validity of the Brief ProQOL-12 were significantly improved over the 30-item measure. Rasch modeling and factor analysis indicated that the measure was reliable, valid interculturally and occupationally. Conclusions The ProQOL-5 (30-item) reliability and validity concerns were resolved in this 12-item measure which was refined to include a more realistic time context of seven days and time definite Likert ranges. These enhancements increased validity, as evidenced by the improvement in all model fit indicators in the excellent range in Study 2. Transcending culture and ethnicity with proven psychometric robustness, the Brief ProQOL-12 serves as a valuable resource for evaluating burnout risk and well-being across heterogeneous ethnic, cultural, and occupational landscapes.

Development and Validation of the Student Well-Being Scale for Elementary School Using Rasch Model Analysis

Development and Validation of the Student Well-Being Scale for Elementary School Using Rasch Model Analysis

The construct validity of self-regulated learning questionnaire for senior high school students: a Rasch model analysis

&lt;span lang="EN-US"&gt;Self-regulated learning (SLR) is a condition in which students actively participate in the process of acquiring knowledge, and it closely relates to students’ metacognitive, motivational, and behavioral aspects. In order to measure this variable, an instrument was developed by referring to the Zimmerman cycle in the form of a questionnaire. Therefore, this study aims to analyze the construct validity of SLR questionnaires designed for high school students through Rasch model analysis. The method employed was descriptive quantitative research. The analyzed questionnaire consists of 50 positive statements, rated on 4-point Likert scale, and arranged of forethought, performance, and self-reflection phases. Furthermore, the construct validity was conducted on 235 third grade (XII) high school students in Gunungsitoli City (Indonesia), with a gender distribution of 58.29% female and 41.70% male. The results showed that the questionnaire with 4-rating scales satisfied the criteria for validity, gender inclusiveness, and unidimensionality based on Rasch model analysis for 25 statements. The implication of this research shows that the SLR questionnaire developed is valid and can be used in wider field research, especially in mathematics learning.&lt;/span&gt;