Rasch Model Analysis Research Articles

Measurement of Psychometric Properties Numerical Aptitude Assessment Scale for Prospective High School Students: A Rasch Model Analysis

Developing structured test instruments is essential in forming a measurable and consistent ability assessment construct. This study examines the psychometric properties of the numerical aptitude test instrument (NAPTIN) utilizing Rasch model analysis to evaluate standardized numerical aptitude assessment test instruments for prospective high school (PHS) students based on validation and construct reliability. Data was collected using an online survey of 228 PHS students. The Rasch model analyses person and item fit, item measure, dimensionality and local independence testing, reliability, and differential item functioning. The results showed that NAPTIN had appropriate and consistent items for measuring the numerical aptitude of PHS students based on gender and school type. This study recommends a systematic measurement tool for researchers. Another contribution is measurement instruments for Indonesian schools and policymakers to choose student program specialties.

Measuring Reliability and Validity of Questionnaire on Teacher Digital Professional Development: A Rasch Model Analysis

Measuring Reliability and Validity of Questionnaire on Teacher Digital Professional Development: A Rasch Model Analysis

Evaluating Psychometric Properties of Raven’s Coloured Progressive Matrices Test in Indonesian Sample using the Rasch Model

Coloured Progressive Matrices (CPM) is a psychological test well known among Indonesian psychologists to measure intelligence. Some researchers who use CPM in their research reveal that CPM has weaknesses in the principle of measurement equivalence. Therefore, the focus of this research is to evaluate the details of the psychometric properties of CPM by using the Rasch model. This research used a secondary data analysis approach, where the primary data sets from a psychological service were collected into a single file for further analysis. Data of 371 boys and 377 girls with an age range of five to seven years old who took an intelligence test to assess their school readiness were collected. The Rasch model analysis showed that CPM showed unidimensionality and local independence, had a fairly good reliability value, and eight items were unsuitable for testing intelligence. Only twenty-eight items of CPM were suitable for measuring children’s intelligence in Indonesia

TOSLS Cognitive Instrument to Measure Students' Scientific Literacy Abilities

Objective: This research is for the development of the TOSLS scientific literacy assessment instrument to determine the scientific literacy abilities of undergraduate science education students. Scientific literacy skills need to be known as a form of evaluation in learning. Scientific literacy skills are one of the abilities students must have after graduating from college and facing the work environment. Method: The quantitative method used in this research uses a Rasch model approach from 29 Bachelor of Science Education program students. Indicators of item suitability with Rasch model analysis measure this quality. The questions consist of 9 multiple choice questions with a correct score of 1 and an incorrect score of 0. Data processing uses MS Excel and SPSS 25 software. Results: The results show that all fit items, according to Rasch analysis, can be maintained without revision and used as a question instrument to measure the level of science literacy ability science for students with moderate scientific literacy abilities. Novelty: The research contribution of this question instrument can be used as reference material for evaluating the lecture process in biophysics courses on fermentation topics based on the level of scientific literacy ability.

Psychometric properties of the mock interview rating scale for autistic transition-age youth.

Employment is a major contributor to quality of life. However, autistic people are often unemployed and underemployed. One potential barrier to employment is the job interview. However, the availability of psychometrically-evaluated assessments of job interviewing skills is limited for autism services providers and researchers. We analyzed the psychometric properties of the Mock Interview Rating Scale that was adapted for research with autistic transition-age youth (A-MIRS; a comprehensive assessment of video-recorded job interview role-play scenarios using anchor-based ratings for 14 scripted job scenarios). Eighty-five transition-age youth with autism completed one of two randomized controlled trials to test the effectiveness of two interventions focused on job interview skills. All participants completed a single job interview role-play at pre-test that was scored by raters using the A-MIRS. We analyzed the structure of the A-MIRS using classical test theory, which involved conducting both exploratory and confirmatory factor analyzes, Rasch model analysis and calibration techniques. We then assessed internal consistency, inter-rater reliability, and test-retest reliability. Pearson correlations were used to assess the A-MIRS' construct, convergent, divergent, criterion, and predictive validities by comparing it to demographic, clinical, cognitive, work history measures, and employment outcomes. Results revealed an 11-item unidimensional construct with strong internal consistency, inter-rater reliability, and test-retest reliability. Construct [pragmatic social skills (r = 0.61, p < 0.001), self-reported interview skills (r = 0.34, p = 0.001)], divergent [e.g., age (r = -0.13, p = 0.26), race (r = 0.02, p = 0.87)], and predictive validities [competitive employment (r = 0.31, p = 0.03)] received initial support via study correlations, while convergent [e.g., intrinsic motivation (r = 0.32, p = 0.007), job interview anxiety (r = -0.19, p = 0.08)] and criterion [e.g., prior employment (r = 0.22, p = 0.046), current employment (r = 0.21, p = 0.054)] validities were limited. The psychometric properties of the 11-item A-MIRS ranged from strong-to-acceptable, indicating it may have utility as a reliable and valid method for assessing the job interview skills of autistic transition-age youth.

Development of a shared item repository for progress testing in veterinary education.

Progress testing in education is an assessment principle for the measurement of students' progress over time, e.g., from start to graduation. Progress testing offers valid longitudinal formative measurement of the growth in the cognitive skills of the individual students within the subjects of the test as well as a tool for educators to monitor potential educational gaps and mismatches within the curriculum in relation to the basic veterinary learning outcomes. Six veterinary educational establishments in Denmark, Finland, Germany (Hannover), the Netherlands, Norway, and Sweden established in cooperation with the European Association of Establishments for Veterinary Education (EAEVE) a common veterinary item repository that can be used for progress testing in European Veterinary Education Establishments (VEEs), linear as well as computer adaptive, covering the EAEVE veterinary subjects and theoretical "Day One Competencies." First, a blueprint was created, suitable item formats were identified, and a quality assurance process for reviewing and approving items was established. The items were trialed to create a database of validated and calibrated items, and the responses were subsequently psychometrically analyzed according to Modern Test Theory. In total, 1,836 items were submitted of which 1,342 were approved by the reviewers for trial testing. 1,119 students from all study years and all partners VEEs participated in one or more of six item trials, and 1,948 responses were collected. Responses were analyzed using Rasch Modeling (analysis of item-fit, differential item function, item-response characteristics). A total of 821 calibrated items of various difficulty levels matching the veterinary students' abilities and covering the veterinary knowledge domains have been banked. The item bank is now ready to be used for formative progress testing in European veterinary education. This paper presents and discusses possible pitfalls, problems, and solutions when establishing an international veterinary progress test.

Teachers As Researchers: Assessing Research Literacy Levels Among Pre-service Teachers in Nigerian Universities

This study assessed the research literacy levels among pre-service teachers. The study adopted a descriptive design and sampled 297 pre-service teachers from two Nigerian South-eastern federal universities. Five research questions guided the study. Data was collected using a semi-structured question guide and a self-developed Research Literacy Test (RLT). Rasch model analysis established the unidimensionality and reliability of the research literacy test (RLT). Moreover, mean scores, standard deviations, t-tests and content analysis were used to answer the study research questions. The study findings showed that pre-service teachers' research literacy level was at the information literacy level. Significant differences were not found between males and females in their research literacy components except at the information level; males were generally more research literate than females. Finally, pre-service teachers indicated that they believe their faculties hold a positive disposition towards their acquisition of research literacy. Keywords: Content Analysis, Gender, Information Literacy, Knowledge of Research Literacy, Pre-service Teachers, Rasch Model Analysis, Research Literacy, Statistical literacy

Science Literacy Assessment Instrument for Additives and Addictive Substances: Development, Validation, and Rasch Model Analysis

Science literacy is one of the areas of basic literacy that PISA focuses on. Science literacy is essence of learning in the 21st century. The purpose of this study is to develop and validate instruments to assess science literacy in the area of additives and addictive substances. The researcher refers to the modified PISA framework and Bloom's Revised Taxonomy. The developed instrument was tested for reliability and validity using the Aiken V method and the Rasch model. The research respondents consisted of 47 students from two junior high schools: a public school and a private school that has been accredited A. The results of the analysis showed that the instrument has validity. The results of the analysis showed that of the 25 items given there were 24 valid items, the resulting Cronbach's alpha value of 0.86 showed very good criteria, it can be concluded that the instrument has high validity and good reliability. This instrument is expected to be used to measure the quality of education and science literacy of students in Indonesia.

Development and Validation of a Test of Science Process Skills for Secondary Students on Cellular Living System Organization Matter: Rasch Model Analysis

This study aims to develop and implement the instrument of science process skills tests on cellular living system organization matter, explicitly designed for secondary students. The indicators developed include formulating hypotheses, communicating, applying concepts, asking questions, observing, planning experiments, using laboratory tools, classifying, and interpreting data. The steps of this research are theoretical construction, determining the purpose of the assessment, constructing indicator items, compiling the items, expert judgment, instrument revision, field trials, and finalization of the instrument. The Science Process skills test consisted of 27 multiple-choice questions. The subject was 30 secondary school students. Data were analyzed using Rasch analysis. Evidence of the validity and reliability of the tests are presented. The conclusion is that the science process skills test instrument is theoretically and empirically valid and reliable

Expert judgement of collaborative cloud classroom quality and its criteria using the many-facets rasch model

Collaborative Cloud Classroom (3CR) is an educational application that provides flexibility in the learning process. This specialized tool is designed to improve analytical and teambuilding skills, as well as enable students to document ideas, which are subsequently developed into bubble diagrams and text outlines. This study aims to assess and analyze the menu qualities possessed by the 3CR application using the Many-Facet Rasch Model (MFRM) analysis using 5 raters and 6 menus. A rating assessment was applied to 5 criteria, namely usability, functionality, visual communication, learning design, and security. In this case, a total of 135 from 150 data points were coded into FACETS software. The results showed that the six menus in the 3CR application were valid and reliable, as well as met the quality standards of learning tools. Security criteria were also the most difficult standards for raters to measure in all the application menus. The study also found that Rater A was more severe, and Rater E was more lenient in rating the 3CR menu. Rater B has overfitting and detected bias between rater gender and 3CR's menu (Discussion Forum and Learning Reflection menu) based on MFRM analysis.

Pre-Service Teachers’ Instructional Innovation Capabilities: A Many-Faceted Rasch Model Analysis

Innovation capability has become a necessary requirement for qualified teachers in the context of informatization. However, the validity and objectivity of existing assessments are unclear. Therefore, this study selected nine researchers to evaluate the instructional innovation capabilities of 60 pre-service teachers from a Chinese normal university. The Many-Faceted Rasch Model (MFRM) was used to measure and analyze the rater severity, difficulty of evaluation criteria, and instructional innovation capability of pre-service teachers. Combined with qualitative analysis, the results showed that most participants had moderate instructional innovation capability, and only certain individuals demonstrated a high level. In addition, innovation related to teaching content and learning activities was easier than innovation related to instructional and evaluation strategies. These results suggest directions for fostering pre-service teachers’ instructional innovation capabilities.

Panelist Analysis of Higher Education Governance Instruments Based on Good University Governance Principles Using Many Facet Rasch Measurements

This research aims to develop instruments related to measuring higher education governance so that it can fulfill the principles of good governance. This research used 3 panelists to measure the content validity of the instrument developed. The criteria for selecting experts/validators are adjusted to the expertise and needs of the instrument development being carried out. The study on testing the content validity of panelists uses Rasch Model analysis with the Facets application which aims to test subjectively rated statement items by looking at several aspects including unidimensional assumptions, testing model fit (item fit), and rater/expert consistency. The results of the content validity analysis carried out show that the unidimensional criteria (the raw variance explained by measure value is more than 20%), the item fit value (the outfit value is in the range of 0.5-1.5), and the rater consistency assessment show that all raters/experts meet the fitness criteria (outfit and infit value categories are in the range 0.5-1.5). The results of the content validity analysis carried out show that the quality of the instrument developed is suitable for use to measure higher education governance.

Science Learning in Answering Digital Competency Needs of Preservice Math and Science Teachers in the STEM Context

These study purposes are to map the digital competency of pre-service math and science teachers and investigate how science learning can answer the challenge of digital competencies (DigComp) needs. This research used Rasch Model analysis to make knowledgeability mapping of the subject and analyzed using Wright map output by Winsteps 5.3.4. Data was gathered from a survey of 328 pre-service teachers of science majors, e.g., Biology, Chemistry, Geography, or Math, using the DigComp Framework-Based Questionnaire (DFBQ). The responses were based on respondents’ diverse demographic profiles (gender, region, living area, and field of study). The findings identify several differences in teacher training students’ knowledgeability of digital competencies that the Wright map in the Rasch model can map. Knowledgeability mapping is essential to determine which part of DigComp still needs to be strengthened with science education in the context of Science, Technology, Engineering, and Mathematics (STEM) implementation. The study yields two main conclusions: 1) The mapping study of pre-service math and science teachers’ knowledgeability in the DigComp framework shows Logit Value Person (0.31LVP1.11) and Logit Value Item (LVI 0.66 logits); (0 ≥ LVI ≥ 0.66) that are reflecting a middle to lower level of competency, and 2) Science learning has high potential to address this challenge through its learning strategy implementations. The findings can be recommendations for future research of knowledgeability mapping and policy development and discuss implications for digital competency framework practices.

Young Muslims' Consumer Ethnocentrism and World-Mindedness

Consumers&amp;#39; knowledge of the country of origin of a product is widely considered to be an important influence on their behavior in selecting products. However, sometimes the perspective of considering product selection is also influenced by consumer ethnocentrism. This study aims to find the effect of world-mindedness on consumer ethnocentrism. The survey method was used in this study by using a questionnaire to 268 Muslim youth respondents. Data were analyzed using the Rasch Model and Regression Analysis. The study shows that around 0.74% of the younger generation of Muslims think very broadly and 1.12% think very differently. There are 3.73% of respondents tend to have the most extreme ethnocentrism. This study also confirms that world-mindedness has a negative and significant effect on consumer ethnocentrism (p &amp;lt;0.05).

Assessing Students Understanding of Chemical Bonds Material by Rasch Modeling

This study aimed to assess high school student’s understanding of Banjarmasin by Rasch modeling, preci-sely the cognitive aspect. The research method was descriptive with a quantitative approach to assess the pattern of reactions and symptoms of Rasch data. According to the research findings, the person reliability (students) was +0.66 based on the Rasch modeling analysis, with the criterion satisfactory, indicating that the students knowledge was adequate. At the same time, Cronbachs’ alpha score had a value of +0.71 and met the criteria of excellent. In addition, the students with the highest level of understanding were coded 127P12B with a logit value (person measure) of 2.52 and average students with logit value of -0.77 or 0. In contrast, the students with low abilities were coded 030P10B and 059L11B (same logit value, -3.27 or 0). Other data were INFIT MNSQ and OUTFIT MNSQ (person) with average values of +0.99 and +1.14 (closer to 1, the better), while the INFIT and OUTFIT ZSTD values were -0.1 and 0.0 respectively (closer to 0, better). The most difficult question was Q16, which had a logit score of 1.96; students’ logit values carried this question. Students were regarded to have appropriate knowledge even though their ability excee-ded the problem ability. In conclusion, the Rasch model-based pre-learning evaluation was found to be useful in measuring students’ cognitive grasp of chemical bonding material. This study could serve as the primary reference for teachers in assessing students’ level of knowledge before they begin learning. In addi-tion to interpreting student knowledge through various Rasch data presentations, a study of the structure of questions with varying difficulty levels could be used to assess the full group of students’ understanding of chemistry.

Validity and Reliability of The Teacher Competency Instrument (TECOMS) for Small Schools in Malaysia : Rasch Model Analysis

Validity and Reliability of The Teacher Competency Instrument (TECOMS) for Small Schools in Malaysia : Rasch Model Analysis

Rasch 모형을 이용한 뷰티전문가의 자기관리 측정도구 타당화

The purpose of this study was to validate the self-management measurement tool's validity for beauty professionals using the Rasch model. Data was collected from 679 participants who were beauty professionals working in the metropolitan area. Kim Hyun-jeong's (2021) self-management scale was used as the measurement tool. Data analysis methods included testing for normality, exploratory factor analysis, and confirmatory factor analysis. In the Rasch model analysis, the validation of the data included testing for unidimensionality, item fit and difficulty, and the validity of response category thresholds.&#x0D; The results obtained from the research are as follows. First, the normality testing of the self-management measurement tool's sub-factors for beauty professionals confirmed that the data showed no skewness and was within the normal distribution range. Second, both the exploratory factor analysis and confirmatory factor analysis of the self-management measurement tool for beauty professionals confirmed that all items were appropriate, thus validating its construct validity. Third, the analysis of item fit using the Rasch model confirmed that the self-management measurement tool for beauty professionals established validity in terms of unidimensionality, item fit and difficulty, and the appropriateness of item category thresholds.&#x0D; In summary, based on the comprehensive findings of this study, Kim Hyun-jeong's (2021) self-management measurement tool for beauty professionals was found to be highly valid.

Physical Education Teacher Perspective On Kurikulum Merdeka and Sport Education Model: A Rasch Model Analysis

Changes and implementations of the new curriculum require a careful preparation and information about the Independent Curriculum, known as Kurikulum Merdeka. The learning model used needs to be explored as an illustration of teacher understanding. This study aimed to examine teacher perspectives regarding their understanding of the Kurikulum Merdeka and the Sport Education Learning Model. The cross-sectional survey method was used to collect data. Data were collected using Google Form filled by 74 teachers. The research instrument was in a Likert scale form and consisted of 30 questions focused on exploring teacher understanding of the Kurikulum Merdeka, the Sport Education (SEM) Learning Model, the SEM Phases/Syntax, and the roles existing in the implementation of the SEM Model and the importance of understanding curriculum and learning models in school. Then, 15 open-ended questions related to models often used by teachers, training expectations to follow, and obstacles experienced during learning at school were given. The research data were processed using Winsteps 5.2.3 software. The Rasch Model analysis was used to analyze the instruments made and the results of the teacher understanding perspective. The results showed that the instrument was valid for use. Related to the results of the teacher perspective, 48.6% of physical education teachers had an understanding and belief in the importance of the Kurikulum Merdeka and the SEM model in elementary schools at a high level. Meanwhile, 12. 2% were at the moderate level and 39.2% were at the low level. These findings can be used as a reflection related to the teacher understanding of the Kurikulum Merdeka and the Sport Education Learning Model and as a reference in arranging training needed for Physical Education teachers in the field.

Self-Efficacy in Self-Care and Its Related Factors Among Patients with Coronary Heart Disease in Indonesia: A Rasch Analysis.

Low self-efficacy (SE) can impact decreasing health status, poor self-care, and quality of life among patients with Coronary Heart Disease (CHD). Many factors can affect SE. However, studies on SE in CHD patients with Rasch Model analysis have not been carried out widely. This study aims to identify the SE in self-care and its related factors that correlate SE among CHD patients. Cross-sectional study was conducted on 104 adult patients (≥18 years) diagnosed with CHD. Selection of the sample using convenience sampling technique with several predetermined criteria. SE was measured using the SEQ-CHDM questionnaire, with high validity and reliability results. Data were analyzed using the Rasch model and chi-square test. The results showed that most respondents had a moderate SE (51.5%). Care units (p=0.003) and duration of illness (p=0.049) were significantly correlated to SE among patients with CHD. "Maintaining an ideal body weight" is the most challenging thing. On the other hand, stop smoking is the most confident thing to be performed by the respondents. We conclude that CHD patients in the acute care unit and patients with a duration of illness >6 months have a lower tendency for SE. Health interventions such as raising awareness about the disease, modifying health behavior, and immediately screening can improve patients' SE. Besides that, proper diagnosis and ongoing treatment are crucial to improving SE and CHD care outcomes.

Rasch Model Analysis: Development of Literacy Numeracy Assessment Instrument of Electromagnetic Induction

This research aimed to describe the validity of a numeracy literacy assessment instrument of electromagnetic induction based on minimum capability assessment (MCA) which was develop using Rasch model analysis assisted by Winstep software. The research employed the research and development design with a modified Mardapi model. The instrument was constructed by developing fifteen questions. The analysis of the instrument was utilized to measure the construct and item validity, reliability, item difficulty level, and item discriminatory. This research involved 32 participants in the small-scale test and 206 participants in the large-scale trial. The test result showed that the fifteen questions were suitable to use because it has values that meet the criteria for all measurement variables both on small scale and large-scale tests and could be assess students' numeracy literacy skills.