Rasch Model Research Articles

Understanding science data literacy: a conceptual framework and assessment tool for college students majoring in STEM

BackgroundIn the era defined by the fourth paradigm of science research, the burgeoning volume of science data poses a formidable challenge. The established data-related requisites within science literacy now fall short of addressing the evolving needs of researchers and STEM students. Consequently, the emergence of science data literacy becomes imperative. However, notwithstanding the escalating importance of science data literacy, a comprehensive definition and conceptual framework are still elusive, posing challenges to effective assessment.ResultsThis study analyzes the science literacy frameworks of six international organizations and countries, including the OECD, and reviews 16 data literacy-related literature sources identified using the PRISMA process. We also consider the characteristics of science data. Based on these sources, we clarify the connotation of science data literacy and construct a tailored conceptual framework for college students majoring in STEM. The framework undergoes two rounds of Delphi method refinement to enhance its applicability. Subsequently, we created and piloted a set of questions using physics, astronomy, geography, and other STEM subjects as examples to assess science data literacy. The revised assessment tool was then used in a formal test with 198 university students, employing Rasch modeling to evaluate its effectiveness. The tool’s validity in assessing science data literacy was confirmed.ConclusionsThis study offers a systematic and comprehensive conceptual framework for science data literacy tailored to STEM undergraduates. Endorsed by experts, the framework outlines essential literacies for STEM students in handling science data. The developed assessment tool enables educators to measure students’ science data literacy levels and serves as a scientific guide to enhance their competencies in this area.

A nonparametric Bayesian item response modelling approach for clustering items and individuals simultaneously

Item response theory (IRT) is a popular modelling paradigm for measuring subject latent traits and item properties according to discrete responses in tests or questionnaires. There are very limited discussions on heterogeneity pattern detection for both items and individuals. In this paper, we introduce a nonparametric Bayesian approach for clustering items and individuals simultaneously under the Rasch model. Specifically, our proposed method is based on the mixture of finite mixtures (MFM) model, which obtains the number of clusters and the clustering configurations for both items and individuals simultaneously. The performance of parameter estimation and parameter clustering under the MFM Rasch model is evaluated by simulation studies, and two real examples are applied to further illustrate the MFM Rasch modelling.

To What Extent Collateral in PLS Financing Brings Maṣlaḥah? An Analytical Comparison from Islamic Law Outlook with Maqāṣid al-Sharī’ah Index

This study aims to compare the level of maṣlaḥah regarding the existence of collateral in Profit and Loss Sharing (PLS) financing from Islamic banking customers and employees standpoints. This is crucial as PLS financing becomes the main characteristic product of Islamic banks, but it emerges as an unfavorable product due to the high financial risk. The existence of collateral plays a pivotal role in this situation to realize maṣlaḥah. This is an explanatory study with a quantitative approach and survey method, considering 400 respondents, particularly 200 Islamic banking customers and 200 Islamic banking employees, who were conveniently obtained at certain time intervals for a one-shot. An Independent sample t-test is used to calculate the comparison level of maṣlaḥah among these two groups. At the same time, Rasch Model analysis is also used to measure the data based on the demographic characteristics of respondents. The result discovers that although there is a slightly different level of maṣlaḥah between Islamic banking customers and employees in PLS finance practice regarding collateral, it still significantly brings benefit, as it aligns with the concept of ta’widh to prevent financial risk. This finding contributes to Islamic banks as a guideline to understand how to operate and enhance the market share of this financing product based on the Rasch Model Analysis.

Reliability and validity of the Cybersecurity Awareness INventory (CAIN)

ABSTRACT The Cybersecurity Awareness INventory (CAIN) was introduced by Tempestini and colleagues (2023, “The Cybersecurity Awareness INventory (CAIN): Early Phases of Development of a Tool for Assessing Cybersecurity Knowledge Based on the ISO/IEC 27032.” Journal of Cybersecurity and Privacy 3 (1): 61–75) as a proficiency test designed to estimate individuals’ knowledge of cyber threats. The questionnaire was based on ISO/IEC 27032:2012 (Information Technology – Security Techniques – Guidelines for Cybersecurity) and consisted of 46 items. In this paper, we present the results of a survey study conducted on a sample of 1511 Italian workers belonging to the same large organisation. The administration of the CAIN to an Italian sample forced us to make an adaptation to the Italian language of the questionnaire. Using Rasch analysis, we improved the CAIN by eliminating some items that did not fit the expected response patterns, leading to a newer and more parsimonious 30-item version. the CAIN score was also used as the dependent variable in a series of ANOVA designs using several security behaviours as factors. That allowed us to test its predictive validity, confirming the ability of the instrument to discriminate between different user profiles with respect to reported security behaviours. Overall, the revised test showed better performance in terms of reliability and validity.

Development and psychometric evaluation of the adjuvant endocrine therapy beliefs scale for breast cancer survivors.

Adjuvant endocrine therapy has a vital role in reducing breast cancer mortality. The beliefs in adjuvant endocrine therapy is a very important factor in the medication adherence of breast cancer survivors. Therefore, it is necessary to develop a standardized scale for assessment of adjuvant endocrine therapy. The purpose of this study was to identify the attributes of adjuvant endocrine therapy beliefs, and to evaluate adjuvant endocrine therapy beliefs scale psychometric properties. A hybrid model was applied to identify the concept of adjuvant endocrine therapy beliefs and measurement question were developed by the scale development process. Statistical analysis using validity analysis and Rasch analysis based on item response theory were performed. A total of 228 breast cancer survivors in South Korea participated in the study. The finally developed adjuvant endocrine therapy beliefs scale consisted of 22 items. The items extracted by 4 factors explained 59.72% of the total variance. The model fit showed an acceptable level. The adjuvant endocrine therapy beliefs scale was excellent in convergent and discriminant validity with reliability. This scale is expected to be practical and useful in identifying adjuvant endocrine therapy beliefs and developing intervention strategies to promote adjuvant endocrine therapy adherence. In addition, continuous education and support should be accompanied so that breast cancer survivors can maintain positive beliefs in adjuvant endocrine therapy adherence.

A Rasch-based validation of the University of Tehran English Proficiency Test (UTEPT)

Recently, psychometricians and researchers have voiced their concern over the exploration of language test items in light of Messick’s validation framework. Validity has been central to test development and use; however, it has not received due attention in language tests having grave consequences for test takers. The present study sought to examine the validity of the University of Tehran English language proficiency test (UTEPT) using 6 aspects of Messick’s validation framework (i.e., content, structural, external, generalizability, substantive, and consequential). To examine each validity dimension, the Rasch model was considered. To this end, the data of 394 test takers who had taken the test in 2023 were cleaned and analyzed for the assumptions of the Rasch model and then for the 6 aforementioned validity aspects. The results indicated that the structural (i.e., unidimensionality), generalizability (i.e., measurement invariance), and substantive aspects held. However, the evidence for the content, external, and consequential aspects of validity was less definitive. The implications of the findings for the validity of the test, for the realm of language testing, and for item developers and item writers will be discussed.

Validation of the Korean version of the impact of event scale-revised in disaster victims using the Rasch model

ABSTRACT This study is to verify the validity of the Impact of Event Scale-Revised (IES-R) by reflecting the Post-Traumatic Stress Disorder (PTSD) structure and gender differences of disaster victims with the Rasch model. The analysis is based on 2,253 disaster victims in South Korea. Results reveal that items on sleep problems are inappropriate in accounting for the concept of this scale. The difficulty of the item was highly recognised by disaster victims. It was found that there were items that functioned differently by gender. Further studies need to verify the items considering cultural and disaster-related contexts for adjusting the difficulty of the items.

Rasch validation of the Keratoconus End-Points Assessment Questionnaire in a Spanish population with keratoconus

ObjectiveTo carry out a methodologically complete validation of the Spanish version of the Keratoconus End-Points Assessment Questionnaire (KEPAQ) in a Spanish population with keratoconus. MethodsAnalytical, prospective study, including patients with keratoconus without previous surgical history, in which a measurement of quality of life was performed using the KEPAQ questionnaire, a complete exploration of the anterior pole and a corneal elevation topography with the Galilei G6 topographer. The evaluation of the psychometric characteristics of the scale in the studied population was carried out using Rasch modeling. ResultsA total of 140 patients with keratoconus were included, with a median age of 26.0 years, the majority (57.6%) being men. For the KEPAQ-E subscale, the median score was 69.3, with a reliability of 0.85 and an eigenvalue of the first contrast of 2.34. For the KEPAQ-F, the median score was 56.4, with a reliability of 0.88 and an eigenvalue of 2.00. All infit and outfit parameters were within normal limits for both subscales. A significant evaluation was found between the evaluations of both subscales (rho = 0.696; p < 0.001). The evaluations of the subscales and various clinical and tomographic characteristics showed a significant classification between them (p value between 0.048 y 0.001). ConclusionThe KEPAQ is a psychometrically robust and valid scale to evaluate quality of life in the Spanish population with keratoconus. This questionnaire can be easily used for both clinical and research aims.

Digital literacy scale: Validity and reliability study with the rasch model

AbstractThe study aimed to develop a Likert-type measurement tool (Digital Literacy Scale, DLS) to determine the digital literacy levels of secondary school students. The validity and reliability of the developed measurement tool were verified using the Rasch model. The Rasch Model can estimate missing data and allow for small study groups (Rasch, Rasch, Probabilistic models for some intelligence and attainment tests, Danish Institute for Educational Research, 1960). Additionally, this model can verify the expected pattern of the measurement tool by calibrating it across participants and items. The content validity of DLS was ensured by expert opinion, and the construct validity was ensured by using the Rach model. In the content validity study, a 25-item pool was created for the draft DLS, and 5 items were removed from the draft DLS at this stage. Construct validity studies were carried out with the remaining 20 items using the classical test theory and item response theory (Rasch model). Exploratory Factor Analysis (EFA) and first-level Confirmatory Factor Analysis (CFA) were used within the scope of classical test theory. Then some Rasch assumptions such as dimensionlessness, local independence, monotonicity, and bias were tested for DLS. In the validity and reliability analysis of DLS, no items were eliminated, and the quantitative theoretical results were statistically confirmed. The analysis results showed that the minimum statistical values required for a good measurement tool were met. Accordingly, 20 valid and reliable compatible items that can be used to determine the digital literacy status of secondary school students were produced.

Analisis Butir Soal Ulangan Harian Siswa Mata Pelajaran PPKn Kelas IV Menggunakan Rach Model di Madrasah Ibtidaiyah Negeri

The study aimed to analyze the items on students' daily test questions on PPKN subjects made by the teacher to measure learning outcomes during the previous lesson process. Conducted on fourth-grade students at the 01 West Pasaman State Madrasah Ibtidayah. The method used in this research is descriptive quantitative using Rasch model data analysis with Win Step software. The results of the data analysis carried out were on the reliability data analysis using Winstep Cronbach Alpha 0.49 software. From the existing data, it is clear that the level of reliability of the questions is still relatively weak. The results of the Rasch model of person reliability and item reliability are 0.47 and 0.52. The results of the analysis of the quality of the items in terms of the validity of the items using the Rasch model obtained 6 questions that were said to be valid and 24 questions that were said to be invalid because they did not meet the requirements for the MNSQ outfit, ZSTD outfit, and the Point Measure correlation (PT measure Corr). In using the Rasch model it is felt to be more accurate because it meets these three criteria, so it is considered valid. above the P value or the level of difficulty of the items below 0.25, there are 3 items, meaning the item is very difficult. Questions that are above 0.75 have 5 items meaning the question is too easy. And questions that are scored between 0.25-0.75 have 17 questions. That is, 18 questions have a good level of difficulty. Analysis of distracting items using the Winstep Rasch Model software for 25 daily test questions for students of Civics class IV at Madrasah Ibtidayah Negeri 01 Pasaman Barat shows that there are 23 questions with good functioning distractors. Items where the distractor does not function properly are 5 questions.

Is it True That My Students don't Understand the Static Fluid Concepts? Rasch Modeling Perspective

Students' conceptual understanding can be analysed using the Classical Test Theory (CTT) and Item Response Theory (IRT) approaches. CTT can only explain students' conceptual understanding at the group level, while IRT can at the individual level. Therefore, this research evaluates students' conceptual understanding using the Rasch model. This research used a survey research type and was carried out in class XI. Students' conceptual understanding was collected using 10 multiple-choice questions on static fluid material. The level of concept understanding was analyzed in stages using Rasch modelling. Understanding analysis begins by analyzing the difficulty level of the questions and then continues with descriptive analysis. The Wright map is used to see the hierarchy between students' understanding of concepts and the difficulty level of the questions used. Finally, a person diagnostic map will be used to see students' understanding of concepts in detail. The analysis results show that students' conceptual understanding is generally in the high and medium categories. Person diagnostic maps have identified patterns of correct and incorrect answers from students that do not reflect their true abilities. So, it is necessary to analyze the exact location of the weaknesses and strengths of students' conceptual understanding. This has implications for the suitability of the learning plan that the teacher will use.

Measurement properties of the Thai translation of the Recovery Assessment Scale - Domains and Stages (RAS-DS) and comparison of recovery experiences between Thai and Australian consumers living with serious mental illness.

Personal recovery, a western conceptualisation that focuses on hope and living meaningful lives of choice rather than focusing on symptom reduction, is a more recent concept in many Asian countries including Thailand. One way to promote recovery-oriented service delivery is to use outcome measures that capture self-reported personal recovery. This study aimed to evaluate a Thai translation of a self-report measure of mental health recovery, the Recovery Assessment Scale - Domains and Stages (RAS-DS). The study also explored the cultural similarities and differences between Thai (n = 190) and Australian (n = 301) recovery experiences by comparing Thai and Australian participant responses to RAS-DS items. Data were analysed using Rasch analysis. Analyses revealed that the Thai version of the RAS-DS had adequate measurement properties. Cultural comparisons suggested that most aspects contained within the RAS-DS appear to be applicable across both Thai and Australian contexts. Three findings suggest linguistic or cultural differences in Thai and Australian recovery experiences: (i) a ceiling effect for Thai participants, (ii) some items were "harder" or "easier" for one cultural group to endorse than the other, and (iii) a few items were "misfitting" for Thai participants.

Psychometrics of the modified Family-Centered Care Assessment short version for childhood obesity.

Incorporating principles of family-centered care into pediatric weight management interventions can improve the effectiveness and quality of treatment and reduce attrition rates. To assess the family-centeredness of interventions, reliable, valid, and easy-to-administer scales are needed. The purpose of the study was to develop a shortened version of the modifed Family Centered Care Assessment (mFCCA) and assess its psychometric properties. The mFCCA, a scale to assess the family-centeredness of interventions for childhood obesity, was administered to families following the Connect for Health randomized control trial evaluating the effectiveness of a primary care-based pediatric weight management intervention. We iteratively removed items from the mFCCA and used Rasch modeling to examine the reliability and validity of the shortened scale. We included data from 318 parents and the exploratory factor analysis showed the presence of a single factor. The results of the Rasch modeling demonstrated acceptable internal consistency of the scale (0.7) and strong validity as evidenced by the overall model fit and range of item difficulty. Following the psychometric analyses, we reduced the number of items from 24 to 8 items. The mFCCA short version demonstrates good psychometrics and can be used to evaluate the family-centeredness of childhood obesity interventions with reduced participant burden, thereby improving outcomes for children with obesity. Clinicaltrials.gov NCT02124460 registered on April 24, 2014.

Development and Validation of Instruments for Assessing the Impact of Artificial Intelligence on Students in Higher Education

&lt;p style="text-align:justify"&gt;The role of artificial intelligence (AI) in education remains incompletely understood, demanding further evaluation and the creation of robust assessment tools. Despite previous attempts to measure AI's impact in education, existing studies have limitations. This research aimed to develop and validate an assessment instrument for gauging AI effects in higher education. Employing various analytical methods, including Exploratory Factor Analysis, Confirmatory Factor Analysis, and Rasch Analysis, the initial 70-item instrument covered seven constructs. Administered to 635 students at Nueva Ecija University of Science and Technology – Gabaldon campus, content validity was assessed using the Lawshe method. After eliminating 19 items through EFA and CFA, Rasch analysis confirmed the construct validity and led to the removal of three more items. The final 48-item instrument, categorized into learning experiences, academic performance, career guidance, motivation, self-reliance, social interactions, and AI dependency, emerged as a valid and reliable tool for assessing AI's impact on higher education, especially among college students.&lt;/p&gt;

Opportunities and threats for agrifood firms. The case of wineries applying Rasch analysis

Opportunities and threats for agrifood firms. The case of wineries applying Rasch analysis

The ethos brief index—validation of a brief questionnaire to evaluate wellness based on a holistic perspective in patients with restless legs syndrome

PurposeThe aim of this study was to validate the Ethos Brief Index (EBI) in patients with Restless Legs Syndrome (RLS).MethodsA cross-sectional design, including 788 subjects with RLS (65% women, 70.8 years, SD 11.3) from the Swedish RLS Association, was used. A postal survey was sent out to collect data regarding socio demographics, comorbidities, and RLS-related treatment data. Questionnaires included were EBI, the Restless Legs Syndrome-6 Scale (RLS-6), Restless Legs Syndrome—Quality of Life questionnaire (RLSQoL), the Insomnia Severity Index (ISI), and the Epworth Sleepiness Scale (ESS). The validity and reliability of the EBI were investigated using Rasch and confirmatory factor analysis (CFA) models. Measurement invariance, unidimensionality, and differential item functioning (DIF) across age and gender groups, as well as insomnia, daytime sleepiness, RLS-related QoL and RLS severity were assessed.ResultsThe results supported the unidimensionality of the EBI in the CFA (i.e., explaining 61.5% of the variance) and the Rasch model. The reliability of the EBI was confirmed using composite reliability and Cronbach’s alpha. No DIF was identified for gender, age, insomnia, daytime sleepiness, RLS severity or RLS-related QoL.ConclusionThe EBI showed good validity and reliability and operated equivalently for male and female patients with RLS. Accordingly, healthcare professionals can use the EBI as a psychometrically sound tool to explore and identify patient-centered problems related to the whole life situation.

Development and Validation of the Jawline Subject Satisfaction Scale

Introduction: Research in aesthetic medicine commonly includes evaluations of subject satisfaction with treatment results. However, conventional analytic methods typically generate statistically imprecise ordinal scores. To overcome this limitation, researchers have begun employing the Rasch model, an analytical framework grounded in item response theory. The Rasch model permits scale modifications capable of enhancing measurement accuracy. This study focuses on using the Rasch model to evaluate a scale measuring subject satisfaction following aesthetic treatments to the jawline. Objective: To develop and validate a multiitem, self-administered questionnaire measuring patient satisfaction with aesthetic treatment of the jawline. Methods: A 10-item questionnaire [The Jawline Subject Satisfaction Scale (JS3)] was devised to measure subject satisfaction following aesthetic treatments of the jawline. Each question was responded to using a 5-point Likert scale, with response selections ranging from “very much satisfied” to “very much dissatisfied” or “strongly agree” to “strongly disagree.” The scale's psychometric properties (reliability and separation for items and persons, item and person fit statistics, and unidimensionality and local independence) were validated using a Rasch model based on a dataset collected from a sample of forty subjects. Results: The results of the Rasch analysis revealed high internal consistency of the JS3, with a person reliability estimate of 0.86 and an item reliability estimate of 0.96. The separation estimates for persons and items were 2.50 and 4.72, respectively, demonstrating the scale's ability to differentiate between high and low responders and validating the instrument's construct. All infit and outfit values fell within the established range (0.5-1.5), and the data fit the model of unidimensionality and local independence. Raw score transformations into logits were conducted, which were then converted to Rasch measurements. These measurements are available for use in practice for conducting standard statistical analyses evaluating treatment and/or group effects. Conclusions: The application of the Rasch model produced a valid and reliable scale (ie, JS3) for measuring satisfaction with the appearance of the jawline following aesthetic treatments.

Psychometric properties of the Beliefs About Adults with ID Scale in American physicians: Application of classical test and Rasch measurement theories.

Physicians' erroneous assumptions about individuals with intellectual disability (ID) negatively impact the quality of care provided to this population. This study aimed to investigate the psychometric properties of the Beliefs About Adults with ID (BAID), an instrument we developed for measuring physicians' erroneous assumptions about adults with ID. Two hundred ninety-two American physicians participated. Classical test theory and Rasch measurement theory were used to refine the scale (through item analysis, exploratory factor analysis, infit and outfit mean-squares statistics, and differential item functioning) and investigate its psychometric properties (functioning of the response scale, reliability, and validity). The BAID provided a unidimensional, reliable, valid, and precise measure in assessing high levels of erroneous assumptions. It showed convergent and divergent validity with the different factors of a scale measuring attitudes towards ID. The BAID items were discriminant, non-redundant, unambiguous, and invariant across gender and previous ID training. The BAID response scale was found to be appropriate for measuring physicians' erroneous assumptions about adults with ID. BAID is a brief instrument with good psychometric properties to assess erroneous assumptions about adults with ID in physicians of different genders and who have/have not previously received ID training. Therefore, it might be helpful for research and medical education purposes.

Psychometric validation of the Danish version of the Major Depression Inventory using data from the Lolland-Falster health study (LOFUS)

Purpose The Major Depression Inventory (MDI) is a widely used self-rating depression scale commonly in primary care in Denmark. It has not been subject to robust psychometric validation in a general population setting. The aim of this study was to evaluate the psychometric measurement properties of the MDI when applied in the general population. Methods We evaluated statistical psychometric validity using modern test theory (confirmatory factor analysis, item response theory models and Rasch measurement theory) testing local independence and differential item function across groups defined by gender, age, education, and chronic disease status. Separate analyses across different strata and across different statistical models were employed. Results Regarding structural validity we consistently identified local dependence for the item two pairs (MDI2,MDI3) and (MDI4,MDI5) across strata. This result was confirmed by bifactor CFA models and item screening. We further identified substantial differential item functioning with respect to age group and with respect to chronic disease. We identified quantified the magnitude of this lack of measurement invariance. Conclusion The MDI is psychometrically valid in homogenous sub populations, but the disclosed evidence of local dependence means that published estimates of its reliability cannot be trusted. The lack of measurement invariance means that the instrument cannot be used to compare individuals or groups unless they are similar in terms of age group and chronic disease status.

Knowledge, Ability and Motivation for the Practice of Checking Mud Content in Sand on Learning Outcomes of Concrete Specifications and Characteristics

Sand mining in the area around Sumber Ringin Village, Sri Gading Village, Jati Dukuh Village and around the coast of the Rantas River is a blessing for the people around Mojokerto City and Regency. Mining activities in this area cannot be separated from the attention of many parties and the community, so researchers want to know how much silt content is in the sand and apply it to the basic competencies of concrete specifications and characteristics. This study aims to determine the knowledge, abilities and learning motivation of students in carrying out the practice of silt content in sand on the learning outcomes of basic competency specifications and characteristics of concrete. including research ex post facto. The research design used path analysis. The prerequisite test used is the homogeneity of the F test and the normality test of the Chi Square distribution. Instrument validation was analyzed using the Rasch Model. The research was conducted in class X Modeling Design and Building Information (DPIB) for State Vocational High School (SMK) 1 Mojokerto with a total of 100 students. The results of the research: (1) there is an effect of knowledge on learning outcomes, (2) there is an effect of ability on learning outcomes, (3) there is an effect of knowledge on student learning motivation, (4) there is an influence of ability on student learning motivation, (5) there is an effect of student learning motivation on learning outcomes, (6) there is an influence of knowledge on learning outcomes through learning motivation, (7) there is an effect of ability on learning outcomes through learning motivation.