Credit Model Research Articles

IRT Observed‐Score Equating for Rater‐Mediated Assessments Using a Hierarchical Rater Model

AbstractWhile significant attention has been given to test equating to ensure score comparability, limited research has explored equating methods for rater‐mediated assessments, where human raters inherently introduce error. If not properly addressed, these errors can undermine score interchangeability and test validity. This study proposes an equating method that accounts for rater errors by utilizing item response theory (IRT) observed‐score equating with a hierarchical rater model (HRM). Its effectiveness is compared to an IRT observed‐score equating method using the generalized partial credit model across 16 rater combinations with varying levels of rater bias and variability. The results indicate that equating performance depends on the interaction between rater bias and variability across forms. Both the proposed and traditional methods demonstrated robustness in terms of bias and RMSE when rater bias and variability were similar between forms, with a few exceptions. However, when rater errors varied significantly across forms, the proposed method consistently produced more stable equating results. Differences in standard error between the methods were minimal under most conditions.

A Novel CREDIT MODEL RISK MEASURE: Do more Data lead to lower model risk?

A Novel CREDIT MODEL RISK MEASURE: Do more Data lead to lower model risk?

Research on Digital Technology to Promote Low-Carbon Transformation of Manufacturing Industries Under the Perspective of Green Credit: An Evolutionary Game Theory Approach

With the increasing global concern for environmental protection and sustainable development, the low-carbon transformation of the manufacturing industries has become a top priority. The rapid development of green digital technology (GDT) provides new opportunities and a strong impetus for the low-carbon transformation of the manufacturing industries. Meanwhile, green credit, as an important financial tool to promote the development of the green economy, plays a key role in guiding resource allocation. In order to respond to the urgent global demand for environmental protection and sustainable development and to accelerate the pace of the low-carbon transformation of manufacturing industries, based on evolutionary game theory, this paper constructs a three-party evolutionary game model of commercial banks (CBs), digital businesses (DBs) and manufacturing industries (MIs); further subdivides the MIs into two categories of non-polluting MIs and polluting Mis; and performs a numerical simulation using Python to analyze the influence of the main parameters on the evolutionary stabilization strategy. The results of the study are as follows: (1) Changes in the interest rate of the green credit have a greater impact on the strategic evolution process of polluting MIs than non-polluting MIs. The green credit model contributes to the introduction of GDT for the low-carbon transformation by non-polluting MIs, although for polluting MIs, the model hinders, to some extent, their introduction of GDT for the low-carbon transformation. (2) Polluting MIs are more sensitive to the investment cost of introducing GDT than non-polluting MIs. When the support benefits of GDT are too low, polluting MIs are more inclined to choose independent innovation to realize the low-carbon transition. (3) Government subsidies to DBs in terms of GDT innovation are crucial to the DBs’ strategy choices. High subsidies can significantly accelerate the cooperation process between DBs and Mis. The findings reveal the challenges and opportunities faced by both non-polluting and polluting manufacturing industries in the process of the low-carbon transformation. In addition, the study provides theoretical references for the behavioral decisions of commercial banks, digital businesses, and manufacturing industries, and proposes corresponding management suggestions to promote the sustainable development of the manufacturing industries.

Investigating item response theory model performance in the context of evaluating clinical outcome assessments in clinical trials.

Item response theory (IRT) models are an increasingly popular method choice for evaluating clinical outcome assessments (COAs) for use in clinical trials. Given common constraints in clinical trial design, such as limits on sample size and assessment lengths, the current study aimed to examine the appropriateness of commonly used polytomous IRT models, specifically the graded response model (GRM) and partial credit model (PCM), in the context of how they are frequently used for psychometric evaluation of COAs in clinical trials. Data were simulated under varying sample sizes, measure lengths, response category numbers, and slope strengths, as well as under conditions that violated some model assumptions, namely, unidimensionality and equality of item slopes. Model fit, detection of item local dependence, and detection of item misfit were all examined to identify conditions where one model may be preferable or results may contain a degree of bias. For unidimensional item sets and equal item slopes, the PCM and GRM performed similarly, and GRM performance remained consistent as slope variability increased. For not-unidimensional item sets, the PCM was somewhat more sensitive to this unidimensionality violation. Looking across conditions, the PCM did not demonstrate a clear advantage over the GRM for small sample sizes or shorter measure lengths. Overall, the GRM and the PCM each demonstrated advantages and disadvantages depending on underlying data conditions and the model outcome investigated. We recommend careful consideration of the known, or expected, data characteristics when choosing a model and interpreting its results.

Psychometric testing of the caring for the caregiver survey for healthcare providers

AimThe purpose of this study was to assess the factorial structure and model fit of the Caring for the Caregiver Scale. BackgroundCaregiving is an all-encompassing role that results in negative health consequences. Healthcare providers require specific knowledge and clinical skills to support caregivers with health promotion. There is no instrument to measure healthcare providers' knowledge, attitudes, and practices on assessing and supporting caregivers. The Healthcare Provider Caring for the Caregiver Scale was developed to identify gaps in healthcare providers' knowledge and clinical practices and help pinpoint areas where interventions may be necessary. MethodsExploratory factor analysis and absolute fit measures were used to evaluate the factorial structure and model fit of the instrument. A multidimensional ordinal Rasch and differential item functioning test was used to fit a three-dimensional partial credit model and ensure the validity of the use of the instrument across subgroups. ResultsA three-factor model was confirmed based on fit statistics and the clustering of the inter-item correlations. The composite measure across 30 items demonstrated strong internal consistency and excellent reliability. Item fit indices estimated from the Rasch model indicated an overall good fit except for two items showing a misfit. ConclusionsThe Caring for the Caregiver Scale demonstrates sufficient factorial structure and model fit making it valuable in evaluating healthcare providers' knowledge, attitudes, and practices with assessing and supporting caregivers. Data collected from this instrument may identify gaps in knowledge and clinical skills and guide intervention development that enhances healthcare providers' ability to deliver quality care to caregivers.

Leveraging Multi-Agent Systems and Decentralised Autonomous Organisations for Tax Credit Tracking: A Case Study of the Superbonus 110% in Italy

This study aims to develop a Secured Fiscal Credits Model to address the challenges of managing Italy’s “Superbonus 110%” tax credit. Using a decentralised governance approach, our research objective is to provide a feasible system to track and control the entire tax credit process, from generation to redemption. The method integrates Artificial Intelligence and blockchain technology within a Decentralised Autonomous Organisation architecture, combined with a Multi-agent System to establish a tokenomics model. The system is structured to prevent accidental errors, such as double spending or overspending, and detect fraudulent behaviours, like false claims of completed work. Our main findings indicate that deploying two Decentralised Autonomous Organisations on the Algorand blockchain significantly enhances trust and security, supporting effective oversight of the Superbonus process and facilitating transparent value exchange among stakeholders. This decentralised governance model introduces substantial automation, reduces biases, and offers a viable solution to strengthen tax credit management. This work proposes an innovative, technology-driven framework that can be generalised to similar fiscal and governance contexts, enhancing transparency and control.

Development of an assessment tool for collaborative problem-solving skills in chemistry

Collaborative problem-solving (CPS) skills are recognised as an essential aspect of 21st century skills and STEM education. In the STEM framework, students’ CPS skills in the learning process need to be appropriately monitored, yet assessment tools for students’ CPS skills are currently not widely developed in chemistry. This study aimed to develop and validate the assessment tool for measuring high school students’ collaborative problem-solving skills in chemistry (CPS-C). Fifty-two students participated in the assessment, which included tasks on Coke Titration, Fruit Battery, and Soap Making. The data were analyzed using Multidimensional Item Response Theory (MIRT) models and the Generalized Partial Credit Model (GPCM). The results indicated that the CPS-C tool showed good internal consistency and item fit. Additionally, ten students were interviewed, and the interview content was analyzed using MAXQDA, emphasizing the importance of providing clear and concise instructions, reducing the number of tasks, and offering meaningful and relevant options. Moreover, this study also validated the reliability of cognitive ability tests, mental load and effort tests, and interest and motivation tests. The results showed satisfactory reliability for all tests except for the mental load and effort tests, which had a lower reliability coefficient of 0.68. Despite some limitations, the CPS-C tool showed great potential for effectively assessing CPS skills. Further research is needed to validate the tool’s effectiveness and reliability across larger, more diverse samples and different cultural contexts.

Psychometric properties of a COVID-19 health literacy scale in a sample of German school principals applying Rasch analysis

BackgroundDuring the COVID-19 pandemic, health literacy was found to be an asset to manage health-related information. The HLS-COVID-Q22 has been developed to measure COVID-19 health literacy. External validation needs to be assessed in different populations to verify the questionnaire’s functioning. The present study aimed to evaluate the psychometric properties of the HLS-COVID-Q22 in a sample of German school principals.MethodsThe sample consisted of 2187 German school principals who completed the HLS-COVID-Q22 online from April to March 2021. The data was analyzed using Rasch analysis, applying the Partial Credit Model for polytomous data. Dimensionality, item fit statistics and rating scale functioning was tested. Values for item difficulty and person ability as well as reliability indices were computed.ResultsUnidimensionality could be confirmed. The rating scale categories worked as intended, participants used every rating step category. Generally, item fit was verified. One item showed potential misfit but could remain in the questionnaire as excluding the item did not reduce reliability. A person separation index of 3.41 and person reliability of 0.92 showed excellent differentiation between COVID-19 health literacy levels. Furthermore, the values for item separation of 20.08 and item reliability of 1.0 indicate good construct validity.ConclusionsThe German version of the HLS-COVID-Q22 appears to be a reliable measurement tool for the target population. Evidence for construct, statistical and fit validity was collected. Future studies need to test additional types of validity like convergent and divergent validity to further evaluate the questionnaire. Moreover, the psychometric properties of the translated versions of the HLS-COVID-Q22 should be compared using Rasch analysis.

Mplus and the R mirt Package: A Comparison of Model Parameter Estimation for Generalized Partial Credit Model (GPCM)

This article aims to carry out an empirical demonstration to calibrate data using the generalized partial credit model (GPCM) and compare the results of GPCM analysis on paid software, namely Mplus, and open-source software, namely R Package Mirt. The data used in this study used secondary data in the form of item scores from the future orientation scale or Skala orientasi masa depan (S-OMD) with a total of 326 participants using a Likert scale with 4 response options. The results of this study show that GPCM is fit for OMD scale data. Comparison of analysis results using Mplus and R Package mirt shows the same output, such as discrimination parameters and item difficulty levels. The resulting factor score correlation also has a perfect correlation or one. In conclusion, open-source software is capable of having the same computing performance as paid software, and even has several additional features that are not found in paid software.

Psychometric validation of the French version of two scales measuring general (HLS19-Q12) and navigational (HLS19-NAV) health literacy using the Rasch model

BackgroundTo examine the psychometrics properties of the French version of two scales measuring general (HLS19-Q12) and navigational (HLS19-NAV) health literacy (HL) using validation methods based on modern psychometric test theories: a Rasch model analysis.MethodsThe data on representative samples of the French adult population came from the Health Literacy Survey (N = 2 003), conducted in France in two waves (2020 and 2021), and from the third wave of SLAVACO study (N = 2 022), conducted in December 2021. A Rasch analysis was performed using a partial credit model with marginal maximum likelihood estimation adapted to polytomous data. Category probability curves were used to examine whether participants found it consistently difficult to distinguish between response options. Differential item functioning (DIF) was also examined for each item in the two scales as a function of age, gender, and economic status. A Person Separation Index (PSI) of 0.7 has conventionally been considered to be the minimum acceptable PSI level. Chi-square fit statistics, expressed as infit and outfit mean square statistics, were calculated for each item in the two scales. An acceptable fit corresponded to average values between 0.7 and 1.3. The dimensionality of the scales was assessed using Principal Components Analysis (PCA) on the residual correlation matrix to identify whether items are locally independent. Additionally, Smith’s method was employed to detect potential multidimensionality.ResultsThe PSI values for both scales were greater than 0.90. No disordered categories were observed. No evidence of significant DIF was found when associated with gender and economic status. However, two items for both scales had a DIF which was dependent on age but did not have a significant impact on scale scores between younger and older people. Minor local dependence was noted on the HLS19-NAV scale. However, a PCA and Smith’s method demonstrated the unidimensionality of the French version of the scales measuring general and navigational HL.ConclusionsOur results, based on a rigorous statistical analysis, verified the psychometric parameters of the French version of the HLS19-Q12 and HLS19-NAV scales. Despite strong correlation between both scales, each measured a different latent trait. Moreover, despite the fact that presence of DIF was weak, attention must nonetheless be paid when comparing scale response scores between young and older respondents.

IMPLEMENTASI METODE NAÏVE BAYES UNTUK PREDIKSI KELAYAKAN BANTUAN MODAL DAN KREDIT USAHA DI DESA SELAT ABIANSEMAL BADUNG

Small, Micro, and Medium Enterprises (MSMEs) are small-scale businesses carried out by individuals or business entities with a certain amount of net worth and sales proceeds. MSMEs in Abiansemal Strait Village, Badung, still use manual methods in analyzing data. Conditions like this are the basis for this study to create a system that can help make it easier for the Abiansemal Badung Strait Village if there is an assistance program from the government, both business capital assistance, and business credit applications to be given to MSME actors. Naïve Bayes' method of classifying MSMEs is built on concepts explored from the interview process, literature studies and system implementation. The design of this system was made using the Python programming language. The CRISP-DM model is a system development method used in the classification of MSMEs. The Feasibility Prediction Model for Capital Assistance and Business Credit using the Naïve Bayes Method can make it easier for users to find out whether the existence of a web-based classification and recommendation system can make it easier for users to carry out the MSME classification process easily and users can see the solutions recommended by referral MSMEs that have better business conditions directly.

Psychometric evaluation of a COVID-19 health literacy scale applying Rasch analysis

Abstract Background Assessing health literacy in times of the COVID-19 pandemic is vital, since health literacy has been highlighted as a personal resource for managing health information. The HLS-Covid-Q22 questionnaire was developed to measure self-reported COVID-19 health literacy in adults. It has been used and validated, mainly using confirmatory factor analysis, in some studies with different target groups. However, several comprehensive validation studies are needed to ensure the feasibility of a novel measurement tool. In the current study, Rasch analysis has been chosen, as this analysis transforms the categorical responses of the items of the HLS-Covid-Q22 into linear measures. Methods The psychometric properties of the HLS-Covid-Q22 were evaluated in a sample of 2187 German adults applying Rasch analysis. The data is from the German COVID-HL school survey that was generated by employing an online survey in school principals during the third infection wave from March to April 2021 in four federal states. Applying the Partial Credit Model for polytomous data, the responses to the HLS-Covid-Q22 items were transformed into linear measures, unidimensionality was tested and several indices to assess the quality of the measurement tool were computed. Results Unidimensionality of the questionnaire could be confirmed. The rating scale steps worked as intended. Participants used every rating category. Item fit could be verified. High reliability indices were found with a person separation index of 3.41, person reliability of .92, item separation of 20.08 as well as item reliability of 1.0. Conclusions The HLS-Covid-Q22 appeared to be a reliable and valid questionnaire which has not been validated by applying Rasch analysis before. Even though Rasch analysis is already a well-established validation method in other disciplines, it should be used to evaluate other health literacy questionnaires more often to gain comprehensive insights into the intended functioning of measurement tools. Key messages • The HLS-Covid-Q22 questionnaire appears to be a valid and reliable measurement tool. • Rasch analysis is an appropriate method to evaluate the psychometric properties of likert-type rating-scale questionnaires.

A convexity-constrained parameterization of the random effects generalized partial credit model.

An alternative closed-form expression for the marginal joint probability distribution of item scores under the random effects generalized partial credit model is presented. The closed-form expression involves a cumulant generating function and is therefore subjected to convexity constraints. As a consequence, complicated moment inequalities are taken into account in maximum likelihood estimation of the parameters of the model, so that the estimation solution is always proper. Another important favorable consequence is that the likelihood function has a single local extreme point, the global maximum. Furthermore, attention is paid to expected a posteriori person parameter estimation, generalizations of the model, and testing the goodness-of-fit of the model. Procedures proposed are demonstrated in an illustrative example.

Item Response Theory Analysis of the Cultural Humility and Enactment Scale – Supervision

Objective This study aimed to validate the Cultural Humility and Enactment Scale - Supervision (CHES-S). Method The sample included a total of 201 post-masters counselors who were currently engaged in clinical supervision across 11 U.S. states, who were recruited from the state licensing board email lists. Analyses included confirmatory factor analysis (CFA) and item response theory (IRT) analysis based on a multidimensional adaptation of the partial credit model. Results We found evidence to support the three-factor structure of the CHES-S. We also found that the CHES-S can be used to distinguish participants scores from low to high CH, and that the thresholds of each item could capture participants responses in the expected order. Conclusion Evidencing reliability and validity with a sample of post-master’s practitioners engaged in supervision for licensure, the CHES-S can be effectively utilized as a tool to monitor supervisory process and outcome in clinical supervision.

Comparison of item characteristic analysis models of reading literacy test with polytomous Item Response Theory

This study aims to compare the analysis model of the characteristics of reading literacy items with the polytomous item response theory, which uses the Graded Response Model (GRM), Partial Credit Model (PCM), Generalized Partial Credit Model (GPCM), and Nominal Reasons Model (NRM). This research is quantitative research in nature, and secondary data were used from about 1000 test takers’ responses to reading literacy items in the 2018 reading literacy study analyzed with the R program. This model comparison was carried out so that the analysis results obtained were more accurate in representing the level of reading literacy skills in Indonesia. The results show that the GPCM model is the fit model with an AIC value of 23753.89 and a BIC value of 24042.45, and the number of suitable testlets is 7 out of a total of 7 testlets. Based on the relationship between information function scores and SEM, reading literacy items provide higher information when participants’ abilities range between -2.3 and +2.

Development of Three-Tier Diagnostic Test to Measure Misconceptions of Grade 5 Students about Fractions

This research developed a measuring instrument in the form of a three-tier diagnostic test to measure students' misconceptions in fraction material in grade V. This research aims to determine (1) the validity, reliability, and item characteristics of the three-tier diagnostic test and (2) the level of misconceptions of grade V elementary school students in Pontianak. This research used quantitative approach and test development model by Mardapi, that has nine steps. The subjects were fifth-grade students in Pontianak City, and the developed test instruments focused on addition-subtraction of fractions, each with two sets of questions A and B. The instrument were analyzed for content and construct validity, reliability, and item characteristics based on IRT using Partial Credit Model (PCM). The results show that the developed test instrument demonstrated strong content and construct validity. Reliability analysis shows high reliability, with Cronbach's Alpha values >0,7. Item fit analysis reveal that 91% and 73% were fit, therefore the unfit items were fixed or eliminated. The analysis of misconceptions shows that 39.52% of students do not understand the concept of fractions’ addition and subtraction, and 29.09% experienced misconceptions. Additionally, 12.61% have false positives, and 8.97% have false negatives. Only 6.30% understood the concept correctly, and 3.52% guessed correctly or understood but had doubts. Generally, the final test instrument demonstrated good validity, reliability and item qualities.

Credit Intelligence and Modelling

Credit Intelligence and Modelling

Patient feasibility as a novel approach for integrating IRT and LCA statistical models into patient-centric qualitative data-a pilot study.

Clinical research increasingly recognizes the role and value of patient-centric data incorporation in trial design, aiming for more relevant, feasible, and engaging studies for participating patients. Despite recognition, research on analytical models regarding qualitative patient data analysis has been insufficient. This pilot study aims to explore and demonstrate the analytical framework of the "patient feasibility" concept-a novel approach for integrating patient-centric data into clinical trial design using psychometric latent class analysis (LCA) and interval response theory (IRT) models. A qualitative survey was designed to capture the diverse experiences and attitudes of patients in an oncological indication. Results were subjected to content analysis and categorization as a preparatory phase of the study. The analytical phase further employed LCA and hybrid IRT models to discern distinct patient subgroups and characteristics related to patient feasibility. LCA identified three latent classes each with distinct characteristics pertaining to a latent trait defined as patient feasibility. Covariate analyses further highlighted subgroup behaviors. In addition, IRT analyses using the two-parameter logistic model, generalized partial credit model, and nominal response model highlighted further distinct characteristics of the studied group. The results provided insights into perceived treatment challenges, logistic challenges, and limiting factors regarding the standard of care therapy and clinical trial attitudes.

Utilizing item response theory for the analysis of self-regulated learning scale in mathematics

In the learning process, students must be able to regulate themselves. This is an effort to improve the quality of student learning, especially their learning achievement. The purpose of this study is to empirically analyze the characteristics of self-regulation skills in learning mathematics. The analysis was carried out by utilizing Item Response Theory (IRT) with a Partial Credit Model (PCM). This study is a descriptive quantitative study with the subjects of the study being 123 students of grade 10 of Senior High Schools in Yogyakarta. Data collection was carried out using a questionnaire that had been developed and its item characteristics were analyzed using IRT. The independent learning scale questionnaire in mathematics consists of 16 statement items and measures 5 aspects. Based on all the questions that are feasible to be analyzed using the PCM model, it is known that there are 5 aspects that fit, namely (1) self-confidence, (2) discipline in learning, (3) active in learning, (4) responsibility, and (5) motivation in learning. This instrument can be used to measure students' self-regulation skills in learning mathematics based on their ability level.

Character Counts: Psychometric-Based Credit Scoring for Underbanked Consumers

Psychometric-based credit scores measure important personality traits that are characteristic of good borrowers’ behaviors. While such data can potentially improve credit models for underbanked consumers, the utility of psychometric data in consumer lending is still largely understudied. The present study contributes to the literature in this respect, as it is one of the first studies to evaluate the efficacy of psychometric-based credit scores for predicting future loan defaults among underbanked consumers. The results from two culturally diverse samples of loan applicants (Sub-Saharan Africa, n = 1113; Western Europe, n = 1033) found that psychometric scores correlated significantly with future loan defaults (Gini = 0.28–0.31) and were incrementally valid above and beyond the banks’ own credit scorecards. These results highlight the theoretical basis for personality in financial behaviors, as well as the practical utility that psychometric scores can have for credit decisioning in general and the facilitation of financial inclusion for underbanked consumer groups in particular.