Three-parameter Logistic Model Research Articles

Detecting latent subpopulations in international large-scale assessments by fitting MixIRT models using NUTS

The focus of this study is to use the mixture item response theory (MixIRT) model while implementing the no-U-turn sampler as a technique for investigating the presence of latent classes (i.e., subpopulations) among eighth-grade students who were administered TIMSS 2019 mathematics subtest in paper format from the gulf cooperation council (GCC) countries. One-, two-, and constrained three-parameter logistic MixIRT models with one to four classes were used to fit to the data, where the model data fit was assessed using Bayesian fit indices. The results indicate that multiple latent classes or subpopulations can better reflect the mathematical proficiency of eighth graders from the four GCC countries, and specifically the two-class constrained three-parameter MixIRT model provides a relatively better fit to the data. The results also indicate that when a mixture of several latent classes present, the conventional unidimensional IRT model is limited in providing information for multiple latent classes and shall be avoided. In addition to adding to the existing literature on MixIRT models for international large-scale assessments such as TIMSS on its heterogenous subpopulations from a fully Bayesian approach, this study sheds light on the limitation of conventional unidimensional IRT models and subsequently directs attention to the use of the more complex MixIRT model for such assessments.

Medication Literacy Test for Older Adults: psychometric analysis and standardization of the new instrument.

Low medication literacy is prevalent among older adults and is associated with adverse drug events. The Medication Literacy Test for Older Adults (TELUMI) was developed and content validated in a previously published study. To evaluate the psychometric properties and provide norms for TELUMI scores. This was a cross-sectional methodological study with older adults selected from the community and from two outpatient services. Descriptive item-analysis, exploratory factor analysis (EFA), item response theory (IRT), reliability, and validity analysis with schooling and health literacy were performed to test the psychometric properties of the TELUMI. The classification of the TELUMI scores was performed using percentile norms. A total of 344 participants, with a mean age of 68.7years (standard deviation = 6.7), were included; most were female (66.6%), black/brown (61.8%), had low schooling level (60.2%) and low income (55.2%). The EFA pointed to the one-dimensional structure of TELUMI. A three-parameter logistic model was adopted for IRT. All items had an adequate difficulty index. One item had discrimination < 0.65, and three items had an unacceptable guessing index (< 0.35) and were excluded. The 29-item version of TELUMI had excellent internal consistency (KR20 = 0.89). There was a positive and strong association between TELUMI scores and health literacy and education level. The scores were classified as inadequate medication literacy (≤ 10.0 points), medium medication literacy (11-20 points), and adequate medication literacy (≥ 21 points). The results suggest that the 29-item version of TELUMI is psychometrically adequate for measuring medication literacy in older adults.

Analysis of differential item functioning in agricultural science examination across southwestern nigeria's senior schools

Differential Item Functioning poses a threat to test fairness and validity in educational assessments. To this end, this study probes the incidence of DIF in the Senior School Certificate Agricultural Science Examination (SSCASE) across school locations in Southwestern Nigeria. It determined the magnitude of DIF using item parameters in SSCASE. The study applies one, two and three-parameter logistic models to analyze the dichotomous data responses of 818 randomly selected students from urban and rural locations. The ex-post facto design adopted the 2015 National Examination Council SSCE in Agricultural Science as its instrument (α =0.887) in this study. The study population comprised SSCE candidates from Osun, Ondo, and Oyo. The results showed the magnitude of DIF in SSCASE was large and moderate (1PLM=0.52 significant DIF, 2PLM=0.45 moderate DIF, 3PLM=0.47 moderate DIF) across school locations. The study concluded that 1, 2 and 3 PLMs can produce fair items in SSCASE.

Personality Measurement of Students Using Item Response Theory Models: Stability Responses from Nigerian Institutions

Item Response Theory (IRT) is utilised to detect bias in assessment tools and address issues such as faked or manipulated responses, enhancing the reliability and stability of conclusions in personality assessment. This article examines the item parameter estimates of a scale and the effectiveness of one-, two-, and three-parameter logistic models in analysing response stability in personality measurement from repeated administration. Three hundred undergraduate students at three tertiary institutions in Nigeria were sampled using a multi-stage sampling procedure. Data was collected using an adapted version of the Big Five Inventory (BFI) with a reliability coefficient of 0.85. The results showed that the item parameter estimates (mean threshold) are within the recommended benchmarks. A comparison of the three IRT models based on the Likelihood ratio (InL), Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC) values revealed that the two-parameter logistic model best fit the personality data among undergraduates from repeated administration. It is recommended that, rather than relying solely on a statistical decision-making process, IRT fit and model comparison should be applied to gain insight into the functioning of items and tests.

Modelling pH Dynamics, SCOBY Biomass Formation, and Acetic Acid Production of Kombucha Fermentation Using Black, Green, and Oolong Teas

Kombucha is a traditional, fermented beverage made with an essential biomaterial known as SCOBY (symbiotic culture of bacteria and yeast). Three different tea types, namely black, green, and oolong, were compared in kombucha fermentation in terms of pH dynamics, the formation of SCOBY biomass, and the production of acetic acid. The rational, exponential, and polynomial models described pH dynamics with good fit, R2 &gt; 0.98. The formation of SCOBY biomass and the production of acetic acid were modelled using sigmoidal functions, with three-parameter logistic and Gompertz models and four-parameter Boltzmann and Richards models. The F-test indicated that the three-parameter models were statistically adequate; thus, the Gompertz model was modified to present the biological meaning of the parameters. The SCOBY biomass formation rates ranged from 7.323 to 9.980 g/L-day, and the acetic acid production rates ranged from 0.047 to 0.049% acid (wt/vol)/day, with the highest values from the non-conventional substrate, oolong tea. The correlations between pH and SCOBY biomass or acetic acid using polynomial models enable the prediction of product formation in kombucha processing.

Small area estimation using multiple imputation in three-parameter logistic models

Small area estimation using multiple imputation in three-parameter logistic models

Interpretation of course conceptual structure and student self-efficacy: an integrated strategy of knowledge graphs with item response modeling

BackgroundThere is a scarcity of studies that quantitatively assess the difficulty and importance of knowledge points (KPs) depending on students’ self-efficacy for learning (SEL). This study aims to validate the practical application of psychological measurement tools in physical therapy education by analyzing student SEL and course conceptual structure.MethodsFrom the “Therapeutic Exercise” course curriculum, we extracted 100 KPs and administered a difficulty rating questionnaire to 218 students post-final exam. The pipeline of the non-parametric Item Response Theory (IRT) and parametric IRT modeling was employed to estimate student SEL and describe the hierarchy of KPs in terms of item difficulty. Additionally, Gaussian Graphical Models with Non-Convex Penalties were deployed to create a Knowledge Graph (KG) and identify the main components. A visual analytics approach was then proposed to understand the correlation and difficulty level of KPs.ResultsWe identified 50 KPs to create the Mokken scale, which exhibited high reliability (Cronbach’s alpha = 0.9675) with no gender bias at the overall or at each item level (p > 0.05). The three-parameter logistic model (3PLM) demonstrated good fitness with questionnaire data, whose Root Mean Square Error Approximation was < 0.05. Also, item-model fitness unveiled good fitness, as indicated by each item with non-significant p-values for chi-square tests. The Wright map revealed item difficulty relative to SEL levels. SEL estimated by the 3PLM correlated significantly with the high-ability range of average Grade-Point Average (p < 0.05). The KG backbone structure consisted of 58 KPs, with 29 KPs overlapping with the Mokken scale. Visual analysis of the KG backbone structure revealed that the difficulty level of KPs in the IRT could not replace their position parameters in the KG.ConclusionThe IRT and KG methods utilized in this study offer distinct perspectives for visualizing hierarchical relationships and correlations among the KPs. Based on real-world teaching empirical data, this study helps to provide a research foundation for updating course contents and customizing learning objectives.Trial registrationNot applicable.

Most quantifiers have many meanings.

In this paper, we investigate, by means of a computational model, how individuals map quantifiers onto numbers and how they order quantifiers on a mental line. We selected five English quantifiers (few, fewer than half, many, more than half, and most) which differ in truth conditions and vagueness. We collected binary truth value judgment data in an online quantifier verification experiment. Using a Bayesian three-parameter logistic regression model, we separated three sources of individual differences: truth condition, vagueness, and response error. Clustering on one of the model's parameter that corresponds to truth conditions revealed four subgroups of participants with different quantifier-to-number mappings and different ranges of the mental line of quantifiers. Our findings suggest multiple sources of individual differences in semantic representations of quantifiers and support a conceptual distinction between different types of imprecision in quantifier meanings. We discuss the consequence of our findings for the main theoretical approaches to quantifiers: the bivalent truth-conditional approach and the fuzzy logic approach. We argue that the former approach neither can explain inter-individual differences nor intra-individual differences in truth conditions of vague quantifiers. The latter approach requires further specification to fully account for individual differences demonstrated in this study.

Multidimensional item Response Theory Calibration of Dichotomous Response Structure Using R Language for Statistical Computing

The study focuses on assessing the proficiency levels of higher education students, specifically the physics achievement test (PHY 101) at the National Open University of Nigeria (NOUN). This test, like others, evaluates various aspects of knowledge and skills simultaneously. However, relying on traditional models for such tests can result in inaccurate interpretations of students’ abilities. The research highlights the importance of exploring the multidimensional nature of the PHY 101 test to improve its accuracy in measuring student proficiency and enhance education and assessment quality at NOUN. Using an ex-post facto research design, the study analyzed 978 responses from NOUN’s Directorate of Examination and Assessment. Through confirmatory and exploratory DETECT techniques, the study found strong evidence supporting the test’s multidimensionality. Three distinct dimensions emerged: cognitive processing, reading ability, and problem-solving skills. A parsimonious multidimensional three-parameter logistic model was used to calibrate the test items, providing valuable insights into item difficulty, discrimination, and resistance to chance influences. While the study primarily focuses on the psychometric aspects of the PHY 101 test, it is important to consider its broader impact on the educational community. The research contributes to educational assessment by emphasizing the significance of recognizing and addressing the multidimensional nature of higher education tests. This approach can result in more accurate assessments of students’ abilities, ultimately improving education quality and fairness. The findings confirm the multidimensional nature of the PHY 101 test and identify three distinct dimensions, aligning with the study’s objective. These insights are relevant to educators and test developers, highlighting the need for a multidimensional approach to effectively assess and enhance student proficiency. For researchers interested in similar studies, it is recommended to explore the broader influence of multidimensional models in educational assessment. Investigating their impact on teaching methods, curriculum development, and student learning experiences can provide valuable insights. Longitudinal studies assessing the long-term effects of multidimensional assessment on student outcomes and success are also recommended.

Simplified functioning assessment for low back pain: ICF-based item response theory modelling.

Despite the growing interest of the item response theory (IRT) in assessment of person abilities and functioning difficulties in screening tools, there is scarcity of research using IRT on ICF-based tools for persons with low back pain (LBP). To generate and validate a parsimonious core set of ICF (PCSI) for LBP based on the IRT modelling. A cross-sectional study. Nine hospitals in Jiangsu Province, China. We recruited patients with LBP. All participants completed the 78 items of the comprehensive ICF Core Set for LBP. The five-point scoring system was converted to dichotomous data with 1 as functioning/independent and 0 as impairment/dependent. Psychometric properties of the data were examined using Mokken Scale analysis and parametric item response modelling. This study recruited 306 participants (185 females and 121 males) with LBP. The overall median age of the study participants was 50.28 (95% CI 23.34; 82.05) years. We constructed a three-parameter logistic model with 28 ICF categories (8 of body function, 18 of activities and participation, and two of body structures). The internal consistency was good with Cronbach's alpha = 0.927 and latent class reliability coefficient (LCRC) = 0.955. The model was validated by significant correlations (P<0.001) of its estimated person abilities with the Oswestry Disability Index (ODI, r=-0.41), the Roland-Morris Disability Questionnaire (RMDQ, r=-0.57), the Physical Component Summary (PCS, r=0.63), and the Mental Component Summary (MCS, r = 0.46) of 12-Item Short Form Survey (SF-12). The person abilities and item difficulties were integrated into a Wright map that offered a background for making individualized clinical decisions. The PCSI of LBP with 28 categories has good construct validity and internal consistency, and is a convenient instrument for assessing functioning among persons with LBP. The IRT model provided theoretical and algorithmic support for deriving a simplified model for functioning assessment hence serving a basis for formulating rehabilitation plans in clinical practice and research. A parsimonious core set of ICF (PCSI) for LBP based on the IRT modelling provides a background for making individualized clinical decisions based on item difficulties.

Intra and Inter-Local Item Dependence of Mathematics Items

Local Item Dependence (LID) is a desecration of Local Item Independence (LII) which can lead to overestimating or underestimating a candidate’s ability in mathematics items and create validity problems. The study investigated the intra and inter-LID of mathematics items. The study made use of ex-post facto research. The population encompassed all 26,086 Senior Secondary School Students three (SSS 3) in Osun State, Nigeria, and 14,936 SSS 3 students were randomly picked as the sample. National Examinations Council (NECO) mathematics items from June/July 2017 served as the research instrument. The calibrated data were subjected to Yen Q3 statistic. Using a multidimensional three-parameter logistic model, the Yen Q3 statistic is the residual correlation of the items calibrated by Jmetrik software. Results revealed that intra-LID was spotted between Item 8 and Item 7, Item 47 and Item 18, Item 28 and Item 21, Item 47 and Item 29, Item 39 and Item 38, Item 60 and Item 47, Item 52 and Item 51, Item 59 and Item 58 with a Q3 index greater than 0.2 rule of thumb. Similarly, inter-LID revealed bundles 3, 6, 7, 13, and 15 with a Yen Q3 index greater than 0.2. The study concluded that intra and inter-LID was discovered in the NECO mathematics item. It was recommended that NECO should consider intra and inter-LID to ensure more credible questions.

A Comparison of Mantel-Haenszel and the Area Difference Between Item Characteristic Curves Index Techniques for Detecting Differential Item Functioning

The study is aimed at comparing between Mantel-Haenszel-MH chi-square and the area difference between three item characteristic curves Index -3ICCs by using Raju’s index to detect the differential item functioning DIF in post-graduate general aptitude test according to gender variable. The study sample of (2280) students was distributed equally between male and female groups. The MH techniques revealed a uniform DIF in (43) items, equal to (45%) of the items in post-graduate general aptitude test. 3ICCs technique, revealed a uniform DIF based on gender in two items, equals to (2%) of item's test. MH technique revealed more efficiency in detecting differential item functioning in post-graduate general aptitude test compared to area difference between 3ICCs. The agreement rate was low between these two techniques. Based on these results, the researchers recommended to use more than one technique when detecting the differential item functioning. Keywords: Differential Item Functioning, Mantel-Haenszel, The three-parameter logistic model, post-graduate general aptitude test.

Exploration and analysis of a generalized one-parameter item response model with flexible link functions.

This paper primarily analyzes the one-parameter generalized logistic (1PGlogit) model, which is a generalized model containing other one-parameter item response theory (IRT) models. The essence of the 1PGlogit model is the introduction of a generalized link function that includes the probit, logit, and complementary log-log functions. By transforming different parameters, the 1PGlogit model can flexibly adjust the speed at which the item characteristic curve (ICC) approaches the upper and lower asymptote, breaking the previous constraints in one-parameter IRT models where the ICC curves were either all symmetric or all asymmetric. This allows for a more flexible way to fit data and achieve better fitting performance. We present three simulation studies, specifically designed to validate the accuracy of parameter estimation for a variety of one-parameter IRT models using the Stan program, illustrate the advantages of the 1PGlogit model over other one-parameter IRT models from a model fitting perspective, and demonstrate the effective fit of the 1PGlogit model with the three-parameter logistic (3PL) and four-parameter logistic (4PL) models. Finally, we demonstrate the good fitting performance of the 1PGlogit model through an analysis of real data.

Comparison of the response time-based effort-moderated IRT model and three-parameter logistic model according to computerized adaptive test performances: a simulation study

Depending on the developments in technology and information, paper-pencil tests leave their place for computerized adaptive tests (CATs). CAT is widely used in the field of health, mainly in psychiatry. Many item response theory models have been proposed in the literature regarding the use of response time focusing on item difficulty and personal characteristics by ignoring the multidimensional interactions, therefore these results may cause bias in estimates of individual ability levels. The present simulation study was conducted to compare the performance of CAT applications of the effort-moderated item response theory (EM-IRT) model, which is based on response time, and the three-parameter logistic (3PL) model. While simulating CAT with the EM-IRT model and the 3PL model, the hybrid method was used for ability estimation and maximum Fisher information (MFI) was used for item selection. The CAT process proceeded until the standard error of the estimation was <0.3 and <0.5, or all items in the item bank were used. The number of individuals was specified as 1000, while the number of items was changed to 50, 100, and 250. All six scenarios were repeated 1000 times. With the increase in the number of items and the decrease in the standard error as a stopping criterion, consistent results were obtained with true ability levels in both methods. The CAT with the EM-IRT model estimated true ability level slightly lower than CAT with the 3PL model. The EM-IRT model enables measuring the response time that could yield additional data to the physician about the mental and cognitive condition of the patient. The CAT method can be a promising method of telemedicine in the era of the pandemic.

Gaining a deeper understanding of the meaning of the carelessness parameter in the 4PL IRT model and strategies for estimating it

Three popular models are used to describe the characteristics of the test items and estimate the ability of examinees under the dichotomous IRT model, namely the one-, two-, and three-parameter logistic models. The three-item parameters are discriminating power, difficulty, and pseudo-guessing. In the development of the dichotomous IRT model, carelessness or upper asymptote parameter was proposed, which forms a four-parameter logistic (4PL) model to accommodate a condition where a high-ability examinee gives an incorrect response to a test item when he/she should be able to respond to the test item correctly. However, the carelessness parameter and the 4PL model have not been widely accepted and used due to several factors, and people’s understanding of that parameter and strategies for estimating it is still inadequate. Therefore, this study aims to shed light on ideas underlying the 4PL model, the meaning of the carelessness parameter, and strategies used to estimate that parameter based on the extant literature. The focus of this study was then extended to demonstrating practical examples of estimating item and person parameters using the 4PL model using empirical data on responses of 1,000 students from the Indonesia Open University (Universitas Terbuka) on 21 of 30 multiple-choice items on the Business English test, a paper-and-pencil test. We mainly analyzed empirical data using the ‘mirt’ package in RStudio. We present the analysis results coherently so that IRT users would have a sufficient understanding of the 4PL model and the carelessness parameter, and they can estimate item and person parameters under the 4PL model.

A Diagnostic Tree Model for Adaptive Assessment of Complex Cognitive Processes Using Multidimensional Response Options

A tree model for diagnostic educational testing is described along with Monte Carlo simulations designed to evaluate measurement accuracy based on the model. The model is implemented in an assessment of inferential reading comprehension, the Multiple-Choice Online Causal Comprehension Assessment (MOCCA), through a sequential, multidimensional, computerized adaptive testing (CAT) strategy. Assessment of the first dimension, reading comprehension (RC), is based on the three-parameter logistic model. For diagnostic and intervention purposes, the second dimension, called process propensity (PP), is used to classify struggling students based on their pattern of incorrect responses. In the simulation studies, CAT item selection rules and stopping rules were varied to evaluate their effect on measurement accuracy along dimension RC and classification accuracy along dimension PP. For dimension RC, methods that improved accuracy tended to increase test length. For dimension PP, however, item selection and stopping rules increased classification accuracy without materially increasing test length. A small live-testing pilot study confirmed some of the findings of the simulation studies. Development of the assessment has been guided by psychometric theory, Monte Carlo simulation results, and a theory of instruction and diagnosis.

Item Response Theory and Modeling with Stata

ABSTRACT This software review discusses the capabilities of Stata to conduct item response theory modeling. The commands needed for fitting the popular one-, two-, and three-parameter logistic models are initially discussed. The procedure for testing the discrimination parameter equality in the one-parameter model is then outlined. The commands for fitting several polytomous models are subsequently indicated, as are those facilitating model comparison. Scoring of individual units of analysis with Stata’s item response theory module is next discussed, and various graphical features of this module are pointed out. The review concludes with an illustration example of using Stata for item response modeling on an empirical data set.

Estimate the noise effect on automatic speech recognition accuracy for mandarin by an approach associating articulation index

The automatic speech recognition (ASR) technology has become one of the fast-growing engineering technologies. The ASR accuracy is often reduced by environmental interference noise. Many studies attempted to improve the ASR word accuracy in noisy backgrounds, but few were concerned about estimating the influence of noise on the word accuracy of ASR systems. This study investigated the effect of stationary noise on the word accuracy for mandarin of two ASR systems (i.e., the Baidu Cloud and Microsoft Azure). We selected the speech material from an open Mandarin speech database. The interference noise contained pink noise, white noise, random noise with equivalent power spectrum to the average power spectrum of mandarin, band-stop filtered noise and several typical household appliances noise (i.e., hairdryer, range hood, blender and water purifier). The signal-to-noise ratios (SNRs) were set to the range −12–3 dB. The Azure ASR system was less affected by noise and tended to have a higher word accuracy under the same noise condition than the Baidu ASR system. We proposed an approach to associate the articulation index (AI) of interference noise with the speech recognition accuracy of ASR systems. A three-parameter logistic model could be established for a specific ASR system, and each parameter is linearly dependent on the AI of interference noise. The performance of ASR systems under different stationary noise conditions could be quantitatively assessed and compared through the approach. Different models should be established for different versions of the ASR systems through this AI-associating approach since the parameters of one model are only applicable to one particular version of an ASR system.

Validation of the Vocabulary Size Test

The Vocabulary Size Test (VST) is one of the most commonly used assessment tools for measuring English vocabulary size in the field of language testing. Despite its common usage, only a limited number of validity and reliability studies have been carried out with regard to the VST. Besides, they were mostly predicated on the Rasch model. This validation study has attempted to reveal evidence for construct validity for the VST, and to this end, item response theory (IRT) analyses were performed based on the three-parameter logistic model (3PLM). The assumptions of IRT were investigated via factor analysis (unidimensionality) and Yen’s Q3 statistic (local independence). Detailed differential item functioning (DIF) analyses were conducted with Mantel-Haenszel, Lord's chi-square test, and Logistic regression methods to add evidence based on internal structure and to check fairness as a lack of measurement bias. The validation results with IRT showed that the 3PLM fitted the data better than the one- and the two-parameter logistic models. DIF results indicated that 10 items exhibited large DIF (seven favoring males and three favoring females). The results further showed that the guessing effect was not negligible for the VST.

Four-Parameter Guessing Model and Related Item Response Models

Guessing effects frequently occur in testing data in educational or psychological applications. Different item response models have been proposed to handle guessing effects in dichotomous test items. However, it has been pointed out in the literature that the often employed three-parameter logistic model poses implausible assumptions regarding the guessing process. The four-parameter guessing model has been proposed as an alternative to circumvent these conceptual issues. In this article, the four-parameter guessing model is compared with alternative item response models for handling guessing effects through a simulation study and an empirical example. It turns out that model selection for item response models should be rather based on the AIC than the BIC. However, the RMSD item fit statistic used with typical cutoff values was found to be ineffective in detecting misspecified item response models. Furthermore, sufficiently large sample sizes are required for sufficiently precise item parameter estimation. Moreover, it is argued that the criterion of the statistical model fit should not be the sole criterion of model choice. The item response model used in operational practice should be valid with respect to the meaning of the ability variable and the underlying model assumptions. In this sense, the four-parameter guessing model could be the model of choice in educational large-scale assessment studies.