Three-parameter Logistic Model Research Articles

Long-Term Impact of Valid Case Criterion on Capturing Population-Level Growth Under Item Response Theory Equating

Using simulated data, this study examined the impact of different levels of stringency of the valid case inclusion criterion on item response theory (IRT)-based true score equating over 5 years in the context of K–12 assessment when growth in student achievement is expected. Findings indicate that the use of the most stringent inclusion criterion generally yielded the most accurate results when overall root mean square error (RMSE) and bias were considered under both zero-growth and growth conditions, for both one-parameter logistic (1PL) and three-parameter logistic (3PL) IRT models, and for both fixed common item parameter (FCIP) and test characteristic curve (TCC) scaling methods. The positive impact of applying the most stringent valid case inclusion criterion was more salient with the 3PL model, under which greater classification accuracy was observed.

The Impact of Three Factors on the Recovery of Item Parameters for the Three-Parameter Logistic Model

ABSTRACTThis article provides a detailed description of three factors (specification of the ability distribution, numerical integration, and frame of reference for the item parameter estimates) that might affect the item parameter estimation of the three-parameter logistic model, and compares five item calibration methods, which are combinations of the three factors, using a simulation study. The five item calibration methods are Normal-Midpoint-Prior (NMPr), Normal-Hermite-Prior (NHPr), Normal-Midpoint-Posterior (NMPo), Normal-Hermite-Posterior (NHPo), and Empirical-Midpoint-Prior (EMPr). In addition, four item response theory computer programs (BILOG-MG, PARSCALE, flexMIRT, and ICL) are compared in terms of their default specifications and available options of the three factors. The EMPr method recovered item parameters accurately regardless of the shape of the population ability distribution and the number of quadrature points. The NMPr, NHPr, NMPo, and NHPo methods returned item parameter estimates with low bias when abilities followed a standard normal distribution, but tended to either underestimate or overestimate the item parameters when the population ability distribution was skewed. Also, unlike the EMPr method, the performance of these four methods depended on the number of quadrature points.

Using the Stan Program for Bayesian Item Response Theory.

Stan is a new Bayesian statistical software program that implements the powerful and efficient Hamiltonian Monte Carlo (HMC) algorithm. To date there is not a source that systematically provides Stan code for various item response theory (IRT) models. This article provides Stan code for three representative IRT models, including the three-parameter logistic IRT model, the graded response model, and the nominal response model. We demonstrate how IRT model comparison can be conducted with Stan and how the provided Stan code for simple IRT models can be easily extended to their multidimensional and multilevel cases.

A comparison of five methods for pretest item selection in online calibration

Many long-term testing programs rely on large item banks that need to be replenished regularly with new items, and these new items need to be pretested before being used operationally. Online calibration is a pretesting strategy in computerised adaptive testing, which embeds pretest items in operational tests and adaptively matches the pretest items with examinees. This paper compares five existing methods for pretest item selection in online calibration. A simulation study was conducted under the one-, two-, and three-parameter logistic models. The effects of two estimation methods, three seeding locations, and five calibration sample sizes were also investigated. Findings from the simulation study are mixed. However, overall, the simplest random selection method appears to be a potential best choice.

Analysis test of understanding of vectors with the three-parameter logistic model of item response theory and item response curves technique

An example of the use of item response theory to assess multiple-choice conceptual instruments.

Item Response Theory Observed-Score Kernel Equating.

Item response theory (IRT) observed-score kernel equating is introduced for the non-equivalent groups with anchor test equating design using either chain equating or post-stratification equating. The equating function is treated in a multivariate setting and the asymptotic covariance matrices of IRT observed-score kernel equating functions are derived. Equating is conducted using the two-parameter and three-parameter logistic models with simulated data and data from a standardized achievement test. The results show that IRT observed-score kernel equating offers small standard errors and low equating bias under most settings considered.

Slow life-history traits of a neritic predator, the bronze whaler (Carcharhinus brachyurus)

Intra-species plasticity in the life-history characteristics of sharks leads to the need for regional estimates to accurately determine resilience to anthropogenic effects. The present study provides the first length-at-age, growth and maturity estimates for the bronze whaler (Carcharhinus brachyurus) from southern Australia. Age estimates were obtained from vertebral sections of 466 individuals spanning 50–308-cm total length. Maximum estimates of age for males and females were 25 and 31 years respectively. The three-parameter logistic model for females (L∞=308cm LT, k=0.15, α=742) and for males (L∞=317cm LT, k=0.13, α=782) provided the best fit to the size at age data. Males matured at a similar age (16 years), but smaller size than females (224v. 270cm LT). Growth parameters and age-at-maturity estimates were similar to those for genetically isolated C. brachyurus populations, and the sympatric dusky shark (C. obscurus). The southern Australian C. brachyurus population is long-lived, slow growing and late maturing. These growth parameters are needed to undertake demographic analyses to assess the resilience of C. brachyurus to fishing, and provide an example of a wide-ranging elasmobranch with similar life-history characteristics across isolated populations.

A simplified version of the maximum information per time unit method in computerized adaptive testing.

In this article, we propose a simplified version of the maximum information per time unit method (MIT; Fan, Wang, Chang, & Douglas, Journal of Educational and Behavioral Statistics 37: 655-670, 2012), or MIT-S, for computerized adaptive testing. Unlike the original MIT method, the proposed MIT-S method does not require fitting a response time model to the individual-level response time data. It is also computationally efficient. The performance of the MIT-S method was compared against that of the maximum information (MI) method in terms of measurement precision, testing time saving, and item pool usage under various item response theory (IRT) models. The results indicated that when the underlying IRT model is the two- or three-parameter logistic model, the MIT-S method maintains measurement precision and saves testing time. It performs similarly to the MI method in exposure control; both result in highly skewed item exposure distributions, due to heavy reliance on the highly discriminating items. If the underlying model is the one-parameter logistic (1PL) model, the MIT-S method maintains the measurement precision and saves a considerable amount of testing time. However, its heavy reliance on time-saving items leads to a highly skewed item exposure distribution. This weakness can be ameliorated by using randomesque exposure control, which successfully balances the item pool usage. Overall, the MIT-S method with randomesque exposure control is recommended for achieving better testing efficiency while maintaining measurement precision and balanced item pool usage when the underlying IRT model is 1PL.

Examination of Goodness of School-based Paper and Pencil Test Applying Classical Test Theory and Three-Parameter Logistic Model in Physical Education: Facilitating Female Student’s Participation in Sp

Examination of Goodness of School-based Paper and Pencil Test Applying Classical Test Theory and Three-Parameter Logistic Model in Physical Education: Facilitating Female Student’s Participation in Sp

New Insights into the Nature of Cerebellar-Dependent Eyeblink Conditioning Deficits in Schizophrenia: A Hierarchical Linear Modeling Approach

Evidence of cerebellar dysfunction in schizophrenia has mounted over the past several decades, emerging from neuroimaging, neuropathological, and behavioral studies. Consistent with these findings, cerebellar-dependent delay eyeblink conditioning (dEBC) deficits have been identified in schizophrenia. While repeated-measures analysis of variance is traditionally used to analyze dEBC data, hierarchical linear modeling (HLM) more reliably describes change over time by accounting for the dependence in repeated-measures data. This analysis approach is well suited to dEBC data analysis because it has less restrictive assumptions and allows unequal variances. The current study examined dEBC measured with electromyography in a single-cue tone paradigm in an age-matched sample of schizophrenia participants and healthy controls (N = 56 per group) using HLM. Subjects participated in 90 trials (10 blocks) of dEBC, during which a 400 ms tone co-terminated with a 50 ms air puff delivered to the left eye. Each block also contained 1 tone-alone trial. The resulting block averages of dEBC data were fitted to a three-parameter logistic model in HLM, revealing significant differences between schizophrenia and control groups on asymptote and inflection point, but not slope. These findings suggest that while the learning rate is not significantly different compared to controls, associative learning begins to level off later and a lower ultimate level of associative learning is achieved in schizophrenia. Given the large sample size in the present study, HLM may provide a more nuanced and definitive analysis of differences between schizophrenia and controls on dEBC.

An Extension of IRT-Based Equating to the Dichotomous Testlet Response Theory Model

ABSTRACTCurrent procedures for equating number-correct scores using traditional item response theory (IRT) methods assume local independence. However, when tests are constructed using testlets, one concern is the violation of the local item independence assumption. The testlet response theory (TRT) model is one way to accommodate local item dependence. This study proposes methods to extend IRT true score and observed score equating methods to the dichotomous TRT model. We also examine the impact of local item dependence on equating number-correct scores when a traditional IRT model is applied. Results of the study indicate that when local item dependence is at a low level, using the three-parameter logistic model does not substantially affect number-correct equating. However, when local item dependence is at a moderate or high level, using the three-parameter logistic model generates larger equating bias and standard errors of equating compared to the TRT model. However, observed score equating is more robust to the violation of the local item independence assumption than is true score equating.

Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications

Computerized adaptive testing (CAT) is a powerful technique to help improve measurement precision and reduce the total number of items required in educational, psychological, and medical tests. In CATs, tailored test forms are progressively constructed by capitalizing on information available from responses to previous items. CAT applications primarily have relied on unidimensional item response theory (IRT) to help select which items should be administered during the session. However, multidimensional CATs may be constructed to improve measurement precision and further reduce the number of items required to measure multiple traits simultaneously. A small selection of CAT simulation packages exist for the R environment; namely, catR (Magis and Raiche 2012), catIrt (Nydick 2014), and MAT (Choi and King 2014). However, the ability to generate graphical user interfaces for administering CATs in realtime has not been implemented in R to date, support for multidimensional CATs have been limited to the multidimensional three-parameter logistic model, and CAT designs were required to contain IRT models from the same modeling family. This article describes a new R package for implementing unidimensional and multidimensional CATs using a wide variety of IRT models, which can be unique for each respective test item, and demonstrates how graphical user interfaces and Monte Carlo simulation designs can be constructed with the mirtCAT package.

An item analysis of the Conditional Reasoning Test of Aggression.

This manuscript uses item response theory (IRT) to estimate item characteristics of the Conditional Reasoning Test of Aggression (CRT-A). Using a sample size of 5,511 respondents, the present analysis provides an accurate assessment of the capability of the CRT-A to measure latent aggression. The one-parameter logistic (1PL) model, two-parameter logistic (2PL) model, and three-parameter logistic (3PL) model are compared before the item analysis. Results suggest that the 2PL model is the most appropriate dichotomous IRT model for describing the item characteristics of the CRT-A. Potential multdimensionality in the CRT-A is also examined. Results suggest that CRT-A items work as theoretically intended, with the probability of selecting an aggressive response increasing with latent trait levels. Information curves indicate that the CRT-A is best suited for use with individuals who are high on latent aggression. Exploratory analyses include an examination of polytomous IRT models and DIF comparing student and employee respondents. The results have implications for future research using the CRT-A as well as the identification of populations appropriate for measurement using this assessment tool.

Differential item functioning analysis of the Vanderbilt Expertise Test for cars.

The Vanderbilt Expertise Test for cars (VETcar) is a test of visual learning for contemporary car models. We used item response theory to assess the VETcar and in particular used differential item functioning (DIF) analysis to ask if the test functions the same way in laboratory versus online settings and for different groups based on age and gender. An exploratory factor analysis found evidence of multidimensionality in the VETcar, although a single dimension was deemed sufficient to capture the recognition ability measured by the test. We selected a unidimensional three-parameter logistic item response model to examine item characteristics and subject abilities. The VETcar had satisfactory internal consistency. A substantial number of items showed DIF at a medium effect size for test setting and for age group, whereas gender DIF was negligible. Because online subjects were on average older than those tested in the lab, we focused on the age groups to conduct a multigroup item response theory analysis. This revealed that most items on the test favored the younger group. DIF could be more the rule than the exception when measuring performance with familiar object categories, therefore posing a challenge for the measurement of either domain-general visual abilities or category-specific knowledge.

Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application.

There has been renewed interest in Barton and Lord's (An upper asymptote for the three-parameter logistic item response model (Tech. Rep. No. 80-20). Educational Testing Service, 1981) four-parameter item response model. This paper presents a Bayesian formulation that extends Béguin and Glas (MCMC estimation and some model fit analysis of multidimensional IRT models. Psychometrika, 66 (4):541-561, 2001) and proposes a model for the four-parameter normal ogive (4PNO) model. Monte Carlo evidence is presented concerning the accuracy of parameter recovery. The simulation results support the use of less informative uniform priors for the lower and upper asymptotes, which is an advantage to prior research. Monte Carlo results provide some support for using the deviance information criterion and [Formula: see text] index to choose among models with two, three, and four parameters. The 4PNO is applied to 7491 adolescents' responses to a bullying scale collected under the 2005-2006 Health Behavior in School-Aged Children study. The results support the value of the 4PNO to estimate lower and upper asymptotes in large-scale surveys.

The Impact of Non-Normality on Extraction of Spurious Latent Classes in Mixture IRT Models.

Unidimensional, item response theory (IRT) models assume a single homogeneous population. Mixture IRT (MixIRT) models can be useful when subpopulations are suspected. The usual MixIRT model is typically estimated assuming a normally distributed latent ability. Research on normal finite mixture models suggests that latent classes potentially can be extracted, even in the absence of population heterogeneity, if the distribution of the data is non-normal. In this study, the authors examined the sensitivity of MixIRT models to latent non-normality. Single-class IRT data sets were generated using different ability distributions and then analyzed with MixIRT models to determine the impact of these distributions on the extraction of latent classes. Results suggest that estimation of mixed Rasch models resulted in spurious latent class problems in the data when distributions were bimodal and uniform. Mixture two-parameter logistic (2PL) and mixture three-parameter logistic (3PL) IRT models were found to be more robust to latent non-normality.

A Stepwise Test Characteristic Curve Method to Detect Item Parameter Drift

An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the existing methods were designed to detect drifts in individual items, which may not be adequate for test characteristic curve–based linking or equating. One example is the item response theory–based true score equating, whose goal is to generate a conversion table to relate number-correct scores on two forms based on their test characteristic curves. This article introduces a stepwise test characteristic curve method to detect item parameter drift iteratively based on test characteristic curves without needing to set any predetermined critical values. Comparisons are made between the proposed method and two existing methods under the three-parameter logistic item response model through simulation and real data analysis. Results show that the proposed method produces a small difference in test characteristic curves between administrations, an accurate conversion table, and a good classification of drifted and nondrifted items and at the same time keeps a large amount of linking items.

Time-Domain Reflectometry for the Prediction of Loblolly Pine and Sweetgum Moisture Content

Time-domain reflectometry (TDR) can be used to predict the moisture content in porous materials, including soil, and is an exciting tool that could be used to measure the moisture content in wet-stored wood. Three-rod probes with 127 mm- or 152 mm-long rods were inserted into 62 loblolly pine and 34 sweetgum saturated bolts. The bolts were air dried over a span of five weeks. TDR waveforms and moisture content were periodically recorded. In total, 534 and 482 readings were taken for the loblolly pine and sweetgum bolts, respectively. An algorithm in R was written to automatically analyze the apparent length of the TDR rods. Calibration models were developed between moisture content and X (apparent length / actual rod length). A three-parameter logistic model was developed for loblolly pine (R2=0.64) and sweetgum (R2=0.84). The process was repeated using shorter bolts and 152 mm-long rods, resulting in improved models for loblolly pine (R2=0.99) and sweetgum (R2=0.97). Overall, TDR and the algorithm written to analyze the waveforms were accurate in predicting moisture content and could be used to monitor moisture in wet-decks.

The Effect of Upper and Lower Asymptotes of IRT Models on Computerized Adaptive Testing.

In this article, the effect of the upper and lower asymptotes in item response theory models on computerized adaptive testing is shown analytically. This is done by deriving the step size between adjacent latent trait estimates under the four-parameter logistic model (4PLM) and two models it subsumes, the usual three-parameter logistic model (3PLM) and the 3PLM with upper asymptote (3PLMU). The authors show analytically that the large effect of the discrimination parameter on the step size holds true for the 4PLM and the two models it subsumes under both the maximum information method and the b-matching method for item selection. Furthermore, the lower asymptote helps reduce the positive bias of ability estimates associated with early guessing, and the upper asymptote helps reduce the negative bias induced by early slipping. Relative step size between modeling versus not modeling the upper or lower asymptote under the maximum Fisher information method (MI) and the b-matching method is also derived. It is also shown analytically why the gain from early guessing is smaller than the loss from early slipping when the lower asymptote is modeled, and vice versa when the upper asymptote is modeled. The benefit to loss ratio is quantified under both the MI and the b-matching method. Implications of the analytical results are discussed.

Direct construction of predictive models for describing growth of Salmonella Enteritidis in liquid eggs – A one-step approach

The objective of this study was to develop a new approach using a one-step approach to directly construct predictive models for describing the growth of Salmonella Enteritidis (SE) in liquid egg white (LEW) and egg yolk (LEY). A five-strain cocktail of SE, induced to resist rifampicin at 100 mg/L, was used to inoculate LEW and LEY. Kinetic studies were conducted isothermally at different temperatures between 8 and 43 °C to generate growth curves at each temperature.This study first solved an inverse problem globally, using the growth curves to estimate the temperature-dependent kinetic parameters, and then applied the parameters to predict growth (a forward problem). Once the growth curves were generated, they were assembled and analyzed using nonlinear regression to determine kinetic parameters of both primary and secondary models in one step, with an objective to minimize the global residual sum of squares (RSS) for the entire data set. For growth in LEW, a three-parameter logistic model was used. For growth in LEY, the Huang model was used as the primary model. The Ratkowsky square-root model was used to evaluate the growth rates.The results showed that the one-step approach resulted in accurate estimation of the kinetic parameters that were used later to successfully predict the growth of SE in LEY and LEW. The estimated nominal minimum growth temperatures of SE were 7.4 °C and 9.9 °C, while the estimated maximum growth temperatures were 45.2 °C and 46.8 °C, respectively, in LEW and LEY. As a validation, the predictive models were tested with independent growth curves of SE in LEY and LEW at 37 °C. The root mean square error (RMSE) was only 0.36 and 0.28 log CFU/ml over a total scale of 8.4 and 7.8 log CFU/ml, respectively, for the growth models of SE in LEY and LEW, suggesting that the one-step approach can generate accurate models for predicting the growth of SE in LEY and LEW. The results from this study can be used to predict the growth of SE and evaluate the safety of LEY and LEW.