Abstract
Assessing significant change (or reliable change) in a person often involve comparing the responses of that person in two administrations of a test or scale. Several procedures have been proposed to determine if a difference between two observed scores is statistically significant or rather is within the range of mere random fluctuations due to measurement error. Application of those procedures involve some knowledge of the test properties. But sometimes those procedures cannot be employed because the properties are unknown or are not trustworthy. In this paper we propose the bootstrap of items procedure to create confidence intervals of the individual's scores without using any known psychometric properties of the test. Six databases containing the responses of several groups to one or more subscales have been analyzed using two methods: bootstrap of items and a classical procedure based on confidence intervals to estimate the true score. The rates of significant change obtained were very similar, suggesting that item bootstrapping is a promising solution when other methods cannot be applied.
Highlights
Often in disciplines like psychology or education it is important to be able to measure a change in a person’s behavior
In clinical and health settings, a distinction is usually made between the presence of some change against the situation in which there is no change or the presence of a change whose magnitude reaches a certain level against smaller amounts of change (Jacobson and Truax, 1991; Kazdin, 1999, 2001; Kendall, 1999; Ogles et al, 2001; Perdices, 2005)
Our most striking result is that the rates in the bootstrap of items (BSI) column are very similar to those for ETS, despite they have been obtained without knowing any of those properties
Summary
Often in disciplines like psychology or education it is important to be able to measure a change in a person’s behavior. If the observed difference falls outside of a predefined range of measurement errors, it is concluded that the change is statistically significant or that a reliable change has occurred. A mere mistake in typing or calculating one or both scores can yield statistically different scores They can be observed because two different persons participate in the before and after tests, in cases of fraud by impersonation. Ferrer and Pardo (Pardo and Ferrer, 2013; Ferrer and Pardo, 2014) have compared, through a simulation study, the relative efficacies of the main methods Application of all those methods requires some knowledge of the test properties, such as the population’s mean and variance, or the reliability or internal consistency. The procedure consists of performing a bootstrap of items (BSI), that is, in applying the bootstrap method to the responses given by an individual to the items of the test or scale
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
