Abstract
Reliability indices for mastery tests depend not only on true-score variance but also on mean and cutoff scores. This dependence was examined in the case of three decision-theoretic indices: (1) the co efficient of agreement; (2) kappa; and (3) the pro portion of correct decisions. The binomial error model was assumed, with a two-parameter beta dis tribution for true scores. The reliability indices were computed at five values of the mean, four values of KR-21, and four cutoff scores. Results show that the dependence of kappa on mean and cutoff scores is opposite to that of the proportion of correct de cisions, which is linearly related to average thresh old loss. Moreover, kappa can be very small when most examinees are classified correctly. Thus, ob jections against the classical reliability coefficient apply even more strongly to kappa.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
