Abstract
The effectiveness of educational interventions has traditionally been evaluated using effect size measures which focus on a single feature of the distribution of the outcomes under intervention and control conditions: a (standardized) mean difference. Recently there has been increased interest in methods which assess the information contained in the full distributions of outcomes under intervention and control, providing measures of separation from these distributions which do not depend on arbitrary cutoffs, hence which are “threshold-free”. We investigate the statistical relationship between several concepts of this type, and discuss how they can be used to estimate alternative effect size metrics as well as their uncertainties in the context of multilevel models as commonly used for the analysis of educational data. A specific aim of this paper is to investigate how the recently proposed “gain index” relates to other measures of separation including the Area under the Curve (AUC) and the overlapping index. A simulation study, using data with an educationally motivated structure, is presented to compare the different methodologies.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
