Analysis of Error Sources and Estimation of Reliability in Peer Review of Forced Connection Method-Sportscasting by Applying Generalizability Theory

Abstract

The purpose of this study which is follow up study of Lee and Kim(2015b)'s was to analyse error sources and estimation of reliability in peer review of forced connection method-sportscasting by applying generalizability theory. Generalizability theory quantify error sources of the data measured under certain specific situation set by the researchers. It is an analysis method that the relative influences of each error sources taking from score is determined(G-study), and the effective measurement condition future applicable is provided(D-study). Participants were 10th high school students(N=216). Data were collected from student's peer review results and analyzed using univariate and multivariate generalizability theory. Results showed that error source for video have a more significant impact than other error sources. But the result by analyzing the gender difference was that error source for the interaction of video and participants have a more significant impact than other error source in the case of girls. Peer review used in this study showed high generalizability coefficient and even when reducing the number of video or participants it can maintain the adequate reliability. But generalizability coefficient of boys was higher than girls and specific measurement conditions leading to enhanced reliability were different when analyzing by gender difference. Also, method of analysis which cannot reflect measurement conditions properly estimates the reliability excessive. Discussions were provided in term of the relative influences of each error sources, the effective measurement condition maintaining the Generalizability coefficient of a certain level, and the comparison the Generalizability coefficient with the way of estimation traditional reliability applying univariate and multivariate Generalizability theory taking from score in peer review of forced connection method-sportscasting.