Reliability Assessment of Arabic Speech Contest

Abstract

This study's assessment model for the Arabic speech competition involved three competent judges. Each of them assessed the same three components, namely al fashahah (fluency), lubbul maudhu' (content/theme discussion), and al harakah (participant movement, including participant expressions). The subjects in this study were all 24 participants in the Arabic speech competition at IAIN Ponorogo in 2019. All of these participants came from MA around Madiun. It is possible to issue differences between scores given by the three judges, considering that the range of values for each component is between 50 to 100. Scoring with a reasonably wide range could affect the consistency of the assessment. It is crucial to estimate the reliability coefficient of the judges' assessment of the Arabic speech contest. This study uses a quantitative approach because the primary data are scores from the judges of the Arabic language competition. The reliability estimation uses a variance analysis approach whose procedure is based on generalizability theory (G-Theory) through the G-Study concept with a multifaceted design. This theory can improve the instrument's quality by testing several sources of variance to make decisions and the consistency of the results of the generalizability coefficient. The data analysis results conclude that the assessment instrument used in the Arabic speech competition in this study is reliable or still feasible to use. The feasibility is based on the validity and reliability of the instruments used. The analysis of some experts regarding the instrument content claims that it was valid. In addition, the reliability coefficient of the combined score of the Arabic speech competition assessment is 0.96708. Therefore, it concludes that the assessment instrument used in this study is reliable.