National Assessment Of Educational Achievement Research Articles

When you are not alone: school composition by family structure and students’ academic achievements in South Korea

ABSTRACT This study examines inequality based on family structure within the school environment, focusing on South Korea’s standardized education system. Previous research has pointed to potential challenges associated with a high concentration of students from single-parent families in schools. However, in the context of a standardized education system that aims to minimize variation across schools, these dynamics may manifest in different patterns. Using a nationwide administrative data set, the National Assessment of Educational Achievement, this study finds a significant interaction effect between the proportion of students from single-parent families within schools and the academic achievement, which is particularly pronounced for students from single-parent families. The results suggest that institutional contexts may play a significant role in reducing educational disparities associated with family structure, thereby providing schools pathways to mitigate the academic challenges prevalent among students from diverse family backgrounds.

Multi-level structural equation modelling for factors affecting science achievement of Korean middle school students before and after COVID-19: based on 2018–2020 national assessment of educational achievement data

ABSTRACT Science academic achievement is a key construct for examining students’ science learning outcomes. While international comparative assessments like TIMSS and PISA have shown high science achievement of Korean middle school students, there is a lack of comprehensive investigation of factors influencing the achievement using domestic data. Additionally, the impact of COVID-19 on science achievement-related factors remains largely unexplored. This study utilises multi-level structural equation modelling to examine the factors affecting science achievement among Korean middle school students. By analysing data from the National Assessment of Educational Achievement (NAEA), we identify individual- and school-level variables that affect student science achievement. The model fitting results were compared before (2018–2019; N = 5771 and N = 5614) and after (2020; N = 4,623) the COVID-19 outbreak. The analysis results reveal that individual-level factors, such as self-efficacy, affective domain of science learning, and self-studying time had significant effects on science achievement consistently throughout years. School-level factors, including principal’s supervision, school climate, and science teacher’s efficacy, demonstrated significantly positive effects on science achievement only in 2020 following the COVID-19 outbreak. This study enhances our understanding of the characteristics of high-achieving Korean middle school students and highlights the role of school community in supporting student learning during uncertain circumstances like the COVID-19 pandemic.

교육과정 개정에 따른 학교단위 수학 성취 수준의 변화

This study examined changes in mathematics achievement at the school level following the implementation of the revised curricula in 2007 and 2009. Using data from the National Assessment of Educational Achievement collected during 2009-2019, segmented linear regression models were applied to analyze the achievements of 9th- and 11thgraders.
 Breakpoints corresponding to the transition from the 2007 revision to the 2009 revision were identified in the data patterns for both grade levels. The proportion of students with below-basic and basic achievement levels in each grade decreased significantly during the implementation of the 2007 revision but increased significantly following the implementation of the 2009 revision. These findings suggest the necessity to reflect on and investigate the impacts of past curriculum revisions on students’ achievement to inform the successful implementation of current curricula in mathematics education.

검사 길이 변화에 따른 국가수준 학업성취도 평가의 안정성 분석1)

The 2019 NAEA(National Assessment of Educational Achievement) for high school is planed to implement a new type of item to measure student subject-specific competency to apply the 2015 Revised National Curriculum. Since the new type of items are implemented to measure complex abilities rather than simple knowledge, the issue of reducing the test length was considered to solve higher-level items. Namely, the major change in the 2019 NAEA is the reduction in the number of items caused by the introduction of the new type of items. This study inspected how the test length change of the 2019 NAEA for high school had an effect on the scale stability. In the NAEA, the media attention is focused on the year trend of the proportion of below basic students, so the scale stability of the NAEA which categorizes students into basic and below basic, is main point. According to the results of this simulation study, in order to precisely categorize basic level students from below basic level students with less than 0.05 estimated RMSE at the cut-off score: at least three items should be included for each subject test with the ‘low’ difficulty level (item difficulty parameter b < -1.5) and the ‘moderate’ discrimination level (item discrimination parameter a ≥ 0.5).

A Comparison of Linguistic Aspects in 2015-2019 NAEA English Listening Texts Between School Levels and Between Language Forms

The current study aims to investigate English listening texts of the National Assessment of Educational Achievement (NAEA) at word, sentence, and text levels. It classifies NAEA listening texts into two school levels (middle school and high school) and two forms of spoken language (dialogues and monologues). It is necessary to examine changes of linguistic aspects in the NAEA for both middle and high school students to figure out whether text difficulty systematically rises as school level increases. The corpus for this study includes 169 listening texts from five academic years from 2015 to 2019. In this period, tests were implemented as paper-based tests. Two research questions are about linguistic differences of NAEA listening texts between middle school and high school and between dialogues and monologues at word, sentence and text levels. Coh-Metrix analyzed descriptive measures related to words, lexical diversity, and word information for word-level analysis, descriptive measures related to sentences and syntactic complexity for sentence-level analysis, and readability and referential cohesion for text-level analysis. Results and educational implications could increase the quality of NAEA listening texts.

코로나19를 전후한 고등학생 수학 성취도 변화: 실태 및 영향요인1)

Drawing on high school data from the National Assessment of Educational Achievement (NAEA) in 2019 and 2020, we examined the deepening educational gap caused by COVID-19. Our results showed that the time spent using electronic devices increased in 2020, when online classes were conducted due to COVID-19 compared to 2019, whereas there was no significant change in the time spent on private education. Also, it turned out that the average math score did not change significantly in the upper and middle ranks in 2020 compared to 2019, but a sharp drop occurred in the lower ranks. In addition, the gap in average math scores between high-performing and low-performing students in schools in metropolitan areas widened significantly in 2020 compared to 2019. Meanwhile, multilevel model analysis results suggest that the deepening of the educational gap during the COVID-19 period was mainly due to student characteristics. Finally, quantile regression results showed that high-performing students had even higher academic achievement, while their low-performing counterparts had lower academic achievement in the wake of COVID-19. Even in the case of low-performing students, however, the negative impact of COVID-19 tended to decrease when school-level variables were controlled for.

중⋅고등학생의 학습동기 수준별 학업성취, 원격수업 경험 및 만족의 차이

Objectives The purpose of this study is to classify the latent groups according to the learning motivation of Korean language and mathematics of middle and high school students, and to analyze the differences in academic achievement, distance learning experiences and satisfaction in 2020 by learning motive groups.
 Methods The subject of analysis is the data of 10,653 middle school students and 10,528 high school students responding to the ｢2020 National Assessment of Educational Achievement｣ of Korea Institute for Curriculum and Evaluation. For data processing, frequency analysis, factor analysis, and reliability analysis were performed using SPSS and Jamovi programs, the latent groups according to the Korean language and math learning motivation were extracted using the Mplus program, and the characteristics of each latent group were analyzed.
 Results The learning motivation groups of middle and high school students were identified as upper, middle, and lower 3 groups in Korean, and upper, middle-high, middle-low, and lower 4 groups in mathematics. As a result of examining the characteristics of each group, the group with high motivation for Korean and math had good Korean and math achievement. Looking at the changes in school level, Korean learning motivation gradually increased, but Korean language achievement decreased, and math learning motivation gradually decreased, but achievement tended to increase. As a result of examining the environmental differences in distance learning experienced in 2020 by the learning motivation group, the group with high Korean and math learning motivation was found to have the least negative influence of distance learning, suggesting the possibility of a learning gap. In addition, there was a difference in satisfaction with distance learning according to the level of motivation, and lower satisfaction with feedback and communication was shown compared to other items.
 Conclusions Middle and high school students' motivation for Korean and mathematics and academic achievement are closely related, suggesting the development of convergence classes between subjects. In addition, the group with high learning motivation appears to be the group that minimizes the unfavorable influence of distance learning, requiring the establishment of strategies to improve learners' learning motivation for each subject.

Exploring the Stability of the Immediate Achievement-Level Results on the National Assessment of Educational Achievement

국가수준 학업성취도 평가는 지금까지 지필평가 방식으로 시행되어 왔고, 평가 시행 후 약 2개월이 지난 시점에 학생들에게 평가 결과(4단계 학업 성취수준)가 제공되었다. 평가결과표가 제공되는 시점에는 학생들의 평가에 대한 관심과 기억이 감소하여 평가 결과에 대한 유용성이 매우 낮았다. 2022년부터 학업성취도 평가에 컴퓨터 기반 검사로 전환됨에 따라 학교 현장의 요구 사항을 수렴한 결과, 평가 시행 직후의 즉각적인 결과 산출에 대한 요구가 높은 것으로 나타났다. 본 연구에서는 컴퓨터 기반 검사 시행 시, 문항모수가 이미 확보되어 있는 연도간 공통문항과 자동채점이 가능한 선다형 및 단답형 문항들을 이용하여 즉각적인 성취수준 결과를 산출할 수 있는 방안을 제시하고, 결과의 안정성을 탐색하기 위한 모의실험을 수행하였다. 자료 생성을 위해 국가수준 학업성취도 평가의 실제 문항모수를 활용하였으며, 모의실험을 위한 조건은 (1) 공통문항의 수, (2) 공통문항의 선정 방법, (3) 피험자 수 등 세 가지를 고려하였다. 그 결과 6개 내외의 공통문항을 문항 정보함수를 활용하여 선정하고, 피험자 수는 검사 유형 당 최소 1000명이 확보될 때 즉각적 결과 산출의 가능성을 확인할 수 있었다. 그러나 현 학업성취도 평가의 문항난이도 분포를 고려할 때 1수준과 2수준의 학생들을 변별하는 기능이 상당히 부족한 것으로 나타났다. 따라서 즉각적인 결과 산출을 위해서 1수준과 2수준 구분에 적합한 저난이도 문항이 추가적으로 확보되어야 함을 시사하였다.

A Simulation Study for Examining Roles of Sampling Weight in Multilevel Analysis with data from a Complex Survey Design

이 연구는 복잡한 조사 설계를 적용한 국가수준 학업성취도(National Assessment of Educational Achievement)를 모집단으로 하여 다단계 층화표집설계에 기반하여 수집된 데이터에 대한 다층모형 시 가중치 적용 분석의 장점을 살펴보고자 하였다. 시뮬레이션 연구를 통해 가중치 미적용, 설계 가중치 적용, 레이크 가중치 적용의 결과를 학교별 2학급 표집과 3학급 표집 조건에 따라 고정효과와 임의효과 추정치에 대한 다층모형 분석 결과를 분석하였다. 레이크 가중치 생성을 위해 외적, 내적 층화 변수와 더불어 보조변수를 활용하였다. NAEA 데이터 기반 시뮬레이션 분석 결과, 가중치 적용 기술 통계값은 가중치 미적용 결과에 비해 모집단 평균에 가까웠지만, 다층분석 모형에서는 고정효과와 임의효과 추정치에서 가중치 적용과 미적용 결과 간에 커다란 차이를 보이지 않았다. 본 연구 결과는 교육분야의 대규모 데이터를 사용한 다층 모형 기반 분석에서 가중치 적용 여부 및 유용성에 대한 실증적 증거를 제공하는 데 도움을 줄 것이다.This study was motivated by the National Assessment of Educational Achievement (NAEA) involving a complex survey design for sampling. The goal of the study was to explore the benefit of weighted analysis in multilevel modeling. Large-scale data such as the NAEA data are often analyzed in multilevel analyses to reflect the nested data structure. A simulation study was conducted incorporating different conditions. Multilevel analysis results were compared without weights, with basic design weights, and with a raked weight, which were generated in consideration of explicit and implicit stratification variables with auxiliary variables. Simulation also compared the analysis results by varying the number of classes per school in a stratified cluster sampling design. In summary, substantial benefits were not observed in weighted multilevel analysis compared to unweighted multilevel analysis, although the weighted descriptive statistics were close to the population means relative to the unweighted descriptive statistics. These findings may help provide scientific evidence for the use of sampling weights in analyses with large-scale educational data.

A Comparative Analysis of NAEA English Reading Passages for Middle and High School Students Using Coh-Metrix

This study aims to identify differences of English reading passages in the National Assessment of Educational Achievement (NAEA) for middle and high school students. For this study, 208 NAEA English reading passages from the five consecutive years from 2015 to 2019 were extracted. The two research questions consider the differences of two tests, and correlations among the selected Coh-Metrix indices such as descriptive measures, lexical characteristics, syntactic complexity, discourse patterns, and readability level. The extracted reading passages were analyzed based on 17 selected Coh-Metrix indices related to each selected index. As for the differences of two tests, descriptive measures, lexical difficulty and readability level showed clear differences, whereas lexical diversity, syntactic complexity, and discourse patterns did not present critical gaps between the two tests. As for the correlations, the positive correlations between readability level and descriptive measures were inevitable in that the way to estimate the Flesch Reading Ease score and the Flesch-Kincaid Grade Level is based on the average number of syllables per word and words per sentence. Lexical diversity was negatively correlated to all the indices of discourse patterns. The implications and limitations of the current study are included.

Predicting secondary school students’ academic achievement from their elementary school performance and learning behaviours: A longitudinal study based on South Korea’s national assessment of educational achievement

ABSTRACT Academic achievement is an important factor strongly related to positive educational experiences that facilitate subsequent learning. Therefore, identifying students who need support at an early stage and promptly providing appropriate intervention play a crucial role in preventing learning deficits. This study examined the longitudinal change in elementary school students’ proficiency status to understand the nature of longitudinal transitions. It explored the feasibility of developing early predictive models for identifying students potentially facing academic difficulties based on information obtained from elementary school. Deep neural network and hierarchical generalized linear models were applied to large-scale national assessment panel data collected within the framework of an accountability system in South Korea. The results showed that although most students maintained their initial proficiency status, a substantial number of students made longitudinal transitions from non-proficiency to proficiency or vice-versa as they progressed from elementary to high school. The predictive models had an acceptable level of accuracy in predicting future academic performance, considering the degree of student mobility across proficiency levels over grades. The study concluded that the predictive models can serve as a starting point to identify students who need assistance and provide educators with a means to intervene promptly.

A Study on the Improvement Plan of the Middle School Science Academic Achievement: Focused on the National Assessment of Educational Achievement(NAEA)

본 연구에서는 학업성취도 평가 개선을 위한 방안을 제시하기 위한 연구로 최근 4년간 중학교 3학년 학생들의 과학 성취수준별 학생 비율의 변화 추이를 분석하였다. 중학교 과학에서 우리나라 학생들의 전반적인 학업성취도는 점진적으로 낮아지는 경향을 보였는데, 우수학력 비율이 줄고 기초학력 미달 비율이 늘어나는 경향이 나타났다. 이러한 학업성취도 하락 추세의 원인을 파악하기 위해서는 인지적 영역의 학업성취도 저하만으로는 전체적인 교육의 성과를 판단하기 어려우므로 현행 지필 평가 체제로 파악할 수 없는 성취기준에 대한 평가 방안도 모색하여야 한다. 이를 위해 과학 학업성취도 평가 문항 분석을 통해 성취수준별 학생들의 학업성취 특성을 분석하고, 학교 교육과정 운영 현황 조사 등을 수행한 후 평가도구 개발 등 학업성취도 평가의 개선을 위하여 출제 과정에 참여한 출제위원과 검토위원을 대상으로 설문조사를 실시하였다. 제기된 주요 개선 과제는 출제위원 대상 사전 워크숍 강화, 인지적 요소 외 교육 목표 달성도를 확인하기 위한 학업성취도 평가 문항 개선 등이다. 이를 토대로 개선 방안에 대한 시사점을 도출하였다.

2009 개정 교육과정과 고등학교 학생들의 영어 학업성취 특성1)

National Assessment of Educational Achievement (NAEA) is a 20-year-old criterion-referenced test, developed on the academic standards of the national curriculum. The test is annually administered to assess the achievement levels of students in the subject area of Language, Social Studies, Mathematics, Science, and English and to survey the perception of teachers and students’ teaching and learning experience. The test results have been used to identify and support the low academic achievers and to provide basic data to education policy makers and curriculum developers. This study intend to identify the achievement features of the high school English learners in the period of the 2009 revised English curriculum in terms of achievement trend and mastered standards. The test results of English from 2012 to 2018, when the 2009 revised English curriculum is effective and practiced, were gathered and analyzed to identify the achievement features of the high school learners of English. Overall there is a tendency of a decrease in the test score and a sustained but bigger gap between cities and other regions. Also, more than half of the academic standards of the English curriculum are not ‘mastered’ by the Good group and all the academic standards are not ‘mastered’ by the Basic group. The findings suggest new educational policies and new English curriculum tailored to the different achievement levels should be considered.

Analysis of the Response Characteristics on Items of Life Science of Science Subject of National Assessment of Educational Achievement (NAEA) according to Item Type and Behavior Dimension

Analysis of the Response Characteristics on Items of Life Science of Science Subject of National Assessment of Educational Achievement (NAEA) according to Item Type and Behavior Dimension

Analysis of Achievement Characteristics by Achievement Standard of the Middle School Curriculum Based on the National Assessment of Educational Achievement

Analysis of Achievement Characteristics by Achievement Standard of the Middle School Curriculum Based on the National Assessment of Educational Achievement

The Analysis of Students’ Responses to Affective Domains and Their Achievement in NAEA English by Using Motivation Theory

The purpose of this study was to analyze the middle school students’ responses to affective domains in English of the 2018 National Assessment of Educational Achievement (NAEA) by using the attribution theory. The purpose of the study was threefold. The first was to examine the relationship between students’ responses of affective domains and their educational achievement. The second was to examine how students’ responses to the affective domains can be interpreted by using the motivation theory. The final purpose was to identify approaches toward enhancing students’ affective characteristics in terms of motivation. As a data source, this study used the ‘2018 NAEA results: middle school’ published by the Korea Institute for Curriculum and Evaluation (KICE) in 2019. The students’ responses to the 16 survey questions of affective domains and their scores in the 2018 NAEA English subject were statistically analyzed and reinterpreted in terms of the three factors of attribution: locus, stability and controllability. Based on the outcome of the analysis, this study interprets how to understand students’ responses to affective domains and suggests approaches toward enhancing students’ affective characteristics in terms of motivation.

Explaining academic achievement gaps in South Korea with multicultural education paradigms

ABSTRACT The academic achievement gap between racial and ethnic majority and minority students in South Korea has received attention over the past decade, with one government-funded report and 10 peer-reviewed articles examining the performance of diverse student groups in the 2011 National Assessment of Educational Achievement. We conducted a qualitative content analysis of these 11 studies to understand how the evident achievement gap is explained by researchers and what assumptions are embedded in these explanations. Using multicultural education paradigms as theoretical lenses, we classified the explanations and advanced understanding; findings indicated that the cultural deprivation paradigm dominated the discussion of the academic performance of ethnic minority students. While other explanations were also found, some of these reinforced stereotypes about ethnic minority groups without corroborating evidence. Emergent efforts to address the strengths of ethnic minority students were insufficient to reflect the authentic view of the cultural difference paradigm. Implications were drawn for researchers and educators who study academic achievement gaps between ethnic minority and majority students in multicultural societies.

수학과 교육과정 성취기준에 대한 중학생의 숙달도 분석

The purpose of this study was to examine the outcomes of the National Assessment of Educational Achievement (NAEA) 2015-2018 when the 2009 Revised National Curriculum was applied and suggested for future curriculum revisions. The characteristics of students’ academic achievement were analyzed based on the results of evaluating test items developed by the achievement standards of the 2009 Revised National Curriculum. The analysis showed that students’ academic achievement was low in the ‘Function’, ‘Probability and Statistics’, and ‘Geometry’ domain. In addition, many achievement standards could be mastered only by the above average group of students. Accordingly, it is necessary to reinforce customized teaching and learning activities in relation to achievement standards which were under achieved. To improve the achievement characteristics, it is necessary to specify detailed achievement standards and to strengthen the linkage between achievement standards.

Alignment Analysis of Vocabulary Items in National Assessment of Educational Achievement

Alignment Analysis of Vocabulary Items in National Assessment of Educational Achievement

2010-2014년 국가수준 학업성취도 평가에서 나타난 고등학생의 수학 학업 특성 분석

2010-2014년 국가수준 학업성취도 평가에서 나타난 고등학생의 수학 학업 특성 분석