Abstract
Although the importance of summary writing is well documented in prior studies, few have investigated the evaluation of written summaries. Due to the complex nature of L2 summary writing, which requires one to read the original material and summarize its content in the L2, raters often emphasize different features when judging the quality of L2 summaries. Therefore, this study examines the ratings of English-language summaries written by Japanese university students in order to identify differences in EFL instructors’ evaluations. Fifty-one Japanese EFL university students read a passage and then wrote an English summary without receiving any instructions concerning summary composition. The raters included three native English speakers (NESs) and three non-native English speakers (NNESs), who individually evaluated each summary using the Educational Testing Service’s holistic rubric. Analysis of inter-rater reliability revealed a lower Cronbach’s alpha coefficient for NNES raters (α = .39) when compared to NES raters (α = .77). Comments were collected from raters regarding the difficulty of evaluating summaries, and the causes of such difficulties were examined. Comments from NNES raters more concerned vocabulary use and paraphrasing, whereas the NES raters concentrated on content and language. This study also explores ways to potentially improve the holistic rubric by examining feedback from raters regarding their rating experiences.
Highlights
Summary writing is widely recognized as an important teaching method, for university students in foreign language classes, and is an effective tool for measuring L2 proficiency
The results of Study I revealed that the rubric was difficult to use, and highlighted the need for an analytic rubric targeting non-native English speakers (NNESs) who evaluate and teach L2 summary writing in higher education
Post-hoc residual analysis indicated that NES raters tended to deem the task of evaluation “easy”; comparatively, NNES raters most often considered it to be “difficult.” The adjusted standardized residuals for the NES and NNES groups were statistically significant (4.59 and 7.86, respectively)
Summary
Summary writing is widely recognized as an important teaching method, for university students in foreign language classes, and is an effective tool for measuring L2 proficiency. This study is one from a series of studies comprising a project for the development of efficient and useful rubrics (or rating scales) for L2 summary writing in EFL academic contexts. Three Japanese raters used the ETS rubric to evaluate summaries written by 51 Japanese EFL university students. The results of Study I revealed that the rubric was difficult to use, and highlighted the need for an analytic rubric targeting non-native English speakers (NNESs) who evaluate and teach L2 summary writing in higher education
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
