Ontario Secondary School Literacy Test Research Articles

The Inequity and Effect of Standardized Literacy Testing for First Nations Students: An Ontario (Canadian) Outlook

The authors illuminate the distinct issues of standardized literacy testing on First Nations (FN) such as the need for equity within Ontario, Canada with regard to the provincial Education Quality and Accountability Office (EQAO - test) and the Ontario Secondary School Literacy Test (OSSLT), both criterion-referenced tests for students in Ontario. The authors address key areas such as historical background, test equity, literacy practices and the effect on FN students. A cursory overview of government and control via testing of First Nations was explored.

Interpreting the Impact of the Ontario Secondary School Literacy Test on Second Language Students Within an Argument-Based Validation Framework

This article draws on Kane’s (2006) argument-based validation framework to synthesize evidence derived from a large-scale, mixed-method explanatory study on the impact of the Ontario Secondary School Literacy Test (OSSLT) on second language (L2) students. The purpose of the OSSLT is to ensure that students have acquired the essential reading and writing skills that apply to all subject areas in the Ontario provincial curriculum up to the end of Grade 9 in Canada. Kane’s framework is used both to specify the proposed interpretations and uses of the OSSLT results by laying out the network of inferences and assumptions involved in this test and to elaborate whether the proposed interpretations and uses have been supported by empirical evidence from the study. Findings from the study show that the results of the OSSLT, a test constructed and normed for first language English speakers, should be interpreted differently and with caution for second language students. By synthesizing the empirical evidence within an argument-based validation framework, we can fully understand the impact of the OSSLT in relation to test design, test accommodation, and literacy classroom practices in the Canadian context.

Student Perceptions of Literacy after the Ontario Secondary Literacy Course: A Qualitative Inquiry

Adolescent literacy has emerged via the high-stakesstandardized test known as the Ontario Secondary School Literacy Test (OSSLT)as a critical area of debate and study. Research has indicated a directconnection between literacy and identity, and that student literacy practicesdiffer from traditional measures of literacy located in school curriculum andevaluated via standardized tests such as the OSSLT. Outcomes such as limitedachievement, difficulties with literacy and the development of literacy skills,and subsequent below standard scorescan diminish student self-concept, lower self-esteem, and impede self-efficacy.This ethnographic case study illuminated the impact of OSSLT and subsequentmandatory enrolment in the Ontario Secondary School Literacy Course usingsemi-structured interviews involving high-school students from a northernOntario secondary school. Previous related research outcomes, whichdemonstrated a connection between standardized test scores and self-concept,were realized via participants’ understanding and perception of literacy, andthrough mitigating factors impacting literacy engagement and achievement.

The construction of ‘illiterate’ and ‘literate’ youth: the effects of high-stakes standardized literacy testing

High-stakes standardized literacy testing is not neutral and continues to build upon the legacy of dominant power relations in the state in its ability to sort, select and rank students and ultimately produce and name some youth as illiterate in contrast to an ideal white, male, literate citizen. I trace the effects of high-stakes standardized testing by using the voices of 16 youth who failed the Ontario Secondary School Literacy Test (OSSLT) to illustrate how the ‘illiterate youth’ revealed to students, schools, and communities by this test is culturally and socially constructed. In an age where multiple literacies are more and more valued, standardized literacy testing acts as a form of social control projected upon the ‘adolescent’ body that has historically been deemed ‘other’ or ‘deficient.’ Just as colonized subjects needed to be ‘civilized,’ so youth now need to acquire a state defined literacy in a competitive and fast paced learning environment. This article helps to demonstrate how power operates on marginalized youth through standardized literacy testing that is being used transnationally.

The Missing Response Patterns in the Ontario Secondary School Literacy Test

Large‐scale assessments are often an important indicator of students’ achievement for schools, states, and provinces. Missing responses can affect the appropriateness of our analysis models and the results of large‐scale educational assessments. The study of missing response patterns (MRPs) can inform the design of a test and interpretation of test results. This study will examine the causes and effects of MRPs based on analyses of students’ responses to the Ontario Secondary School Literacy Test (OSSLT) in 2006. This is a test with high stakes for students. With some preliminary statistical analyses in SPSS (descriptive statistics, plots, cross‐tabs, and multinomial and logistic regressions), we are exploring possible causes of MRPs by examining the relationships between patterns of missing responses and responses to test items and background questionnaire items. All results will be helpful for us to understand more about the test’s construct and internal validity to support improvement of the relevant largescale assessment in the future.

The Inequity and Effect of Standardized Literacy Testing for First Nations Students: An Ontario (Canadian) Outlook

The authors illuminate the distinct issues of standardized literacy testing on First Nations (FN) such as the need for equity within Ontario, Canada with regard to the provincial Education Quality and Accountability Office (EQAO - test) and the Ontario Secondary School Literacy Test (OSSLT), both criterion-referenced tests for students in Ontario. The authors address key areas such as historical background, test equity, literacy practices and the effect on FN students. A cursory overview of government and control via testing of First Nations was explored.

Tracking the Success of English Language Learners within the Context of the Ontario Secondary School Literacy Test

The study tracked 8 high school English Language Learners (ELLs) from 2 Ontario secondary schools located in small Ontario cities for 3 years and closely documented their linguistic, cognitive, and socio-cultural learning experiences. Each student was visited 2-4 times per year and face-to-face interviews were used on each visit. It was found that these students‘ difficulties in academic English were the major challenge to their success in their high school subject-area courses and in the Ontario Secondary School Literacy Test (OSSLT). These challenges, in turn, affected their educational goals. The findings of this study provide a rich understanding of these students‘ educational achievement as they concurrently learned English and their subject-area courses within the context of the OSSLT—a large-scale high-stakes literacy test. Cette étude recense trois années d‘expériences d‘apprentissage linguistiques, cognitives et socio culturelles de huit allophones d‘anglais langue seconde (ALS) issus de deux écoles secondaires situées dans des petites villes de l‘Ontario. Chaque étudiant a été interviewé en face à face deux à quatre fois par an. Les résultats montrent que la difficulté majeure de ces étudiants est l‘apprentissage de l‘anglais académique et que cet apprentissage détermine leur succès dans tous les autres cours au niveau du lycée et au Test Provincial des Compétences Linguistiques (TPCL) de l‘Ontario. Ce défi affecte également leurs objectifs éducatifs. Les résultats de cette étude montrent non seulement les performances éducatives de ces étudiants en ALS mais aussi dans leurs autres cours dans le cadre du TPCL, un test à enjeux élevés.

Differential Functioning of Reading Subskills on the OSSLT for L1 and ELL Students: A Multidimensionality Model‐Based DBF/DIF Approach

The increasing numbers of English language learners (ELLs) in Canadian schools pose a significant challenge to the standards‐based provincial tests used to measure proficiency levels of all students from various linguistic and cultural backgrounds. This study investigated the extent to which reading item bundles or items on the Ontario Secondary School Literacy Test (OSSLT) function differentially for Grade 10 students who speak only or mostly English at home (first language [L1] students; n = 1,969) and those whose home language is something other than English (ELL students; n = 3,675). Based on Roussos and Stout's (1996a) multidimensionality‐based DIF analysis paradigm, a variety of substantive and statistical techniques were employed: (a) content review by English as a second language (ESL) experts, (b) exploratory and confirmatory dimensionality analyses, and (c) confirmatory differential bundle functioning (DBF)/differential item functioning (DIF) procedures. The evidence gathered in the study indicated that items associated with vocabulary knowledge favored L1 students, whereas items requiring grammatical knowledge or integrated reading and writing skill favored ELL students. Instructional implications for the promotion of effective literacy education programs are discussed, as is the development of a literacy curriculum that can meet the needs of linguistically diverse learners in a multilingual context.

Walking in Their Shoes: Students’ Perceptions of Large-Scale High-Stakes Testing

Abstract: With the implementation of the Ontario Secondary School Literacy Test (OSSLT) in 2002, Ontario became the first province in Canada requiring successful completion of a large-scale high-stakes literacy test for high school graduation. We began to explore and analyze students’ perceptions of this testing program in terms of their preparation for the test, the test’s impact and value, and the potential influence of such a testing program on the students’ views about literacy. Our study used qualitative data obtained through focus groups and interviews with students who were either successful or unsuccessful on the OSSLT. The students recalled specific test preparation for the OSSLT, often at the expense of their regular schooling. Following test instructions was a clear message to the students, leading to formulaic test responses and a narrowly expressed view of literacy. Potentially important differences were also found between the successful and unsuccessful students.

Do Test Formats in Reading Comprehension Affect Second-Language Students' Test Performance Differently?

Large-scale testing in English affects second-language students not only greatly but also differently than first-language learners. The research literature reports that confounding factors in such large-scale testing such as varying test formats may differentially affect the performance of students from diverse backgrounds. An investigation of test performance between ESL/ELD students and non- ESL/ELD students on the Ontario Secondary School Literacy Test (OSSLT) was performed to investigate whether test formats in reading comprehension affected the two groups differently. The results indicate that the overall pattern of difficulty levels on the three test formats were the same between ESL/ELD students and non-ESL/ELD students, except that ESL/ELD students performed substantially lower on each format and that more variability was found among ESL/ELD students. Further, discriminant analysis results indicated that only the multiplechoice questions obtained a significant discriminant coefficient in differentiating the two groups. The results suggest a lack of association between test formats and test performance.

Abstracts: Language testing

07–604Abbott, Marilyn (Alberta Education, Canada; marilyn.abbott@gov.ab.ca), A confirmatory approach to differential item functioning on an ESL reading assessment. Language Testing (Sage) 24.1 (2007), 7–36.07–605Barber, Richard (Dubai Women's College, UAE), A practical model for creating efficient in-house placement tests. The Language Teacher (Japan Association for Language Teaching) 31.2 (2007), 3–7.07–606Cheng, Liying, Don Klinger & Ying Zheng (Queen's U, Canada; chengl@edu.queensu.ca), The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing (Sage) 24.2 (2007), 185–208.07–607Cohen, Andrew (U Minnesota, USA) & Thomas Upton, ‘I want to go back to the text’: Response strategies on the reading subtest of the new TOEFL®. Language Testing (Sage) 24.2 (2007), 209–250.07–608Dávid, Gergely (Eötvös Loránd U, Hungary; david.soproni@t-online.hu), Investigating the performance of alternative types of grammar items. Language Testing (Sage) 24.1 (2007), 65–97.07–609Elder, Catherine (U Melbourne, Australia; caelder@unimelb.edu.au), Gary Barkhuizen, Ute Knoch & Janet Von Randow, Evaluating rater responses to an online training program for L2 writing assessment. Language Testing (Sage) 24.1 (2007), 37–64.07–610Qian, David (The Hong Kong Polytechnic U, China; David.Qian@polyu.edu.hk), Assessing university students: Searching for an English language exit test. RELC Journal (Sage) 38.1 (2007), 18–37.07–611Scott Walters, Francis (U New York, USA; Francis.Walters@qc.cuny.edu), A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing (Sage) 24.2 (2007), 155–183.07–612Shiotsu, Toshihiko (Kurume U, Japan; toshihiko_shiotsu@kurume-u.ac.jp) & Cyril Weir, The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing (Sage) 24.1 (2007), 99–128.07–613Vanderveen, Terry (Kangawa U, Japan), The effect of EFL students' self-monitoring on class achievement test scores. JALT Journal (Japan Association for Language Teaching) 28.2 (2006), 197–206.07–614Xi, Xiaoming (Educational Testing Service, USA; xxi@ets.org), Evaluating analytic scoring for the TOEFL® Academic Speaking Test (TAST) for operational use. Language Testing (Sage) 24.2 (2007), 251–286.

Student Accounts of the Ontario Secondary School Literacy Test: A Case for Validation

Abstract: The Ontario Secondary School Literacy Test (OSSLT) is a cross-curricular literacy test issued to all secondary school students in the province of Ontario. The test consists of a reading and a writing component, both of which must be successfully completed for secondary school graduation in Ontario. This study elicited 16 first language and second language student accounts of their OSSLT test-taking processes immediately after the March 2006 test administration. The analysis of these students’ accounts provided valuable information about the validity of the inferences drawn from the Ontario Secondary School Literacy Test. These accounts suggest the complexity of the processes that the students engaged as they attempted to demonstrate their reading and writing skills on the test. The study has implications for test developers and test users regarding the interpretation of student test performance on the Ontario Secondary School Literacy Test.

The challenges of the Ontario Secondary School Literacy Test for second language students

Results from the Ontario Secondary School Literacy Test (OSSLT) indicate that English as a Second Language (ESL) and English Literacy Development (ELD) students have comparatively low success and high deferral rates. This study examined the 2002 and 2003 OSSLT test performances of ESL/ELD and non-ESL/ELD students in order to identify and understand the factors that may help explain why ESL/ELD students failed the test at relatively high rates. The analyses also attempted to determine if there were significant and systematic differences in ESL/ELD students' test performance. The performance of ESL/ELD students was consistently and similarly lower across item formats, reading text types, skills and strategies, and the four writing tasks. Using discriminant analyses, it was found that narrative text type, indirect understanding skill, vocabulary strategy of reading, and the news report writing task were significant predictors of ESL/ELD membership. The results of this study provide direction for further research and instruction regarding English literacy achievement for these second language students within the context of having to complete large-scale English literacy tests designed and constructed for first English language students.

Literacy is Just Reading and Writing, isn't it? The Ontario Secondary School Literacy Test and its Press Coverage

This article examines how the public discourse of print news media defines and shapes the representation of the Ontario Secondary School Literacy Test (OSSLT) based on coverage in three primary newspapers between 1998 and 2004. The data were analysed using qualitative and quantitative measures to identify types of coverage, themes, and inclusion/exclusion of voices. The analysis, which is framed by discourse about conceptions of literacy relating to Dewey's democratic vision for the press, suggests some disappointments on the measure of democratic representation and participation. The article concludes that, if the media is to represent the diversity of voices and provide a wide range of views so as to fulfil its democratic responsibility as envisioned by Dewey, a wider debate over representations of literacy must occur and more perspectives and voices must be included in newspaper coverage.

Did we take the same test? Differing accounts of the Ontario Secondary School Literacy Test by first and second language test‐takers

Within the context of increasing numbers of second language (L2) learners in Canadian schools and expanding standards‐driven testing frameworks, a passing score on the Ontario Secondary School Literacy Test (OSSLT) is a recently imposed secondary school graduation requirement in Ontario. There is evidence, however, that tests designed on the basis of first language (L1) populations may have lower reliability and validity for L2 students. This study elicited accounts of the OSSLT in 33 focus groups of 22 L1 students and 136 L2 students, attending 7 Ontario secondary schools, prior to and immediately after the March 2006 test administration. The results suggest important differences in L1 and L2 accounts of test constructs and suggest a gap between what is valued as literacy on the test and what is valued in classroom literacy practice, raising some concern regarding the test’s consequential validity. By examining how different groups of test‐takers interpret test constructs and the interaction between these interpretations, test design, and accounts of classroom practice, we may better address issues of fidelity in test construct representation (i.e., understand what may constitute construct under‐representation and construct‐irrelevant variance). This study highlights what may make a test more L2‐friendly, i.e. what supports (or impedes) L2 test performance. Although in the washback literature test‐taker accounts of tests have been the least researched, the results of this study suggest that such accounts have the potential to increase test fairness, enhance the validity of inferences drawn from test performance, improve the effectiveness of accommodation strategies, and promote positive washback.