Search and Re-Search

Abstract

NOT UNTIL I wandered through Wikipedia's glossary on statistics did I realize I have been KDD-ing. Recently I have been conducting a random scan of research, especially research related to testing. Turns out, this endeavor has a name--Knowledge Discovery in Databases. Granted, I limited my knowledge discovery to one database, the research posted by CRESST (the National Center for Research on Evaluation, Standards, and Student Testing) at UCLA. Because Congress has told us we must have research evidence to be able to act on or propose any reforms in education, I thought I should look for some research. The fact is that much of what Congress mandated or otherwise covered in the No Child Left Behind (NCLB) Act has no research base. I guess that's not supposed to matter. Still, my dutiful little chore actually proved fascinating, and many nuggets of research that federal policy makers ignored when they crafted NCLB are certainly worth sharing. At this writing, a mini-revolution is taking place among some school boards in Virginia. They are refusing to use standard assessments with English-language learners as prescribed by the feds. If they stick to their principles, they could lose millions of dollars in federal funding. If they cave in, they will be subjecting their English-language learners to an unfair system of testing. Research shows that test items high in language complexity, such as those used to assess reading, may increase the measurement error in testing the skills of these students. While English-language learners generally perform below the level of native speakers, the differences are smaller in math, where language is not as great a barrier as it is in reading. This seems like common sense, but perhaps politicians reject the idea that language background makes a difference in test scores because that just seems like too simple an explanation. Another one of those commonsense ideas that strict exam proctors probably would reject out of hand is the finding that, if students are allowed an opportunity to talk about a story before being tested on it, their scores are much higher. This research was conducted when small-group collaboration was being tried on large-scale tests--a practice since abandoned, along with most other innovative assessments. Even just 10 minutes discussing a story in three-person teams allowed students to share personal reactions and gain understanding from other students' perceptions. Or, if the story emphasized facts, the students improved their skills at decoding and understanding the material. An assessment that includes group discussions has different qualities from traditional assessments. However, it can answer questions that traditional assessments cannot, such as rendering a judgment on students' social interactions and on how well they function in classroom situations. While everyone has been searching for answers to the dropout problem, I wonder whether anyone has thought about asking students. A research study of state standardized tests as they were evolving from assessments of minimum competency to something more rigorous to be used for high school graduation found that educators considered the tests benign--but students didn't. Failing a graduation test the first time around definitely contributed to students' feelings that they would not complete high school and would be likely to drop out. This was an even stronger influence on students' sense of failure than other factors that are usually associated with leaving school, such as school grades, level of family education, and having repeated a grade in the early years. The study also found that the reasons students drop out were almost invisible to educators. Since the mid-1990s, some states moved ahead of others in trying to work with low-performing schools, and researchers who analyzed these earlier efforts drew some cautionary lessons from them. First, the less ambitious initiatives were more stable. …