Validity Of Dynamic Assessment Research Articles

Comparing summative and dynamic assessments of L2 written argumentative discourse: Microgenetic validity evidence

Drawing upon multiple data sources, this case study compares the summative assessment (SA) and dynamic assessment (DA) of L2 writing, in order to examine the validity of DA. There is a dearth of research on the validity of DA of L2 writing. Most DA studies to date have examined its effectiveness in assessing and promoting learning based on a loosely defined construct, which DA aims to assess and develop, and without discussing DA’s validity in relation to the microgenesis that is expected to transpire within DA. This study examines what an L2 learner knows of the constructs and can do with mediation to produce argumentative texts. The two constructs which were assessed and promoted were an adaptation of the Toulmin (1958/2003) model of argument and the knowledge-transforming cognitive composing process (Bereiter & Scardamalia, 1987) that led the production of the texts. After a period of instruction in argumentation and its cognitive processes, SA suggested that the learner’s development, as judged by his scores, was almost flat; thus, DA was performed to verify if, indeed, there was no true development and to simultaneously assess if, with mediation, the learner could produce the missing argumentative discourse features; a series of dialogic and contingent DA interactions were enacted to help the learner identify and produce the missing features; both goals were successfully accomplished. A microgenetic, together with rating and textual analysis, was employed to examine the quality and validity of mediation required to generate the discourse features. An argument-based approach to validity supports the claim that DA complemented SA to make the assessment practice fairer by tapping into emergent learning; the argument approach provides a robust basis for the interpretation and use of the findings of DA.

Validity of a nonspeech dynamic assessment of the alphabetic principle in preschool and school-aged children

Learning to read and spell is an important but difficult achievement for children who have complex communication needs. Given that assessment is a vital part of any intervention program, one major barrier is the lack of reliable and valid assessments for this population. This study evaluated the reliability and validity of Dynamic Assessment of the Alphabetic Principle (DAAP), which does not require spoken responses. For this study, 27 preschool and school-aged children with typical development completed the DAAP and other standard measures of phonemic awareness and early literacy. Results indicated the DAAP had high internal consistency and strong correlations among its subtests, indicating high reliability. Moreover, performance on the DAAP had high correlations with standard measures of phonemic awareness and early literacy, providing evidence of its validity. Consequently, the DAAP represents one approach to fill the important need for assessments of early literacy that do not require speech responses.

Dynamic Assessment of Reading Difficulties: Predictive and Incremental Validity on Attitude toward Reading and the Use of Dialogue/Participation Strategies in Classroom Activities.

Dynamic Assessment (DA) has been shown to have more predictive value than conventional tests for academic performance. However, in relation to reading difficulties, further research is needed to determine the predictive validity of DA for specific aspects of the different processes involved in reading and the differential validity of DA for different subgroups of students with an academic disadvantage. This paper analyzes the implementation of a DA device that evaluates processes involved in reading (EDPL) among 60 students with reading comprehension difficulties between 9 and 16 years of age, of whom 20 have intellectual disabilities, 24 have reading-related learning disabilities, and 16 have socio-cultural disadvantages. We specifically analyze the predictive validity of the EDPL device over attitude toward reading, and the use of dialogue/participation strategies in reading activities in the classroom during the implementation stage. We also analyze if the EDPL device provides additional information to that obtained with a conventionally applied personal-social adjustment scale (APSL). Results showed that dynamic scores, obtained from the implementation of the EDPL device, significantly predict the studied variables. Moreover, dynamic scores showed a significant incremental validity in relation to predictions based on an APSL scale. In relation to differential validity, the results indicated the superior predictive validity for DA for students with intellectual disabilities and reading disabilities than for students with socio-cultural disadvantages. Furthermore, the role of metacognition and its relation to the processes of personal-social adjustment in explaining the results is discussed.

Development of First-Graders’ Word Reading Skills

Dynamic assessment (DA) of word reading measures learning potential for early reading development by documenting the amount of assistance needed to learn how to read words with unfamiliar orthography. We examined the additive value of DA for predicting first-grade decoding and word recognition development while controlling for autoregressive effects. Additionally, we examined whether predictive validity of DA would be higher for students who have poor phonological awareness skills. First-grade students (n = 105) were assessed on measures of word reading, phonological awareness, rapid automatized naming, and DA in the fall and again assessed on word reading measures in the spring. A series of planned, moderated multiple regression analyses indicated that DA made a significant and unique contribution in predicting word recognition development above and beyond the autoregressor, particularly for students with poor phonological awareness skills. For these students, DA explained 3.5% of the unique variance in end-of-first-grade word recognition that was not attributable to autoregressive effect. Results suggest that DA provides an important source of individual differences in the development of word recognition skills that cannot be fully captured by merely assessing the present level of reading skills through traditional static assessment, particularly for students at risk for developing reading disabilities.

Using Dynamic Assessment to Evaluate the Expressive Syntax of Children who use Augmentative and Alternative Communication

The developmental readiness of four 5-year-old children to produce basic sentences using graphic symbols on an augmentative and alternative communication (AAC) device during a dynamic assessment (DA) task was examined. Additionally, the ability of the DA task to predict performance on a subsequent experimental task was evaluated. A graduated prompting framework was used during DA. Measures included amount of support required to produce the targets, modifiability (change in participant performance) within a DA session, and predictive validity of DA. Participants accurately produced target structures with varying amounts of support. Modifiability within DA sessions was evident for some participants, and partial support was provided for the measures of predictive validity. These initial results indicate that DA may be a viable way to measure young children's developmental readiness to learn how to sequence simple, rule-based messages via aided AAC.

Validating Dynamic Assessment of Triadic Gaze for Young Children With Severe Disabilities

This research investigated the use of a dynamic assessment (DA) to identify differences among young children with severe disabilities, which would predict progress in learning behaviors indicating coordinated joint attention (CJA). Six children 10-24 months of age were enrolled in a 16-week treatment for behaviors indicating CJA, specifically triadic gaze (TG), which is a 3-point gaze shift between object and adult. An initial static assessment documented the children's eligibility for the study and their baseline performance of TG. DA procedures were then implemented to determine each child's performance with examiner support in producing behaviors suggesting joint attention (i.e., tracking, gaze toward an object or an adult, scanning between objects, scanning an object and adult, and TG). Results demonstrated differences among children during the DA via a DA score and a behavioral profile. These results were predictive of differences among children in subsequent learning of TG. These data support the validity of DA for describing heterogeneity among young children with severe disabilities who look similar on static assessment but appear differentially ready to learn behaviors associated with joint attention. This knowledge will assist clinicians in planning more efficacious services for young children who struggle to communicate and are at risk for extended therapeutic needs.

Examining the Predictive Validity of a Dynamic Assessment of Decoding to Forecast Response to Tier 2 Intervention

The purpose of this study was to examine the role of a dynamic assessment (DA) of decoding in predicting responsiveness to Tier 2 small-group tutoring in a response-to-intervention model. First grade students (n = 134) who did not show adequate progress in Tier 1 based on 6 weeks of progress monitoring received Tier 2 small-group tutoring in reading for 14 weeks. Student responsiveness to Tier 2 was assessed weekly with word identification fluency (WIF). A series of conditional individual growth curve analyses were completed that modeled the correlates of WIF growth (final level of performance and growth). Its purpose was to examine the predictive validity of DA in the presence of three sets of variables: static decoding measures, Tier 1 responsiveness indicators, and prereading variables (phonemic awareness, rapid letter naming, oral vocabulary, and IQ). DA was a significant predictor of final level and growth, uniquely explaining 3% to 13% of the variance in Tier 2 responsiveness depending on the competing predictors in the model and WIF outcome (final level of performance or growth). Although the additional variances explained uniquely by DA were relatively small, results indicate the potential of DA in identifying Tier 2 nonresponders.

The Predictive Validity of Dynamic Assessment

The authors report on a mixed-methods review of 24 studies that explores the predictive validity of dynamic assessment (DA). For 15 of the studies, they conducted quantitative analyses using Pearson's correlation coefficients. They descriptively examined the remaining studies to determine if their results were consistent with findings from the group of 15. The authors implemented analyses in five phases: They compared the predictive validity of traditional tests and DA, compared two forms of DA, examined the predictive validity of DA by student population, investigated various outcome measures to determine whether they mediate DA's predictive validity, and assessed the value added of DA over traditional testing. Results indicated superior predictive validity for DA when feedback is not contingent on student response, when applied to students with disabilities rather than at-risk or typically achieving students, and when independent DA and criterion-referenced tests were used as outcomes instead of norm-referenced tests and teacher judgment.