Universal Screener Research Articles

Reducing Inequalities Inherent in Literacy Assessment of Multilingual Learners

This 2-year exploratory study of Multilingual Learner’s performance on literacy assessments is grounded in Culturally Responsive Teaching. By examining the protocols for universal screeners and progress monitoring at one midwestern elementary school, teachers and researchers learned more about language vs. literacy-related results and discovered ways to better meet the needs of learners. Data gathered through focus group discussions, classroom observations, literacy assessments, and field notes were analyzed. Findings include: (1) students read passages that lacked cultural and linguistic relevancy, (2) professionals questioned the reliability and validity of assessments for multilingual learners, (3) assessment results perpetuated an idea that students needed to be ‘fixed’ creating a tension between language acquisition and literacy assessment scores, and (4) room for increased awareness and consideration in the selection of literacy assessments for ML learners. Ideas are shared for nuanced improvements in literacy assessments for multilingual learners.

Developing a Whole Child School Screening Instrument: Evaluating Perceived Usability as an Initial Step in Planning for Consequential Validity

We are developing the Equitable Screening to Support Youth (ESSY) Whole Child Screener to address concerns prevalent in existing school-based screenings that impede goals to advance educational equity using universal screeners. Traditional assessment development does not include end users in the early development phases, instead relying on a psychometric approach. In working to develop the ESSY Whole Child Screener, we are integrating a mixed methods approach with attention to consequential validity from the outset of measure development. This approach includes end users in measure development decisions. In this study, we interviewed a diverse sample of school staff (n = 7), administrators (n = 3), and family caregivers (n = 8) to solicit their perceptions of the usability of the initial draft of the ESSY Whole Child Screener. We identified three overarching themes: (1) paving the road for implementation of a whole child screener, (2) potential roadblocks to use, and (3) suggested paths forward to maximize positive intended consequences. Paving the road for implementation of a whole child screener includes subthemes related to alignment with existing initiatives, comprehensive yet efficient design, and potential positive consequences of assessing the whole child. Potential roadblocks to use includes subthemes of staff buy-in, family comfort with contextual screening items, teacher accuracy, and school capacity to provide indicated supports. Suggested paths forward to maximize positive intended consequences include clear and precise messaging to staff and families, optimizing instrumentation and data collection procedures, and strengthening connections to data interpretation and use. We discuss next steps in the design and testing of the initial measure as well as assessment development more broadly.

What Makes for an Effective Gifted and Talented Screener?

Universal screening is one of the most-common topics and well-accepted best practices within the field of gifted and talented education. There appears to be little disagreement that universally screening all students as part of a gifted and talented identification process results in fewer missed students. But surprisingly, there is little guidance on what makes for a quality universal screener—the tool that decides who needs further consideration. In this paper, we provide guidance that can help schools select the universal screener that helps them correctly identify as many students as possible at the lowest possible cost.

A preliminary investigation of the psychometric properties of the Integrated Teacher Rating Form (ITRF) in Greek school children

ABSTRACT Given the dearth of technically adequate universal screening tools in the Greek language, the current study sought to examine the psychometric properties of a 67-item universal screener (i.e. Integrated Teacher Rating Form; ITRF), that was originally developed in American English and then translated and implemented in Greek primary schools. The ITRF, which was designed to identify students at risk and facilitate the planning of interventions targeting classroom behaviors that interfere with academic and social functioning, incorporates four constructs (i.e. Disorganized/Academic Performance Problems, Disruptive behavior, Anxious/Depressed behavior, Socially Withdrawn behavior). To test the classification accuracy of the ITRF, a multi-gated screening procedure was followed, consisting of a brief 4-item screening measure that Greek teachers completed for all students in classroom, followed by completion of the ITRF for those students whose first-gate ratings indicated concern. Results suggested that teachers reported the highest level of concern for students’ academic performance problems, followed by oppositional/disruptive behaviors, socially withdrawn behaviors, and anxious/depressed behaviors. Classification accuracy was found to be excellent for the Disorganized/Academic Productivity Problems Gate 1 measure, good for the Disruptive/Oppositional measure, fair for the Socially Withdrawn measure and poor for the Anxious/Depressed measure.

English Learners’ Performance on a Measure of Dyslexia Risk

Many schools now screen students for dyslexia in early grades. However, there are valid concerns that these screeners may be biased or ineffective at screening students who are not yet proficient in English (i.e., English Learners; ELs). The present study examined the performance of 54 first graders on a dyslexia screener. Results showed that students who were ELs performed similarly to their peers who were proficient in English on many literacy subskills. Additionally, we found that EL students were not significantly more likely to be “flagged” as at risk for dyslexia. These findings have practical implications for using universal screeners to identify students, including students who are designated ELs, as being at risk for learning disabilities, including dyslexia.

The REACH (Resiliency, Engagement, and Accessibility for Comorbid HIV/PTSD/SUD populations) Protocol: Using a Universal Screener to Improve Mental Health and Enhance HIV Care Outcomes.

Among people living with HIV (PLWH), 50% report substance use disorders (SUDs), and 30-61% report posttraumatic stress disorder (PTSD). Comorbid PTSD/SUD/HIV has been linked to faster HIV progression and twice the rate of death, lower medical adherence and retention, and increased viral load compared to PLWH without co-occurring PTSD or SUD. A critical first step in establishing comprehensive mental health services for PLWH is the implementation of an evidence-based screening protocol for PTSD and SUDs to facilitate referrals to specialty mental health providers. Guided by the Consolidated Framework for Implementation Research, this mixed-methods study aimed to examine the feasibility of delivering the REACH (Resiliency, Engagement, and Accessibility for Comorbid HIV/PTSD/SUD populations) Screening protocol. Three case managers were instructed to provide the REACH screening electronically to all patients that they saw for enrollment or re-enrollment appointments over 3 months (n=102). Of the 70 patients who completed the screener, 27% had clinically significant PTSD symptoms and 48.6% had SUD concerns. Qualitative feedback revealed themes related to beliefs about SU and PTSD, attitudes toward screening, comfort in the discussion of SU and PTSD, and referral and treatment considerations. Discussion includes lessons learned for implementation of this assessment of PTSD/SUD in PLWH as a novel approach to increase mental health engagement and promote health equity, with the potential long-term impact of improving HIV care outcomes via ameliorating mental health/SUD, and implications for prevention of HIV transmission. Implementation science can be leveraged to understand the gap in the utilization of existing evidence-based screening tools in HIV care settings.

Usability and social consequences of the early identification system as a universal screener for social, emotional, and behavioral risks.

The Early Identification System (EIS) was developed to overcome many of the usability challenges of school-based behavior screeners. Several prior studies have documented the technical adequacy of the EIS. The present study expanded this work by examining the use, relevance, values implications, and social consequence of EIS implementation in a sample of 54 K-12 schools and 23,104 students in the Midwestern United States. We found that nearly all schools, teachers, and students completed the EIS as planned. Schools used the data to provide universal, selective, and indicated services to a high percentage of students identified with risks as well as to plan professional development for educators based on the screening data. Seventy-nine percent of schools implemented the EIS system with high fidelity, and fidelity was unrelated to student demographic composition within schools. These findings suggest that the EIS may overcome many of the usability barriers that plague common behavior screeners. Limitations and implications for advancing the science of social consequence evaluation are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Universal Screening in Early Childhood Populations: A Systematic Review

Early childhood is an important period for the development of social, emotional and behavioral (SEB) skills. Deficits in these skills often lead to negative outcomes; thus, early identification is essential for the provision of services. Unfortunately, only a fraction of students with deficits are identified and receive services. One cause of this is the methods used to identify students, such as teacher nominations which do not identify all students in need (Dowdy et al., 2013). Proactive practices, such as universal screening, are a more systematic way of identification. The purpose of this review was to examine commonly used early childhood screeners and their evidence base, effectiveness, and the feasibility and accessibility of their use in early childhood settings. This critical review analysed 18 screeners using Southam-Gerow & Prinstein’s (2014) review criteria for evidence-based treatments and a technical adequacy rubric based on Glover and Albers’ (2007) considerations for evaluating universal screening assessments. Of the 18 screening tools reviewed, four screeners are highly recommended based on their technical adequacy and usability within early childhood settings. These results highlight the need for further research in the evaluation of early childhood universal screeners.

Supplemental Material for Usability and Social Consequences of the Early Identification System as a Universal Screener for Social, Emotional, and Behavioral Risks

Supplemental Material for Usability and Social Consequences of the Early Identification System as a Universal Screener for Social, Emotional, and Behavioral Risks

Using a one-minute screener to identify students at risk for academic difficulties

Multiple regression data from a teacher-completed one-minute rating scale (i.e., the Brief Behavior, Academic, and Social Screener; B-BASS) predicted academic risk of 198 third- through fifth-grade rural elementary students; teacher ratings across 16 B-BASS items within cognitive, social/emotional, socioeconomic/family, and executive functions domains predicted 81.4% and 66.2% of the variance in high-stakes End-of-Grade Tests of English Language Arts/Reading and Mathematics, respectively. The psychometric properties of this universal screener (e.g., test-retest reliabilities) are strong given its brief nature; consequently; the B-BASS is recommended for screening at-risk status in a number of academic and social areas. Implications, limitations, and specific recommendations are discussed.

Lexiland: A Tablet-based Universal Screener for Reading Difficulties in the School Context

Massive and timely screening of the student population for early signs of reading difficulties is needed to implement timely effective remediation of these difficulties. However, traditional approaches are costly and hard to apply. Here, we present Lexiland, a tablet-based reading assessment tool for kindergarten and primary school children developed to be applied in school settings with minimal personnel intervention. Following a story line, players help a character of the game perform several tasks that measure different predictors of reading outcomes. Most of the tasks that usually involve a verbal response were switched to receptive tasks to demand a touch-screen response only. The tablet application was administered to a sample of N = 616 5-yo kindergarten children and to a sub-sample of these children twice during the following two years (First and Second Grades). Applying logistic regression and cross-validation, we selected a reduced subset of tasks that can predict with great sensitivity and specificity, whether a five-year-old child will have reading difficulties by the end of first grade (sensitivity 90% and specificity 76%) and two years later (sensitivity 90% and specificity 61%). Importantly, Lexiland is a scalable tool to implement universal screening, given the increasing availability of devices able to run android and iOS applications.

Initial Development of a National Survey on Remote Learning in Early Childhood During COVID-19: Establishing Content Validity and Reporting Successes and Barriers.

This article describes the development and administration of a survey to identify early childhood educators’ successes and barriers when delivering remote instruction (e.g., online whole or small group instruction) during the COVID-19 pandemic to children 2–5 years old. The survey was developed using procedures outlined by the commonly accepted stages of an instrument development process. Content validity was established using four approaches: (a) identification of the purpose of the survey, (b) creation of a blueprint of items, (c) cognitive interviews, and (d) expert panel review. A total of 1,053 early childhood educators began the survey, with 808 (77%) of the responses included because educators met the inclusion criteria of working in the United States and responding to at least one question related to remote instruction. The survey contained 37 closed-ended and six open-ended items covering eight domains: (a) demographic information; (b) preparation, guidelines, and materials for remote learning; (c) caregiver communication and engagement; (d) assessment; (e) instruction; (f) educators’ levels of confidence before and after remote learning; (g) access to services (i.e., wraparound and/or special education); and (h) planning for the return to face-to-face instruction. Both quantitative (descriptive, t-test, regression, ANOVA, and Chi-square tests) and consensual qualitative research analyses were applied to summarize the survey results. Findings from this survey indicated that even with limited or no guidance from administrators, educators successfully adapted to remote instruction and their levels of confidence increased over time. Ongoing improvements need to be made to sustain regular communication with all families, to offer access to technology (i.e., devices and internet), to administer assessments or universal screeners, and to provide cohesive guidelines and expectations. Results from this study begin to shed light on early childhood educators’ adaptation to remote instruction as a result of COVID-19.Supplementary InformationThe online version contains supplementary material available at 10.1007/s10643-021-01216-y.

Predictive Validity of the Student Risk Screening Scale-Internalizing and Externalizing (SRSS-IE) Scores

Schools have become the most common setting for youth to receive additional supports—including access to mental health services serving as an entry point for identification and subsequent provision of these services. Many schools rely on office discipline referral (ODR) or suspension data as informal screeners for students who may need additional supports. Discipline data may be effective at identifying students with externalizing behaviors, yet students with internalizing behaviors may not engage in behaviors that warrant an ODR or suspension. As such, these students may go undetected in the absence of systematic screenings. This study explored whether a universal behavior screener identified students at risk for externalizing and internalizing concerns who may not have been identified through ODRs. The Student Risk Screening Scale for Internalizing/Externalizing (SRSS-IE) was completed for 1,201 elementary students in 3 elementary schools. SRSS-IE data were used to predict year-end ODRs. Results showed externalizing scores (SRSS-E7) on the SRSS-IE predicted year-end ODRs, with higher SRSS-E7 scores predicting with more ODRs. However, internalizing scores (SRSS-I5) on the SRSS-IE did not significantly predict year-end ODRs suggesting ODRs may not be an effective data source for identifying students with internalizing behaviors absent of externalizing behaviors.

Developing Early Reading Profiles for Latinx Kindergarten Students Using Typical Universal Screeners

The purpose of the current study was to determine students’ early reading profiles at the beginning and end of kindergarten using existing school-based measures with a sample of Spanish speaking, Latinx students. Using latent profile analysis, a multivariate analytic technique that empirically identifies profiles based on student responses to multiple measures, students’ performance was compiled using typical universal screening measures of reading, a formal measure of English language development, and information regarding referral for special education evaluation and grade retention. At the beginning of the year, students demonstrating greater English proficiency also had increased ability on initial sound fluency and letter naming fluency tasks in English. By the end of the year, students demonstrating continued deficits in early literacy skills (phoneme segmentation fluency, letter naming, and nonsense word fluency) included students referred for special education or grade level retention than students who demonstrated greater proficiency in these skill areas. Recommendations for research and school-based practices are provided.

Criterion Validity of a Computer Adaptive Universal Screener to an End-of-Year State Mathematics Assessment

ABSTRACTThe current study examined the criterion validity of a computer adaptive universal screener with an end-of-year state mathematics assessment using extant data provided by a local education agency. Participants included 1,195 third through eighth graders. Correlational analyses were used to report predictive and concurrent validity coefficients for the universal screener with the end-of-year state mathematics assessment. Moderation analyses were performed to examine the extent to which criterion validity estimates were differential for certain demographic variables. Criterion validity coefficients were above the 0.70 threshold suggested for universal screeners for 5 of 6 grades. The criterion validity of the universal screener to the state assessment was differential for many demographic groups: gender, special education status, free or reduced meals status, and section 504 status. Implications for future research and practice are discussed.

Bias or Empathy in Universal Screening? The Effect of Teacher–Student Racial Matching on Teacher Perceptions of Student Behavior

This study uses linear probability models with student and teacher fixed effects to assess whether the racial match between teachers and students affects “at-risk” ratings on a teacher-completed universal screener of student internalizing and externalizing behavior. The data are from a large, urban California school district. I find that Asian and Black teachers are more likely to rate their same-race students “at-risk” for internalizing behavior compared with how the same Asian and Black students would be rated by White teachers. These findings have implications for policy and practice aimed at enhancing universal screening for externalizing and internalizing behavior.

Interpreting Reports From Universal Screeners: Roadblocks, Solutions, and Implications for Designing Score Reports

Many teachers struggle with interpreting formative assessment data. As a result, they are often unable to modify instruction that is responsive to student performance. To support valid interpretations and uses of assessments, test developers need to better understand teachers’ ability to read and interpret score reports, their perception of the information that is being conveyed, and how they use the information to change instruction. The purpose of this manuscript is to report on a series of focus groups examining (a) how teachers interact with various score report features and (b) how these features support or hinder their ability to interpret and analyze formative assessment data. Results provide insights into the design of score reports that may facilitate teachers’ ability to interpret and analyze data and, ultimately, use these data to modify their instruction to improve student outcomes. Solutions to the roadblocks teachers encountered are proposed and generalized examples of how score reports can be designed to facilitate teachers’ interpretations are provided. Findings from this study may help test developers anticipate challenges the intended audience could face when interpreting assessment results on score reports, and subsequently improve the quality of score reports to enhance teachers’ interpretations and decision-making at the classroom level.

Teacher-rated school readiness items in a kindergarten sample: Outcomes in first grade.

The purpose of the study was to investigate the effectiveness of a brief and feasible universal screener for kindergarten readiness for predicting follow-up outcomes in first grade. Participants included kindergarten and first-grade teachers and students from 6 urban elementary schools. Students included slightly more males than females (54%; 46%), the race/ethnicity of the students was predominantly African American (74%) or White (23%), and 36% of students qualified for free or reduced lunch. Data included in this study were collected for children in the fall (October) of their kindergarten year and the Spring (April) of their first-grade year. Kindergarten teachers completed single-item ratings of student readiness as well as full-scale ratings of student behaviors and academic competence. Students also completed a standardized academic achievement test. Main findings from this study include that the single-item screener continued to predict negative academic and social behavior outcomes at a follow-up 18 months later. The percentage of unique variance explained by the readiness items was significant and, in the majority of cases, remained significant even after the baseline variables were entered into the analyses. Children rated poor in their academic readiness were over 9-10 times as likely to have low reading performance at the end of their first-grade year. Furthermore, children rated as poor in their behavior readiness were at over 6 times greater risk for low prosocial and high disruptive behavior teacher ratings in first grade. The screener could be used to screen for children low in readiness in order to provide supports and monitoring for early intervention. Kindergarten entry is a critical time to screen children and support them in developing skills to ameliorate deficits and build essential early academic and social behavior skills. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Assessing the Severity of Concerns About Preschool Children’s Self-Regulation of Attention, Behavior, and Emotions Using the ABLE Universal Screener: A Rasch Analysis

Discerning the seriousness of socioemotional symptoms in young children can be difficult. To address this issue, the Attention, Behavior, Language, and Emotions (ABLE) universal behavioral health screener uses a rubric for severity that includes indicators of problem duration, impairment, generalizability, exacerbation, persistence, peer comparison, and need for professional intervention. This report examines whether this rubric can be treated as a unidimensional scale across respondents and problem types and streamlined for more efficient screening. Head Start teachers and parents ( N = 2,643) reported socioemotional, attentional, behavioral, and language problems. Rasch analyses confirmed that the ABLE severity scale is unidimensional but the coherence of severity indicators differed by respondents. Consequently, greater precision in establishing clinical significance can be achieved by weighting severity items differentially for teachers and parents.

Reconsidering the Psychometrics of the GRS-S: Evidence for Parsimony in Measurement

The authors evaluated measurement-level, factor-level, item-level, and scale-level revisions to the Gifted Rating Scales–School Form (GRS-S). Measurement-level considerations tested the extent to which treating the Likert-type scale rating as categorical or continuous produced different fit across unidimensional, correlated trait, and bifactor latent factor structures. Item- and scale-level analyses demonstrated that the GRS-S could be reduced from a 72-item assessment on a 9-point rating scale down to a 30-item assessment on a 3-point rating scale. Reliability from the reduced assessment was high (ω > .95). Receiver operating characteristic (ROC) curve comparisons between the original and reduced versions of the GRS-S showed that diagnostic accuracy (i.e., area under the curve) of the scales was comparable when considering cut scores of 120, 125, and 130 on the WISC-IV Full Scale ( Wechsler Intelligence Scale for Child–Fourth Edition) and verbal IQ and the WIAT-III (Wechsler Individual Achievement Test–Third Edition) composite score. The findings suggest that a brief form of the GRS-S can be used as a universal or selective screener for giftedness without sacrificing key psychometric considerations.