Criterion-referenced Instruments Research Articles

Basic Motor Skills of Children With Down Syndrome: Creating a Motor Growth Curve.

To create a motor growth curve based on the Test of Basic Motor Skills for Children with Down Syndrome (BMS) and estimate the age of achieving BMS milestones. A multilevel exponential model was applied to create a motor growth curve based on BMS data from 119 children with Down syndrome (DS) aged 2 months to 5 years. Logistic regression was applied to estimate the 50% probability of achieving BMS milestones. The BMS growth curve had the largest increase during infancy with smaller increases as children approached the predicted maximum score. The age at which children with DS have a 50% probability of achieving the milestone sitting was 22 months, for crawling 25 months, and for walking 38 months. The creation of a BMS growth curve provides a standardization of the gross motor development of children with DS. Physical therapists then may monitor a child's individual progress and improve clinical decisions.

Incorporating oral language assessment into MTSS/RTI frameworks: The potential of personal narrative assessment

ABSTRACT In Multi-tiered System of Support (MTSS) and Response to Intervention (RTI) models, the reading achievement of English Learners (ELs) is assessed multiple times during the school year to monitor students’ progress toward meeting grade-level reading standards. Most MTSS/RTI frameworks do not include a comparable system for assessing oral language development. The English Learner Oral Narrative Scale (ELONS) was developed as an informal, criterion-referenced instrument to assess the personal narrative skills of ELs (i.e., skills in recounting personal experiences). The ELONS was piloted with 28 third grade ELs in a transitional bilingual education program to describe participants’ personal narrative skills and to explore relationships between narrative skills and reading achievement. Mean scores were at the Advanced level for English global narrative proficiency and vocabulary, grammar, and fluency; mean ratings for oral comprehension and pronunciation were at the Advanced High level. Twelve of 28 ELs produced Spanish narratives of sufficient length to score. Mean global and subskill scores were at the Advanced level except for fluency (Intermediate level). ELONS global proficiency scores showed moderate correlations to state assessments of English language proficiency and English reading. Implications for oral language assessment in MTSS/RTI frameworks and future research directions are discussed.

Teachers' screening estimations of speech-language impairments in primary school children in Nepal.

The prevalence of speech-language impairments in children have been estimated for several languages, primarily in developed countries. However, prevalence data is lacking for developing countries, such as Nepal. (1) To obtain teacher estimates of incidence and overall prevalence of speech-language impairments and its subtypes as a function of gender, age and grade level; and (2) to validate the screening instrument on Nepalese children. The adapted teachers' screening instrument, namely adapted Teachers' Speech and Language Referral Checklist (a-TSLRC), was administered in 2776 (690 and 2086) primary school children aged 5;00-11;11 years (mean = 8;11 years). The screening was conducted at four different points in time, i.e. Incidences I and II, and each incidence consisted of a testing and a retesting phase. Prior to this, teachers were trained in forum meetings, and an information sheet containing an overview of speech-language impairments, and guidelines/criteria for marking the occurrence of speech-language impairments in the TSLRC were disseminated. Overall prevalence of speech-language impairments in children was estimated as 8.11%. Specifically, overall speech problems were estimated as 4.68%, and language problems as 8.0%. Additionally, the prevalence by subtypes of speech-language impairments as categorized in the TSLRC were reported to be 2.95% for an articulation/phonological problems, 2.09% for stuttering, 3.42% for a voice problems, 4.97% for a receptive language problems and 7.74% for an expressive language problems. The internal consistency among items was sufficient and a good intra-rater reliability was obtained. The study indicates that the overall results of speech-language impairments in children via the adapted in-Nepalese criterion-referenced instrument are supported by international studies. In addition, justifiable reliability and validity was obtained. Therefore, based on these overall evidence, this instrument can be useful for the screening of speech-language impairments in primary school children in Nepal.

Development and Psychometric Testing of the Criterion-Referenced Measurement Tool for Genetics.

The purpose of this study was to develop a criterion-referenced instrument to provide effective documentation of knowledge of advanced practice nursing in genetics. This is a methodological psychometric study. The Criterion-Referenced Measurement Tool for Genetics (CRMTG) was found to be valid with a content validity index of .88 and reliable with a Cronbach's alpha of .99 based on the findings of this study. The CRMTG pass rates indicated a significant difference between nurses who practice in a genetic setting and nurses who do not practice in a genetic setting (p < .001). The CRMTG was found to be a valid and reliable instrument to provide effective documentation of knowledge of advanced practice nursing in genetics.

Descriptive and criterion-referenced self-assessment with L2 readers

Brantmeier [Brantmeier, C., 2006. Advanced L2 learners and reading placement: self-assessment, computer-based testing, and subsequent performance. System 34 (1), 15–35] found that self-assessment (SA) of second language (L2) reading ability is not an accurate predictor for computer-based testing or subsequent classroom performance. With 359 advanced L2 university-level learners, the present study attempts to provide new empirical evidence concerning the use of SA, when measured via criterion-referenced items, as a predictor of reading scores on a computer-based placement test and subsequent reading achievement. Before reading, participants completed a contextualized, criterion-referenced instrument for SA that was created from the European Language Portfolio. A four-item contextualized SA questionnaire was also completed after reading. Participants completed the following: self-assessment questionnaire taken prior to computer-based test, computer-based test of 8 readings, 2 in-class readings, written recall, multiple-choice, topic familiarity questionnaire, and post-reading SA questionnaire. Findings indicated that a descriptive and criterion-referenced questionnaire for SA may be a reliable predictor of both reading scores on a computer-based test and subsequent classroom performance when comprehension is measured via sentence completion and multiple choice items. The criterion-referenced questionnaire did not prove to be a reliable predictor of reading comprehension with the written recall task for the short story. The post-reading SA instrument proved reliable for all three comprehension tasks. Results hold important implications for both language placement directors and test constructors.

The Office of Special Education Program's LD Initiative: A Context for Inquiry and Consensus

Memo: Problems in Assessment of Children [T]he fundamental problem in the assessment of the learning disabled is making the initial determination that a child is or is not learning disabled. This appears ... critical for at least two reasons. First, from a service point of view, an adequate diagnostic procedure assures us that appropriate children are being served. Second, from an administrative point of view, one can be assured when one speaks of learning-disabled children, one is referring to an identifiable group. Obviously, people are currently classifying children as with a variety of approaches. [S]ome of these approaches suffer from such conceptual problems that their validity must be questioned. Most ... suffer from sufficient technical problems such as to make them less than adequate. Just as important as the initial decision that a learning disability exists, however, is the determination of appropriate programming. There is little reason for educators to be concerned about whether children are learning disabled unless it somehow relates to the education program that children receive. Decisions concerning appropriate programming for children would undoubtedly [be] best made using criterion-referenced instruments. At this point, [they] are not available. The requirements in PL 94-142 for Protection in Evaluation Procedures, Least Restrictive Environments, and Individual Education Programs make the whole decision-making process for classification, placement and programming exceedingly complex. A critical concern ... is ... [how] assessment information relates to and is integrated with decision-making. Many people have voiced concern over the critical nature of early identification of the learning disabled. I ... [agree] that this is extremely important. However ... we cannot hope to engage in early identification with all of its attendant difficulties unless and until we have resolved our definitional and operational problems. ********** The U.S. Office of Special Education Programs (OSEP) recently relocated their offices. Among the historical treasures found in the move was a carbon copy of a letter written by then junior staffer Lou Danielson to a university professor. Although Lou's letter was written in 1977, 27 years ago, it could well have been composed today. Indeed, points nearly identical to Lou's were made in a letter written during the 1997 Individuals with Disabilities Education Act (IDEA) reauthorization process by the National Joint Committee on Learning Disabilities (NJCLD). OSEP's comprehensive response to the NJCLD letter has become known as the LD a multiyear process involving many activities and numerous stakeholders intended to probe, document, and discuss what we know about the identification and classification of children with learning disabilities (LD). Below, we describe OSEP's Initiative, including the creation of the National Research Center on LD. Background In 1975, Congress passed the Education of All Handicapped Children Act (Public Law 94-142), now known as the IDEA. A central intent of the law was to guarantee a free appropriate public education for all children with disabilities. This would be accomplished by supporting states and localities to improve child outcomes by protecting the rights, and meeting the individual needs, of all children and youth with disabilities and their families. Early and accurate identification of children with disabilities was viewed as critical to ensuring that they receive services essential to their success. Much progress has been made over the past three decades in providing effective programming, special education, and other related services to children with disabilities. In 1997, the reauthorization of IDEA changed the focus of the law from providing access to services to improving results and accountability. Essential to achieving improved results is the accurate and early identification of children with disabilities. …

The development of the School Function Assessment Chinese version for cross-cultural use in Taiwan.

The primary purpose of this research project was to translate and adapt the School Function Assessment (SFA), a standardized criterion-referenced instrument that measures school-related functional skills, for its cross-cultural use in Taiwan. The project consisted of four study phases: translation, cultural adaptation, pilot testing, and field testing for standardization. A series of rigorous procedures including the method of translation and back-translation, team consensus for cultural adaptation, and Rasch modelling techniques were used to address various dimensions of cross-cultural equivalence and psychometric properties of the translated SFA. The protocols that were developed as well as technical issues that were addressed in this project provide useful guidelines for international occupational therapists who are interested in translating and adapting instruments for cross-cultural use.

Measuring Functional Independence of Students with Severe Disabilities: Issues and Methods

This review highlights issues in functional outcome assessment and describes educationally based functional outcome measures applicable to students with severe disabilities. The demand for increased inclusion of students with disabilities in general education settings has resulted in a need for assessments to identify students' capabilities to meet the functional as well as academic demands of school. Students with disabilities often have difficulties at school because of physical, cognitive, behavioral, social, and emotional impairments that interfere with their ability to participate fully in learning activities. Specific norm-referenced and criterion-referenced instruments are reviewed for their usefulness in describing functional independence and functional challenges across educational settings.

Infants and toddlers with disabilities: relationship-based approaches.

The passage of the Education of the Handicapped Act Amendments of 1986 (P.L. 99-457) heralded a new era for early intervention for children with disabilities (De Gangi, Royeen, & Wietlisbach, 1992; Hepp, 1991; Roberts, 1991; Rossetti, 1990; Shonkoff & Meisels, 1990; Trivette, Dunst, Deal, Hamer, & Propst, 1990). This act, renamed the Individuals with Disabilities Education Act (IDEA) (P.L. 101-476) in 1990, was amended again in 1997. The section in the legislation that addresses programs and services offered to infants and toddlers (Part C in the newest reauthorization, P. L. 105-17), emphasizes the importance of the family in the intervention process and requires the involvement and collaboration of multiple disciplines and agencies in meeting the needs of families with young children. The way in which services are delivered and the philosophical and theoretical approaches that underlie service delivery have continued to evolve during the past 15 years. This article discusses current and emerging early inter vention practice in the light of its evolution during the past decade. For many of the disciplines, a change to a radically new approach to service delivery challenges the training that they received. Understanding the evolution of the approach to services to young children will help as social workers collaborate with professionals whose training may have emphasized a different understanding of service delivery. For the past 15 years, the majority of the intervention approaches used by programs serving young children focused on provision of developmental activities and therapies and services requested by the parents. Traditional approaches to early intervention used a discipline-based, normative perspective in assessment and intervention with a heavy focus on developmental milestones. Special education and therapeutic approaches grew from a medical model that was prescriptive, focusing on and disability of the child. The desired outcome of these approaches was an increase in the number of developmental skills and milestones represented on norm-referenced and criterion-referenced instruments. Service providers each focused on their own discipline, acting as experts who assessed, developed, and prescribed interventions for children. Parents and families functioned as recipients of expert information and services. They might be involved in carrying out some recommendations--and indeed would be considered non compliant if they did not follow through on recommendations--but theirs was a more passive role in the traditional model. With the advent of family-centered care, the role of the family in children's development received more attention. Family goals and needs are solicited more actively in assessment and in the development of the individual family service plan mandated for infants and toddlers. The developmental perspective continues to be organized around developmental milestones with more emphasis on social--emotional development and the addition of consideration of cultural influences on development. Child deficits with some consideration for child and family strengths continue to be a central focus of many professional recommendations. Often, provision of services has been cross-disciplinary, meaning that there has been some sharing of perspectives across disciplines. Although parental involvement in the assessment and intervention process is mandated by IDEA, parental input on assessment continues to be viewed skeptically by many professionals (Sexton, Thompson, Perez, & Rheams, 1990), and as yet parents are not widely acce pted as the experts on their child and collaborators in the development of programs. Outcomes for service have expanded to include parental satisfaction, decreases in family stressors, and increases in family resources and supports. The parental role involves more participation in problem solving and in program development within controlled choices (that is, parents may choose from among the options offered by the agencies involved). …

Evaluating Planned Change Theories

A criterion-referenced instrument was developed for determining to what extent a planned change theory meets predetermined criteria of significance, agreement with nursing's perspective, clarity and consistency, economy, generality, practicality and testability. These criteria form the basis of a quantitative instrument composed of 31 evaluation questions to determine a planned change theory's suitability for nursing situations. This instrument combines well with qualitative evaluations of theory goodness and with change theorists' ratings of their own theories. Psychometric evaluation of the instrument is in progress.

Reliability and validity of the WATSMART three-dimensional optoelectric motion analysis system.

Reliability and validity of the WATSMART (Waterloo Spatial Motion Analysis Recording Technique) system was evaluated under static and dynamic conditions. In experiment 1, infrared light-emitting diodes (IREDs) were placed at the axis and along the arms of a clinical goniometer. Twelve angles in 5-degree increments were each recorded 10 times at each of three spatial locations. Reliability was assessed using intraclass correlation coefficients (ICCs) and analysis-of-variance procedures to determine within-trial variability. The ICCs for all spatial locations exceeded .99. The 95% confidence interval for each angle was less than 0.5 degree in all cases. Criterion-referenced instrument validity was assessed with regression analysis. Slopes of the regression of reconstructed angle on reference angle were close to unity for each spatial location. A systematic error, however, that increased as the goniometer was rotated 45 degrees away from the cameras was evident. In experiment 2, a robotic arm was fitted with four IREDs and made to repeat a defined movement trajectory 10 times at each of three spatial locations. The ICCs for portions of each trajectory ranged from .20 to .99. The results show that reliable and valid results can be obtained from this motion analysis system if adequate precautions are taken to reduce unwanted light reflections. Reliability and validity decreased somewhat as the object was rotated further away from the plane in which the cameras were mounted.

Issues Related to Designing and Conducting School Health Education Research

Abstract Investigators interested in conducting school health research face many important challenges. First, an appropriate research course for school health must be charted so the most important research issues are addressed in a systematic way. Second, there is a continuing need to develop scientifically sound research methods that can be used in the school setting. Third, there is the immediate need to identify ways of overcoming the usual problems encountered in the conduct of school health research. This paper focuses on the third challenge, via an analysis of the research design and measurement issues that most frequently confront school health researchers. The research design issues addressed include randomization of experimental units to treatments and selection of the appropriate statistical unit of analysis. The measurement issues addressed included use of existing versus newly developed instruments, use of norm-referenced versus criterion-referenced instruments, use of self-report techniques, ...

Issues Related to Designing and Conducting School Health Education Research

Investigators interested in conducting school health research face many important challenges. First, an appropriate research course for school health must be charted so the most important research issues are addressed in a systematic way. Second, there is a continuing need to develop scientifically sound research methods that can be used in the school setting. Third, there is the immediate need to identify ways of overcoming the usual problems encountered in the conduct of school health research. This paper focuses on the third challenge, via an analysis of the research design and measurement issues that most frequently confront school health researchers. The research design issues addressed include randomization of experimental units to treatments and selection of the appropriate statistical unit of analysis. The measurement issues addressed included use of existing versus newly developed instruments, use of norm-referenced versus criterion-referenced instruments, use of self-report techniques, and appropriate use of affective instruments. Following an analysis of the issues a series of relevant questions are posed.

A nutrition knowledge test for nutrition educators

This paper describes the development of a criterion-referenced instrument for assessing nutrition knowledge of potential nutrition educators. The test consists of 50 multiple-choice items. It addresses 3 content areas—basic nutrition principles, sources of nutrients, and function of nutrients; and it measures 4 cognitive levels—recall, comprehension, application, and analysis. The results of an administration of the test to 576 college graduates demonstrate that the instrument discriminates between groups of professionals who are trained in nutrition and those who are not.

Assessment of multiply handicapped deaf students for program development.

Comprehensive educational assessment underlies the development of accurate Individualized Education Programs and instructional plans for multiply handicapped students, yet it is problematic for many reasons. The purpose of this article is to suggest guidelines for integrating assessment data into the development of prescriptive programs. These include: (a) administer criterion-referenced instrument; (b) view the categorical or domain areas of instruments as potential goal statements; (c) view the sub-categories or subdomains as potential long-term objectives; (d) view the items of the sub-categories as potential short-term objectives; (e) select sub-categories to serve as long-term objectives, and items to serve as short-term objectives, based on certain priority concerns for multiply handicapped students; (f) convert the selected sub-categories and items to behavioral statements representing long- and short-term pupil objectives.

SEVTC Assessment Scales for Community Integration: Development and Validation of the Post-Entry Scales

Currently, there is considerable interest among professionals in the field of mental retardation in developing and expanding community programs as alternatives to institutional placement for individuals with mental retardation. This paper describes the development of a second of a series of three criterion-referenced instruments designed to facilitate community integration of persons with moderate-profound mental retardation. Empirical data demonstrating reliability and validity of the instrument are presented. The results are discussed in terms of a number of assessment-related applications which are of immediate relevance to individuals undergoing training for community integration.

Published Tests and the Needs of Educational Accountability

A match was made between consensual indexes of elementary education goal priorities and the availability of published tests that held promise for their assessment. A Q sort was utilized to obtain priority ratings on 106 pre-established goals from a nationwide sample. Independently, published tests for the elementary level were sorted into the same 106 goal categories. Results of the matching techniques indicated that most goals, whether high or low in priority, are measured by few or no published tests. This failure on the part of test publishers to respond to the needs of education is seen as an important contributor to the current popularity of criterion-referenced instruments.