Stability Of Test Scores Research Articles

(In)Stability of Test Scores

Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school’s control. This study examined year to year changes in school level results on standardized tests delivered in Ontario, Canada. G-theory analyses found that test scores are not stable enough for meaningful conclusions to be made based on year to year changes in school level results. For small and medium sized schools, years of data need to be collected before defensible decisions can be made about trends in test scores. The authors introduce a ‘bounce’ statistic that provides a simple, easy to interpret measure of test score stability.

بناء اختبار محکي المرجع لقياس الکفايات المهنية لدى الأخصائيين النفسيين في مدينة الرياض

هدفت الداراسة الحالية إلى بناء اختبار محکي المرجع لقياس الکفايات المهنية لدى الأخصائيين النفسيين في مدينة الرياض، والتحقق من صدق درجات الاختبار )الصدق الوصفي وصدق انتقاء النطاق السلوکي(، وکذلک التحقق من ثبات درجات الاختبار وفقا لمعامل ليفنجستون، بالإضافة إلى تحديد درجة القطع المناسبة لکل کفاية مهنية، والتي من خلالها يتم تصنيف الأخصائيين النفسيين إلى متقنين وغير متقنين لهذه الکفايات، والتعرف على مدى تمکن الأخصائيين النفسيين في مدينة الرياض من الکفايات المهنية التي تضمنها الاختبار الذي تم بناؤه. وتم استخدام المنهج المسحي )الوصفي(، کونه المنهج المناسب لتحقيق أهداف البحث، وتکونت عينة الدارسة من ( 134 ) أخصائيا نفسيا ذکورا- وإنا ثا، وقد قامت الباحثة ببناء الاختبار مرورا بأربع مراحل هي(مرحلة التحليل - مرحلة البناء- مرحلة التجريب- مرحلة الإخراج )، وبعد المرور بهذه الخطوات لبناء الاختبار أنتجت الدراسة اختبارا محکي المرجع لقياس الکفايات المهنية لدى الأخصائيين النفسيين في مدينة الرياض يتکون من (61) مفردة اختبارية، تقيس تسع کفايات مهنية هي )الکفاية المعرفية: بنظريات علم النفس والتعلم ونظريات الإرشاد، والکفاية الشخصية: القيم والأخلاقيات والتواصل، وکفاية استخدام أدوات القياس والتحقق من خصائصها وتحليل نتائجها، وکفاية مناهج البحث في علم النفس، وکفاية إدارة المقابلة التوجيهية والعلاقة مع العميل، وکفاية القدرة على تطوير الذات والنمو المهني، وکفاية الوعي بالثقافة الاجتماعية المحلية والدولية وسعة الإطلاع، وکفاية التعريف بأساليب العلاج النفسي، وکفاية التخطيط للبرامج وکتابة التقاريروإتخاذ القرار )، کما توصلت الدراسة إلى أن هذا الاختبار يتمتع بصدق وصفي وصدق نطاق سلوکي جيد، وتبين أن الاختبار يتمتع بثبات عال جدا وفقا لمعامل ليفنجستون وصل إلى درجة 0.98 ) ) للاختبار ککل، وبلغت درجة القطع للاختبار وفقا لطريقة أنجوف(41.49)، واتضح أن هناک تدني واضح في الکفايات المهنية لدى الأخصائيين النفسيين في مدينة الرياض. الکلمات المفتاحية : اختبار محکي المرجع - الکفايات المهنية - الأخصائيين النفسيين. Abstract The Present study aimed to construct a criterion – Referenced to measure the professional competence for the psychologists in Al Riyadh and check the veracity of the test scores ( honesty descriptive and ratified the selection of scale behavioral ) as well as checking the stability of test scores according to the coefficient of Livingstone as well as to determine the cut-off grade appropriate both professional and adequacy of which the psychologists are classified to professional and is unprofessional for this caliber and to identify the extent to which psychologists in Riyadh of professional competence which is included in the test that was built . Curriculum survey was used (descriptive), being a curriculum appropriate to achieve the objectives of the research, the study sample consisted of (134) psychologists - male and female - have the researcher to build the test through the four stages are: (analysis phase - the construction phase - experimental stage - the stage output). After applying these steps to build the test produced a study criterion – Referenced test to measure the professional competence to the psychologists in the city of Riyadh consists of (61) Single test measures the nine competencies professional is ( enough knowledge definition theories of psychology and learning theories and guidance , and enough personal values and ethics and communication the adequacy of the use of measurement tools and verification of characteristics and analysis of their results, the adequacy of research methods in psychology, the adequacy of management of the corresponding guidelines and the relationship with the client, the adequacy of the ability to self-development and professional growth, insufficient awareness of the culture of local social international and erudition inadequate

The implementation of grassroots product-oriented social innovations by non-governmental organisations: proposal of a measurement tool

The aim of this article is to present a tool for measuring the implementation of grassroots product-oriented social innovations by non-governmental organisations (NGOs). This tool will enable a quantitative assessment of the activities of NGOs in regard to the implementation of these innovations as well as a comparison of various areas based on the number of organisations which implement these innovations. The validation of the tool was performed during studies conducted in 2017 on a random sample of NGOs located in Łódź (a city in central Poland). The studies resulted in the establishment of a two-factor structure for the Index of Implementation of Grassroots Product-Oriented Social Innovation. The tool is characterised by satisfactory psychometric properties in regard to factor validity, the internal consistency of each subscale and the stability of test scores over time.

Validation of an Expressive and Receptive Tablet Assessment of Early Literacy

Touch-screen tablets are used in the classroom for assessment. Little is known about the psychometric properties of tablet-based assessments. This study examined the validity and reliability of an expressive and receptive assessment app designed to measure literacy skills. Children (N = 45; 3–5 years) completed the app assessments for alphabet and word skills using a tablet. Children (n = 15) were posttested 1 to 2 weeks later to check test score stability. The tablet measures showed good internal consistency (all α > .90) and test–retest reliability (intraclass correlation coefficient [ICC] range = .87 to .98). Agreement between scores for the tablet- and paper-based tests was also high (ICC range = .91 to .98). This indicates that expressive and receptive tablet tests provide valid and reliable measures.

Validation of a touch screen tablet assessment of early literacy skills and a comparison with a traditional paper-based assessment

ABSTRACTTouch screen tablets are being increasingly used in schools for learning and assessment. However, the validity and reliability of assessments delivered via tablets are largely unknown. The present study tested the psychometric properties of a tablet-based app designed to measure early literacy skills. Tablet-based tests were also compared with traditional paper-based tests. Children aged 2–6 years (N = 99) completed receptive tests delivered via a tablet for letter, word, and numeral skills. The same skills were tested with a traditional paper-based test that used an expressive response format. Children (n = 35) were post-tested 8 weeks later to examine the stability of test scores over time. The tablet test scores showed high internal consistency (all α’s > .94), acceptable test-retest reliability (ICC range = .39–.89), and were correlated with child age, family SES, and home literacy teaching to indicate good predictive validity. The agreement between scores for the tablet and traditional tests was high (ICC range = .81–.94). The tablet tests provides valid and reliable measures of children’s early literacy skills. The strong psychometric properties and ease of use suggests that tablet-based tests of literacy skills have the potential to improve assessment practices for research purposes and classroom use.

THE EFFECTS OF COGNITIVE MATURITY ON CONCUSSION ASSESSMENT IN LOW SOCIOECONOMIC URBAN HIGH SCHOOL ATHLETES

BackgroundImplementing concussion assessment programs for youth athletes are necessary to manage clinical care in sports. However, confounding factors such as cognitive maturity and socioeconomic status may complicate clinical interpretation of...

Validation of the Narcissistic Personality Questionnaire for Children—Revised among U.S. students.

In the present studies, the Narcissistic Personality Questionnaire for Children-Revised (NPQC-R), which was previously validated with 7th- and 9th-grade students in Singapore, was examined with a sample of U.S. students. The NPQC-R is a self-report measure and consists of 2 (Exploitativeness and Superiority) factors. In Study 1, a confirmatory factor analysis (CFA) was performed on the responses of 275 U.S. students in Grades 4-8. The results of the CFA indicated that a 2-factor model provided the best fit to the data. In addition, the NPQC-R was found to be invariant across U.S. students in Grades 4-6 and 7-8. In Study 2, support for the convergent and discriminant validity and test score stability of the NPQC-R scores was found among a sample of 112 U.S. students. Implications of the findings of the studies for mental health professionals who work with children and adolescents in clinical and school settings are discussed.

Utility of Gene Expression Profiling Test Score Stability To Predict Future Clinical Events in Heart Transplant Recipients

Purpose Gene-expression profiling (GEP) test scores have primarily been used to identify heart transplant recipients at low risk of rejection at the time of surveillance testing. We hypothesized that the stability of two or more intra-patient GEP test scores may also predict a lower risk of future allograft dysfunction or death. Methods and Materials Patients from the IMAGE study population with at least two surveillance GEP test scores were selected for this cohort study. GEP tests were performed at 3, 4, or 6 month intervals. Intra-patient GEP score stability was defined as the standard deviation of an individual’s cumulative test scores. GEP score (range 0-39), GEP score ≥34, and GEP score stability were studied in multivariate Cox regression models to predict future clinical events. Results In the cohort of 369 patients, race, age at transplantation, and time post-transplantation were significantly associated with future events. In the multivariate models, the GEP score stability gave a significant incremental accuracy in predicting future clinical events (p-value Figure 1 , where the upward shift of stability in event patients can be seen. The hazard ratio for a 1 unit increase in GEP score stability was 1.76 (1.4-2.3, 95% CI).[ figure1 ] Conclusions The stability of intra-individual GEP scores over time may enable personalized prediction of future risk of allograft dysfunction or death and may provide prognostic information beyond that rendered from individual GEP test scores. (ClinicalTrials.gov NCT00351559).

Examination of the Psychometric Properties of the Environmental School Transition Anxiety Scale

A pilot study was conducted to examine the psychometric properties of the Environmental School Transition Anxiety Scale (E-STAS) with a sample of 220 fourth- to sixth-grade students who were about to or had completed their school transition. The results of an exploratory factor analysis (EFA) of the students’ responses on the E-STAS produced a two-factor (Academic and School Organization) structure. The E-STAS scores demonstrated adequate internal consistency reliability and 1- to 3-week test score stability. Gender differences were also found on the E-STAS, with females outscoring males. In addition, the results of correlational analyses provided support for the convergent and discriminant validity of the E-STAS scores. Implications of the findings for school professionals are discussed.

Psychometric Analyses of the Test Anxiety Scale for Elementary Students (TAS-E) Scores among Singapore Primary School Students

The psychometric properties of the Test Anxiety Scale for Elementary Students (TAS-E) scores were examined among Singapore primary school students. In Study 1, an exploratory factor analysis (EFA) was performed to determine the factor structure of the TAS-E in a sample of 540 Singapore students. In Study 2, a confirmatory factor analysis (CFA) was performed on the TAS-E scores in another sample of 540 Singapore students to determine whether the findings would support the factor structure reported in Study 1. The results of the EFA and Schmid-Leiman transformation in Study 1 and the CFA in Study 2 suggest that the TAS-E has four factors (Physiological Hyperarousal, Social Concerns, Task Irrelevant Behavior, and Worry) and a higher-order factor, the Total Test Anxiety factor. These findings are similar to the results reported in validation studies of the TAS-E scores with U.S. elementary students. In addition, the test score stability and convergent and discriminant validity of the TAS-E scores were examined in Study 3 among 1,080 Singapore primary school students. The results indicated that the TAS-E scores appear to have adequate test score stability over a 2-week test-retest period. Evidence supporting the convergent and discriminant validity of the TAS-E scores was also found. Implications of the findings of the three studies are discussed.

Examination of the Psychometric Properties of the Test Anxiety Scale for Elementary Students (TAS-E) Scores

The psychometric properties of the Test Anxiety Scale for Elementary Students (TAS-E) scores were examined. In Study 1, an exploratory factor analysis (EFA) was performed on the responses of 997 students in Grades 2 to 6 on the TAS-E. The results of the EFA produced a four-factor solution: Physiological Hyperarousal, Social Concerns, Task Irrelevant Behavior, and Worry. Gender differences were also found on each factor, except the Social Concerns factor. In Study 2, a confirmatory factor analysis (CFA) was performed on the TAS-E scores with a sample of 421 elementary students. The CFA results provided support for the four-factor solution in Study 1. The results also indicated that the TAS-E scores had adequate internal consistency reliability and test score stability. Evidence for convergent and discriminant validity of the TAS-E scores was found.

Reliability and Validity Evidence of the Chinese Piers-Harris Children’s Self-Concept Scale Scores Among Taiwanese Children

The Piers—Harris Children’s Self-Concept Scale—Second Edition (Piers—Harris 2) was designed to measure self-concept among children and adolescents. This study aimed to assess the reliability and validity of the scores of the Chinese version of the Piers—Harris 2 (Chinese Piers—Harris). The Chinese Piers—Harris 2 was administered to 243 Taiwanese students in Grades 1 to 4. The results of factor analysis indicated a six-factor promax solution was the most appropriate solution for the Chinese Piers—Harris 2. The results also revealed acceptable internal consistency and test score stability of the Chinese Piers—Harris 2 scores. In addition, the scores of the Chinese Piers—Harris 2 had significant correlations with the scores of the Elementary School Children’s Self-Concept Scale and the Behavioral and Emotional Rating Scale. Overall, the findings indicate that the Chinese Piers—Harris 2 scores have good psychometric properties for a sample of Taiwanese children.

Test Score Stability and the Relationship of Adult Manifest Anxiety Scale-College Version Scores to External Variables Among Graduate Students

A sample of 79 individuals participated in the present study to evaluate the test score stability (8-week test-retest interval) and construct validity of the scores of the Adult Manifest Anxiety Scale-College Version, a new measure used to assess anxiety in college students, for application to graduate-level students. Results of the study indicated for a sample of graduate students that the AMAS-C test scores have adequate to good temporal stability ( rs = .70 to .87). Evidence supporting the construct validity of the AMAS-C test scores was found. Validity coefficients ranged from -.67 to .88. Implications of the findings are discussed.

Development and Evaluation of the Listening in Spatialized Noise Test

The goal of this study was to design and develop an audiological test that provides an ecologically valid measure of speech understanding in background noise while minimizing the effects of between-listener variation in factors such as linguistic skills and attention on test performance. The Listening in Spatialized Noise Test (LISN) creates a three-dimensional auditory environment under headphones and was designed to be totally software driven, so that it can be delivered in any audiology clinic with the use of only a PC and an audiometer. The extent to which the LISN was able to simulate free-field conditions and the effect of learning on the test were also examined. : In a three-alternative forced choice adaptive procedure, 20 adults with normal hearing were required to indicate the intelligibility level of target continuous discourse presented at 0 degrees azimuth in the presence of distracter sentences simultaneously presented at either 0 degrees azimuth (0 degrees condition) or at both +90 degrees and -90 degrees azimuth (+/-90 degrees condition). The target story was always spoken by female 1, whereas there were three conditions of speaker for the distracter sentences: the "same female speaker" as the target (same voice condition); two "different female speakers" (different female voices condition); and a "male speaker" (male voice condition). In a separate study, 16 adults with normal hearing who had not participated in the first study were assessed on the same voice and different female voices conditions, which were presented and then retested in the same order and test session to determine the effect of practice on performance on the LISN. The 20 adults were able to understand the target story at a significantly lower threshold in the +/-90 degrees condition than the 0 degrees condition. The degree of this spatial separation advantage (SSA) decreased significantly as the vocal quality of speakers of the target and the distracter sentences became more different (10.4 dB in the same voice condition, compared with 5.6 dB in the different female voices condition, and only 3.3 dB in the male voice condition). The SSA for the different female voices and male voice conditions were comparable to measurements previously reported in a free-field environment. There was no significant difference in SSA between the first and second presentations for either the same voice condition (at 10.3 dB and 10.2 dB) or the different female voices condition (at 4.7 and 5.7 dB). For adults with normal hearing, the ability to comprehend the story in the separate condition was facilitated by the use of binaural cues, such as interaural time differences, to distinguish the target from the spatially separated distracters. When a target and masker are distinguishable on the basis of features of the various speakers' voices (such as large differences in fundamental frequency), listeners are less reliant on spatial cues to recognize the target, and the SSA in dB is reduced. The stability of test scores with practice, the comparable levels of performance to those achieved in free-field environments, and the ability of the test to utilize difference scores to assess binaural processing while minimizing differences between participants in variables such as linguistic skills make the LISN a potentially valuable tool for assessing auditory processing disorders.

Test Score Stability and Construct Validity of the Adult Manifest Anxiety Scale-College Version Scores Among College Students: A Brief Report

The authors investigated the temporal stability and construct validity of the Adult Manifest Anxiety Scale-College Version (AMAS-C; C. R. Reynolds, B. O. Richmond, & P. A. Lowe. 2003b) scores Results indicated that the AMAS-C scores had adequate to excellent test score stability, and evidence supported the construct validity of the AMAS-C test scores. Implications for counselors are discussed.

Relationship between Private Self-Consciousness and Indices of Reliability and Homogeneity

Previous research has shown that scores on private self-consciousness were related to increased test-retest reliability and internal consistency of responses. To extend these findings to other measures, split-half reliability and Cronbach alpha 84 women and 27 men completed the Self-consciousness Scale and the NEO-Five Factor Inventory. It was hypothesized that subjects scoring high in private self-consciousness would show greater split-half reliability and homogeneity. The results did not support the hypotheses as scores for private self-consciousness were not significantly related to split-half reliability or homogeneity on the NEO-Five Factor Inventory. The present results, combined with previous findings, suggest that private self-consciousness is related to temporal stability of test scores but not to estimates of reliability associated with parallel forms of a test or to internal consistency.

Three-Year Stability of WISC-R IQs for Handicapped Children from Three Racial/Ethnic Groups

Score stability is an important psychometric quality of tests intended to measure relatively enduring characteristics such as intelligence. The purpose of the present study was to examine the long-term stability of WISC-R IQs (Full Scale, Verbal, and Performance) for a sample (N = 382) of male and female subjects from three racial or ethnic groups (Anglo, black, Mexican-American). The three-year stability coefficients for the varied sample of subjects (VIQ = .81, PIQ = .78, FSIQ = .85) compared well with those established with a three-week interval during the standardization of the WISC-R. Specific findings indicated that Anglo subjects' IQs were significantly more stable than those of blacks on all three IQ scales and also more stable than those of Mexican-Americans on the Performance and Full Scales. Sex of the subject had minimal influence on test score stability; only females' Verbal performances resulted in significantly larger stability coefficients than those of males. These and other results are discussed from educational, developmental, and psychometric perspectives.

Wais Test-Retest Reliability for a Clinical Out-Patient Sample

WAIS test-retest reliabilities were calculated for a clinical out-patient sample with testing intervals varying from 1 to 10 yr. There was no relationship between test-retest interval and the stability of test scores and the correlations between IQs were quite satisfactory (Full Scale IQ = .897, Verbal IQ = .906, Performance IQ = .876). Reliabilities remained high even when the sample was divided by diagnosis into organic, neurotic, personality disorder, and schizophrenic subgroups.

The Reliability and Validity of Objective Indices of Moral Development

The present paper addresses three issues sur rounding Rest's Defining Issues Test, an objective test of moral development based on Kohlberg's six- stage theory of moral development. Those issues are (1) the stability of test scores over time; (2) correla tion of scores with Kohlberg's interview measure of moral development; and (3) the insensitivity of its scoring procedure, which ignores responses to all items keyed to lower stages. In two age heterogene ous samples, total score test-retest reliabilities were generally in the high .70's or low .80's, regardless of which of several scoring schemes was used. In another age heterogeneous sample, the correlation with scores on Kohlberg's test was .70; but in two age homogeneous samples, the correlations were about .35 and .20. These validity coefficients sug gest that (1) the common variance shared by Rest's and Kohlberg's tests in age heterogeneous samples can be attributed to the fact that scores on both tests increase with age and (2) the two tests cannot be considered equivalent measures of the same con struct differing only in format. Results also in dicated that an empirically weighted scoring scheme is more sensitive to longitudinal change than is Rest's P score. This sensitivity to longi tudinal trends is an important property for tests such as Rest's which claim to be developmental and are frequently used to assess educational change. The empirically weighted sum had a significantly higher test-retest reliability (p < .05) than did a simple sum of item responses, and it had a signifi cantly higher correlation with Kohlberg's measure than did a theoretically weighted sum.

Access, attrition, test scores, and grades of college entrants and persisters: 1965?1973

This study contrasts the distributions of indices of academic abilities and achievements of entering freshmen classes and of classes at the end of the first year of college over an eight-year period. The data reported are based on two groups of students: one comprised of all students who completed the ACT Assessment Program and subsequently enrolled in college, and the other group comprised of all students in the first group completing their first year of college. Data collected show a decline in ACT test scores, an increase in the high school grades of college entrants, and the stability of test scores and increases in college grades of freshmen completing their first year of college.