Parallel Form Reliability Research Articles

Prevalence of Two-Syllable Digits Affecting Forward Digit Span Test Score

The word length effect shows a connection between word length and working memory performance. Although the relationship between digit verbal length and digit span has been investigated between languages, it has not been investigated within a language. It was hypothesized that this effect can be shown as a connection between the prevalence of digits with two syllables and digit span score. The study examined the effect of amount of syllables on Norwegian digit span test scores by altering the prevalence of two-syllable digits using three conditions in a repeated measures design ( N = 54). Results suggest that an elimination of two-syllable digits in a digit span test significantly reduced forward span test score (Cohen’s d = 0.36), but had no effect on backward span scores. These results suggest that a balanced distribution of two-syllable digits in a forward digit span tests should theoretically increase the test’s comparability and reduce language-related biases thus increasing the test’s parallel-form reliability. A peak-span model is proposed to integrate the findings into previous research on the word length effect.

Measurement Reliability of Phonation Quotient Derived From Three Aerodynamic Instruments

The purpose of this study was to examine parallel forms reliability between two hand-held spirometers and a pneumotachograph-based system for vital capacity and derived phonation quotient measurements. This is a prospective, repeated measures design. A total of 20 adult males were tested using three aerodynamic instruments-Baseline windmill-type spirometer, Contec SP10 digital spirometer and the Pentax Medical Phonatory Aerodynamic System (PAS), Model 6600 for measures of vital capacity. Phonation quotient was calculated using vital capacity from each instrument along with maximum phonation time. Repeated measures analyses of covariance (ANCOVAs) were performed to test for main effects of the instruments on vital capacity and phonation quotient, with age as a covariate. Pearson Product Moment correlation was performed to assess measurement reliability between the instruments. Statistically significant differences on ANCOVA were seen in vital capacity measures for the digital spirometer compared with the windmill spirometer and PAS. No differences were found between any of the instruments for phonation quotient. Large and positive correlations were present between all three instruments for both vital capacity and phonation quotient measurements. Strong parallel forms reliability in measures of vital capacity and derived phonation quotient was seen among the three instrument systems, although measurement precision was different when comparing the digital spirometer to two other instrument types.

Reliability of an Adapted Version of the Modified Six Elements Test as a Measure of Executive Function

The Modified Six Elements Test (MSET) is used to examine executive deficits—more specifically, planning deficits. This study investigates the reliability of an adapted version of the MSET and proposes a novel scoring method. Two parallel versions of the adapted MSET were administered in 60 healthy participants in a counterbalanced order. Test–retest and parallel-form reliability were examined using intraclass correlation coefficients, Bland-Altman analyses, standard errors of measurement, and smallest real differences, representing clinically relevant changes over time. Moreover, the ecological validity of the adapted MSET was evaluated using the Executive Function Index, a self-rating questionnaire measuring everyday executive performance. No systematic differences between the test occasions were present, and the adapted MSET including the proposed scoring method was capable of detecting real clinical changes. Intraclass correlations for the test–retest and parallel-form reliability were modest, and the variability between the test scores was high. The nonsignificant correlations with the Executive Function Index did not confirm the previously established ecological validity of the MSET. We show that both parallel versions of the test are clinically equivalent and can be used to measure executive function over the course of time without task-specific learning effects.

Validity and Reliability of the Seizure Self-Efficacy Scale for Children with Epilepsy.

This study aims to adapt the Seizure Self-Efficacy Scale for Children (SSES-C) into Turkish and then assess its validity and reliability in children with epilepsy. The study sample consisted of 166 children (aged 9-17 years) with epilepsy who attended of Akdeniz University Hospital, Antalya Training and Research Hospital, and Bursa Dortcelik Children's Hospital Pediatric Neurology Clinics between July 2012 and March 2013. All research data were collected by a researcher in face-to-face interviews using Child Information Form, Seizure Self-Efficacy Scale for Children and Children's Depression Scale. The Seizure Self-Efficacy Scale for Children is a 15-item, 5-point Likert scale designed by Caplin et al. (2002). The linguistic adaptation and validation of the scale was conducted by seven experts. To evaluate the content validity of the scale, we elicited judgments from a panel of 10 content experts. The expert judgments showed that the correlation between the items on the scale was fairly good (Kendall's W=0.411, p<0.001, ki-kare: 57.495). Load factor of 40% and a large factor analysis included analysis of substances and two factors accounting for 49.67% of the total variance explained. We calculated Cronbach's alpha coefficient for the internal consistency and the full-scale score showed good internal consistency (alpha 0.89). Within the context of reliability studies, it was found correlations varying between 0,98-0,74 for the two sub-factors of the scale. Test/retest correlation coefficients were significant (p<0,01) and high (r=0.99). In parallel forms reliability, the correlations between the Seizure Self-Efficacy Scale for Children and Children's Depression Rating Scale were found to be negative, moderate and statistically significant (r=-0.58, p<0.001). The measurements conducted on the Turkish version of the Seizure Self-Efficacy Scale for Children showed that it is consistent with the original scale, valid and reliable for Turkish society.

The Trojan Lifetime Champions Health Survey: development, validity, and reliability.

Self-report questionnaires are an important method of evaluating lifespan health, exercise, and health-related quality of life (HRQL) outcomes among elite, competitive athletes. Few instruments, however, have undergone formal characterization of their psychometric properties within this population. To evaluate the validity and reliability of a novel health and exercise questionnaire, the Trojan Lifetime Champions (TLC) Health Survey. Descriptive laboratory study. A large National Collegiate Athletic Association Division I university. A total of 63 university alumni (age range, 24 to 84 years), including former varsity collegiate athletes and a control group of nonathletes. Participants completed the TLC Health Survey twice at a mean interval of 23 days with randomization to the paper or electronic version of the instrument. Content validity, feasibility of administration, test-retest reliability, parallel-form reliability between paper and electronic forms, and estimates of systematic and typical error versus differences of clinical interest were assessed across a broad range of health, exercise, and HRQL measures. Correlation coefficients, including intraclass correlation coefficients (ICCs) for continuous variables and κ agreement statistics for ordinal variables, for test-retest reliability averaged 0.86, 0.90, 0.80, and 0.74 for HRQL, lifetime health, recent health, and exercise variables, respectively. Correlation coefficients, again ICCs and κ, for parallel-form reliability (ie, equivalence) between paper and electronic versions averaged 0.90, 0.85, 0.85, and 0.81 for HRQL, lifetime health, recent health, and exercise variables, respectively. Typical measurement error was less than the a priori thresholds of clinical interest, and we found minimal evidence of systematic test-retest error. We found strong evidence of content validity, convergent construct validity with the Short-Form 12 Version 2 HRQL instrument, and feasibility of administration in an elite, competitive athletic population. These data suggest that the TLC Health Survey is a valid and reliable instrument for assessing lifetime and recent health, exercise, and HRQL, among elite competitive athletes. Generalizability of the instrument may be enhanced by additional, larger-scale studies in diverse populations.

Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke

This study aimed to develop and evaluate a Hong Kong Chinese version of the Cambridge Prospective Memory Test (CAMPROMPT-HKCV). Thirty-three subjects at least one year post-stroke participated in the study. They were simultaneously rated on version A of the CAMPROMPT-HKCV by two testers to establish its internal consistency and inter-rater reliability. Raters used the parallel versions of the test (A and B), in rating 10 patients within 2 weeks to establish the parallel form reliability. Another 10 were also assessed on the same day using both version A of the CAMPROMPT-HKCV and the Rivermead Behavioural Memory Test–Chinese version (RBMT-CV) to establish concurrent validity. A new group of 40 stroke patients and 44 healthy controls was recruited to establish its sensitivity and specificity. Results indicated that test–retest reliability on time-based, event-based and total scores, and inter-rater reliability for versions A and B of the test were high. Cronbach's alpha of the event-based score was higher than that of the time-based score. The reliability and concurrent validity of the parallel forms were established. There was a significant difference in performance on CAMPROMPT-HKCV (version A) between the stroke group and the healthy control group. ROC analysis showed that the ability of the cut-off CAMPROMPT-HKCV (total score) to differentiate PM problems was 20.5 (out of 36) with sensitivity at 95.5% and specificity at 55.9%. Further study in developing stratified norms across different age groups in Chinese-speaking stroke patients is recommended.

Psychometric Evaluation of the Symptoms and Functioning Severity Scale (SFSS) Short Forms with Out-of-Home Care Youth.

There is a need for brief progress monitoring measures of behavioral and emotional symptoms for youth in out-of-home care. The Symptoms and Functioning Severity Scale (SFSS; Bickman et al., 2010) is one measure that has clinician and youth short forms (SFSS-SFs); however, the psychometric soundness of the SFSS-SFs with youth in out-of-home care has yet to be examined. The objective was to determine if the psychometric characteristics of the clinician and youth SFSS-SFs are viable for use in out-of-home care programs. The participants included 143 youth receiving residential treatment and 52 direct care residential staff. The current study assessed internal consistency and alternate forms reliability for SFSS-SFs for youth in a residential care setting. Further, a binary classification test was completed to determine if the SFSS-SFs similarly classified youth as the SFSS full version for low- and elevated-severity. The internal consistency for the clinician and youth SFSS-SFs was adequate (α = .75 to .82) as was the parallel forms reliability (r = .85 to .97). The sensitivity (0.80 to 0.95), specificity (0.88 to 0.97), and overall accuracy (0.89 to 0.93) for differentiating low and elevated symptom severity was acceptable. The clinician and youth SFSS-SFs have acceptable psychometrics and may be beneficial for progress monitoring and additional research should clarify their potential for progress monitoring of youth in out-of-home programs.

Construction and updating of a public events questionnaire for repeated measures longitudinal studies

Impairments of retrospective memory and cases of retrograde amnesia are often seen in clinical settings. A measure of the proportion of memories retained over a specified time can be useful in clinical situations and public events questionnaires may be valuable in this respect. However, consistency of retention of public events memory has rarely been studied in the same participants. In addition, when used in a research context, public events questionnaires require updating to ensure questions are of equivalent age with respect to when the test is taken. This paper describes an approach to constructing and updating a Public Events Questionnaire (PEQ) for use with a sample that is recruited and followed-up over a long time-period. Internal consistency, parallel-form reliability, test-retest reliability, and secondary validity analyses were examined for three versions of the PEQ that were updated every 6 months. Versions 2 and 3 of the questionnaire were reliable across and within versions and for recall and recognition. Change over time was comparable across each version of the PEQ. These results show that PEQs can be regularly updated in a standardized fashion to allow use throughout studies with long recruitment periods.

Factorial Validity and Reliability of the Tamil Version of Multidimensional Scale of Perceived Social Support Among a Group of Participants in University Malaya Medical Centre, Malaysia

Background and Objectives:This study was done to validate the Tamil version of Multidimensional Scale of Perceived Social Support (MSPSS-TV) among a group of Indian participants in University Malaya.Materials and Methods:Ninety-four people who took part in this study were given MSPSS-TV, Medical Outcome Study (MOS) social support survey, Malay version of Beck Depression Inventory (BDI), Malay version of General Health Questionnaire (GHQ), and English version of MSPSS. After a week, these participants were again required to complete the MSPSS-TV, but with the item sequences shuffled.Results:Internal consistency of this scale was excellent (Cronbach's alpha = 0.924), and both test-retest reliability (0.71) and parallel form reliability (0.92) were high (Spearman's rho, P<0.01). MSPSS-TV was negatively correlated with GHQ and BDI.Interpretation and Conclusions:The MSPSS-TV demonstrated sound psychometric properties in gauging the social support among a group of participants. It could therefore be used as a simple screening tool among the young educated Malaysians.

The Development of Test, Investigating the Accuracy of Professional Evaluation

The aim of this article is to describe test construction process. It could be used to estimate appraisers’ performance. The authors used 5 scales of the 16 Personality Factors Raymond Cattell's test “Warmth”, “Reasoning”, “Emotional Stability”, “Dominance” “Rule-Consciousness”. Stimulus material for this test consists of a set of video records showing behavior of four respondents in three different situations: (interview), work in a pair (role playing game), work in small group (panel discussion). Each video record lasts from 4 to 7 minutes. Testee estimates people's bechaviour on the video record. Test has two forms. Test was proved to have internal consistency, parallel-form reliability, test- retest reliability, criteria and construct validity

A Test of Everyday Reading Comprehension (TERC)

The aim of this study was to develop a standardised test of everyday reading comprehension for children in Australian schools. To this end, we developed two forms for the Test of Everyday Comprehension (TERC) and developed norms for children aged 6 to 12 and in grades 1 to 6. Measures of parallel form reliability indicated that the two TERC forms were equal in difficulty and measured a common concept. Measures of inter-rater reliability indicated that scoring the two TERC forms was simple enough to minimise differences between testers. Ideally, the TERC should be used (1) as a screening test for poor reading comprehension in primary-school children and (2) as a tool to communicate to parents the impact that a child's reading difficulty can have on their everyday life. Poor performance on the TERC should be followed-up by tests that target other aspects of the child's reading and language skills to locate the source of their reading comprehension problem.

Reliability and validity of theMalay version of Attitudes toward Lesbians and Gay Men (MVATL/MVATG): A study on a group of medical students inMalaysia

This study aims to examine the validity and reliability of the Malay version of Attitudes toward Lesbians and Gay Men (MVATL/MVATG) among a group of medical students in Malaysia. It is a cross-sectional study of 173 medical students in the Faculty of Medicine, University of Kuala Lumpur, Malaysia. The participants were given the MVATL/MVATG, Index of Attitudes toward Homosexuals (IATH), Homosexuality Attitude Scale (HAS) and the English version of Attitude toward Lesbians and Gay Men. Two weeks later, these students were given the MVATLG again. Significant correlation was found between the individual scores of MVATL and MVATG with IATH and HAS in the results. The scale was able to differentiate Muslim and Non-Muslim subjects. The internal consistency (Cronbach's alpha) of both the MVATL and MVATG were good, at 0.76 and 0.82, respectively. The parallel form reliability (Pearson's correlation) of MVATL was 0.0.73 and 0.74 for MVATG. The test-retest reliability of MVATL/MVATG was good (Intraclass correlation coefficient, ICC = 0.67 for MVATL and 0.60 for MVATG). The MVATLG demonstrated good psychometric properties in measuring attitudes toward homosexuality among a group of medical students in Malaysia and it could be used as a simple instrument on young educated Malaysian adults.

Factorial Validity and Reliability of the Malaysian Simplified Chinese Version of Multidimensional Scale of Perceived Social Support (MSPSS-SCV) Among a Group of University Students

This study was aimed at validating the simplified Chinese version of the Multidimensional Scale of Perceived Support (MSPSS-SCV) among a group of medical and dental students in University Malaya. Two hundred and two students who took part in this study were given the MSPSS-SCV, the Medical Outcome Study social support survey, the Malay version of the Beck Depression Inventory, the Malay version of the General Health Questionnaire, and the English version of the MSPSS. After 1 week, these students were again required to complete the MSPSS-SCV but with the item sequences shuffled. This scale displayed excellent internal consistency (Cronbach's α = .924), high test-retest reliability (.71), parallel form reliability (.92; Spearman's ρ, P < .01), and validity. In conclusion, the MSPSS-SCV demonstrated sound psychometric properties in measuring social support among a group of medical and dental students. It could therefore be used as a simple screening tool among young educated Malaysian adolescents.

Preliminary Validation of the Defense and Veterans Pain Rating Scale (DVPRS) in a Military Population

The Army Surgeon General released the Pain Management Task Force final report in May 2010. Among military providers, concerns were raised that the standard numeric rating scale (NRS) for pain was inconsistently administered and of questionable clinical value. In response, the Defense and Veterans Pain Rating Scale (DVPRS) was developed. The instrument design integrates pain rating scale features to improve interpretability of incremental pain intensity levels, and to improve communication and documentation across all transitions of care. A convenience sample of 350 inpatient and outpatient active duty or retired military service members participated in the study at Walter Reed Army Medical Center. Participants completed the five-item DVPRS-one pain intensity NRS with and without word descriptors presented in random order and four supplemental items measuring general activity, sleep, mood, and level of stress and the Brief Pain Inventory seven interference items. Using systematic sampling, a random sample was selected for a word descriptor validation procedure matching word phases to corresponding pain intensity on the NRS. Parallel forms reliability and concurrent validity testing demonstrated a robust correlation. When the DVPRS was presented with the word descriptors first, the correlation between the two ratings was slightly higher, r = 0.929 (N = 171; P < 0.001), than ordering first without the descriptors, r = 0.882 (N = 177; P < 0.001). Intraclass correlation coefficient was 0.943 showing excellent alignment of word descriptors by respondents (N = 42), matching them correctly with pain level. The DVPRS tool demonstrated acceptable psychometric properties in a military population.

Interrater reliability and parallel forms reliability of two noninvasive hemodynamic monitoring devices in clinically stable emergency department patients

To evaluate the interrater reliability and parallel forms reliability of transcutaneous Doppler ultrasonography (TCDU) and impedance cardiography (ICG) in clinically and hemodynamically stable emergency department patients. We enrolled 30 emergency department patients over a 2-day period. Patients had three consecutive simultaneously blinded measurements of stroke volume (SV) and heart rate (HR) recorded by TCDU (USCOM) and ICG (Cardiodynamics). Two physicians, with basic familiarity but no clinical experience with either device recorded three measurements of SV and HR on each device. Intraclass correlation coefficients (ICC), mixed linear models for repeated measures, and Bland-Altman plots were used to assess interrater reliability and nature of relationships between measures from the devices (parallel forms reliability). The ICC for TCDU was 0.96 for HR and 0.95 for SV, whereas the ICC for ICG was 0.93 for HR and 0.98 for SV. The device HR estimates were significantly related (P<0.0001 for all slopes) for all phases, but SV failed to reach significance following the first 50 trials [t(94.2)=2.72, P=0.0077]. Although HR estimates were within reasonable clinical tolerances (bias 0.5%, limits of agreement -15.4 to 16.4%) SV disagreement was concerning (bias 3.8%, limits of agreement -58 to 66%). Both TCDU and ICG have fair interrater reliability of SV independent of operator experience. A statistically significant relationship exists between the two devices but this does not produce predictable values in SV. Over time comparative results become less biased but remain limited by a great degree of variability.

Reliability and Validity of Turkish Versions of the Child, Parent and Staff Cancer Fatigue Scales

This study was designed to adapt the Turkish versions of scales to evaluate fatigue in children with cancer from the perspectives of the children, parents and staff. The objective of this study was to validate "Child Fatigue Scale-24 hours" (CFS-24 hours), "Parent Fatigue Scale-24 hours" (PFS-24 hours) and "Staff Fatigue Scale-24 hours" (SFS-24 hours) for use in Turkish clinical research settings. Translation of the scales into Turkish and validity and reliability tests were performed. The validity of the translated scales was assessed with language validity and content validity. The reliability of the translated scales was assessed with internal consistency. The scales were evaluated by considering the following: calculation of the Cronbach alpha coefficient for parallel form reliability with 52 pediatric cancer patients, 86 parents and 43 nurses. The internal consistency was estimated as 0.88 for the Child Fatigue Scale-24 hours, 0.77 for the Parent Fatigue Scale-24 hours, and 0.72 for the Staff Fatigue Scale-24 hours (Cronbach's α). The Turkish version of the Child Fatigue Scale-24 hours, the Parent Fatigue Scale-24 hours and the Staff Fatigue Scale-24 hours were judged reliable and valid instruments to assess fatigue in children and showed good psychometric properties. These scales should assist in understanding to what extent initiatives can minimize or eliminate fatigue. Our scales are recommended for further studies and use in pediatric oncology clinics as routine measurements and nursing initiatives should be planned accordingly.

A Note on the Reliability Coefficients for Item Response Model-Based Ability Estimates

Assuming item parameters on a test are known constants, the reliability coefficient for item response theory (IRT) ability estimates is defined for a population of examinees in two different ways: as (a) the product-moment correlation between ability estimates on two parallel forms of a test and (b) the squared correlation between the true abilities and estimates. Due to the bias of IRT ability estimates, the parallel-forms reliability coefficient is not generally equal to the squared-correlation reliability coefficient. It is shown algebraically that the parallel-forms reliability coefficient is expected to be greater than the squared-correlation reliability coefficient, but the difference would be negligible in a practical sense.

Reliability and Validity of a Computer-Mediated, Single-Word Intelligibility Test: Preliminary Findings for Children with Repaired Cleft Lip and Palate

To determine the reliability and validity of a computer-mediated, 50-word intelligibility test designed to be a global measure of severity of speech disability in children with repaired cleft lip and palate. A prospective between-group design was used with convenience sampling of patients from a university craniofacial center. A total of 38 children between the ages of 4 and 9 years participated. Of these, 22 had repaired cleft lip and palate; whereas, 16 had no clefts. Twenty adults served as listeners. Speech intelligibility scores were calculated for repeated administrations of a single-word test based upon the number of correct orthographically transcribed words by four groups of five listeners per child. Measures of parallel forms, interlistener, and intralistener reliabilities were estimated; measures of construct validity were also determined. All measures of reliability were adequate. Parallel forms reliability of the test based upon mean scores from five listeners per child was high (r = .97). Thirty-seven of 38 children had differences between forms of 11 percentage points or less. Construct validity of the test was shown by (1) significantly lower speech intelligibility scores for children with cleft lip and palate than controls and (2) a moderately high correlation (r = .79) between intelligibility scores and percent consonants correct for all children. A computerized, single-word intelligibility test was described that appears to be a reliable and valid measure of global speech deficits in children with cleft lip and palate. Additional development of the test may further facilitate standardized assessment of children with cleft lip and palate.

Reliability of a science admission test (HAM-Nat) at Hamburg medical school.

Objective: The University Hospital in Hamburg (UKE) started to develop a test of knowledge in natural sciences for admission to medical school in 2005 (Hamburger Auswahlverfahren für Medizinische Studiengänge, Naturwissenschaftsteil, HAM-Nat). This study is a step towards establishing the HAM-Nat. We are investigating parallel forms reliability, the effect of a crash course in chemistry on test results, and correlations of HAM-Nat test results with a test of scientific reasoning (similar to a subtest of the "Test for Medical Studies", TMS).Methods: 316 first-year students participated in the study in 2007. They completed different versions of the HAM-Nat test which consisted of items that had already been used (HN2006) and new items (HN2007). Four weeks later half of the participants were tested on the HN2007 version of the HAM-Nat again, while the other half completed the test of scientific reasoning. Within this four week interval students were offered a five day chemistry course.Results: Parallel forms reliability for four different test versions ranged from rtt=.53 to rtt=.67. The retest reliabilities of the HN2007 halves were rtt=.54 and rtt =.61. Correlations of the two HAM-Nat versions with the test of scientific reasoning were r=.34 und r=.21. The crash course in chemistry had no effect on HAM-Nat scores.Conclusions: The results suggest that further versions of the test of natural sciences will not easily conform to the standards of internal consistency, parallel-forms reliability and retest reliability. Much care has to be taken in order to assemble items which could be used interchangeably for the construction of new test versions. The test of scientific reasoning and the HAM-Nat are tapping different constructs. Participation in a chemistry course did not improve students’ achievement, probably because the content of the course was not coordinated with the test and many students lacked of motivation to do well in the second test.

Iranian version of Form B of the Multidimensional Health Locus of Control Scales among the youth

The purpose of this study was to translate the Form B of Multidimensional Health Locus of Control Scales into Persian and then to measure its psychometric characteristics among the Iranian youth. The Health Locus of Control is regarded as an effective variable in developing health behaviours, clinical capacities and in determining health problems. Many studies have been conducted on Form B in the world, which have led to valuable results in health behaviours, health outcomes and health cares. Questionnaire survey. Undergraduate students (n=448) participated in this study. The reliability of the scale was calculated through three methods of estimating reliability, namely, test-retest, parallel forms and Cronbach's alpha. To calculate the validity of the scale, we used content, concurrent, construct and predictive methods. We determined the content validity of the scale by forward and backward-translating each item from English into Persian. The concurrent validity of the questionnaire was estimated as acceptable. To determine the predictive validity, the relationship between Form B and health status was used, an action that showed that health status correlated positively with Internal but negatively with Chance and did not correlate with Powerful others Health Locus of Control. The exploratory factor analysis indicated that the factor structure of the translated scale was quite similar to that of the original one. The internal consistency ranged from 0·69-0·72 and parallel forms reliability indices ranged from 0·70-0·72 for its subscales. The results indicated that psychometric characteristics of the Iranian version of Form B were estimated acceptable among the youth. The findings will assist professional health educators and nurses to develop educational interventions that should be designed to support and strengthen students' abilities so that they could recognise, understand and predict health behaviours.