Cross-validation Group Research Articles

Lao Depression Inventory.

There are no measurement tools that accurately measure depression among Lao refugees. The overall purpose of this research was to complete the development and validation procedures for the Lao Depression Inventory (LDI). The study consisted of 216 Ethnic Lao refugees. A clinical interview and 164 true/false questions were administered to identify specific items which could identify depression among the Ethnic Lao people. All items were administered in both English and Lao. Overall, 78 of the 164 items differentiated groups of depressed and nondepressed Lao at the .01 level. Results of validation procedures showed that a 30-item scale had an accuracy rate of 89% in identifying the presence of depression in the validation group; the hit-rate for the same items and cutoff was 92% in the cross-validation group. Potential uses of the scale are discussed.

Predicting Performance on the National Teacher Examinations Core Battery

AbstractThis study examined factors that predict performance on the National Teacher Examinations (NTE) Core Battery. We found strong relationships between a student’s undergraduate grade point average (GPA), American College Test (ACT) subtests, and the NTE Core Battery tests. We developed and used regression equations to predict NTE Core Battery test scores for a cross-validation group of students whose scores were not used in developing the original equations. Predictions ranged in accuracy from r = .73 for professional knowledge to r = .92 for general knowledge. Results are discussed in terms of comparing current predictors of success on the NTE Core Battery with past predictors of success on the NTE Common Examinations and the implications for teacher educators.

ENRICH MARITAL INVENTORY: A DISCRIMINANT VALIDITY AND CROSS‐VALIDATION ASSESSMENT

To assess the validity and clinical utility of the marital inventory ENRICH, a discriminant validity study was conducted using a national sample of 5039 married couples. The sample was randomly split in order to form a cross-validation group. ENRICH is a multidimensional scale and two types of analysis were conducted to assess the value of these various scales. Results from discriminant analysis indicated that using either the individual scores or couples' scores, happily married couples could be discriminated from unhappily married couples with 85-95% accuracy. These results were cross-validated with a second sample. Using regression analysis, it was clearly demonstrated that background factors account for little of the variance in discriminating happy from unhappily married couples compared to their relationship dynamics, i.e., scale scores. All ENRICH scales except equalitarian roles proved significant, indicating the validity of a multidimensional inventory.

Total and visceral adipose-tissue volumes derived from measurements with computed tomography in adult men and women: predictive equations.

Total and visceral adipose-tissue (AT) volumes were determined by computed tomography (CT) by a multiscan technique in 17 men and 10 women with a wide range of body weights. In these primary materials, weight, height, and various diameters, circumferences, and subcutaneous AT thicknesses of the trunk were examined for their relationships to CT-determined total and visceral AT volumes. Predictive AT equations from the primary materials were then tested on two cross-validation groups consisting of another 7 men and 9 women. For the prediction of the total AT volume, weight/height was the superior single predictor, with errors less than 11% in primary and cross-validation materials. For the prediction of visceral AT volume, simple equations based entirely on the sagittal diameter of the trunk at the L3-L5 level resulted in errors less than 21% in both sexes.

Predicting Aerobic Capacity During the Modified Naughton Treadmill Protocol in Patients with Coronary Artery Disease

The Naughton treadmill protocol has become a popular treadmill test for use with patients with coronary artery disease (CAD). Frequently, the stages are reduced from 3 to 2 minutes, but there is little information on the prediction of aerobic capacity (AC) based on this abbreviated protocol. To determine the relationship between AC and exercise time (ET) with the modified Naughton protocol, 50 men with CAD, evenly divided between β blocked (β group) and non-β blocked patients (control group), completed a maximal graded exercise test with measurement of AC. These 50 patients were the validation group (VAL). An equation predicting AC, based on the VAL group, was calculated with stepwise regression equation using ET, β blocker usage did not significantly contribute to the regression model. The equation predicting AC from ET and age is: AC (ml ± kg-1 ± min-1) = 17.66 + ET (.985) -age (.169), r =.93, standard error of the estimate (SEE) = 3.2. A cross-validation (CV) group sample of 20 subjects (evenly divided between subjects taking β blockers and nonblocked subjects) completed the modified Naughton protocol, and was used to cross-validate the regression equation. The VAL group's equation was used to predict the CV group's AC, and this value was compared with the CV group's measured AC. The CV group's measured and predicted AC were not statistically different, indicating the appropriateness of the validation equation. Completing a given speed and grade with the shortened Naughton protocol yielded a lower AC than that predicted by the standard Naughton protocol. The results indicate that with trained patients with CAD: (1) AC can be predicted for clinical use from the modified Naughton protocol using E T and age, (2) minimal to moderate doses of β blocking medications did not alter the relationship between ET and AC, and (3) separate equations predicting AC are needed for the Naughton protocols with 3− and 2-minute stages.

Sensitivity of two visual reproduction tests to alcoholic brain impairment.

The Wechsler Memory Scale Visual Reproduction subtest and the Rey-Osterreith Complex Figure Test were administered to alcoholics to examine visual reproduction impairment. The initial test group was comprised of 28 inpatients in an alcohol treatment program while 16 subsequent inpatients served as the cross-validation group. The Visual Reproduction subtest correlated significantly with an average impairment index both in the initial group and the less-impaired cross validation group. The Complex Figure scores showed significant correlations with the average impairment index only in the initial group. It was concluded that the most likely reason for this outcome was the emphasis upon intentional learning in the Visual Reproduction subtest as opposed to the emphasis upon incidental learning in the Complex Figure recall task. The results were discussed in the context that alcoholics may have an impaired ability to use efficient encoding strategies spontaneously.

The Factorial Invariance of Responses by Males and Females to a Multidimensional Self-Concept Instrument: Substantive and Methodological Issues.

The factorial invariance of responses by preadolescent males and females to a multidimensional self-concept instrument was examined for responses to the Self Description Questionnaire (SDQ). Sets of responses by 500 males and by 500 females were each randomly divided in half to form four groups (M1, M2, Fl, and F2). The factorial invariance of an a priori structure demonstrated the replicability of the structure across cross-validation groups (M1 and M2, F1 and F2) and the generality of the structure across sex (M1 and Fl, M2 and F2). Additional a posteriori structures that better fit the data were derived on the basis of the initial analyses, but the estimated values of the new parameters were not invariant across cross-validation groups or across sex. This suggests that some of the improved fit was illusory and due to capitalizing on chance. However, factor loadings and factor correlations were invariant across sex for a priori and a posteriori structures. Hence the results support the replicability of SDQ factor structure across cross-validation samples and its generality across responses by preadolescent males and females.

Estimation of &OV0312;O2max from a one-mile track walk, gender, age, and body weight

The purpose of this investigation was to explore an alternative field test to estimate maximal oxygen consumption (VO2max) using a one-mile walk test. VO2max was determined in 343 healthy adult (males = 165, females = 178) subjects 30 to 69 yr using a treadmill protocol (mean +/- SD: VO2max = 37.0 +/- 10.7 ml X kg-1 X min-1). Each subject performed a minimum of two, one-mile track walks as fast as possible. The two fastest walks (T1, T2) with elapsed times within 30 s were used for subsequent analyses. Heart rates were monitored continuously and recorded every one-quarter mile. Multiple regression analysis (best sub-sets) to estimate VO2max (l X min-1) yielded the following predictor variables: track walk-1 time (T1); fourth quarter heart rate for track walk-1 (HR 1-4); age (yr); weight (lb); and sex (1 = male, 0 = female). The best equation (N = 174) was: VO2max = 6.9652 + (0.0091*WT) - (0.0257*AGE) + (0.5955*SEX) - (0.2240*T1) - (0.0115*HR1-4); r = 0.93, SEE = 0.325 l X min-1. Comparing observed and estimated VO2max values in a cross-validation group (N = 169) resulted in r = 0.92, SEE = 0.355 l X min-1. Generalized and sex-specific equations to estimate VO2max (ml X kg-1 X min-1) were also generated. The accuracy of estimation as expressed by SEE was similar among the equations. The results indicate that this one-mile walk test protocol provides a valid sub-maximum assessment for VO2max estimation.

A model for predicting delayed intracranial hypertension following severe head injury.

A graph is presented for predicting delayed intracranial hypertension (intracranial pressure (ICP) greater than 30 mm Hg) for severely head-injured patients, based on a logistic regression model. Data gathered during the first 24 hours of patient observation are used to predict patient status during the subsequent 48 hours. The best predictor out of 10 factors analyzed was the peak ICP level during the first 24 hours (p less than 0.0001). Other predictors used in the final model were the presence of hypotension (p = 0.045) and abnormal ventricles--defined as ventricles which were either absent, small, or enlarged (p = 0.086). Error rates of 24% and 20% were obtained initially and by means of a separate cross-validation group, respectively. Use of a conservative cut point (25% estimated chance of developing excess ICP) for designating high-risk patients provided a procedure with sensitivity of 86% to 89% for the two groups.

Brief assessment of drinking problems in male and female psychiatric inpatients.

Used discriminant function analysis to identify and cross-validate a subset of items on the Michigan Alcoholism Screening Test (MAST) that in combination maximally distinguish between male and female psychiatric inpatients (N = 120) diagnosed as having alcohol-related and non-alcohol-related disorders. Classification of Ss by use of the composite variate showed 88% overall accuracy with an 83% valid positive rate in the validation sample, but only 65% overall accuracy and a 10% valid positive rate in an independent cross-validation group. The reliability of the MAST interview did not decline at higher levels of alcohol involvement.

Separation of subtypes of depression using discriminant analysis: Separation of bipolar endogenous depression from nonendogenous (“Neurotic”) depression

We derived a discriminant function separating patients with bipolar endogenous depression (“melancholia”) from patients with nonendogenous (“neurotic”) depression, and showed that the difference between the groups was not one of overall severity of illness alone. The discriminant function (DF) included 5 clinical items. We reduced the DF to a discriminant index (DI) with integral item weights, and trichotomized the DI scores into two definite classifications and an intermediate, uncertain classification. We cross-validated this DI in a separate group of patients, and found no decrease in the accuracy of classification on cross-validation. Thirty-three of 41 (80%) of the patients in the cross-validation group were classified by the DI; 26 of 33 (79%) correctly. We also validated the DI classification against an external, biological marker, the dexamethasone suppression test (DST). The DI predicted the DST result with the same accuracy as the clinical diagnoses did, supporting the validity of the DI.

COMPARISON OF TRADITIONAL AND ITEM RESPONSE THEORY METHODS FOR EQUATING TESTS

Achievement test batteries are typically published in several parallel forms with different levels for different grades. The forms and levels of each different test of the battery must be equated, that is, every score on a given form or level must be translatable into a score value on any other form or level of that test. Equipercentile and linear methods have been used traditionally to equate tests (Angoff, 1971), but item response (latent trait) theory methods have been recently advocated as a potential improvement over traditional methods (Lord, 1977; Wright, 1977). Lord (1977) argued from theoretical considerations that traditional equating methods are not appropriate for equating tests of differing difficulty, whereas item response theory methods have the capacity to provide an appropriate equating in this case. Lord's (1977) definition of equating implies that exact equating is possible only when the tests to be equated measure the same unidimensional ability. Achievement tests covering abilities encountered over a range of grades are probably not unidimensional. The usefulness of a score scale is severely limited, however, unless it spans all of the levels for which the battery is intended. It appears necessary, therefore, to attempt equating across levels even when unidimensionality does not hold. The present study was designed to compare the end results of two traditional equating schemes and seven item response theory equating schemes. The data used came from the 1978 equating project of the Iowa Tests of Educational Development (ITED). The study entailed both the equating of forms of similar difficulty and the equating of levels of differing difficulty. A cross-validation group was used to establish a criterion for comparing the results of different equating methods. A cross-validation summary statistic was calculated which measured the closeness of converted score distributions for stratified randomly equivalent groups. The goal of the study was to identify the method or methods which were best according to this criterion and to examine idiosyncracies of the equating schemes for the equating of a two-level high school achievement test battery.

Checking the Reliability and Validity of HSPQ Profiles

Developed a scale designed to detect invalid response patterns on the High School Personality Questionnaire (HSPQ). Using a large sample of protocols drawn from the HSPQ standardization group, ten items were found to satisfy the necessary criterion for inclusion in the resulting "random" or "validity" (RV) scale. When applied to separate cross-validation groups of nearly 2,000 valid and randomly generated response protocols, RV was found to correctly classify 79% of the profiles. As a further test of the scale's utility, 100 profiles classified as valid and 100 classified as invalid were selected for comparison. In the invalid group, KR-20 reliabilities of the 14 trait scales were almost uniformly lowered, Also, for the invalid group the correlations among the trait scales were attenuated and altered to the extent that the underlying factor structure of the test was upset.

Predicting Freshman GPA of Law/Justice Students

This study was concerned with the validity of the Verbal (V) and Mathematics (M) scales of the College Board Scholastic Aptitude Test (SAT) as predictors of college freshman grade point average (GPA) at Glassboro State College. A double cross validation was performed on data from 204 first-year law/justice students. Results indicated that SAT-V and SAT-M together were somewhat more valid predictors of GPA than was either score separately. Multiple Rs were .33 for the criterion group and .37 for the cross-validation group. Both SAT and GPA distributions were found to be skewed, the former positively and the latter negatively. This circumstance was a possible factor influencing the size of the correlation coefficients.

Physiological Biorhythm as a Correlate of Pilot Error Accidents and Incidents

Aircraft accidents and incidents attributed to pilot error were hypothesized to have occurred while the pilot was in a critical phase for one or more biorhythms. From screening accident and incident reports for a large military unit, two groups of 25 pilots who had been involved in pilot error accidents and one group of 50 pilots who had been involved in pilot error incidents were identified. 13 of the accident validation group and 12 of the cross validation group were found to have been in a critical physiological phase at the time of accident, or twice the number expected by chance. For the incident group, 20 of the 50 pilots were in a critical physiological phase at the time of incident. Results for all three groups exceeded chance at the .025 level. Results for emotional and intellectual biorhythms, and for double critical phases were found not significant.

Carrier detection in classic hemophilia by combined measurement of immunologic (VIII AGN) and procoagulant (VIII AHF) activities.

Accurate carrier detection in classic hemophilia has been difficult because of (1) technical problems related to the performance of both immunologic (VIII AGN) and procoagulant (VIII AHF) determinations, and (2) statistical problems related to the analysis of these data. VIII AHF was determined by a one-stage assay based on the partial thromboplastin time (PTT). VIII AGN was measured by the method of quantitative immunoelectrophoresis. The discriminant function U. = 0.67 1n (AHF) -- 3.17 AGN X 10(-3) was calculated for a validation group of 20 normal persons and for seven obligate carriers, and tested for accuracy of prediction on a cross-validation group of seven additional normal women and ten additional obligate carriers. A U. score of greater than or equal to 2.54 correctly identified 25 of 27 normal persons. Sixteen of 17 obligate carriers had U. scores below 2.54. In addition, of seven possilbe carriers, four were identified as normal and three as carriers. Five normal women taking oral contraceptives had disproportionately high U. scores. It is concluded that detection of carriers of classic hemophilia should be possible in the clinical laboratory by calculation of a discriminant function from combined measurements of VIII AGN and VIII AHF.

Negative response bias and the MMPI Overcontrolled-Hostility scale: A response to Deiker.

Tested the hypothesis of T. E. Deiker (see record 1974-27935-001) that naysaying accounts for the MMPI Overcontrolled-Hostility (O-H) scale's ability to discriminate among criminal groups differing in their propensity for violence. 4 balanced O-H scales were constructed by adding true or deleting false keyed items according to different selection strategies. The 2 extended scales discriminated cross-validation groups as well or better than the regular imbalanced O-H scale, and the shortened scales were significant or approached significance in 3 of 4 comparisons. It is concluded that the validity of the O-H scale is not dependent on naysaying. (22 ref) (PsycINFO Database Record (c) 2006 APA, all rights reserved)

Behavioral correlates of WAIS profile patterns: an exploratory study.

To assess the feasibility of both an empirical derivation of WAIS subtest patterns and an actuarial assignment of behavioral descriptions to these patterns, 11 potential patterns were delineated. Three of these were studied in depth, and one of these was the focus of this report. The derivation and cross validation groups of this one pattern displayed highly similar WAIS scores on all 11 subtests, but differences between the groups in age, marital status, and some output descriptors were noted. These results were explained in terms of age acting as a moderator variable. In general, the results suggest that patterns of WAIS subtest scores can be identified and that these can be reliably associated with specific behavioral descriptors.

Prediction of individual job termination from measured job satisfaction and biographical data

The Minnesota Satisfaction Questionnaire (MSQ) was administered to a group of 475 employees of a discount store chain at the same time that biographical data were collected. After a lapse of one year, personnel records indicated about 20% of the employees had terminated. “Leavers” were significantly less satisfied on 10 of the 27 MSQ scales and differed from “stayers” on 3 of the 11 biographical items. Several discriminant functions were developed using sets of biographical data alone, the MSQ scales alone and both sets of predictors in combination, to predict termination. The MSQ scales alone resulted in the greatest improvement in the hit rate for predicting “leave” in the cross-validation group.

The measurement of vocational maturity in adults

This investigation was designed to develop a measure of vocational maturity in adults. An Adult Vocational Maturity Inventory was constructed so that its items would reflect on the respondents past vocational decisions. The inventory was then administered to 200 unemployed men, 100 vocational trainees, and 100 graduate students. The total sample was equally divided into item analysis and cross-validation groups. Based on an analyses of the scores for the cross-validation group, the graduate students had the highest level of vocational maturity, the unemployed men had the lowest, and the vocational trainees fell in between the other two. Results of this study lent support for the hypothesis that vocational maturity can be quantitatively measured in adults. Evidence was also given in support of vocational choice theorists who have suggested that occupational maturity can continue into later life stages than adolescence.