National Intelligence Test Research Articles

Speed and the Flynn Effect

We investigated the role of test-taking speed on the Flynn Effect (FE). Our study compared two cohorts of Estonian students (1933/36, n = 888; 2006, n = 912) using 9 subtests from the Estonian adaptation of the National Intelligence Tests (NIT). The speededness of the items and the subtests was found by determining the proportion of unreached items from among the total number of errors (Stafford, 1971). The test-taking speed of the younger cohort was higher in all 9 of the subtests. This suggests that the younger cohort is able to solve more items than the older one. The lack of measurement invariance at the item and subtest level was quantitatively estimated using a method proposed by Dimitrov (2017). The test-taking speed and the non-invariance of the items was strongly, yet inversely correlated (up to - 0.89). The subtests versions that consisted of only invariant items showed no, or a small positive, FE. The subtest versions consisting of only speeded items showed a large positive FE, with cohort differences of up to 50%. If the requirement of measurement invariance is ignored then this effect becomes apparent. The rise in test-taking speed between cohorts can be attributed to an increase in automated responses, which is an outgrowth of modern education (differences in the mandatory age of school attendance, and in the student's readiness to solve abstract items also affected the test-taking speed of the cohorts). We were able to conclude that the younger cohort is faster than the older one.

Predicting the Flynn Effect through word abstractness : Results from the National Intelligence Tests support Flynn's explanation

The current study investigates the Flynn Effect (FE) and its relation to abstract thinking ability. We compare two cohorts of Estonian students (1933/36, n=888; 2006, n=912) using the Concepts (Logical Selection) subtest of the Estonian adaptation of the National Intelligence Tests (NIT). The item presentation order of the subtest correlates with the abstractness of the words used in the items (r=.609) of the subtest. The different test results (right, wrong and missing answers) were analysed in order to make an estimate of the FE magnitude. The FE for abstract thinking ability of those samples was 1.06 Hedges' g (adjusted for guessing). The magnitude of the FE is dependent upon the degree of difficulty of the items (an item's difficulty is estimated by determining its abstractness and its familiarity to students). The more difficult part of the subtest (the second half) showed a FE=1.80 whereas the easier part (the first half) of the subtest showed a FE=.72. Word abstractness was a strong predictor of all the testing results in both cohorts (Beta=.700). The familiarity of words used in the test items has no correlation with the test results if word abstractness is controlled in both cohorts. Our findings support Flynn's explanation that the FE is primarily an indicator of the rise in abstract thinking ability.

A NIT-picking analysis: Abstractness dependence of subtests correlated to their Flynn effect magnitudes

We examine the association between the strength of the Flynn effect in Estonia and highly convergent panel-ratings of the ‘abstractness’ of nine subtests on the National Intelligence Test, in order to test the theory that the Flynn effect results in part from an increase in the use of abstract reference frames in solving cognitive problems. The vectors of abstractness ratings and Flynn effect gains, controlled for guessing) exhibit a near-zero correlation (r=−.02); however, abstractness correlates positively with (and is therefore confounded by) g-loadings (r=.61). A General Linear Model is used to determine the degree to which the abstractness vector predicts the Flynn effect vector, independently of subtest g-loadings and the portion of the secular IQ gain due to guessing (the Brand effect). Consistent with the abstract reasoning model of the Flynn effect, abstractness positively predicts Flynn effect magnitudes, once controlled for confounds (sr=.44), which indicates an increasing tendency to utilize factors external to the items in order to abstract their solutions.

An Investigation of Growth Mixture Models for Studying the Flynn Effect

The Flynn effect (FE) is the well-documented generational increase of mean IQ scores over time, but a methodological issue that has not received much attention in the FE literature is the heterogeneity in change patterns across time. Growth mixture models (GMMs) offer researchers a flexible latent variable framework for examining the potential heterogeneity of change patterns. The article presents: (1) a Monte Carlo investigation of the performance of the various measures of model fit for GMMs in data that resemble previous FE studies; and (2) an application of GMM to the National Intelligence Tests. The Monte Carlo study supported the use of the Bayesian information criterion (BIC) and consistent Akaike information criterion (CAIC) for model selection. The GMM application study resulted in the identification of two classes of participants that had unique change patterns across three time periods. Our studies show that GMMs, when applied carefully, are likely to identify homogeneous subpopulations in FE studies, which may aid in further understanding of the FE.

Data from “Changes in test-taking patterns over time” concerning the Flynn Effect in Estonia

The dataset from our previous Intelligence paper consists of data collected from the National Intelligence Tests (NIT, Estonian adaptation) at two historical time points: in 1933/36 (N=890) and in 2006, (N=913). The average age of the students was 13 years. The data-file consists of information about the cohort, age, and gender and test results at the item level for nine of the ten NIT subtests and subtest scores for the 10 th subtest. The three answer types are separated into three categories: right answers, wrong answers and missing answers. Data can be used for the psychometric research of cohort and sex differences at the scale and item level.

Controlling for increased guessing enhances the independence of the Flynn effect from g: The return of the Brand effect

The cause of the Flynn effect is one of the biggest puzzles in intelligence research. In this study we test the hypothesis that the effect may be even more independent from g than previously thought. This is due to the fact that secular gains in IQ result from at least two sources. First, an authentic Flynn effect that results from environmental improvements and should therefore be strongly negatively related to the g loading (and therefore the heritability) of IQ subtests. Second, a “Brand effect”, which results from an increase in the number of correct answers simply via enhanced guessing. As harder items should encourage more guessing, secular gains in IQ stemming from this Brand effect should be positively associated with subtest g loadings. Analysis of Estonian National Intelligence Test data collected between 1933 and 2006, which includes data on guessing, g loadings and secular IQ gains, corroborates this hypothesis. The correlation between gains via the Brand effect and g loadings is .95, as predicted. There is a modest negative association between raw secular gain magnitude and subtest g loadings (−.18) that increases to −.47 when these are controlled for the Brand effect. Applying five psychometric meta-analytic corrections to this estimate raises it to −.82 indicating that the authentic Flynn effect is substantially more independent from g than previously thought.

An item-level examination of the Flynn effect on the National Intelligence Test in Estonia

This study examined the Flynn effect (FE; i.e., the rise in IQ scores over time) in Estonia using the Estonian version of the National Intelligence Tests (NIT; Haggerty, Terman, Thorndike, Whipple & Yerkes, 1919; National Research Council, 1920). Using secondary data from two cohorts (1934, n=890 and 2006, n=913) of students, we analyzed the NIT's subtests using item response theory (IRT). For each subtest, we first examined invariance in all the items and then linked the latent variable (θ) scores between the two cohorts using the invariant items. The results showed that there was a FE in θ for all subtests except one, although there was much variability in the FE magnitude, ranging from an effect size of 0.24 (3.60 IQ points) to 1.05 (15.75 IQ points). In addition, this study showed there was a decrease in the variability of θ for all the subtests, although only two of the subtests showed large decreases (approximately .50 standard deviations). Last, the subtests' precision of measuring θ was very similar at both time points.

Changes in test-taking patterns over time

The current study aims to investigate the relationship between right, wrong and missing answers to cognitive test items (test-taking patterns) in the context of the Flynn Effect (FE). We compare two cohorts of Estonian students (1933/36, n=890; 2006, n=913) using an Estonian adaptation of the National Intelligence Tests and document three simultaneous trends: fewer missing answers (−1 Cohen's d averaged over subtests), and a rise in the number of right and wrong answers to the subtests (average ds of .86 and .30, respectively). In the Arithmetical Reasoning and Vocabulary subtests, adjustments for false-positive answers (the number of right minus the number of wrong answers) reduced the size of the Flynn Effect by half. These subtests were supposed to be high g-loading subtests. Our conclusion is that rapid guessing has risen over time and influenced tests scores more strongly over the years. The FE is partly explained by changes in test-taking behavior over time.

Establishing an association between the Flynn effect and ability differentiation

The relationship between the Flynn effect and ability differentiation is investigated in a reanalysis of published data on Estonian student cohorts tested in 1933/36, 1997/98 and 2006 on the National Intelligence Test (Must, te Nijenhuis, Must, & van Vianen, 2009). To determine whether there was a relationship we computed the vector correlation between the Flynn effects (d) and the change in the g loading (Δg) between measurement occasions for each of the 10 NIT subtests and for each of the seven cohort comparisons, giving a total N of 70 effect sizes. The association between d and Δg was robustly negative (indicating that the Flynn effects were negatively associated with changes in the g loading of subtests) for all cohort comparisons, with values of r ranging from −.100 to −.461 (N=10). When all effect sizes were analyzed together, the vector correlation was found to be −.281 (p⩽.05, N=70). This indicates a significant association between the Flynn effect and ability differentiation. Possible causes of this association are discussed.

Comparability of IQ scores over time

This study investigates the comparability of IQ scores. Three cohorts (1933/36, 1997/98, 2006) of Estonian students ( N = 2173) are compared using the Estonian National Intelligence Test. After 72 years the secular rise of the IQ test scores is.79 SD. The mean .16 SD increase in the last 8 years suggests a rapid increase of the Flynn Effect (FE) in Estonia. The measurement is not strictly invariant, which means that the IQ scores of different cohorts are not directly comparable. Less than perfect comparability of test scores is caused by at least two factors: time between measurements and societal/educational changes between cohorts. As was to be expected, the meaning of subtests and the meaning of the g score have changed over time.

The secular rise in IQs: In Estonia, the Flynn effect is not a Jensen effect

This study investigated the secular rise in IQ scores over a 60-year period in 12- to 14-year-old Estonian schoolchildren. In 1934/1936, Juhan Tork adapted the U.S. National Intelligence Test for Estonia and administered it to more than 6000 schoolchildren. We administered the same test to 449 students in 1997/1998 and compared the results of 381 of these with a carefully matched sample of 307 from the testing in the 1930s. We found a rise of nearly 1 S.D. on subtests using basic information-processing algorithms such as Comparison and Symbol–Number, but only 0.50 S.D.s on verbal subtests such as Sentence Completion and Concept Comprehension. The secular gains were most pronounced on the low g-loaded subtests. In two compared age groups of children, the rank order correlations between the secular changes on the various subtests and the rank of those subtests on the g factor are negative and nonsignificant, the mean r s=−.40 (one-tailed P=.13). As such, these results supported Rushton's [Pers. Individ. Differ. 26 (1999) 381] finding that the secular rise over time is not occurring on the g factor. In Estonia, the Flynn effect is not a Jensen effect.

Discussion

The relationship between the Flynn effect and ability differentiation is investigated in a reanalysis of published data on Estonian student cohorts tested in 1933/36, 1997/98 and 2006 on the National Intelligence Test (Must, te Nijenhuis, Must, & van Vianen, 2009). To determine whether there was a relationship we computed the vector correlation between the Flynn effects (d) and the change in the g loading (Δg) between measurement occasions for each of the 10 NIT subtests and for each of the seven cohort comparisons, giving a total N of 70 effect sizes. The association between d and Δg was robustly negative (indicating that the Flynn effects were negatively associated with changes in the g loading of subtests) for all cohort comparisons, with values of r ranging from −.100 to −.461 (N = 10). When all effect sizes were analyzed together, the vector correlation was found to be −.281 (p ⩽ .05, N = 70). This indicates a significant association between the Flynn effect and ability differentiation. Possible causes of this association are discussed.

The effects of war upon the intelligence of youth.

A Dutch version of the National Intelligence Test has been given each year to 13-14-year-old candidates of the boys' industrial school attached to the Philips works. IQ's in the post-war years average 4 points lower than in pre-war years, presumably as a result of educational setback during the war. Other tests closely related to school training showed more decline from 1944 to 1947 than tests unrelated to schooling, thus supporting the contention that lack of training was responsible. The evidence shows that IQ reflects any conditions which alter the rate of mental development. (PsycINFO Database Record (c) 2006 APA, all rights reserved)

College degrees and elementary-school intelligence quotients.

Of an original group of 1505 subjects (median IQ 102) tested in elementary school, with the National Intelligence Test, 150 received 1 college degree (median IQ 116) and 17 received 2 college degrees (median IQ 114). (PsycINFO Database Record (c) 2006 APA, all rights reserved)

A Study of Retardation and Special Instruction in Reading

The study reported in this article was undertaken in the interests of the Committee on Retardation appointed by William J. O'Shea, superintendent of schools in New York City. The function of the committee was to find a way to decrease the amount of retardation in the public schools. In May, 1930, all the children in Grades I A-IV B, inclusive, in Public School 208, Brooklyn, were given group tests of intelligence and reading attainment. The group intelligence tests used were the National Intelligence Tests; the Haggerty Intelligence Examination, Delta I; and the Detroit First-Grade Intelligence Test. The group tests of reading attainment selected were the Gates Primary Reading Test; the Gates Advanced Reading Test; the Haggerty Reading Examination, Sigma I; and the Thorndike-McCall Reading Scale. On the basis of their scores on the group intelligence tests, their scores on the tests of reading attainment, and the teachers' estimates of their ability, the children were grouped homogeneously for the September semester.

A Study of Various Factors Which Influence the Use of the Accomplishment Quotient as a Measure of Teaching Efficiency

The present article reports changes in Accomplishment Quotient over a oneand two-year period, for pupils of Grades 3A to 6A in the public schools of Shaker Heights, Ohio.1 The average ability of the children studied is considerably higher than that in the average school system, since the median IQ for the total enrollment of the four elemen tary schools involved is approximately 114, and less than 25 percent of the pupils have IQ's below 100. The teachers who taught these classes were unusually well trained, and perhaps represented a higher selection among teachers in general than did the children among children in gen eral. The school buildings were modern and well-equipped. Classes were small; the average number of pupils per teacher at the end of the period was 19, and the largest class included in this study numbered only 29. In general, conditions for teaching and learning were un usually good. The Stanford Achievement Test was administered in May of each 3rear to all pupils in Grades 2A to 6A, inclusive. Form A was used in 1924 and 1926, and Form B in 1925. In 1926 the 3A pupils took the Advanced Form of the test, since experience in the preceding years indicated that the Primary Form was too easy for the 3A pupils. The mental ages used in computing the AQ were, in general, cal culated from IQ's based on group tests. Stanford-Binet IQ's were avail able for about 15 percent of the pupils, and these were used in preference to the group test IQ's. Most of the latter were obtained from the McCall Multi-mental Scale which was administered during October and November, 1925. In the few cases where the record for that was not available, IQ's were based on the National Intelligence Test A for the older children, and on the Pintner-Cunningham Primary Test for the younger pupils. The two or three mental ages used for each child were based on the same IQ. Thus, in the case of the AQ comparison from May, 1925 to May, 1926, the mental ages were based on a single in telligence test which had been taken half way between the two educa

On the Technique of Establishing the “Goodness” of an Intelligence Test

The following discussion on the technique of certain methods of evalu ating tests is the result of examining somewhat critically the methods pro posed by Dr. V. A. Jones in his recent monograph, The Effect of Age and Experience on Tests of Intelligence.1 The purpose of Dr. Jones' monograph is to investigate the influence of chronological age upon mental age. The methods used are those of partial correlation and the K index. The latter will be explained presently. It is, of course, important to measure mental maturity as distinct as possible from the effects of chronological age, environmental factors, health, etc. Jones has a group of twenty-six tests which he ranks from one to twenty-six as he considers best for measuring MA independently from CA. The total population used in this study is 487. The author classifies the population into three groups: (1) Richmond Sampling, 208 white children from the lower half of the fifth grade. (2) Miscellaneous Sampling, grades 5-7, population 179. (3) Miscellaneous Sampling, grades 3-4, population of 100. The tests studied were the subtests, totaling twenty-six, from the following: National Intelligence Test, Scale A; National Intelligence Test, Scale B ; Haggerty Intelligence Examination, Delta 2 ; Otis Group Intelligence Scale, Advanced; and the Otis Group Intelligence Scale, Primary, for grades 3 and 4.2 Jones uses two criteria to establish the goodness of a test ; first, that the partial correlation between rsma<ca should approach 1, where s can stand for any subtest ; second, that the partial correlation rs ca.ma should approach zero.

Sex Differences in Intelligence-Test Scores in the Elementary School

The outcome of any intelligence test may be regarded as con ditioned by a series of factors, such as age, sex, native ability, school training, practice, and race. Of these factors, comparatively little attention has been paid to sex, perhaps because the sex difference was regarded as of little magnitude or of little practical import. In the classifications of pupils, for example, the scores of either sex have been, doubtless quite properly, compared with the single standard score for the age or grade in question, which has been itself derived from the scores obtained by hundreds of children of both sexes combined, to furnish the standard age score or the standard grade score. Most investigators, however, have been aware of the existence of a small sex difference, and this difference has at least an aca demic interest when it is considered in its possible theoretical rela tions to the rate of physical and mental maturity of the two sexes during childhood and adolescence. Moreover, it possesses still greater interest when the direction of the sex difference is discovered to be reversed in the secondary school. It seems worth while, there fore, to assemble pertinent facts concerning the sex differences in elementary-school scores and then to compare this situation with that revealed in the high school. In this paper, accordingly, I shall present facts concerning two well-known intelligence tests designed for elementary-school use, namely, the National Intelli gence Test, Scale A, and the Illinois General Intelligence Scale. Subsequently, I shall present facts concerning sex differences dis covered by the use of Army Alpha in the high school.

Standard Deviation versus Age as a Score Unit

The purpose of this article1 is to show by certain concrete cases that mental or educational age is often a misleading unit for use in comparing an individual's score on one test with his score on another test, and that a more significant comparison can be made by expressing each score as a deviation from the mean score of the individual's age group. The standard deviation of scores is shown to be a better unit for this purpose than is the percentile.2 A comparison of the mental ages of two individuals, and a comparison of their deviations from the mean of their age group as well, may give contradictory results. The Stanford Binet Test,3 the National Intelligence Test, Scale A, Form I, and the Healy Picture Completion Test, Number II, were given to 78 eleven-year-old public-school boys selected from grades in to viii inclusive in an eastern city. These boys were selected from each grade in proportion to the incidence of eleven-year old boys in that grade in the city. The latest available age norms for the National Intelligence Test and the Healy Picture Com pletion Test were used. The scores obtained are summarized in Table I. According to Table II, on the Stanford Binet Test Indi vidual A attained a raw score of 149, and his mental age was therefore 149 months ; his standard score was + .92. A standard

The Effectiveness of Directed Study

It is the purpose of this article to present the results obtained in a comparative study made of two eighth-grade arithmetic classes, one of which was conducted as a directed-study class and the other as an ordinary recitation class. The classes were of the same size and were made up of boys only. When the seventh-grade records in arithmetic were compared, it was found that there was a slight advantage in favor of the recitation class. This finding was further substantiated by the scores made on the National Intelligence Tests. A sixty-minute period was used for the directed-study class. Part of the time was given to reviewing the work of the preceding day, part to assigning new work, and the remainder to directing the study of the assignment made. There was some variation in the amount of time devoted to each of these procedures, but the time available for study was never less than twenty-five minutes. The class was divided into three groups on the basis of ability. The pupils in the inferior group were given work suited to their ability and essential as part of the course. Those in the middle group were given some work in addition to that given to the pupils in the inferior group. The superior group was given part of the work assigned to the inferior and the middle groups and, in addition, supplementary problems from other books. Home work was not expected. The work done each day was collected at the close of the period. Suggestive directions for the study of arithmetic were given to each pupil in mimeographed form. These directions were discussed, and frequent reference was made to them during the study period. This training in the application of study directions did much to build up proper study habits. The class period for the recitation group was forty minutes in length so as to conform more nearly to the average recitation period. Most of the period was used in reviewing the problems that had