The Effect of Sample Size on Parametric and Nonparametric Factor Analytical Methods

Abstract

Educational and psychological tests are extensively used and effectively contribute to many fields. For example, in clinical applications, they can detect severe emotional disorders and behavioural problems, assess teaching programs, determine learning deficiencies, classify students by their capabilities and select eligible recipients of diplomas. Psychological tests are also used to select industrial personnel by classifying and determining a potential employee's professional skills. Traditionally, psychological tests measure differences among individuals or the responses of the same individuals under different conditions. Well-structured tests can provide an accurate measurement of these individual differences (Anastasi & Urbina, 1997).Investigating the constructs underlying the responses is one of the most important stages of assessing test structures as well as developing, evaluating and continuing large-scale tests. Such an assessment offers empirical evidence for the cognitive processes and content aspects of the test validity (as cited in Tate, 2003). Determining the dimensionality of a group of variables is important when constructing a psychological theory and developing a scale (Timmerman & Lorenzo-Seva, 2011).Dimensionality is also applied in hypotheses testing of homogeneity in classical test theory (CTT), in item response theory (IRT). The former provides logical justification for behaviours related to psychological constructs. From the CTT perspective, the items on a psychometric homogeneous test measure only one attribute of a common factor. This type of item set can be defined as unidimensional, because it indicates variation of respondents on a single dimension (McDonald, 1999). The items of CTT models have been hypothesized to measure the same dominant dimension (Nandakumar & Stout, 1993). If evidence for unidimensionality is obtained, Cronbach alpha coefficient (which determines the reliability in CTT) could be calculated for these measures. In this case, the obtained value is close to the real reliability (Cotton, Campbell, & Malone, 1957; Yang & Green, 2011). The of basic assumption of IRT directly affects on the IRT models, the obtained items and ability parameters, the test equating and test scaling parameters and the model-data fit indices (Embretson & Reise, 2000; Hambleton & Swaminathan, 1985; Kolen & Brennan, 2004). Therefore, it is important to assess the dimensionality from the CTT and IRT perspectives.Factor analysis was developed as a tool for determining psychological attributes and is particularly related to the construct validity process. Construct validity is important when considering the nature and number of dimensions underlying the responses of an aptitude test or an attitude scale (Anastasi & Urbina, 1997; Embretson & Reise, 2000; McDonald, 1999). The dimensionality of a test can be assessed from the test specifications, (which envelope the achievement domain and determine a representative sample of items from this domain), content analysis (performed by a test development specialist) and psychological analysis methods (which formulate a hypothesized item structure from a psychological perspective) (Ackerman, Gierl, & Walker, 2003). The factor structure can be analyzed by principal component analysis (PCA), principal axis factoring (PAF), unweighted least squares (ULS), generalized least squares (GLS), maximum likelihood (ML), alpha factoring (AF), image factoring (IF) techniques. The factor number is then determined by the Kaiser criterion, scree plot, eigenvalues and parallel analysis, of which parallel analysis is the most recommended method (Timmerman & Lorenzo-Seva, 2011). However, linear factor analysis models used to examine the dimensionality of a test are not suitable for item-level data, because most achievement tests are dichotomous and because personality and behavior scale items have dichotomous or polytomous response formats. …