Number Of Response Categories Research Articles

Evaluation of manual ability in stroke patients in Benin: cultural adaptation and Rasch validation of the ABILHAND-Stroke questionnaire.

ABILHAND is a self-reported questionnaire assessing manual ability. It was validated and calibrated using the Rasch analysis for European stroke patients. After a stroke, performing upper limb activities of daily living is influenced by personal and environmental contextual factors. It is thus important to conduct a contextual validation to use this questionnaire outside of Europe. The aim of this study was to perform a cross cultural validation of the ABILHAND-Stroke questionnaire for post-stroke patients living in Benin, a West-African country. Observational cross-sectional study. Outpatient rehabilitation centres. 223 Beninese chronic stroke patients. The experimental questionnaire was made of 59 items evaluating manual activities. Patients had to estimate their difficulty of performing each activity according to four response categories: impossible, very difficult, difficult and easy. For construct validity analysis, patients were also evaluated with other assessment tools: Box and Block Test, the motor subscale of the Functional Independence Measure, the Stroke Impairment Assessment Set, and ACTIVLIM-Stroke. Data were analysed with the Rasch partial credit model. The response categories very difficult and difficult were merged and the number of response categories was reduced from 4 to 3 (impossible, difficult and easy). The Rasch analyses selected 16 bimanual activities that fit the Rasch model (chi square=42.35; P=0.10). The item location ranged from -1.10 to 2.24 logits. The standard error ranged from 0.15 to 0.22 logits. There is no differential item functioning between subgroups (age, sex, dexterity, affected side, time since stroke). The person separation index is 0.82. The questionnaire can measure 3 levels of manual ability, similarly to the occidental version. The ABILHAND-stroke is a Rasch validated, unidimensional and invariant questionnaire to assess manual ability among Beninese patients. The ordinal score can be transformed into linear score using a conversion table. This assessment tool is clinically relevant in Benin, a developing country, since it requires no specific equipment or training. It should promote and standardize assessments for stroke patients in clinical practice and research in this African country.

Effects of varying numbers of Likert scale points on factor structure of the Rosenberg Self‐Esteem Scale

Likert‐type rating scales are among the most widely used tools in psychological research. Different numbers of response categories would likely affect response style, data distribution, reliability, and construct validity. There is a lack of research in factor structure invariance under Likert scales with different numbers of categories. The purpose of this study is to examine the effects of varying numbers of Likert points (4–11) on scale properties such as factor structure, external validity, and latent means based on the Rosenberg Self‐Esteem Scale (M. Rosenberg, ). The sample consists of 1,807 students from secondary schools in Macau. Confirmatory factor analysis shows that the correlated two‐factor model is the most appropriate one; longitudinal invariance analysis reveals that measurement invariance across Likert scales was satisfied at the scalar level. In addition, latent mean scores on the two factors as well as observed means on the subscales are comparable across Likert scales. Moreover, the measurement model across Likert scales exhibit similar external validity. Although psychometric properties are mostly similar among a different number of points, the 4‐point Likert scale is not recommended for its higher skewness and lower loadings; the 11‐point Likert scale from 0 to 10 is slightly preferred for its higher loadings and composite reliability.

Categorical Omega With Small Sample Sizes via Bayesian Estimation: An Alternative to Frequentist Estimators.

When item scores are ordered categorical, categorical omega can be computed based on the parameter estimates from a factor analysis model using frequentist estimators such as diagonally weighted least squares. When the sample size is relatively small and thresholds are different across items, using diagonally weighted least squares can yield a substantially biased estimate of categorical omega. In this study, we applied Bayesian estimation methods for computing categorical omega. The simulation study investigated the performance of categorical omega under a variety of conditions through manipulating the scale length, number of response categories, distributions of the categorical variable, heterogeneities of thresholds across items, and prior distributions for model parameters. The Bayes estimator appears to be a promising method for estimating categorical omega. Mplus and SAS codes for computing categorical omega were provided.

Ordinal response generalized difference in differences with varying categories: The health effect of a disability program in Korea.

We consider the use of difference in differences (DD) to evaluate the effect of an activity assistance program on the health of severely disabled Koreans. There are, however, 2 problems in the data: the number of response categories for self-assessed health changed over the waves of the repeated cross-section survey and the "parallel untreated response path" assumption for DD is suspect. We show how to overcome these problems by renormalizing parameters and applying "generalized difference in differences (GDD)." We find a significantly positive effect of the program with DD, but not with GDD. Our solutions should prove useful in applications in which one or other of these problems arise.

The Effect of Small Sample Size on Measurement Equivalence of Psychometric Questionnaires in MIMIC Model: A Simulation Study

Evaluating measurement equivalence (also known as differential item functioning (DIF)) is an important part of the process of validating psychometric questionnaires. This study aimed at evaluating the multiple indicators multiple causes (MIMIC) model for DIF detection when latent construct distribution is nonnormal and the focal group sample size is small. In this simulation-based study, Type I error rates and power of MIMIC model for detecting uniform-DIF were investigated under different combinations of reference to focal group sample size ratio, magnitude of the uniform-DIF effect, scale length, the number of response categories, and latent trait distribution. Moderate and high skewness in the latent trait distribution led to a decrease of 0.33% and 0.47% power of MIMIC model for detecting uniform-DIF, respectively. The findings indicated that, by increasing the scale length, the number of response categories and magnitude DIF improved the power of MIMIC model, by 3.47%, 4.83%, and 20.35%, respectively; it also decreased Type I error of MIMIC approach by 2.81%, 5.66%, and 0.04%, respectively. This study revealed that power of MIMIC model was at an acceptable level when latent trait distributions were skewed. However, empirical Type I error rate was slightly greater than nominal significance level. Consequently, the MIMIC was recommended for detection of uniform-DIF when latent construct distribution is nonnormal and the focal group sample size is small.

Latent Growth Curve Modeling of Ordinal Scales: A Comparison of Three Strategies

Ordinal scales can be used in latent growth curve modeling in three ways: mean, weighted mean scores, and factors measured by scale items. Sum and mean scores are commonly used in growth curve modeling in spite of certain discouragement. It was unclear how much bias these practices could produce in terms of the change rates and patterns. This study compared three methods with Monte Carlo Simulations under different number of response categories of the items, in terms of five key parameters of growth curve modeling. The hypothetical population models were derived from real empirical data to generate datasets of binary, trichotomous, five- and seven-point scales with sample size of 300. Latent growth curve modeling of mean, weighted mean, and factors measured by the ordinal scales were respectively fit to these datasets. Results indicated that modeling the factors that are measured with ordinal scales yield the fewest biases. Biases of modeling the means and weighted of the scales were under one decimal point in the change rates, whereas biases in the variances and covariance of the intercept and slope factors were large. In conclusion, it is inadvisable to use means or weighted means of ordinal scales for latent growth curve modeling. It produces the best results modeling the factors that are measured with the ordinal scales.

Critical Values for Yen's Q3: Identification of Local Dependence in the Rasch Model Using Residual Correlations.

The assumption of local independence is central to all item response theory (IRT) models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3, there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence (LD), and for this reason, a variety of arbitrary rules of thumb are used. In this study, an empirical data example and Monte Carlo simulation were used to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about LD during scale development and validation. A parametric bootstrapping procedure should be implemented in each separate situation to obtain the critical value of LD applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic, no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories. Furthermore, the results show that LD should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic.

The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables.

Three estimation methods with robust corrections-maximum likelihood (ML) using the sample covariance matrix, unweighted least squares (ULS) using a polychoric correlation matrix, and diagonally weighted least squares (DWLS) using a polychoric correlation matrix-have been proposed in the literature, and are considered to be superior to normal theory-based maximum likelihood when observed variables in latent variable models are ordinal. A Monte Carlo simulation study was carried out to compare the performance of ML, DWLS, and ULS in estimating model parameters, and their robust corrections to standard errors, and chi-square statistics in a structural equation model with ordinal observed variables. Eighty-four conditions, characterized by different ordinal observed distribution shapes, numbers of response categories, and sample sizes were investigated. Results reveal that (a) DWLS and ULS yield more accurate factor loading estimates than ML across all conditions; (b) DWLS and ULS produce more accurate interfactor correlation estimates than ML in almost every condition; (c) structural coefficient estimates from DWLS and ULS outperform ML estimates in nearly all asymmetric data conditions; (d) robust standard errors of parameter estimates obtained with robust ML are more accurate than those produced by DWLS and ULS across most conditions; and (e) regarding robust chi-square statistics, robust ML is inferior to DWLS and ULS in controlling for Type I error in almost every condition, unless a large sample is used (N = 1,000). Finally, implications of the findings are discussed, as are the limitations of this study as well as potential directions for future research. (PsycINFO Database Record

A Meta-Analytic Investigation of the Relationship Between Scale-Item Length, Label Format, and Reliability

Abstract. Using two meta-analytic datasets, we investigated the effect that two scale-item characteristics – number of item response categories and item response-category label format – have on the reliability of multi-item rating scales. The first dataset contained 289 reliability coefficients harvested from 100 samples that measured Big Five traits. The second dataset contained 2,524 reliability coefficients harvested from 381 samples that measured a wide variety of constructs in psychology, marketing, management, and education. We performed moderator analyses on the two datasets with the two item characteristics and their interaction. As expected, as the number of item response categories increased, so did reliability, but more importantly, there was a significant interaction between the number of item response categories and item response-category label format. Increasing the number of response categories increased reliabilities for scale-items with all response categories labeled more so than for other item response-category label formats. We explain that the interaction may be due to both statistical and psychological factors. The present results help to explain why findings on the relationships between the two scale-item characteristics and reliability have been mixed.

Construct validity and reliability of Olweus Bully/Victim Questionnaire – Brazilian version

The Revised Olweus Bully/Victim Questionnaire (OBVQ) is among the few bullying assessment instruments with well-established psychometric properties in different countries. Nevertheless, the psychometric properties of the Brazilian version (Questionario de Bullying de Olweus - QBO) have not been determined. We aimed at verifying the construct validity and reliability of the bully and victim scales of the QBO. To achieve that goal, the victim and bully scales were assessed using polytomous item response theory (IRT). The best fit was obtained with a generalized partial credit model that is capable of measuring the specific discriminating power for each item in these scales. The QBO was administered to 703 public school students (mean age: 13 years; standard deviation = 1.58). Based on IRT analysis, the number of response categories in each item was reduced from four to three. Cronbach reliability scores were satisfactory: α = 0.85 (victim scale) and α = 0.87 (bully scale). In this study, hurtful comments, persecution, or threats had high power to discriminate victims and bullies. For both QBO scales, higher severity parameters were observed for direct bullying items. The results also show that the construct of both QBO scales measures the same construct proposed for the overall instrument. Thus, the QBO can be administered to different Brazilian populations to assess the main characteristics of bullying: repetition of behavior over time and intentionally acting to humiliate, threaten, or harm somebody.

Comparing the Effects of Different Smoothing Algorithms on the Assessment of Dimensionality of Ordered Categorical Items with Parallel Analysis.

The analysis of polychoric correlations via principal component analysis and exploratory factor analysis are well-known approaches to determine the dimensionality of ordered categorical items. However, the application of these approaches has been considered as critical due to the possible indefiniteness of the polychoric correlation matrix. A possible solution to this problem is the application of smoothing algorithms. This study compared the effects of three smoothing algorithms, based on the Frobenius norm, the adaption of the eigenvalues and eigenvectors, and on minimum-trace factor analysis, on the accuracy of various variations of parallel analysis by the means of a simulation study. We simulated different datasets which varied with respect to the size of the respondent sample, the size of the item set, the underlying factor model, the skewness of the response distributions and the number of response categories in each item. We found that a parallel analysis and principal component analysis of smoothed polychoric and Pearson correlations led to the most accurate results in detecting the number of major factors in simulated datasets when compared to the other methods we investigated. Of the methods used for smoothing polychoric correlation matrices, we recommend the algorithm based on minimum trace factor analysis.

Modelling scale effects and uncertainty in rating surveys

In social and behavioural sciences the optimal use of rating scales is an important issue. Discussions on the number of response categories that optimizes the psychometric proprieties of the scales and sums up the global amount of information deriving by the response patterns are the aim of several studies. In a parametric context, this paper faces two specific topics: verify if the number of modalities affects the interpretation of responses and suggest a transformation of the observed ratings distribution when a comparison of results with a different numbers of response categories is necessary. Analysis is carried out within the statistical framework introduced for the study of feeling and uncertainty components in the process of responses which generate ordinal scores. Some empirical evidences and a simulation experiment support the usefulness of the approach.

Can we augment web responses with telephonic responses to a graduate destination survey?

The 2010 Western Cape graduate destination survey utilised a sequential mixed-mode design in which an initial web survey was augmented with an equivalent telephonic survey. This article examines mode effect in the Western Cape survey in terms of overall effect size and the bearing it had on the main outcome of the study. Standardised residuals and Cramér’s V are used to determine mode effect across two scenarios, a full sample vs. a subsample, and using two categorical questions with different numbers of response categories. Overall effect size appears to be small in the first question, but increases noticeably together with non-responses in the second question that has many more response categories. Web responses to alumni or graduate destination surveys can perhaps be augmented with telephonic responses if necessary, provided response categories are kept to a minimum, and interviewers are trained properly and monitored for possible interviewer misbehaviour. The benefit of obtaining larger samples should then also outweigh the benefit of using web surveys alone.

Are judgments a form of data clustering? Reexamining contrast effects with the k-means algorithm.

A number of theories have been proposed to explain in precise mathematical terms how statistical parameters and sequential properties of stimulus distributions affect category ratings. Various contextual factors such as the mean, the midrange, and the median of the stimuli; the stimulus range; the percentile rank of each stimulus; and the order of appearance have been assumed to influence judgmental contrast. A data clustering reinterpretation of judgmental relativity is offered wherein the influence of the initial choice of centroids on judgmental contrast involves 2 combined frequency and consistency tendencies. Accounts of the k-means algorithm are provided, showing good agreement with effects observed on multiple distribution shapes and with a variety of interaction effects relating to the number of stimuli, the number of response categories, and the method of skewing. Experiment 1 demonstrates that centroid initialization accounts for contrast effects obtained with stretched distributions. Experiment 2 demonstrates that the iterative convergence inherent to the k-means algorithm accounts for the contrast reduction observed across repeated blocks of trials. The concept of within-cluster variance minimization is discussed, as is the applicability of a backward k-means calculation method for inferring, from empirical data, the values of the centroids that would serve as a representation of the judgmental context.

Refinement of the SWN-20 based on the Rasch rating model

ObjectivesThe aim of the present study was to refine the 20-item Subjective Well-Being under Neuroleptic Treatment Scale (SWN-20) using the Rasch rating model to validate measurements of subjective well-being in patients with schizophrenia undergoing antipsychotic treatment. MethodsIn total, 854 (403 males, 451 females) inpatients (n=213) and outpatients (n=641) with schizophrenia participated in this study, which was designed as an open-label investigation of paliperidone extended release. The participants completed the Korean version of the SWN-20 themselves. Refinement of the Korean version of the SWN-20 was accomplished using the Rasch rating model. ResultsInfit and outfit statistics for all 20 items satisfied the criterion for construct validity. Second, all items except items 2 and 20 had suitable point–measure correlations, reflecting content validity. Third, item characteristic curves indicated that roughly 18 items were evenly distributed along the person ability continuum. Finally, option analysis of the category characteristics showed that categories 3 and 4 in the SWN-20 response format were unnecessary. ConclusionsWe offer several recommendations for improving the SWN-20: (a) items 2 and 20 should be omitted to ensure construct validity; (b) easier items would be added related to the person ability estimates in the process of validating a short form of the SWN scale based on item response theory; and (c) the number of response categories should be reduced for schizophrenic patients.

Longitudinal Models for Ordinal Data With Many Zeros and Varying Numbers of Response Categories

Ordinal response scales are often used to survey behaviors, including data collected in longitudinal studies. Advanced analytic methods are now widely available for longitudinal data. This study evaluates the performance of 4 methods as applied to ordinal measures that differ by the number of response categories and that include many zeros. The methods considered are hierarchical linear models (HLMs), growth mixture mixed models (GMMMs), latent class growth analysis (LCGA), and 2-part latent growth models (2PLGMs). The methods are evaluated by applying each to empirical response data in which the number of response categories is varied. The methods are applied to each outcome variable, first treating the outcome as continuous and then as ordinal, to compare the performance of the methods given both a different number of response categories and treatment of the variables as continuous versus ordinal. We conclude that although the 2PLGM might be preferred, no method might be ideal.

Bayesian Estimation of a Multidimensional Additive Graded Response Model for Correlated Traits

The aim of the article is to propose a Bayesian estimation through Markov chain Monte Carlo of a multidimensional item response theory model for graded responses with an additive structure with correlated latent traits. A simulation study is conducted to evaluate the model parameter recovery under different conditions (sample size, test and subtest length, number of response categories, and correlation structure). The results show that the parameters are well reproduced when the sample size is sufficiently large (n = 1, 000), while the worst recovery is observed for small sample size (n = 500), and four response categories with a short number of test items.

Identifying Guttman Structures in Incomplete Rasch Datasets

In applications of IRT, it often happens that many examinees omit a substantial proportion of item responses. This can occur for various reasons, though it may well be due to no more than the simple fact of design incompleteness. In such circumstances, literature not infrequently refers to various types of estimation problem, often in terms of generic “convergence problems” in the software used to estimate model parameters. With reference to the Partial Credit Model and the instance of data missing at random, this article demonstrates that as their number increases, so does that of anomalous datasets, intended as those not corresponding to a finite estimate of (the vector parameter that identifies) the model. Moreover, the necessary and sufficient conditions for the existence and uniqueness of the maximum likelihood estimation of the Partial Credit Model (and hence, in particular, the Rasch model) in the case of incomplete data are given – with reference to the model in its more general form, the number of response categories varying according to item. A taxonomy of possible cases of anomaly is then presented, together with an algorithm useful in diagnostics.

A new look at Horn’s parallel analysis with ordinal variables.

Previous research evaluating the performance of Horn's parallel analysis (PA) factor retention method with ordinal variables has produced unexpected findings. Specifically, PA with Pearson correlations has performed as well as or better than PA with the more theoretically appropriate polychoric correlations. Seeking to clarify these findings, the current study employed a more comprehensive simulation study that included the systematic manipulation of 7 factors related to the data (sample size, factor loading, number of variables per factor, number of factors, factor correlation, number of response categories, and skewness) as well as 3 factors related to the PA method (type of correlation matrix, extraction method, and eigenvalue percentile). The results from the simulation study show that PA with either Pearson or polychoric correlations is particularly sensitive to the sample size, factor loadings, number of variables per factor, and factor correlations. However, whereas PA with polychorics is relatively robust to the skewness of the ordinal variables, PA with Pearson correlations frequently retains difficulty factors and is generally inaccurate with large levels of skewness. In light of these findings, we recommend the use of PA with polychoric correlations for the dimensionality assessment of ordinal-level data.

Performance of Velicer’s Minimum Average Partial Factor Retention Method With Categorical Variables

Despite strong evidence supporting the use of Velicer’s minimum average partial (MAP) method to establish the dimensionality of continuous variables, little is known about its performance with categorical data. Seeking to fill this void, the current study takes an in-depth look at the performance of the MAP procedure in the presence of ordinal-level measurement. Using Monte Carlo methods, seven factors related to the data (sample size, factor loading, number of variables per factor, number of factors, factor correlation, number of response categories, and skewness) as well as two factors related to the MAP method (type of correlation matrix and power) were systematically manipulated. The results indicate that using polychoric correlations and the squared partial correlations leads to considerably more accurate estimations than using Pearson correlations and/or raising the partial correlations to the fourth power. Additionally, the MAP method is shown to be a biased estimator of dimensionality in two conditions: (a) for low factor loadings (.40) and (b) for medium factor loadings (.55) and a small number of variables per factor (≤ 6). The applicability of this method with categorical variables is discussed in the context of these findings.