Extreme Response Style Research Articles

Cross-Cultural Variations in Extreme Rejecting and Extreme Affirming Response Styles

Previous research has discussed cultural differences in moderacy vs extremity response styles. The present research found that cultural differences in response styles were more complex than previously speculated. We investigated cross-cultural variations in extreme rejecting versus affirming response biases. Although research has indicated that overall Chinese have less extreme responses than Westerners, the difference may be mainly driven by extreme rejecting responses because respondents consider answering survey questions as a way of interacting with researchers, and extreme rejecting responses may disrupt harmony in relationships, which is valued more in Chinese collectivistic culture than in Western individualistic cultures. Studies 1 and 2 revealed that Chinese had less extreme rejecting response style than did British, whereas they did not differ in extreme affirming response style. Study 2 further revealed that the cross-cultural asymmetry in extreme rejecting versus affirming response styles was partially accounted for by individualism orientation at the individual level. Consistently, Study 3 revealed that at the country level, individualism was positively associated with extreme rejecting response style, but was not associated with extreme affirming response style, suggesting that individualism accounted for the asymmetric cultural variation in extreme rejecting versus affirming response styles.

Combining mixture distribution and multidimensional IRTree models for the measurement of extreme response styles.

Personality constructs, attitudes and other non-cognitive variables are often measured using rating or Likert-type scales, which does not come without problems. Especially in low-stakes assessments, respondents may produce biased responses due to response styles (RS) that reduce the validity and comparability of the measurement. Detecting and correcting RS is not always straightforward because not all respondents show RS and the ones who do may not do so to the same extent or in the same direction. The present study proposes the combination of a multidimensional IRTree model with a mixture distribution item response theory model and illustrates the application of the approach using data from the Programme for the International Assessment of Adult Competencies (PIAAC). This joint approach allows for the differentiation between different latent classes of respondents who show different RS behaviours and respondents who show RS versus respondents who give (largely) unbiased responses. We illustrate the application of the approach by examining extreme RS and show how the resulting latent classes can be further examined using external variables and process data from computer-based assessments to develop a better understanding of response behaviour and RS.

Examination of the Extreme Response Style of Students using IRTree: The Case of TIMSS 2015

In the literature, response style is one of the factors causing an achievement-attitude paradox and threatens the validity of the results obtained from studies. In this regard, the aim of this study is two-fold. Firstly, it attempts to determine which item response tree (IRTree) models based on the generalized linear mixed model (GLMM) approach (random intercept, random intercept with the fixed effect of extreme response and random intercept-slope model) best fit the Trends in International Mathematics and Science Study (TIMSS) 2015 data. Secondly, it purports to explore how the extreme response style affects students’ attitudes toward mathematics of students. This study is both basic research and descriptive research in terms of seeking for answers for two different research questions. For the sample of this research, 15 countries were randomly selected among countries participated in TIMSS 2015. The students’ responses to items measuring attitude in the student questionnaire were analyzed with the packages “lme4” and “irtrees” in R software. When the model fit indices were evaluated, the random intercept-slope model was found to be the best fit to the data. According to this model, the extreme response style explains a significant amount of variances in the students’ attitude toward mathematics. Additionally, students with a negative attitude toward mathematics were found to have an extreme response style. It was concluded that an extreme response style had an effect on students’ attitude.

Evaluation on types of invariance in studying extreme response bias with an IRTree approach.

In recent years, item response tree (IRTree) approaches have received increasing attention in the response style literature for their ability to partial out response style latent variables as well as associated item parameters. When an IRTree approach is adopted to measure extreme response styles, directional and content invariance could be assumed at the latent variable and item parameter levels. In this study, we propose to evaluate the empirical validity of these invariance assumptions by employing a general IRTree model with relaxed invariance assumptions. This would allow us to examine extreme response biases, beyond extreme response styles. With three empirical applications of the proposed evaluation, we find that relaxing some of the invariance assumptions improves the model fit, which suggests that not all assumed invariances are empirically supported. Specifically, at the latent variable level, we find reasonable evidence for directional invariance but mixed evidence for content invariance, although we also find that estimated correlations between content-specific extreme response latent variables are high, hinting at the potential presence of a general extreme response tendency. At the item parameter level, we find no directional or content invariance for thresholds and no content invariance for slopes. We discuss how the variant item parameter estimates obtained from a general IRTree model can offer useful insight to help us understand response bias related to extreme responding measured within the IRTree framework.

A Scaled Threshold Model for Measuring Extreme Response Style

Extreme response style is the tendency of individuals to prefer the extreme categories of a rating scale irrespective of item content. It has been shown repeatedly that individual response style differences affect the reliability and validity of item responses and should, therefore, be considered carefully. To account for extreme response style (ERS) in ordered categorical item responses, it has been proposed to model responder-specific sets of category thresholds in connection with established polytomous item response models. An elegant approach to achieve this is to introduce a responder-specific scaling factor that modifies intervals between thresholds. By individually expanding or contracting intervals between thresholds, preferences for selecting either the outer or inner response categories can be modeled. However, for a responder-specific scaling factor to appropriately account for ERS, there are two important aspects that have not been considered previously and which, if ignored, will lead to questionable model properties. Specifically, the centering of threshold parameters and the type of category probability logit need to be considered carefully. In the present article, a scaled threshold model is proposed, which accounts for these considerations. Instructions on model fitting are given together with SAS PROC NLMIXED program code, and the model’s application and interpretation is demonstrated using simulation studies and two empirical examples.

Extreme response style bias in burn survivors.

This paper explores extreme response style to the Life Impact Burn Recovery Evaluation (LIBRE) Profile, a measure of social participation in burn survivors. We fit a Multidimensional Generalized Partial Credit Model (MGPCM) with a positive extreme response style (PERS) factor and compared this model with the original MGPCM, estimated the impact that PERS has on scores, and examined the personal characteristics that may result in an individual more likely to respond in a fashion that would inflate their true low scores. The average impact of the PERS, based upon the root mean squared bias, ranged from 0.27 to 0.50 of a standard deviation of the scale. Individuals who were older, had participated in a burn survivor support group, and had selected to self-administer the measure were less likely to have a high PERS bias that masks low scores. Future work can consider PERS when measuring the psychosocial impacts of burn injuries and other health conditions.

Clustering preference data in the presence of response-style bias.

Preference data, such as Likert scale data, are often obtained in questionnaire-based surveys. Clustering respondents based on survey items is useful for discovering latent structures. However, cluster analysis of preference data may be affected by response styles, that is, a respondent's systematic response tendencies irrespective of the item content. For example, some respondents may tend to select ratings at the ends of the scale, which is called an 'extreme response style'. A cluster of respondents with an extreme response style can be mistakenly identified as a content-based cluster. To address this problem, we propose a novel method of clustering respondents based on their indicated preferences for a set of items while correcting for response-style bias. We first introduce a new framework to detect, and correct for, response styles by generalizing the definition of response styles used in constrained dual scaling. We then simultaneously correct for response styles and perform a cluster analysis based on the corrected preference data. A simulation study shows that the proposed method yields better clustering accuracy than the existing methods do. We apply the method to empirical data from four different countries concerning social values.

Modeling Response Styles in Cross‐Country Self‐Reports: An Application of a Multilevel Multidimensional Nominal Response Model

AbstractWe examined the feasibility and results of a multilevel multidimensional nominal response model (ML‐MNRM) for measuring both substantive constructs and extreme response style (ERS) across countries. The ML‐MNRM considers within‐country clustering while allowing overall item slopes to vary across items and examination of whether certain items were more prone to ERS. We applied this model to survey items from TALIS 2013. Results indicated that self‐efficacy items were more likely to trigger ERS compared to need for professional development, and the between‐country relationships among constructs can change due to ERS. Simulations assessed the estimation approach and found adequate recovery of model parameters and factor scores. We stress the importance of additional validity studies to improve the cross‐cultural comparability of substantive constructs.

Item Response Tree Models to Investigate Acquiescence and Extreme Response Styles in Likert-Type Rating Scales.

Item response tree (IRTree) models are recently introduced as an approach to modeling response data from Likert-type rating scales. IRTree models are particularly useful to capture a variety of individuals' behaviors involving in item responding. This study employed IRTree models to investigate response styles, which are individuals' tendencies to prefer or avoid certain response categories in a rating scale. Specifically, we introduced two types of IRTree models, descriptive and explanatory models, perceived under a larger modeling framework, called explanatory item response models, proposed by De Boeck and Wilson. This extends the typical application of IRTree models for studying response styles. As a demonstration, we applied the descriptive and explanatory IRTree models to examine acquiescence and extreme response styles in Rosenberg's Self-Esteem Scale. Our findings suggested the presence of two distinct extreme response styles and acquiescence response style in the scale.

IRTree models with ordinal and multidimensional decision nodes for response styles and trait-based rating responses.

IRTree models decompose observed rating responses into sequences of theory-based decision nodes, and they provide a flexible framework for analysing trait-related judgements and response styles. However, most previous applications of IRTree models have been limited to binary decision nodes that reflect qualitatively distinct and unidimensional judgement processes. The present research extends the family of IRTree models for the analysis of response styles to ordinal judgement processes for polytomous decisions and to multidimensional parametrizations of decision nodes. The integration of ordinal judgement processes overcomes the limitation to binary nodes, and it allows researchers to test whether decisions reflect qualitatively distinct response processes or gradual steps on a joint latent continuum. The extension to multidimensional node models enables researchers to specify multiple judgement processes that simultaneously affect the decision between competing response options. Empirical applications highlight the roles of extreme and midpoint response style in rating judgements and show that judgement processes are moderated by different response formats. Model applications with multidimensional decision nodes reveal that decisions among rating categories are jointly informed by trait-related processes and response styles.

MIMIC approach to assessing differential item functioning with control of extreme response style

Likert or rating scales may elicit an extreme response style (ERS), which means that responses to scales do not reflect the ability that is meant to be measured. Research has shown that the presence of ERS could lead to biased scores and thus influence the accuracy of differential item functioning (DIF) detection. In this study, a new method under the multiple-indicators multiple-causes (MIMIC) framework is proposed as a means to eliminate the impact of ERS in DIF detection. The findings from a series of simulations showed that a difference in ERS between groups caused inflated false-positive rates and deflated true-positive rates in DIF detection when ERS was not taken into account. The modified MIMIC model, as compared to conventional MIMIC, logistic discriminant function analysis, ordinal logistic regression, and their extensions, could control false-positive rates across situations and yielded trustworthy true-positive rates. An empirical example from a study of Chinese marital resilience was analyzed to demonstrate the proposed model.

Methods for the Control of Extreme Response Styles in Self-Report Instruments: A Review

Abstract Response styles are systematic ways of responding to self-report items that may impact the validity and the precision of scores from instruments. One of these biases is extreme responding (ER), which occurs when a person tends to use only extreme rating categories from a response scale (e.g., totally disagree or totally agree), irrespective of item content. Many different methods were developed that aim to identify and control extreme responses to provide a more accurate assessment of an individual's trait. The aim of this study is to perform a systematic review of these main techniques for statistical control of extreme responses in psychometric instruments of self-report. We identified several analytical approaches, which we organized into seven clusters, from simple count of the numbers of extreme response to the use of modern statistics methods, as Item Response Theory uni and multidimensional. Advantages and limitations of each method are discussed. We also present a general diagram that summarizes the distinct available methods we found.

Dysfunctional attitudes or extreme response style as predictors of depressive relapse and recurrence after mobile cognitive therapy for recurrent depression

BackgroundAccording to previous research, dysfunctional attitudes and/or scoring extreme on the end-point anchors of questionnaires of dysfunctional thinking predict depressive relapse/recurrence. Evidence that these two methods represent a risk for depressive relapse/recurrence is however mixed, due to differential or poorly defined concepts. The current study aimed to test the two methods. MethodsRemitted recurrently depressed patients with low residual depressive symptoms (N = 264) were recruited as part of a randomized controlled trial of the effectiveness of mobile Cognitive Therapy for recurrent depression versus treatment as usual. In the current secondary analysis, Cox regression models were conducted to test dysfunctional attitudes and extreme responding variables (assessed on the Dysfunctional Attitudes Scale [DAS]) as predictors of depressive relapse/recurrence within two years after randomization. ResultsData from 255 participants were analyzed. Results showed that DAS total scores at baseline significantly predicted depressive relapse/recurrence (Hazard Ratio [HR] = 1.01, p = .042). An index that reflects endorsement of habitual relative to functional responses was a significant predictor of depressive relapse/recurrence (HR = 2.11, p = .029). LimitationsThe current study employed a single measure to identify extreme responses and dysfunctional attitudes. Secondly, various statistical analyses were performed without correcting for multiple testing, which in turn increased the likelihood to finding significant results. ConclusionsCurrent study confirmed both methods: People who scored higher on the DAS or had relatively more habitual than functional responses on the extreme positive ends of the DAS had a decreased time to depressive relapse/recurrence.

The heritability of response styles and its impact on heritability estimates of personality: A twin study

Response styles (RS) are an important research matter in questionnaire research and have even been interpreted as implicit personality variables aside from explicit personality measurement. To better understand RS and their impact on explicit personality measurement, it is necessary to explore causes of variability in RS, including heritability. Therefore, the current study followed two aims: First, we investigated the heritability of two of the most common RS (acquiescence RS and extreme RS; ARS & ERS). Second, we tested whether correcting for RS affects the genetic component for and correlations among explicit personality measures. We used an extended classical twin design with twins (n = 464) and non-twin siblings (n = 242). Personality was measured administering the Affective Neuroscience Personality Scales (ANPS). Results indicated a heritability of 37% for ARS and 57% for ERS. This difference between the two RS might be explained by their genesis: ARS has been associated with social adaption, and thus might depend primarily on environmental experiences, while ERS has been related to anxiety and temperament, which are majorly hereditary. Correcting explicit personality for RS did not change estimations of heritability for the ANPS traits suggesting that RS does not majorly influence estimations of heritability for explicitly measured personality.

276 Extreme Response Style Bias in Burn Survivors: The LIBRE Project

This study explores the extreme response style (ERS) of burn survivors who completed the Life Impact Burn Recovery Evaluation (LIBRE) Profile. The LIBRE Profile is an innovative patient reported outcome measure (PROM) that assesses social participation in burn survivors. Clinicians have recently begun to use PROMs to optimize care, however, PROMs can be sensitive to response style biases. ERS is a bias that occurs when an individual tends to select either the most positive or most negative response option of an item, when that choice does not reflect the person’s true ability or score. The objective of this study is to determine if the ERS distorts the LIBRE Profile scores. We fit a Multidimensional Generalized Practical Credit Model (MGPCM) with a high extreme response style (HERS) factor. We looked only at HERS because there were few participants who selected the low extreme response option. Next, we compared this model with the original MGPCM, which is not adjusted by the HERS bias, to estimate the impact that HERS has on scores. Lastly, we examined the personal characteristics, such as gender, age, time since burn, mode of administration and participation in peer support groups that may influence if an individual more likely to respond in a particular response style fashion would inflate their true low scores. The average impact of the HERS, based upon the root mean squared bias, ranged from 0.27–0.50 of a standard deviation of the scale. Individuals who were older, had participated in a burn survivor support group, and had selected to self-administer the measure were less likely to have a high HERS bias that masks low scores. Burn survivors displayed evidence of the HERS bias impacting their true LIBRE Profile scores. Burn survivors are a clinically unique group with special personal characteristics and clinical histories. In order to improve the clinical usefulness of the LIBRE Profile, measurement and understanding of extreme response styles is essential to aid interpretation of the scaled results. In future administrations, the HERS bias should be estimated to assess the occurrence of the bias and if it is affecting scores. Future work can consider this bias when measuring the psychosocial impacts of burn injuries and other health conditions.

Bayesian Analysis of Multidimensional Item Response Theory Models: A Discussion and Illustration of Three Response Style Models

ABSTRACTInterest in Bayesian analysis of item response theory (IRT) models has grown tremendously due to the appeal of the paradigm among psychometricians, advantages of these methods when analyzing complex models, and availability of general-purpose software. Possible models include models which reflect multidimensionality due to designed test structure, construct-irrelevant variance, and mixed item formats. Using Markov Chain Monte Carlo methods, models can be estimated and evaluated for model-fit. In addition to discussing Bayesian analysis, analyses of three IRT models designed to account for extreme response style are illustrated: IRTree, multidimensional nominal response model (MNRM), and modified generalized partial credit model (MPCM). While results indicated there may be little impact of model choice on substantive trait estimates for the data set studied herein, model-fit results favored the MNRM and MPCM over the IRTree Model.

Extreme Response Style and the Measurement of Intra-Individual Variability in Affect

ABSTRACTExtreme response style (ERS) has the potential to bias the measurement of intra-individual variability in psychological constructs. This paper explores such bias through a multilevel extension of a latent trait model for modeling response styles applied to repeated measures rating scale data. Modeling responses to multi-item scales of positive and negative affect collected from smokers at clinic visits following a smoking cessation attempt revealed considerable ERS bias in the intra-individual sum score variances. In addition, simulation studies suggest the magnitude and direction of bias due to ERS is heavily dependent on the mean affect level, supporting a model-based approach to the study and control of ERS effects. Application of the proposed model-based adjustment is found to improve intra-individual variability as a predictor of smoking cessation.

Multidimensional Modeling of Traits and Response Styles

Abstract. Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

The Graded Response Differential Discrimination Model Accounting for Extreme Response Style

ABSTRACTExtreme response style or, more generally, individual differences in response spacing have been shown to be an influential bias when analyzing questionnaire data. Recently a promising model adjusting for this bias — the differential discrimination model — has been proposed. An advantage to other related approaches is that the model can be fitted using standard structural equation modeling software. However, the model is designed for analyzing continuous item responses, whereas graded response formats are certainly more prominent in behavioral sciences. To resolve this limitation, the present article extends the differential discrimination model to analyzing graded responses. Empirical examples as well as a small simulation study are presented.

Modified Logistic Regression Approaches to Eliminating the Impact of Response Styles on DIF Detection in Likert-Type Scales.

Extreme response styles (ERS) is prevalent in Likert- or rating-type data but previous research has not well-addressed their impact on differential item functioning (DIF) assessments. This study aimed to fill in the knowledge gap and examined their influence on the performances of logistic regression (LR) approaches in DIF detections, including the ordinal logistic regression (OLR) and the logistic discriminant functional analysis (LDFA). Results indicated that both the standard OLR and LDFA yielded severely inflated false positive rates as the magnitude of the differences in ERS increased between two groups. This study proposed a class of modified LR approaches to eliminating the ERS effect on DIF assessment. These proposed modifications showed satisfactory control of false positive rates when no DIF items existed and yielded a better control of false positive rates and more accurate true positive rates under DIF conditions than the conventional LR approaches did. In conclusion, the proposed modifications are recommended in survey research when there are multiple group or cultural groups.