Distinguishing Between Models for Extreme and Midpoint Response Styles as Opposite Poles of a Single Dimension versus Two Separate Dimensions: A Simulation Study.

Response styles, the general tendency to use certain categories of rating scales over others, are a threat to the reliability and validity of self-report measures. The mixed partial credit model, the multidimensional nominal response model, and the item response tree model are three widely used models for measuring extreme and midpoint response styles and correcting their effects. This research aimed to examine and compare their validity by fitting them to empirical data and correlating the content-related factors and the response style-related factors in these models to extraneous criteria. The results showed that the content factors yielded by these models were moderately related to the content criterion and not related to the response style criteria. The response style factors were moderately related to the response style criteria and weakly related to the content criterion. Simultaneous analysis of more than one scale could improve their validity for measuring response styles. These findings indicate that the three models could control and measure extreme and midpoint response styles, though the validity of the mPCM for measuring response styles was not good in some cases. Overall, the multidimensional nominal response model performed slightly better than the other two models.

To improve the validity of self-report measures, researchers should control for response style (RS) effects, which can be achieved with IRTree models. A traditional IRTree model considers a response as a combination of distinct decision-making processes, where the substantive trait affects the decision on response direction, while decisions about choosing the middle category or extreme categories are largely determined by midpoint RS (MRS) and extreme RS (ERS). One limitation of traditional IRTree models is the assumption that all respondents utilize the same set of RS in their response strategies, whereas it can be assumed that the nature and the strength of RS effects can differ between individuals. To address this limitation, we propose a mixture multidimensional IRTree (MM-IRTree) model that detects heterogeneity in response strategies. The MM-IRTree model comprises four latent classes of respondents, each associated with a different set of RS traits in addition to the substantive trait. More specifically, the class-specific response strategies involve (1) only ERS in the "ERS only" class, (2) only MRS in the "MRS only" class, (3) both ERS and MRS in the "2RS" class, and (4) neither ERS nor MRS in the "0RS" class. In a simulation study, we showed that the MM-IRTree model performed well in recovering model parameters and class memberships, whereas the traditional IRTree approach showed poor performance if the population includes a mixture of response strategies. In an application to empirical data, the MM-IRTree model revealed distinct classes with noticeable class sizes, suggesting that respondents indeed utilize different response strategies.

Studies concerning the impact of the length of response scales on the measurement of attitudes have primarily focused on the method bias associated with question format. At the same time another line of research has focused on the issue of response styles that affect how respondents answer to attitude questions. So far, research has paid less attention to the issue of whether the length of the response scales is related to response styles. In this study, we explore if differences in length of the response scale (i.e., method factor) have differential effects in evoking extreme and midpoint response style behavior (i.e., style factor). Our hypotheses read as follows. As the number of response categories increases, we expect subjects to be more likely to exert extreme response style. Furthermore, we expect subjects to be more likely to adopt a midpoint response style when they are offered a middle response category. To investigate these hypotheses we developed a split ballot experiment in which the number of response categories is manipulated from 5 to 11 categories. Data are collected by a random sample, large-scale web survey which allows for random assignment to the experimental conditions. The results show clear evidence of extreme response style and moderate evidence of midpoint response style. Extreme response style is not affected by the length of response scales, whereas the exertion of midpoint response style only popped up in the longer scale versions.

This paper investigates the integration of response styles (extreme and midpoint responding and socially desirable responding) and their effects on self-reports among 76,887 teachers from 18 countries in the Teaching and Learning International Survey (TALIS). Socially desirable responding (with a positive and a negative impression management factor) and 17 core constructs related to the teaching profession were measured with Likert scales; extreme and midpoint response styles were derived from these measures. Using factor analysis, a general response style was extracted with socially desirable and extreme response styles as positive indicators and midpoint response style as a negative indicator. This general response style was more strongly correlated with constructs of personal involvement, such as teacher efficacy and job satisfaction, than constructs with less personal involvement, at both the individual and country level; however, statistical correction for response styles had negligible effects on the size of cross-cultural differences and country rankings in any construct. We conclude that the general response style can be interpreted as response amplification versus moderation, and that there is no indication that correcting for the general response style increases the validity of cross-cultural comparisons of TALIS teacher data.

From a cross-cultural perspective, we studied the psychological meaning of a generalized response style that combines acquiescent, extreme, and midpoint response styles. Individual- and country-level indexes of the three specific response styles were constructed from eight multicountry surveys for correlation and multilevel analyses. At both levels, we confirmed a general response style factor with a positive loading of extreme response style, a negative loading of midpoint response style, and acquiescent response style in between. This general response style refers to a communication filter that moderates or amplifies expressions and it is associated with both individual and cultural factors. At the country level, the general response style was negatively related to the socioeconomic development and the percentage of atheists and positively related to aggregated values and personality traits pertinent to “fitting in” and avoidance of ambiguity. At the individual level, the general response style was positively associated with age and negatively associated with education. We conclude that integrating specific response styles to a general response style can help to create consistency in findings across styles and studies. Implications for cross-cultural management research and practices are discussed.

反应风格是指被试基于某些固定的倾向而不是基于具体的测验内容对题目做出选择。传统观点认为反应风格是一种系统的误差，而新近的观点发现反应风格很有可能是一种独特的人格变量。鉴于此，本研究在后者的视角下试图检验反应风格的类特质属性以及反应风格的影响因素。本研究采用大五人格问卷NEO-FFI以及语义反转的NEO-FFI分别对1200名高中生和1200名大学生进行了前后两次施测，时间间隔为1个月。计算两次测验中同一对题目相同答案的次数为反应风格的指标。结果发现：1) 中国被试默认肯定反应风格最多，依次是默认否定、折中、极端否定和极端肯定风格，且各种反应风格并不是独立存在的。2) 随着受教育程度的提高，各种反应风格均显著下降。性别在默认肯定风格和折中风格上主效应显著，性别与受教育程度在极端肯定风格以及折中风格上的交互作用显著。3) 各反应风格与人格特质的不同维度均存在不同程度和不同方向的相关。 Response styles refer to respondents’ systematic tendency to answer questionnaire items on some basis other than the specific item content. The traditional opinions view response styles as a kind of system error, but the new viewpoints find that the response styles are more likely to be unique personality variables. Due to this, the present study is trying to examine the trait-like properties and the effect factors of response styles in the latter perspectives. This research adopts the Big Five Personality questionnaire NEO-FFI and semantic reversal of NEO-FFI measures 1200 high school students and 1200 undergraduates twice one month in between calculating the number of the same answer at each pair of items as response styles indicators in two tests. The findings were as follows: 1) The acquiescent response style is more than other response styles in Chinese respondents, then the disacquiescence response style, midpoint response style, positive extreme response style and negative extreme response style. All kinds of response styles are not independent. 2) All response styles were significantly reduced with the higher education. The main effect of gender on the acquiescent response style and midpoint response style were significant, and the interaction effect was significant on midpoint response style and positive extreme response style. 3) Each response styles were related to personality traits at different degree and direction.

Response styles pose great threats to psychological measurements. This research compares IRTree models and anchoring vignettes in addressing response styles and estimating the target traits. It also explores the potential of combining them at the item level and total‐score level (ratios of extreme and middle responses to vignettes). Four models were evaluated: three multidimensional IRTree models with different levels of using vignette data and a nominal response model (NRM) addressing extreme and midpoint response styles with item‐level vignette responses. Simulation results indicated that the IRTree model using item‐level vignette responses outperformed others in estimating the target trait and response styles to different extents, with performance improving as the number of vignettes increased. Empirical findings further demonstrated that models using item‐level vignette information yielded higher reliability and closely aligned target trait estimates. These results underscore the value of integrating anchoring vignettes with IRTree models to enhance estimation accuracy and control for response styles.

Abstract. Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

Individual response style behaviors, unrelated to the latent trait of interest, may influence responses to ordinal survey items. Response style can introduce bias in the total score with respect to the trait of interest, threatening valid interpretation of scores. Despite claims of response style stability across scales, there has been little research into stability across multiple scales from the beneficial perspective of item response trees. This study examines an extension of the IRTree methodology to include mixed item formats, providing an empirical example of responses to three scales measuring perceptions of social media, climate change, and medical marijuana use. Results show extreme and midpoint response styles were not stable across scales within a single administration and 5-point Likert-type items elicited higher levels of extreme response style than the 4-point items. Latent trait of interest estimation varied, particularly at the lower end of the score distribution, across response style models, demonstrating as appropriate response style model is important for adequate trait estimation using Bayesian Markov chain Monte Carlo estimation.

Contamination of responses due to extreme and midpoint response style can confound the interpretation of scores, threatening the validity of inferences made from survey responses. This study incorporated person-level covariates in the multidimensional item response tree model to explain heterogeneity in response style. We include an empirical example and two simulation studies to support the use and interpretation of the model: parameter recovery using Markov chain Monte Carlo (MCMC) estimation and performance of the model under conditions with and without response styles present. Item intercepts mean bias and root mean square error were small at all sample sizes. Item discrimination mean bias and root mean square error were also small but tended to be smaller when covariates were unrelated to, or had a weak relationship with, the latent traits. Item and regression parameters are estimated with sufficient accuracy when sample sizes are greater than approximately 1,000 and MCMC estimation with the Gibbs sampler is used. The empirical example uses the National Longitudinal Study of Adolescent to Adult Health's sexual knowledge scale. Meaningful predictors associated with high levels of extreme response latent trait included being non-White, being male, and having high levels of parental support and relationships. Meaningful predictors associated with high levels of the midpoint response latent trait included having low levels of parental support and relationships. Item-level covariates indicate the response style pseudo-items were less easy to endorse for self-oriented items, whereas the trait of interest pseudo-items were easier to endorse for self-oriented items.

Previous research has discussed cultural differences in moderacy vs extremity response styles. The present research found that cultural differences in response styles were more complex than previously speculated. We investigated cross-cultural variations in extreme rejecting versus affirming response biases. Although research has indicated that overall Chinese have less extreme responses than Westerners, the difference may be mainly driven by extreme rejecting responses because respondents consider answering survey questions as a way of interacting with researchers, and extreme rejecting responses may disrupt harmony in relationships, which is valued more in Chinese collectivistic culture than in Western individualistic cultures. Studies 1 and 2 revealed that Chinese had less extreme rejecting response style than did British, whereas they did not differ in extreme affirming response style. Study 2 further revealed that the cross-cultural asymmetry in extreme rejecting versus affirming response styles was partially accounted for by individualism orientation at the individual level. Consistently, Study 3 revealed that at the country level, individualism was positively associated with extreme rejecting response style, but was not associated with extreme affirming response style, suggesting that individualism accounted for the asymmetric cultural variation in extreme rejecting versus affirming response styles.

IRTree models decompose observed rating responses into sequences of theory-based decision nodes, and they provide a flexible framework for analysing trait-related judgements and response styles. However, most previous applications of IRTree models have been limited to binary decision nodes that reflect qualitatively distinct and unidimensional judgement processes. The present research extends the family of IRTree models for the analysis of response styles to ordinal judgement processes for polytomous decisions and to multidimensional parametrizations of decision nodes. The integration of ordinal judgement processes overcomes the limitation to binary nodes, and it allows researchers to test whether decisions reflect qualitatively distinct response processes or gradual steps on a joint latent continuum. The extension to multidimensional node models enables researchers to specify multiple judgement processes that simultaneously affect the decision between competing response options. Empirical applications highlight the roles of extreme and midpoint response style in rating judgements and show that judgement processes are moderated by different response formats. Model applications with multidimensional decision nodes reveal that decisions among rating categories are jointly informed by trait-related processes and response styles.

Existing models, such as the item response tree (IRTree), have been extensively developed to analyze response styles in Likert-scale data. However, less attention has been given to questionnaires employing continuous measurement formats. These continuous bounded response formats include the visual analogue scale (VAS), slider bars, and probability judgments. We propose a novel item response model framework that leverages a hierarchical structure and constructs pseudo-responses. This framework enables the flexible incorporation of content traits, extreme response style (ERS), and midpoint response style (MRS), while isolating the effect of response style from observed responses. An empirical study was conducted to validate the ability of the new model to assess ERS and MRS. The results demonstrated that the model achieves a superior fit to continuous bounded response data and provides effective estimates of ERS and MRS. Furthermore, a simulation study was conducted to test the recovery of model parameters in various situations. The results demonstrated that the Markov chain Monte Carlo method can accurately estimate model parameters. In general, the trait of interest and response styles estimated by the new models demonstrate robust validity, and our models successfully mitigate the adverse effects of response styles on observed responses.

An increased use of models for measuring response styles is apparent in recent years with the multidimensional nominal response model (MNRM) as one prominent example. Inclusion of latent constructs representing extreme (ERS) or midpoint response style (MRS) often improves model fit according to information criteria. However, a test of absolute model fit is often not reported even though it could comprise an important piece of validity evidence. Limited information test statistics are candidates for this task, including the full (), ordinal (), and mixed () statistics, which differ in whether additional collapsing of univariate or bivariate contingency tables is conducted. Such collapsing makes sense when item categories are ordinal, which may not hold under the MNRM. More generally, limited information test statistics have gone unevaluated under nominal data and non-ordinal latent trait models. We present a simulation study evaluating the performance of and with the MNRM. Manipulated conditions included sample size, presence and type of response style, and strength of item slopes on substantive and style dimensions. We found that sometimes had inflated Type I error rates, always had little power, and lacked power under some conditions. and may provide complementary and valuable information regarding model fit.

Cross-mode surveys are on the rise. The current study compares levels of response styles across three modes of data collection: paper-and-pencil questionnaires, telephone interviews, and online questionnaires. The authors make the comparison in terms of acquiescence, disacquiescence, and extreme and midpoint response styles. To do this, they propose a new method, namely, the representative indicators response style means and covariance structure (RIRSMACS) method. This method contributes to the literature in important ways. First, it offers a simultaneous operationalization of multiple response styles. The model accounts for dependencies among response style indicators due to their reliance on common item sets. Second, it accounts for random error in the response style measures. As a consequence, random error in response style measures is not passed on to corrected measures. The method can detect and correct cross-mode response style differences in cases where measurement invariance testing and multitrait multimethod designs are inadequate. The authors demonstrate and discuss the practical and theoretical advantages of the RIRSMACS approach over traditional methods.