Extreme Response Style Research Articles

Comparing acquiescent and extreme response styles in face-to-face and web surveys

Likert scales are popular for measuring attitudes, but response style, a source of measurement error associated with this type of question, can result in measurement bias. This study investigates the effect of data collection mode on both types of response styles using data from the 2012 American National Election Studies (ANES). 2012 was the 1 year in which ANES conducted two parallel surveys, one through face-to-face interviews and another through Web, using two independent national probability samples and an identical questionnaire. We used three sets of balanced Likert scales from the survey to measure the acquiescent and extreme response styles. Using the latent class analysis modeling approach, we find that: (1) both acquiescent and extreme response styles exist in both face-to-face and Web survey modes; (2) face-to-face respondents demonstrate more acquiescent and extreme response styles than Web respondents; (3) the mode effect is similar for white, black and Hispanic respondents.

Response styles in factual items: Personal, contextual and cultural correlates.

This study investigated response styles in factual items and explored their associations with personal, contextual and cultural factors. Responses on various factual questions, cognitive tests and interviewers' observational data from a total of 152,514 respondents in 22 countries in the Programme for the International Assessment of Adult Competencies (PIAAC) were analysed. Indexes of extreme, midpoint and acquiescent response styles were extracted from Likert-scale and dichotomous responses of factual items. A general response style (GRS) with a positive loading of extreme response style and negative loadings of midpoint and acquiescent response styles was confirmed. This factor showed a similar cross-cultural patterning as another general factor from attitudinal and self-evaluative items of Likert scales in a previous study, which indicated the pervasiveness of response styles irrespective of types of survey items. In a multilevel analysis, the individual-level GRS was found to be negatively related to being male, educational level and literacy competency, and positively related to 3rd-person presence and background noise, and at country level negatively associated with socioeconomic development. Cross-level interactions were also found. Implications on the pervasiveness and nature of response styles are discussed.

The Effect of Survey Design on Extreme Response Style: Rating Job Satisfaction

This paper explores the relationship between survey rating scale and Extreme Response Style (ERS) using experimental data from Understanding Society (Innovation Panel 2008), where a self-assessment questionnaire measuring job satisfaction uses two alternative (7 and 11 points) rating options. Our results suggests that when shifting from a shorter to a longer scale, the survey design generates a tendency to choose response scales at the extreme of the distribution, thus creating a misleading quantification of the variable of interest. The experimental design of the data enables us to test our hypothesis using a non-linear estimation approach where age, gender and education level are shown to affect ERS.

反应风格与人格特质的关系

反应风格是指被试基于某些固定的倾向而不是基于具体的测验内容对题目做出选择。传统观点认为反应风格是一种系统的误差，而新近的观点发现反应风格很有可能是一种独特的人格变量。鉴于此，本研究在后者的视角下试图检验反应风格的类特质属性以及反应风格的影响因素。本研究采用大五人格问卷NEO-FFI以及语义反转的NEO-FFI分别对1200名高中生和1200名大学生进行了前后两次施测，时间间隔为1个月。计算两次测验中同一对题目相同答案的次数为反应风格的指标。结果发现：1) 中国被试默认肯定反应风格最多，依次是默认否定、折中、极端否定和极端肯定风格，且各种反应风格并不是独立存在的。2) 随着受教育程度的提高，各种反应风格均显著下降。性别在默认肯定风格和折中风格上主效应显著，性别与受教育程度在极端肯定风格以及折中风格上的交互作用显著。3) 各反应风格与人格特质的不同维度均存在不同程度和不同方向的相关。 Response styles refer to respondents’ systematic tendency to answer questionnaire items on some basis other than the specific item content. The traditional opinions view response styles as a kind of system error, but the new viewpoints find that the response styles are more likely to be unique personality variables. Due to this, the present study is trying to examine the trait-like properties and the effect factors of response styles in the latter perspectives. This research adopts the Big Five Personality questionnaire NEO-FFI and semantic reversal of NEO-FFI measures 1200 high school students and 1200 undergraduates twice one month in between calculating the number of the same answer at each pair of items as response styles indicators in two tests. The findings were as follows: 1) The acquiescent response style is more than other response styles in Chinese respondents, then the disacquiescence response style, midpoint response style, positive extreme response style and negative extreme response style. All kinds of response styles are not independent. 2) All response styles were significantly reduced with the higher education. The main effect of gender on the acquiescent response style and midpoint response style were significant, and the interaction effect was significant on midpoint response style and positive extreme response style. 3) Each response styles were related to personality traits at different degree and direction.

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

In this paper, we consider a two-level multidimensional item response model that examines country differences in extreme response style (ERS) as a possible cause for the achievement-attitude paradox in PISA 2006. The model is an extension of Bolt & Newton (2011) that uses response data from seven attitudinal scales to assess response style and to control for its effects in estimating correlations between attitudes and achievement. Despite detectable variability in ERS across countries and detectable biasing effects of ERS on attitudinal scores, our results suggest that the unexpected between-country correlation between attitudes and achievement is not attributable to country differences in ERS. The remaining between-country correlations between mean attitudes and mean achievement once controlling for ERS can be explained by the observation that (1) despite detectable country differences, most variability in ERS occurs within, as opposed to between, countries, and (2) ERS appears to be only weakly correlated with achievement. The methodological approach used in this paper is argued to provide an informative way of studying the effects (or lack thereof) of cross-country variability in response style.

A Simulation Study on Methods of Correcting for the Effects of Extreme Response Style.

The impact of response styles such as extreme response style (ERS) on trait estimation has long been a matter of concern to researchers and practitioners. This simulation study investigated three methods that have been proposed for the correction of trait estimates for ERS effects: (a) mixed Rasch models, (b) multidimensional item response models, and (c) regression residuals. The methods were compared with respect to their ability of recovering the true latent trait levels. Data were generated according to a unidimensional model with only one trait, a mixed Rasch model with two populations of ERS and non-ERS, and a two-dimensional model incorporating a trait and an ERS dimension. The data were analyzed using the same models as well as linear regression where the trait estimate is regressed on an ERS score and the resulting residual is considered the corrected trait estimate. Over all conditions, the two-dimensional model achieved the best trait recovery, though the difference to the unidimensional model was rather small. Mixed Rasch models were in general inferior to the other correction methods. When the trait and ERS showed no to weak correlations, ERS appeared to have a minor impact on trait estimation.

What is the optimal number of response alternatives for rating scales? From an information processing perspective

Rating scales are measuring instruments that are widely used in social science research. However, many different rating scale formats are used in the literature, differing specifically in the number of response alternatives offered. Previous studies on the optimal number of response alternatives have focused exclusively on the participants’ final response results, rather than on the participants’ information processing. We used an eye-tracking study to explore this issue from an information processing perspective. We analyzed the information processing in six scales with different response alternatives. We compared the reaction times, net acquiescence response styles, extreme response styles and proportional changes in the response alternatives of the six scales. Our results suggest that the optimal number of response alternatives is five.

The Issue of Fuzzy Concepts in Test Construction and Possible Remedies

With the first issue of 2015, I want to continue the series ofeditorials aimed at highlighting specific topics relevant dur-ing test construction. More importantly, I focus on issuesthat repeatedly lead to paper rejections. In this issue, I wantto take up the cudgels for a mixed methods approach in testconstruction. The quantitative aspects of testing a newlyconstructed assessment tool range prominently within thisjournal (Alonso-Arbiol & van de Vijver, 2010). However,some issues arising during the test development processcannot be dealt with using quantitative methods alone.Along with the coauthors of this editorial – experts in theapplication of a mixed methods approach in test construc-tion – I want to explore these issues.Many test constructions show a lack of awareness con-cerning the comprehensibility of items, specifically in per-sonality tests. However, this lack of awareness can haveimplications for psychometric quality. Frequently, it is sim-ply assumed that respondents’ understanding of a test itemmatches the meaning implied by the test developer. It is,however, rarely ever tested whether all respondents of thetargeted population actually understand the test items cor-rectly and in a similar way. As pointed out in the last edito-rial (Ziegler, 2014), item content should be precisely tunedto the needs of every potential respondent in a populationtargeted by a newly developed test. More specifically, per-son variables such as age, gender, and education (Ramm-stedt & Kemper, 2011) have to be taken into accountwhen constructing test items in order to assure that eachpotential respondent fully understands the meaning andmay respond accordingly. If these person variables areneglected in the process of test construction, the psycho-metric quality of a test may be substantially affected byfuzzy concepts. The goal of this editorial is to raise aware-ness for the detrimental effects fuzzy concepts can have intest development and possible remedies.

The Stability of Extreme Response Style and Acquiescence Over 8 Years.

This study investigated the stability of extreme response style (ERS) and acquiescence response style (ARS) over a period of 8 years. ERS and ARS were measured with item sets drawn randomly from a large pool of items used in an ongoing German panel study. Latent-trait-state-occasion and latent-state models were applied to test the relationship between time-specific (state) response style behaviors and time-invariant trait components of response styles. The results show that across different random item samples, on average between 49% and 59% of the variance in the state response style factors was explained by the trait response style factors. This indicates that the systematic differences respondents show in their preferences for certain response categories are remarkably stable over a period of 8 years. The stability of ERS and ARS implies that it is important to consider response styles in the analysis of self-report data from polytomous rating scales, especially in longitudinal studies aimed at investigating stability in substantive traits. Furthermore, the stability of response styles raises the question in how far they might be considered trait-like latent variables themselves that could be of substantive interest.

Effects of a General Response Style on Cross-Cultural Comparisons: Evidence from the Teaching and Learning International Survey

This paper investigates the integration of response styles (extreme and midpoint responding and socially desirable responding) and their effects on self-reports among 76,887 teachers from 18 countries in the Teaching and Learning International Survey (TALIS). Socially desirable responding (with a positive and a negative impression management factor) and 17 core constructs related to the teaching profession were measured with Likert scales; extreme and midpoint response styles were derived from these measures. Using factor analysis, a general response style was extracted with socially desirable and extreme response styles as positive indicators and midpoint response style as a negative indicator. This general response style was more strongly correlated with constructs of personal involvement, such as teacher efficacy and job satisfaction, than constructs with less personal involvement, at both the individual and country level; however, statistical correction for response styles had negligible effects on the size of cross-cultural differences and country rankings in any construct. We conclude that the general response style can be interpreted as response amplification versus moderation, and that there is no indication that correcting for the general response style increases the validity of cross-cultural comparisons of TALIS teacher data.

Comparing Extreme Response Styles between Agree-Disagree and Item-Specific Scales

Although Likert scales in agree-disagree (A/D) format are popular in surveys, the data quality yielded by them is controversial among researchers. Recognizing the measurement issues involved with the A/D format, researchers have developed other question formats to measure attitudes. In this study, we focused on an alternative question type, the item-specific (IS) question, which asks the respondent to choose an option that best describes his or her attitude. Using political efficacy items from the American National Election Studies (ANES), we compare extreme response style (ERS) between A/D and IS scales. Latent class factor analysis showed that ERS exists in both A/D and IS scale formats, but differs slightly across the two. Also, when analyzing ERS within subjects across two waves, there is only a single ERS for both question formats, after controlling for the correlation within respondents. The last finding suggests that ERS is a stable characteristic

Are Chinese university students more likely to exhibit a Surface approach to learning than other international students in Finland?

This article is based on a study which investigated whether Chinese international students at a university in Finland are more likely to rely on a Surface approach to learning and dismiss a Deep approach than are other international students in the same university educational context. In responding to a survey, students’ scores with respect to the Deep approach and Organized Studying did not significantly differ between the groups. However, the Chinese students appeared slightly to exhibit the Surface approach to a greater extent, although this result was tentative when the students’ extreme response styles were taken into account.

Toward a unification of acquiescent, extreme, and midpoint response styles

From a cross-cultural perspective, we studied the psychological meaning of a generalized response style that combines acquiescent, extreme, and midpoint response styles. Individual- and country-level indexes of the three specific response styles were constructed from eight multicountry surveys for correlation and multilevel analyses. At both levels, we confirmed a general response style factor with a positive loading of extreme response style, a negative loading of midpoint response style, and acquiescent response style in between. This general response style refers to a communication filter that moderates or amplifies expressions and it is associated with both individual and cultural factors. At the country level, the general response style was negatively related to the socioeconomic development and the percentage of atheists and positively related to aggregated values and personality traits pertinent to “fitting in” and avoidance of ambiguity. At the individual level, the general response style was positively associated with age and negatively associated with education. We conclude that integrating specific response styles to a general response style can help to create consistency in findings across styles and studies. Implications for cross-cultural management research and practices are discussed.

Measuring Response Styles Across the Big Five: A Multiscale Extension of an Approach Using Multinomial Processing Trees

This study shows how to address the problem of trait-unrelated response styles (RS) in rating scales using multidimensional item response theory. The aim is to test and correct data for RS in order to provide fair assessments of personality. Expanding on an approach presented by Böckenholt (2012), observed rating data are decomposed into multiple response processes based on a multinomial processing tree. The data come from a questionnaire consisting of 50 items of the International Personality Item Pool measuring the Big Five dimensions administered to 2,026 U.S. students with a 5-point rating scale. It is shown that this approach can be used to test if RS exist in the data and that RS can be differentiated from trait-related responses. Although the extreme RS appear to be unidimensional after exclusion of only 1 item, a unidimensional measure for the midpoint RS is obtained only after exclusion of 10 items. Both RS measurements show high cross-scale correlations and item response theory-based (marginal) reliabilities. Cultural differences could be found in giving extreme responses. Moreover, it is shown how to score rating data to correct for RS after being proved to exist in the data.

The serotonin transporter polymorphism (5-HTTLPR) and personality: response style as a new endophenotype for anxiety

Although the serotonin transporter length polymorphic region (5-HTTLPR) polymorphism is an extensively-investigated genetic marker of anxiety related personality traits (neuroticism and harm avoidance) and affective disorders, effect sizes in meta-analyses are small, if present at all, and all available primary studies to date lack mandatory statistical power. Moreover, questionnaire data is prone to confounding by variables such as social desirability. Therefore, extreme response style (ERS) is suggested as a new approach to elucidate the relationship between 5-HTTLPR and negative emotionality, as it is more implicit and of high reliability. N = 1075 healthy subjects were genotyped for 5-HTTLPR and a flanking polymorphism (rs25531) and filled out the NEO Five Factor Inventory and the Temperament Character Inventory. As dependent variable the number of extreme responses across all items was calculated. Using the common genotype or the triallelic approach (including rs25531) the meta-analytic findings could not be replicated. However, there was a significant association between 5-HTTLPR and extreme response style. Carriers of the L-allele or the L'-allele, respectively, had a significantly higher number of extreme responses than homozygous SS carriers across all items of the NEO Five Factor Inventory. This finding could be replicated in an alternative personality questionnaire (Affective Neuroscience Personality Scales, ANPS). There is a long tradition in psychological assessment indicating that ERS is an implicit measure of personality. Given the positive findings of the present study, ERS qualifies as a promising endophenotype in future genetic association studies on personality and affective disorders.

The Effect of Labeling and Numbering of Response Scales on the Likelihood of Response Bias

Extreme response style (ERS) and acquiescence response style (ARS) are among the most encountered problems in attitudinal research. The authors investigate whether the response bias caused by these response styles varies with following three aspects of question format: full versus end labeling, numbering answer categories, and bipolar versus agreement response scales. A questionnaire was distributed to a random sample of 5,351 respondents from the Longitudinal Internet Studies for the Social Sciences household panel, of which a subsample was assigned to one of five conditions. The authors apply a latent class factor model that allows for diagnosing and correcting for ERS and ARS simultaneously. The results show clearly that both response styles are present in the data set, but ARS is less pronounced than ERS. With regard to format effects, the authors find that end labeling evokes more ERS than full labeling and that bipolar scales evoke more ERS than agreement style scales. With full labeling, ERS opposes opting for middle response categories, whereas end labeling distinguishes ERS from all other response categories. ARS did not significantly differ depending on test conditions.

Extreme response style and symptom return after depression treatment: The role of positive extreme responding.

Evidence linking extreme response style (ER) to depressive relapse has been mixed. One reason might be high levels of extreme responses that are positive in nature (ER-Ps) compared with those negative in nature (ER-Ns) at posttreatment. ER-Ps likely consist of both maladaptive "style" responses and adaptive "content" responses (i.e., legitimate denials of dysfunction). The composition of ER-Ps might confound measures of total extreme responding as well as conventional scores on cognitive questionnaires. In the current study, we assessed ER in a new sample by (a) disambiguating ER-Ps that reflect style from those that reflect content and (b) assessing the contribution of ER-Ps to the prediction of relapse/recurrence. Responders (N = 104) to a randomized controlled trial of cognitive therapy versus medications for moderate to severe depression had an average age of 40 years (SD = 12), and they were 58% female, 38% married/cohabitating, and 85% Caucasian. ER variables were calculated using the Dysfunctional Attitudes Scale (DAS; Weissman, 1979), with ER-Ps categorized as either content or style responses. ER indices and DAS scores were used to predict symptom return over 2 years. No standard extreme responding variables (e.g., an index of total extreme responding) predicted symptom return, but higher relative levels of style ER-P predicted relapse/recurrence. Total DAS scores also predicted relapse/recurrence but only when high levels of style ER-P responses were controlled. ER-Ps, at least on the DAS, appear to contain indicators of both adaptive and maladaptive positive responses. Future research should attend to the valence of the extreme responses as well as to the content of extreme positive responses, which may reflect either healthy or unhealthy tendencies.

A Cross-National Comparison of Extreme Response Style Measures

Five measures of extreme response style were compared across 6,146 study participants from 36 countries: the traditional measure, a modified traditional measure, the individual standard deviation, an index of dispersion and an index of entropy. The traditional measure of extreme response style, whereby the two extreme categories of an item or rating scale are assigned a value of ‘1’, all interior categories are assigned a value of ‘0’ and the sum of the ‘1’ values reflects the extent of extreme responding behaviour, performed slightly better than the other extreme response style measures examined with respect to reliability and ability to discriminate. The traditional measure of extreme response style was positively related to the variance of an attitudinal variable but unrelated to its mean. It was also related to Hofstede's cultural orientation variables of individualism-collectivism and power distance. Future cross-cultural and cross-national empirical research should systematically incorporate measures of extreme responding so that more is learned about the phenomenon and its possible effects.

Generalized IRT Models for Extreme Response Style

Extreme response style (ERS) is a systematic tendency for a person to endorse extreme options (e.g., strongly disagree, strongly agree) on Likert-type or rating-scale items. In this study, we develop a new class of item response theory (IRT) models to account for ERS so that the target latent trait is free from the response style and the tendency of ERS is quantified. Parameters of these new models can be estimated with marginal maximum likelihood estimation methods or Bayesian methods. In this study, we use the freeware program WinBUGS, which implements Bayesian methods. In a series of simulations, we find that the parameters are recovered fairly well; ignoring ERS by fitting standard IRT models resulted in biased estimates, and fitting the new models to data without ERS did little harm. Two empirical examples are provided to illustrate the implications and applications of the new models.

Surface- and deep-level diversity in panel selection — Exploring diversity effects on response behaviour in foresight

In addition to foresight research endeavours that focus on the application of the Delphi survey technique, numerous research articles have dealt with the method itself: namely improving the Delphi technique's task and process characteristics. Particularly in Policy Delphi surveys and related variations that strive to explore opposing views, the diversity of the Delphi panel has been scrutinised. In the majority of earlier Policy Delphi studies, expertise accounted for the most predominant panel selection criterion. However, further surface- and deep-level diversity dimensions discussed in related social science research need to be incorporated for steering diversity in panel selection. In this article, the main effects of surface- and deep-level panel diversity on response behaviour are examined, focussing on extreme response style (ERS). Moreover, interaction phenomena of diversity variables are considered. By conducting a Policy Delphi in real-time format on the future of multimodal mobility, we demonstrate that value attributes significantly influence extreme response behaviour while expertise is especially important in combination with various other diversity variables. Furthermore, we identified a moderating effect of age on the relationship between environmental value characteristics and Delphi panellists' response behaviour.