Extreme Response Style Research Articles

Do Individual Response Styles Matter?

The occurrence of differential item functioning (DIF) for gender indicates that an instrument may not be functioning equivalently for men and women. Aside from DIF effects, item responses in personality questionnaires can also be influenced by response styles. This study analyzes the German NEO-PI-R regarding its differential item functioning for men and women while taking response styles into account. To this end, mixed Rasch models were estimated first to identify latent classes that differed in their response style. These latent classes were identified as extreme response style (ERS) and nonextreme response style (NERS). Then, DIF analyses were conducted separately for the different response styles and compared with DIF results for the complete sample. Several items especially on Neuroticism, Agreeableness, and Conscientiousness facets showed gender-DIF and thus function differentially between men and women. DIF results differed mainly in size between the complete sample and the response style subsamples, though DIF classification was overall consistent between ERS, NERS, and the complete sample.

Do culture-dependent response styles distort substantial relationships?

Survey data are frequently distorted by answering tendencies, such as acquiescence, disacquiescence, midpoint and extreme response style. Cross-cultural research projects may be particularly vulnerable to misinterpretations. This paper provides empirical insights into the manner and degree to which culture systematically distorts findings from survey data. The study is based on data from 1027 respondents from Austria, France, Germany, India, Mexico, Russia, Spain, and the United States. The investigation firstly demonstrates that Hofstede's and project GLOBE's cultural dimensions explain variations in response styles across different countries. Secondly, and most importantly, the paper explores the impact of culture-dependent response styles on findings. Remarkably, the extent of distortion in correlation analysis and mean comparisons is less severe than expected. Nonetheless, cross-cultural researchers would be well advised to control at least for (dis-)acquiescence before analyzing and interpreting their data.

Intra-individual variation of extreme response style in mixed-mode panel studies

It is well known that the self-report survey method suffers from many idiosyncratic biases, such as varying response styles due to different survey modes used. Using latent state-trait theory it is argued that response styles will also vary intra-individually, depending on the particular survey situation. In this study we examine intra-individual variation in extreme response style behavior (ERS) using mixed-mode survey panel data as a quasi-experimental setting. Data from the Irish National Election Study panel are used, which consists of repeated face-to-face and mail-back surveys. Latent transition analysis is used to detect switches in ERS, distinguishing ‘stable’ and ‘volatile’ respondents in terms of their response style. Overall, ERS is inflated in the intermediate mail component of the panel, whereas preliminary analyses suggest that low education and ideological extremity are drivers of that change. Results are discussed with regards to measurement errors in mixed-mode and longitudinal surveys.

Consistency of extreme response style and non-extreme response style across traits

The consistency of extreme response style (ERS) and non-extreme response style (NERS) across the latent variables assessed in an instrument is investigated. Analyses were conducted on several PISA 2006 attitude scales and the German NEO-PI-R. First, a mixed partial credit model (PCM) and a constrained mixed PCM were compared regarding model fit. If the constrained mixed PCM fit better, latent classes differed only in their response styles but not in the latent variable. For scales where this was the case, participants’ membership to NERS or ERS on each scale was entered into a latent class analysis (LCA). For both instruments, this second order LCA revealed that the response style was consistent for the majority of the participants across latent variables.

Leader–member exchange (LMX) and culture: A meta-analysis of correlates of LMX across 23 countries.

This study extends leader-member exchange (LMX) research by meta-analyzing the role of national culture in moderating relationships between LMX and its correlates. Results based on 282 independent samples (N = 68,587) from 23 countries and controlling for extreme response style differences indicate that (a) relationships of LMX with organizational citizenship behavior, justice perceptions, job satisfaction, turnover intentions, and leader trust are stronger in horizontal-individualistic (e.g., Western) contexts than in vertical-collectivistic (e.g., Asian) contexts; and (b) national culture does not affect relationships of LMX with task performance, organizational commitment, and transformational leadership. These findings highlight that although members are universally sensitive to how their leaders treat them, members' responses in Asian contexts may also be influenced by collective interests and role-based obligations.

The effect of response style bias on the measurement of transformational, transactional, and laissez-faire leadership

Transformational, transactional, and laissez-faire leadership are considered to be three distinct leadership styles. In this research we argue that response style behaviour in the form of acquiescence and extreme response style can distort the measurement of these dimensions of leadership. Using a sample of 864 employees selected from 135 work teams, this research demonstrates that (1) response styles affect measurement; (2) divergent validity of the three dimensions increases when response styles are taken into account; (3) gender is spuriously related to leadership upon response styles; and (4) team ratings substantially change when controlling for response styles bias. As a secondary topic of this research, we elaborate on a relatively new approach in diagnosing response styles, i.e., a confirmatory latent-class factor analysis. We explain the advantages of this approach and illustrate the steps a researcher has to take in conducting this type of analysis.

Exploring of Students' Extreme Response Style on Questionnaire

The presence of individuals who have an extreme response style or who likely choose an end point response option will contaminate the research inference. Hitherto, such individual was difficult to detect since there is no statistical analysis can accommodate for this purpose. This study aimed to apply the mixture Rasch Model Analysis techniques to identify the proportion of students who possess extreme response styles when completing the questionnaire. Total 2.981 high school students from 30 cities in 15 provinces were instructed to complete questionnaires measuring self-esteem. Self-Self-Esteem Scale consists of four self-reported sub-scales using Likert's model. Analysis suggest that based on how to respond to the scale, student in this study was grouped into three classes: extreme response style class, normal class, and mixture class. These numbers of class were consistent on all four sub-scales. The proportion of students who consistently gave an extreme response on four sub-scales was 4 percent; 6 percent was on three sub-scale, 13 percent on two sub-scales and 53 percent on one sub-scale. The small percentage of students who responded consistently gave an extreme response suggest that high-school students can choose an option response on the questionnaire appropriately.

Response Style Differences in Cross-National Research

International Management researchers often rely on surveys to collect their data. However, responses to survey questions can be biased by response styles, a respondent’s tendency to provide a systematic response to questions regardless of their content. Response styles vary across countries and individuals, but there is limited systematic research that investigateswhy they vary. Our study investigates middle (MRS) versus extreme response styles (ERS), the tendency to use the middle or extreme categories on rating scales. We examine the impact of culture, different types of scale anchors and the level of knowledge of the topic in question on MRS and ERS. We asked five groups of respondents (Chinese in China, Chinese in Australia, Anglo-Australians in Australia, and two groups of German students in Germany) to indicate on a 10-point scale whether certain employee attitudes or behaviour were more typically Australian (left-hand of the scale) or Chinese (right-hand of the scale). We then asked them how they would rate the performance (low to high on a 10-point scale) of an employee who displayed this attitude or behaviour. Asian respondents showed higher MRS than Western respondents. When scale anchors referred to naturally opposing and mutually exclusive constructs (Australian versus Chinese) respondents showed more ERS than when they referred to level or degree of a construct (low-high performance). Knowledge of cross national differences resulted in higher ERS on behavioural questions but not on performance questions.

The Impact of Controlling for Extreme Responding on Measurement Equivalence in Cross-Cultural Research

Prior research has shown that extreme response style can seriously bias responses to survey questions and that this response style may differ across culturally diverse groups. Consequently, cross-cultural differences in extreme responding may yield incomparable responses when not controlled for. To examine how extreme responding affects the cross-cultural comparability of survey responses, we propose and apply a multiple-group latent class approach where groups are compared on basis of the factor loadings, intercepts, and factor means in a Latent Class Factor Model. In this approach a latent factor measuring the response style is explicitly included as an explanation for group differences found in the data. Findings from two empirical applications that examine the cross-cultural comparability of measurements show that group differences in responding import inequivalence in measurements among groups. Controlling for the response style yields more equivalent measurements. This finding emphasizes the importance of correcting for response style in cross-cultural research.

Multiscale Measurement of Extreme Response Style

This article extends a methodological approach considered by Bolt and Johnson for the measurement and control of extreme response style (ERS) to the analysis of rating data from multiple scales. Specifically, it is shown how the simultaneous analysis of item responses across scales allows for more accurate identification of ERS, and more effective control of ERS effects on the substantive trait estimates, than when analyzing just one scale. Moreover, unlike a competing approach presented by Greenleaf, the current strategy can accommodate conditions in which the substantive traits across scales correlate, as is almost always the case in social sciences research. Simulation and real data analyses are used for illustration.

An empirical assessment of the cross-national measurement validity of graded paired comparisons

The popular use of graded paired comparisons in empirical studies assessing consumers’ preferences, and the potential effect of cross-national differences in (extreme) response styles on the quality of graded paired comparison data, supply ample reasons for an empirical verification of the cross-national validity of such scales. Using data from a cross-national margarine brand study including fourteen different nations (N = 4,560), we found sufficient statistical evidence for cross-national bias due to existing cross-national differences in extreme responses. However, the low values reported for effect size measures (intra-class correlation coefficient, R2 value) indicated that the impact of the cross-national bias is marginal. The findings from our study provided empirical support for the hypothesis that graded paired comparison data can be meaningfully compared across nations.

Dealing with Extreme Response Style in Cross-Cultural Research: A Restricted Latent Class Factor Analysis Approach

Cross-cultural comparison of attitudes using rating scales may be seriously biased by response styles. This paper deals with statistical methods for detection of and correction for extreme response style (ERS), which is one of the well-documented response styles. After providing an overview of available statistical methods for dealing with ERS, we argue that the latent class factor analysis (LCFA) approach proposed by Moors (2003) has several advantages compared to other methods. Moors' method involves defining a latent variable model which, in addition to the substantive factors of interest, contains an ERS factor. In LCFA the observed ratings can be treated as nominal responses, which is necessary for modeling ERS. We find strong evidence for the presence of ERS and, moreover, find that the groups differ not only in their attitudes but also in ERS. These findings underscore the importance of controlling for ERS when examining attitudes in cross-cultural research.

Measurement Equivalence of Ordinal Items: A Comparison of Factor Analytic, Item Response Theory, and Latent Class Approaches

Three distinctive methods of assessing measurement equivalence of ordinal items, namely, confirmatory factor analysis, differential item functioning using item response theory, and latent class factor analysis, make different modeling assumptions and adopt different procedures. Simulation data are used to compare the performance of these three approaches in detecting the sources of measurement inequivalence. For this purpose, the authors simulated Likert-type data using two nonlinear models, one with categorical and one with continuous latent variables. Inequivalence was set up in the slope parameters (loadings) as well as in the item intercept parameters in a form resembling agreement and extreme response styles. Results indicate that the item response theory and latent class factor models can relatively accurately detect and locate inequivalence in the intercept and slope parameters both at the scale and the item levels. Confirmatory factor analysis performs well when inequivalence is located in the slope parameters but wrongfully indicates inequivalence in the slope parameters when inequivalence is located in the intercept parameters. Influences of sample size, number of inequivalent items in a scale, and model fit criteria on the performance of the three methods are also analyzed.

Extreme Response Style among U.S. Hispanics vs. Non-Hispanics

It is well documented that U.S. Hispanics tend to give extreme rating scale responses more so than white non-Hispanics (Marin et al. 1992). However, there is no agreement on how to best measure extreme response style (ERS) among Latinos and control for its effects in the data analysis and interpretation. Furthermore, there are a small number of documented cases and academic papers written on the subject, and the few related to U.S. Hispanics are very limited or flawed from a study design perspective. To shed new light, Encuesta, Inc. carried out a small scale experiment within a nationally representative study conducted for its pro bono nonpartisan Americanos Poll series. The study was conducted in such a manner as to obtain proper representation of all U.S. Hispanics by language usage and acculturation level. The findings will provide evidence of extreme response style and highlight differences and possible solutions when conducting research among Hispanics.

Variations in Response Style Behavior by Response Scale Format in Attitude Research

Studies concerning the impact of the length of response scales on the measurement of attitudes have primarily focused on the method bias associated with question format. At the same time another line of research has focused on the issue of response styles that affect how respondents answer to attitude questions. So far, research has paid less attention to the issue of whether the length of the response scales is related to response styles. In this study, we explore if differences in length of the response scale (i.e., method factor) have differential effects in evoking extreme and midpoint response style behavior (i.e., style factor). Our hypotheses read as follows. As the number of response categories increases, we expect subjects to be more likely to exert extreme response style. Furthermore, we expect subjects to be more likely to adopt a midpoint response style when they are offered a middle response category. To investigate these hypotheses we developed a split ballot experiment in which the number of response categories is manipulated from 5 to 11 categories. Data are collected by a random sample, large-scale web survey which allows for random assignment to the experimental conditions. The results show clear evidence of extreme response style and moderate evidence of midpoint response style. Extreme response style is not affected by the length of response scales, whereas the exertion of midpoint response style only popped up in the longer scale versions.

Adolescent Self-esteem: Differences by Race/Ethnicity, Gender, and Age

Large-scale representative surveys of 8th-, 10th-, and 12th-grade students in the United States show high self-esteem scores for all groups. African-American students score highest, Whites score slightly higher than Hispanics, and Asian Americans score lowest. Males score slightly higher than females. Multivariate controls for grades and college plans actually heighten these race/ethnic/gender differences. A truncated scoring method, designed to counter race/ethnic differences in extreme response style, reduced but did not eliminate the subgroup differences. Age differences in self-esteem are modest, with 12th graders reporting the highest scores. The findings are highly consistent across 18 annual surveys from 1991 through 2008, and self-esteem scores show little overall change during that period.

The effect of rating scale format on response styles: The number of response categories and response category labels

Questionnaires using Likert-type rating scales are an important source of data in marketing research. Researchers use different rating scale formats with varying numbers of response categories and varying label formats (e.g., 7-point rating scales labeled at the endpoints, fully labeled 5-point scales, etc.) but have few guidelines when selecting a specific format. Drawing from the literature on response styles, we formulate hypotheses on the effect of the labeling of response categories and the number of response categories on the net acquiescence response style, extreme response style and misresponse to reversed items. We test the hypotheses in an online survey ( N = 1207) with eight experimental conditions and a follow-up study with two experimental conditions ( N = 226). We find evidence of strong effects of scale format on response distributions and misresponse to reversed items, and we formulate recommendations on the choice of a scale format.

Self-report Response Bias: Learning How to Live with its Diagnosis in Chaplaincy Research

Chaplaincy research is dominated by self-report data collected directly from research subjects or participants. Self-report response bias is the research measurement inaccuracy that originates with the respondent. A review of research published in The Journal of Pastoral Care & Counseling (1998–2008) found that all but one of thirty-eight research articles used self-report data. Of this total, less than half acknowledged methodological limitations, and only two acknowledged the potential impact of self-report response bias.This article focuses on seven categories of self-report response bias that may impact chaplaincy research: social desirability, acquiescence, leniency or harshness, critical event or recency, halo effect, extreme response style and midpoint response style. Although these biases have the potential to impact self-report data, the data themselves are not inherently flawed. This discussion offers recommendations for addressing self-report response bias during the research process. It also s...

The stability of individual response styles.

Modeling capabilities for longitudinal data have progressed considerably, but questions remain on the extent to which method bias may negatively affect the validity of longitudinal survey data. The current study addresses the stability of individual response styles. We set up a longitudinal data collection in which the same respondents filled out 2 online questionnaires with nonoverlapping sets of heterogeneous items. Between data collections, there was a 1-year time gap. We simultaneously modeled 4 response styles that capture the major directional biases in questionnaire responses: acquiescence, disacquiescence, midpoint, and extreme response style. Drawing from latent state-trait theory, we specified a 2nd-order factor model with time-invariant and time-specific response style factors and a specifically designed covariance structure for the residual terms. The results indicate that response styles have an important stable component, a small part of which can be explained by demographics. The meaning and implications of these findings are discussed.

Response Styles and how to Correct them

Abstract Cross-mode surveys are on the rise. Unfortunately, data obtained from different modes of data collection (e.g., telephone and online data) may not be comparable due to measurement bias, especially differences in acquiescence, disacquiescence, extreme and midpoint response styles. This article discusses a study that finds response style differences between data based on the same questionnaire, but obtained by different modes of data collection: paper-and-pencil questionnaires, telephone interviews, and online questionnaires. Similar problems may also occur in cross-national data. We propose a new method to measure response styles and correct for them: the representative indicators response style means and covariance structure (RIRSMACS) method