Understanding Acquiescence Bias: The Mediating Role of Careless Responding in Demographic Differences

From a cross-cultural perspective, we studied the psychological meaning of a generalized response style that combines acquiescent, extreme, and midpoint response styles. Individual- and country-level indexes of the three specific response styles were constructed from eight multicountry surveys for correlation and multilevel analyses. At both levels, we confirmed a general response style factor with a positive loading of extreme response style, a negative loading of midpoint response style, and acquiescent response style in between. This general response style refers to a communication filter that moderates or amplifies expressions and it is associated with both individual and cultural factors. At the country level, the general response style was negatively related to the socioeconomic development and the percentage of atheists and positively related to aggregated values and personality traits pertinent to “fitting in” and avoidance of ambiguity. At the individual level, the general response style was positively associated with age and negatively associated with education. We conclude that integrating specific response styles to a general response style can help to create consistency in findings across styles and studies. Implications for cross-cultural management research and practices are discussed.

The severity of bias in respondents’ self-reports due to acquiescence response style (ARS) and extreme response style (ERS) depends strongly on how consistent these response styles are over the course of a questionnaire. In the literature, different alternative hypotheses on response style (in)consistency circulate. Therefore, nine alternative models are derived and fitted to secondary and primary data. It is found that response styles are best modeled as a tau-equivalent factor complemented with a time-invariant autoregressive effect. This means that ARS and ERS are largely but not completely consistent over the course of a questionnaire, a finding that has important implications for response style measurement and correction.

Acquiescent response style (ARS), the tendency for survey respondents to agree with survey items, is of particular concern for increasing measurement error in surveys with populations who are more likely to acquiesce, such as Latino respondents in the U.S. In order to develop methods for reducing ARS, this study addressed two questions: (1) Does administering a questionnaire using conversational interviewing (CI) yield less ARS than standardized interviewing (SI)? (2) Do bipolar disagree/agree (DA) response scales lead to higher ARS than unipolar response scales that do not assess agreement (non-AG)? A total of 891 Latino telephone survey respondents were screened for ARS and randomly assigned to four experimental groups determined by crossing interviewing technique (CI or SI) and response format (non-AG or DA): (1) SI/non-AG (n=301); (2) SI/DA (n=295); (3) CI/non-AG (n=149); and (4) CI/DA (n=146). CI yielded lower ARS than SI (p<0.001), but there was no difference in ARS between DA and non-AG response scales. A subset of coded interview recordings indicated that the CI interviewers reduced ARS by clarifying questions even in the absence of evidence of respondent confusion and helping with response mapping. These results suggest that difficulty answering questions associated with cognitive decline and cultural norms may have prompted higher use of ARS, but that conversational interviewers were able to mitigate these difficulties and cultural tendencies. Findings from this study suggest that using CI to administer survey questions may decrease ARS and improve data quality among survey respondents who are more likely to engage in ARS.

Item response tree (IRTree) models are recently introduced as an approach to modeling response data from Likert-type rating scales. IRTree models are particularly useful to capture a variety of individuals' behaviors involving in item responding. This study employed IRTree models to investigate response styles, which are individuals' tendencies to prefer or avoid certain response categories in a rating scale. Specifically, we introduced two types of IRTree models, descriptive and explanatory models, perceived under a larger modeling framework, called explanatory item response models, proposed by De Boeck and Wilson. This extends the typical application of IRTree models for studying response styles. As a demonstration, we applied the descriptive and explanatory IRTree models to examine acquiescence and extreme response styles in Rosenberg's Self-Esteem Scale. Our findings suggested the presence of two distinct extreme response styles and acquiescence response style in the scale.

Keywords: Careless responding; data cleaning; data integrity; invalid responding; online survey; research outcome quality

Careless response and attrition as sources of bias in online survey assessments of personality traits and performance

Promotion of on-line pharmacies in social media - frequency and content

Developing a short screener for acquiescent respondents.

The present study aimed to identify the differences in the ability of Iranian students based on their response styles in the TIMSS (Trends in International Mathematics and Science Study) Grade 8 science test for the years 2011, 2015, and 2019. This study examined various response styles, including extreme, moderate, and acquiescent styles, as well as their effects on item psychometric parameters and individual abilities. Data were analyzed using various Item Response Theory (IRT) models. The results showed that in the 2011 TIMSS Grade 8 science test, individuals with a typical response style exhibited the highest ability, followed by those with a moderate response style. The lowest ability was observed among individuals with an acquiescent response style. In the 2015 TIMSS Grade 8 science test, individuals with an extreme response style had the highest average ability among all groups, suggesting their superior ability compared to other individuals. Furthermore, in the 2019 TIMSS Grade 8 science test, there was no significant difference in the abilities of individuals with a moderate response style and those with an acquiescent response style (P &gt; 0.05). However, a statistically significant difference was found between the abilities of individuals with an extreme response style and those of other groups (P &lt; 0.05). A statistically significant difference was also observed between the abilities of individuals with a typical response style and those with an acquiescent response style. These findings can contribute to the improvement of standardized test design, reduction of cultural biases, and provision of strategies for more accurate assessments at both national and international levels.

Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sources of cross-cultural differences on survey-type instruments. Differences in ERS and ARS, if undetected, may give rise to spurious results that do not reflect genuine differences in attitudes or perceptions. Multiple-group confirmatory factor analysis is recommended as the most effective method of testing for ERS and ARS and determining whether cultural groups can be meaningfully compared on the basis of factor (latent) means. A detailed numerical example is provided.

Assessing the measurement model (MM) of self-report scales is crucial to obtain valid measurements of individuals' latent psychological constructs. This entails evaluating the number of measured constructs and determining which construct is measured by which item. Exploratory factor analysis (EFA) is the most-used method to evaluate these psychometric properties, where the number of measured constructs (i.e., factors) is assessed, and, afterward, rotational freedom is resolved to interpret these factors. This study assessed the effects of an acquiescence response style (ARS) on EFA for unidimensional and multidimensional (un)balanced scales. Specifically, we evaluated (a) whether ARS is captured as an additional factor, (b) the effect of different rotation approaches on the content and ARS factors recovery, and (c) the effect of extracting the additional ARS factor on the recovery of factor loadings. ARS was often captured as an additional factor in balanced scales when it was strong. For these scales, ignoring extracting this additional ARS factor, or rotating to simple structure when extracting it, harmed the recovery of the original MM by introducing bias in loadings and cross-loadings. These issues were avoided by using informed rotation approaches (i.e., target rotation), where (part of) the rotation target is specified according to a priori expectations on the MM. Not extracting the additional ARS factor did not affect the loading recovery in unbalanced scales. Researchers should consider the potential presence of ARS when assessing the psychometric properties of balanced scales and use informed rotation approaches when suspecting that an additional factor is an ARS factor.

PurposeExtant research shows that acquiescence response style (ARS) is culture-bound and may bias the results of comparative cross-cultural studies. Conventional measures of ARS are difficult to apply in practice. To overcome this limitation, the purpose of this paper is to propose an alternative, practice-oriented measure, namely, pARS. The authors apply Hofstede’s cultural dimensions (Hofstede et al., 2010) to test whether pARS is culture-bound. The cross-cultural study provides a high level of cross-cultural generalisability due to the extensive number of surveyed countries (n=30) and subjects (n=236.089). The authors run multi-level analysis to identify within- and between-country-level predictors.Design/methodology/approachOn the individual level, the authors use data of a large-scale cross-cultural study, including 236.089 consumers from 30 countries worldwide. The authors apply several methods to test for the culture-boundness of pARS. First, they apply correlation analysis to replicate existing cross-cultural results and to ensure nomological validity. Second, applying ordinary least square regression, the authors simultaneously test the six Hofstede cultural dimensions (Hofstede et al., 2010) and investigate interactions between the dimensions. Finally, they use multi-level analysis to confirm the stability of culture-bound results, controlling for individual- and country-level variability.FindingsThe paper introduces an alternative measure for acquiescence (pARS), which is particularly suitable for shorter questionnaires. A large-scale consumer study with 236.089 respondents in 30 countries supports the culture-bound validity of pARS. The authors confirm construct validity and the nomological network of pARS. Contrasting existing studies, multi-level analysis demonstrates that a high level of power distance majorly leads to ARS. Therefore, cross-cultural researchers need to control for ARS in countries high in power distance, especially when paired with high uncertainty avoidance.Originality/valueA large-scale consumer study with 236.089 respondents from 30 countries shows that respondents from various countries differ significantly in their level of acquiescence. The study confirms that power distance is the most relevant cultural dimension to explain these differences. Although ARS may bias the results of comparative cross-cultural studies, it is rarely controlled by market research studies outside the academic realm. The present work proposes and establishes the validity of a practice-oriented measure of acquiescence, namely, pARS. pARS is particularly suitable for shorter questionnaires. In contrast to prior approaches, applying pARS does not require adding non-substantive items to the questionnaire.

This study explores the acquiescent response style (ARS) among respondents in the Czech Republic. To analyse ARS, confirmatory factor analysis (CFA) was employed and the response style (RS) was modelled as a latent variable. The RS factor in the CFA model must be validated by its relationship to education and age, i.e. proxies of cognitive ability. The two studies presented in this article use large amount of data as all available balanced batteries of items were analysed. In Study 1, the RS factor showed no correlation with age/education in most of the models. Study 2 employed additional measures of cognitive ability and education and confirmed the overall absence of the relationship. Author concludes that the modelled RS factor then cannot be considered the ARS and deduces that the most likely explanation of the identified systematic error variance is the respondents’ carelessness manifested by automatically choosing the extreme agreement, or choosing the first presented answer as a result of memory effect.

Careless responding (CR) in surveys has been identified as a serious threat for the validity of survey data. It occurs when participants respond without following the instructions or reading the content of items. The purpose of this article is to assess and explain the amount of CR at the individual and country level. In our study, individual CR (measured by indices using a short Big 5 personality questionnaire) correlated with education level and personality traits. In addition, using a sample of 8,320 participants from 34 countries, CR at the country level was investigated and it was strongly correlated with the cultural dimensions: human development, individualism, gender inequality, and power distance. CR can be seen as a powerful predictor of differences between countries. Finally, a comparison between CR and response styles (extreme and midpoint answers, acquiescence, and socially desirable responses) was conducted. CR and response styles showed some overlap, but loaded on different factors. CR is compatible with extreme answers, midpoint answers, and acquiescence, but different from social desirability. Social desirability is the only response style which requires careful reading of items.

Extreme response style (ERS) and acquiescence response style (ARS) are among the most encountered problems in attitudinal research. The authors investigate whether the response bias caused by these response styles varies with following three aspects of question format: full versus end labeling, numbering answer categories, and bipolar versus agreement response scales. A questionnaire was distributed to a random sample of 5,351 respondents from the Longitudinal Internet Studies for the Social Sciences household panel, of which a subsample was assigned to one of five conditions. The authors apply a latent class factor model that allows for diagnosing and correcting for ERS and ARS simultaneously. The results show clearly that both response styles are present in the data set, but ARS is less pronounced than ERS. With regard to format effects, the authors find that end labeling evokes more ERS than full labeling and that bipolar scales evoke more ERS than agreement style scales. With full labeling, ERS opposes opting for middle response categories, whereas end labeling distinguishes ERS from all other response categories. ARS did not significantly differ depending on test conditions.