Extreme Response Style Research Articles

Investigating Directional Invariance in an Item Response Tree Model for Extreme Response Style and Trait-Based Unfolding Responses.

Item response tree (IRTree) approaches have received increasing attention in the response style literature due to their capability to partial out response style latent traits from content-related latent traits by considering separate decisions for agreement and level of agreement. Additionally, it has shown that the functioning of the intensity of agreement decision may depend upon the agreement decision with an item, so that the item parameters and person parameters may differ by direction of agreement; when the parameters across direction are the same, this is called directional invariance. Furthermore, for non-cognitive psychological constructs, it has been argued that the response process may be best described as following an unfolding process. In this study, a family of IRTree models to handle unfolding responses with the agreement decision following the hyperbolic cosine model and the intensity of agreement decision following a graded response model is investigated. This model family also allows for investigation of item- and person-level directional invariance. A simulation study is conducted to evaluate parameter recovery; model parameters are estimated with a fully Bayesian approach using JAGS (Just Another Gibbs Sampler). The proposed modeling scheme is demonstrated with two data examples with multiple model comparisons allowing for varying levels of directional invariance and unfolding versus dominance processes. An approach to visualizing the final model item response functioning is also developed. The article closes with a short discussion about the results.

The Effects of Questionnaire Length on the Relative Impact of Response Styles in Ambulatory Assessment

Ambulatory assessment (AA) is becoming an increasingly popular research method in the fields of psychology and life science. Nevertheless, knowledge about the effects that design choices, such as questionnaire length (i.e., number of items per questionnaire), have on AA data quality is still surprisingly restricted. Additionally, response styles (RS), which threaten data quality, have hardly been analyzed in the context of AA. The aim of the current research was to experimentally manipulate questionnaire length and investigate the association between questionnaire length and RS in an AA study. We expected that the group with the longer (82-item) questionnaire would show greater reliance on RS relative to the substantive traits than the group with the shorter (33-item) questionnaire. Students (n = 284) received questionnaires three times a day for 14 days. We used a multigroup two-dimensional item response tree model in a multilevel structural equation modeling framework to estimate midpoint and extreme RS in our AA study. We found that the long questionnaire group showed a greater reliance on RS relative to trait-based processes than the short questionnaire group. Although further validation of our findings is necessary, we hope that researchers consider our findings when planning an AA study in the future.

Exploring the Influence of Response Styles on Continuous Scale Assessments: Insights From a Novel Modeling Approach

The use of discrete categorical formats to assess psychological traits has a long-standing tradition that is deeply embedded in item response theory models. The increasing prevalence and endorsement of computer- or web-based testing has led to greater focus on continuous response formats, which offer numerous advantages in both respondent experience and methodological considerations. Response styles, which are frequently observed in self-reported data, reflect a propensity to answer questionnaire items in a consistent manner, regardless of the item content. These response styles have been identified as causes of skewed scale scores and biased trait inferences. In this study, we investigate the impact of response styles on individuals’ responses within a continuous scale context, with a specific emphasis on extreme response style (ERS) and acquiescence response style (ARS). Building upon the established continuous response model (CRM), we propose extensions known as the CRM-ERS and CRM-ARS. These extensions are employed to quantitatively capture individual variations in these distinct response styles. The effectiveness of the proposed models was evaluated through a series of simulation studies. Bayesian methods were employed to effectively calibrate the model parameters. The results demonstrate that both models achieve satisfactory parameter recovery. Neglecting the effects of response styles led to biased estimation, underscoring the importance of accounting for these effects. Moreover, the estimation accuracy improved with increasing test length and sample size. An empirical analysis is presented to elucidate the practical applications and implications of the proposed models.

Answering tendencies on questionnaires: Comparing Mandarin and Bahasa Malaysia versus English

Aims Conducting psychological research in different countries and cultures necessitates measures in different languages. However, the language of a measure might influence responses, even within the same multilingual individual. The cultural accommodation theory proposes that one’s association with a language influences their responses. Moreover, response styles (RSs), such as an extreme or acquiescence RS, might systematically affect responses regardless of the content of the measure. These effects were reported on culture-related measures but are unclear on culture-free measures. Methodology and analyses We aimed to investigate the effects of language on psychological measures that do not explicitly examine cultural factors. Multilingual Malaysians ( n = 111) filled in the Adult Executive Functioning Inventory (ADEXI), the Depression, Anxiety, and Stress Scale—21 items (DASS-21), the Brief Coping Orientation to Problems Experienced Inventory (Brief COPE), the Pediatric Quality of Life Inventory (PedsQL), and the Traditional Masculinity-Femininity Scale (TMF) in Bahasa Malaysia and English, or in Mandarin and English Findings. There were no language differences on the ADEXI and TMF. However, several subscales of the Brief COPE, the Stress subscale of the DASS, and the PedsQL scores were higher in Mandarin than in English. On the Brief COPE and the PedsQL, there were also differences in RS between Mandarin and English, which might explain (part of) these differences. There were no differences between Bahasa Malaysia and English in scores. However, there was a more extreme RS in English than in Bahasa Malaysia and a more acquiescence RS in Bahasa Malaysia than in English on the Brief COPE. These differences suggest that the measures are not culture-free or that previously reported language differences did not result from culture alone. The language of a measure might be an additional important factor. When using different translations of the same measure, it is important to take cultural accommodation and RS into account.

코로나19 시기 청소년이 지각한 재난안전에 대한 응답양식 분석

The purpose of this study was to analyze the survey data on the perception of disaster safety by adolescents during the COVID-19 period, using the data from the 2020 Korean Children and Youth Panel Survey (KCYPS). As a large social disaster, COVID-19 could affect the adolescents' responses in addition to their perceptions of disasters. Therefore, we applied the item response tree model to analyze the response patterns. The results showed that adolescents generally felt safe from disasters such as COVID-19, but their perceptions of disaster safety differed between safety of individuals and that of society. In addition, it was confirmed that there were negative extreme response styles and positive extreme response styles, which represent excessive anxiety and excessive indifference to disasters, respectively. Moreover, the adolescents' perceptions of disaster safety were related to life satisfaction, happiness, and the degree of knowledge of disaster response measures. We also proposed the utilization and follow-up studies of the research results.

Separation of Traits and Extreme Response Style in IRTree Models: The Role of Mimicry Effects for the Meaningful Interpretation of Estimates.

Item response tree (IRTree) models are a flexible framework to control self-reported trait measurements for response styles. To this end, IRTree models decompose the responses to rating items into sub-decisions, which are assumed to be made on the basis of either the trait being measured or a response style, whereby the effects of such person parameters can be separated from each other. Here we investigate conditions under which the substantive meanings of estimated extreme response style parameters are potentially invalid and do not correspond to the meanings attributed to them, that is, content-unrelated category preferences. Rather, the response style factor may mimic the trait and capture part of the trait-induced variance in item responding, thus impairing the meaningful separation of the person parameters. Such a mimicry effect is manifested in a biased estimation of the covariance of response style and trait, as well as in an overestimation of the response style variance. Both can lead to severely misleading conclusions drawn from IRTree analyses. A series of simulation studies reveals that mimicry effects depend on the distribution of observed responses and that the estimation biases are stronger the more asymmetrically the responses are distributed across the rating scale. It is further demonstrated that extending the commonly used IRTree model with unidimensional sub-decisions by multidimensional parameterizations counteracts mimicry effects and facilitates the meaningful separation of parameters. An empirical example of the Program for International Student Assessment (PISA) background questionnaire illustrates the threat of mimicry effects in real data. The implications of applying IRTree models for empirical research questions are discussed.

Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles.

The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from "Strongly Disagree" to "Strongly Agree". Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research.

Investigating Heterogeneity in Response Strategies: A Mixture Multidimensional IRTree Approach.

To improve the validity of self-report measures, researchers should control for response style (RS) effects, which can be achieved with IRTree models. A traditional IRTree model considers a response as a combination of distinct decision-making processes, where the substantive trait affects the decision on response direction, while decisions about choosing the middle category or extreme categories are largely determined by midpoint RS (MRS) and extreme RS (ERS). One limitation of traditional IRTree models is the assumption that all respondents utilize the same set of RS in their response strategies, whereas it can be assumed that the nature and the strength of RS effects can differ between individuals. To address this limitation, we propose a mixture multidimensional IRTree (MM-IRTree) model that detects heterogeneity in response strategies. The MM-IRTree model comprises four latent classes of respondents, each associated with a different set of RS traits in addition to the substantive trait. More specifically, the class-specific response strategies involve (1) only ERS in the "ERS only" class, (2) only MRS in the "MRS only" class, (3) both ERS and MRS in the "2RS" class, and (4) neither ERS nor MRS in the "0RS" class. In a simulation study, we showed that the MM-IRTree model performed well in recovering model parameters and class memberships, whereas the traditional IRTree approach showed poor performance if the population includes a mixture of response strategies. In an application to empirical data, the MM-IRTree model revealed distinct classes with noticeable class sizes, suggesting that respondents indeed utilize different response strategies.

Relationships Between Response Styles and the Hofstede and GLOBE Dimensions of Culture in a Sample of Adolescents From 33 Countries

A number of studies have examined the relationships between response styles and the Hofstede and GLOBE cultural indices; however, studies involving adolescent samples or examining the effects of national wealth on observed relationships are scarce. This study addresses these gaps by applying simple and partial correlation analysis to the data of 15-year-olds in 33 PISA 2006 countries. The study found that the relationships between response styles and cultural indices in the two frameworks are similar to those in past studies of adult populations. After accounting for GDP per capita, the majority of relationships remained unchanged. However, others, such as Hofstede’s power distance and acquiescence and dis-acquiescence, lost significance, and Hofstede’s masculinity and extreme response styles only gained significance when GDP per capita was held constant. The findings highlight the influence of cultural values on students’ questionnaire-response behaviours, which should be recognised in comparative studies.

A Systematic Review of Response Styles Among Latinx Populations.

Culture influences responses to psychological measurements in ways unrelated to target constructs, thus biasing test scores and potentially contributing to under- and over-diagnosis of mental health problems in populations for which measures have not yet been normed. We conducted a systematic review of publications addressing response style among Latinx population groups in North and South America. In a final corpus of 24 studies, Latinx/Latin American populations were generally found to exhibit higher levels of extreme response style (n = 17), acquiescent response style (n = 10), and socially desirable responding (n = 5). The few publications (n = 3) that investigated midpoint responding reported no differences. Seven publications (29%) attempted to adjust scores to mitigate response style bias, using both scale design and statistical techniques. Findings suggest that researchers and clinicians should directly assess culturally patterned response style as a construct, rather than inferring style indirectly using other measures. For clinicians, knowledge of response style represents another facet of case conceptualization.

Moving beyond Likert and Traditional Forced-Choice Scales: A Comprehensive Investigation of the Graded Forced-Choice Format

The graded forced-choice (FC) format has recently emerged as an alternative that may preserve the advantages and overcome the issues of the dichotomous FC measures. The current study presented the first large-scale evaluation of the performance of three types of FC measures (FC2, FC4 and FC5 with 2, 4 and 5 response options, respectively) and compared their performance to their Likert (LK) counterparts (LK2, LK4, and LK5) on (1) psychometric properties, (2) respondent reactions, and (3) susceptibility to response styles. Results showed that, compared to LK measures with the same number of response options, the three FC scales provided better support for the hypothesized factor structure, were perceived as more faking-resistant and cognitive demanding, and were less susceptible to response styles. FC4/5 and LK4/5 demonstrated similarly good reliability, while LK2 provided more reliable scores than FC2. When compared across the three FC measures, FC4 and FC5 displayed comparable psychometric performance and respondent reactions. FC4 exhibited a moderate presence of extreme response style, while FC5 had a weak presence of both extreme and middle response styles. Based on these findings, the study recommends the use of graded FC over dichotomous FC and LK, particularly FC5 when extreme response style is a concern.

Correcting for Extreme Response Style: Model Choice Matters.

Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results. For this reason, various item response theory (IRT) models have been proposed to model ERS and correct for it. Comparisons of these models are however rare in the literature, especially in the context of cross-cultural comparisons, where ERS is even more relevant due to cultural differences between groups. To remedy this issue, the current article examines two frequently used IRT models that can be estimated using standard software: a multidimensional nominal response model (MNRM) and a IRTree model. Studying conceptual differences between these models reveals that they differ substantially in their conceptualization of ERS. These differences result in different category probabilities between the models. To evaluate the impact of these differences in a multigroup context, a simulation study is conducted. Our results show that when the groups differ in their average ERS, the IRTree model and MNRM can drastically differ in their conclusions about the size and presence of differences in the substantive trait between these groups. An empirical example is given and implications for the future use of both models and the conceptualization of ERS are discussed.

Cultural Bias in Parent Reports: The Role of Socialization Goals When Parents Report on Their Child’s Problem Behavior

It has been called into question whether widely used screening instruments for child mental health can provide comparable results across countries and cultures. Socialization goals can influence whether and to what extent a parent considers a behavior to be problematic and thus might influence parental reports on their child’s behavior. We tested comparability of parental reports between native German (N = 116) and Turkish origin (N = 77) parents in Germany in an online study using a vignette approach. Parents were asked to rate the perceived problem severity of the same behavior depicted in the vignettes. We expected and found that parents of Turkish origin in Germany rate the externalizing problem behaviour depicted in the vignettes as more problematic compared to native German parents. The effect was fully mediated by parental approval of the socialization goals obedience and collectivism. We also controlled for social desirability responding and an extreme response style.

Personality, culture and extreme response style: A multilevel modelling analysis

This study investigated the dual contribution of personality traits and cultural dimensions to Extreme Response Style, while controlling for the effects of age, gender, and GDP per capita. Participant data was extracted from the myPersonality Project (n = 173,075). Multilevel modelling compared 10 models to determine which variables best predicted Extreme Response Style. Models that included personality demonstrated significantly better fit than those with cultural values alone. Personality factors predicted Extreme Response Style more strongly than the other tested variables, with Openness to Experience showing the strongest relationship. Effect sizes for all fixed effects across all models were small. Limitations included the inability to determine participants’ subcultures or migration statuses, online recruitment method, and small sample sizes in some countries.

New Customer Satisfaction Index for the Japanese Market: From Net Promoter Score to Promoter Score Japan

Objective - This paper aims to examine the applicability of the Net Promoter Score (NPS) to Japanese consumers. The NPS measures the relationship between customer loyalty and corporate growth as an alternative to the conventional customer satisfaction index because of its simplicity and ease of understanding for managers. However, because of its simplicity, the NPS has been criticized for not considering the culture and consumer characteristics of the various countries in which it is used. In this study, we review Japanese consumers' response styles and examine a more adaptable measurement scale to the Japanese market. Methodology/Technique - The sales data and customer NPS ratings of four companies in four industries—a cosmetics brand, a fashion e-commerce company, a daily necessities manufacturer, and a credit card company—were collected, and the relationship between them was analyzed to examine the validity of NPS classifications for the Japanese market. Findings –The study yielded the following results. (1) Customers who give the highest ratings tend to exert a more positive impact on corporate earnings through sales (spending amount). (2) Median spending amount intersects with the recommendation score at around 6. (3) On an ordinal scale from 0 to 10, the recommendation frequency significantly increases at 5. Moreover, this study reveals that the NPS classification of consumers is inconsistent with Japanese customers' actual situation. Novelty - As an alternative to the Net Promoter Score (NPS), the Promoter Score Japan (PSJ) is proposed to measure customer satisfaction and produce more accurate recommendations in the Japanese market. Type of Paper: Empirical JEL Classification: M31, M39. Keywords: Net Promoter Score; customer satisfaction; customer recommendation; response bias; net promoter ratio; extreme response style; midpoint response style Reference to this paper should be made as follows: Kimura, T. (2022). New Customer Satisfaction Index for the Japanese Market: From Net Promoter Score to Promoter Score Japan, J. Mgt. Mkt. Review, 7(3), 129 – 137. https://doi.org/10.35609/jmmr.2022.7.3(5)

Individual-Level Cultural Factors and Use of Survey Response Styles Among Latino Survey Respondents

Acquiescent (ARS) and extreme response styles (ERS) can have detrimental effects on survey data and, for unknown reasons, are more frequently used by Latino than non-Latino white respondents. This exploratory study examined the influence of culture on these response styles by investigating their associations with individual-level cultural factors and ARS and ERS among 1,296 Mexican American, Puerto Rican, and Cuban American telephone survey respondents. Principal components representing stronger endorsement of marianismo/ machismo and social attentiveness ( simpatía, personalismo, respect for elders, value for sincerity, collectivism, individualism) were associated with higher ARS and ERS, while higher trust in strangers and more limited health literacy were associated with lower ERS. Findings from this study will enable survey designers to better anticipate ARS and ERS in surveys with Latino populations and, in turn, guide the selection of data collection and analysis methods to mitigate measurement error in the presence of these response styles.

다척도 심리검사 자료의 이질적 극단응답양식 탐색을 위한 혼합문항반응나무모형의 적용

Item response tree model (IRTree) is a sort of a multidimensional item response theory that incorporates a tree structure depicting a respondent's cognitive response process into item response theory. IRTree has recently received considerable attention from researchers for measuring extreme response style (ERS) that can threaten reliability and factor structure in self-report survey data. In addition to the work of IRTree, we extend IRTree as the mixture item response tree model (MixIRTree), which combines latent class analysis with IRTree for measuring and investigating the possible heterogeneity of data caused by ERS. For analyses, this study used the body symptom and depression scales of the emotional problems in the third wave of KCYPS 2018. The results of the study are as follows. First, the three latent classes were captured, showing the different levels of ERS. The latent class 3 did not show the ERS tendency than the latent classes 1 and 2. The latent class 1 showed the largest ERS tendency. Second, the ERS tendency across the latent classes was different between the negative affect scales. For example, the ERS tendency of the latent class 1 was larger in the depression scale than body symptom scale, while the ERS tendency of the latent class 2 was larger in the body symptom scale than the depression scale. Based on these findings, we provide practical implications of the MixIRTree for measuring the heterogeneity of ERS and discuss the future direction for further study.

The Impact of Extreme Response Style on the Mean Comparison of Two Independent Samples

Extreme response style (ERS) is prevalent in survey research using rating scales. It may cause biased results in group comparisons. This research conducted two sets of simulation studies to explore the magnitude of the ERS impact on mean comparisons between two independent samples. Data were generated from a multidimensional nominal response model. Study 1 examined the influence of ERS on the estimate of group differences in the variable of interest. The results indicated that ERS led to biased estimates, especially when these groups differed significantly in ERS. The correlation between ERS and the variable of interest also moderated the ERS impact. The results were illustrated with an empirical example. Study 2 investigated the impact of ERS on the type I error and type II error in the independent t-test based on scale scores. When the variable of interest had no true difference between groups, ERS inflated the type I error rate. When the difference existed, ERS inflated the type II error rate. Two groups’ true difference in ERS and the variable of interest, unequal ERS variances, the correlation between ERS and the variable of interest, and the number of items moderated the impact of ERS on type I and II error rates. The implications for practices and further research are discussed.

Exploring the patterns and degrees of extreme response styles of TIMSS 2019 data using the IRTree model

본 연구에서는 IRTree 모형을 활용하여 극단응답양식과 이를 통제한 응답자 모수를 추정하고 동일한 자료를 GRM으로 분석한 결과와 비교하였다. 또한 극단응답양식이 성별, 학년, 성취수준에 따라 어떤 양상으로 나타나는지 탐색하였다. 이를 위해 TIMSS 2019 설문 자료 중 과학에 대한 자신감과 흥미 척도를 이용하였고, 참여국 중 유럽, 아시아, 북미 문화권에서 9개 국가를 선정하였다. 연구 결과, GRM 모형보다 IRTree 모형이 자료에 더 적합한 것으로 나타났다. 과학에 대한 자신감 척도와 흥미 척도에 대해 IRTree 모형을 적용하여 추정된 극단응답양식과 응답자 모수는 정적 상관을 보였으며, IRTree 모형과 GRM 모형에서 산출된 응답자 모수들도 정적 상관을 나타내었다. 성별, 학년, 과학 성취수준에 따라 극단응답양식의 크기는 차이를 보였으며, 극단응답양식의 크기에 따라 응답자 모수의 왜곡 정도가 다름을 확인할 수 있었다.

Kişi Uyumu ve Uç Yanıt Stili Arasındaki İlişki

Extreme response style (ERS) refers to a disproportionate preference for endpoint options in Likert type or rating scale items, regardless of their content. The ERS has the potential to deteriorate the fit of the measurement model in use. In the current study, it was proposed that the use of the ERS is one possible cause of aberrant response patterns. In the context of item response theory (IRT), aberrant response patterns are also known as misfit responses, and these patterns are investigated using person-fit statistics. Accordingly, the aim of this study was determined to investigate whether the ERS is associated with misfit response patterns. Using a convenient sampling method, personality data from 727 university students were collected for this aim using the Basic Personality Traits Inventory (BPTI). The findings revealed that people who use the ERS were more likely to produce misfit response patterns. The results were consistent for both the lower-tail and the upper-tail ERS use. Based on the findings, it was concluded that the ERS is one plausible reason for a person's misfit in the context of personality m. More in-depth analyses of the link between the ERS and person-fit are encouraged.