Midpoint Response Research Articles

Measuring health-related quality of life of older people with frailty receiving acute care: feasibility and psychometric performance of the EuroQol EQ-5D

BackgroundAlthough outcome goals for acute healthcare among older people living with frailty often include Health-Related Quality of Life (HRQoL) and other patient-reported outcome measures (PROMs), current quality metrics usually focus on waiting times and survival. Lay and patient review have identified the EuroQol EQ-5D as a candidate measure for this setting. This research appraised the EQ-5D for feasibility, psychometric performance, and respondents’ outcomes in the acute frailty setting.MethodsPeople aged 65 + with Clinical Frailty Scale (CFS) 5–8 were recruited from eight UK hospitals’ emergency care and acute admissions settings. They completed the five-level EQ-5D and the EQ-VAS. Feasibility was assessed with completion times and completeness. For reliability, response distributions and internal consistency were analysed. Finally, EQ-Index values were compared with demographic characteristics and service outcomes for construct validity.ResultsThe 232 participants were aged 65–102. 38% responded in emergency departments and 62% in admissions wards. Median completion time was 12 (IQR, 11) minutes. 98% responses were complete. EQ-5D had acceptable response distribution (SD 1.1–1.3) and internal consistency (Cronbach’s alpha 0.69). EQ-VAS demonstrated a midpoint response pattern. Median EQ-Index was 0.574 (IQR, 0.410) and was related positively with increasing age (p = 0.010) and negatively with CFS (p < 0.001). Participants with higher CFS had more frequent problems with mobility, self-care, and usual activities.ConclusionsAdministration of the EQ-5D was feasible in these emergency and acute frailty care settings. EQ-5D had acceptable properties, while EQ-VAS appeared problematic. Participants with more severe frailty had also poorer HRQoL.

Person Misfit and Person Reliability in Rating Scale Measures: The Role of Response Styles

ABSTRACT Person misfit and person reliability indices in item response theory (IRT) can play an important role in evaluating the validity of a test or survey instrument at the respondent level. Prior empirical comparisons of these indices have been applied to binary item response data and suggest that the two types of indices return very similar results. In this paper, however, we demonstrate an important applied distinction between these methods when applied to polytomously-scored rating scale items, namely their varying sensitivities to response style tendencies. Using several empirical datasets, we illustrate settings in which these indices are in one case highly correlated and in two other cases weakly correlated. In the datasets showing a weak correlation between indices, the primary distinction appears due to the effects of response style behavior, whereby respondents whose response styles are less common (e.g. a disproportionate selection of the midpoint response) are found to misfit using Drasgow et al’s person misfit index, but often show high levels of reliability from a person reliability perspective; just the opposite frequently occurs for respondents that over-select the rating scale extremes. It is suggested that person misfit reporting should be supplemented with an evaluation of person reliability to best understand the validity of measurement at the respondent level when using IRT models with rating scale measures.

Analysis of the tendency to select the “neither nor” option in agreement/disagreement scales on a low – salience topic: The contribution of individual differences

This paper examines the frequency of midpoint responses in agree/disagree scales in a survey measuring attitudes toward science and scientific policy, a low-salience and difficult topic. It also examines the contribution of individual differences to explain the tendency to select this option. It is assumed that the use of the midpoint “Neither agree nor disagree” (NA/ND) in Agree/Disagree (A/D) scales is to some extent an indication of satisficing. It is also assumed that there are individual differences in respondents’ tendency to select the NA/ND response. Using a Generalized Linear Mixed Model we include Krosnick’s regulators of satisficing, socio-demographics and individual differences as predictors. We find that the contribution of the regulators of satisficing identified by Krosnick is small. In turn, factors associated with individual differences explain a great amount of variance on the number of NA/ND responses. We conclude that the presence of this option in a survey of a low-salience topic increases satisficing as strategy of the respondents to deal with the cognitive burden of both the A/D scale and the difficulty of the topic. Furthermore, the number of NA/ND responses may be understood as an indicator of individual differences in the propensity of respondents to satisficing when answering a survey on a low-salience or difficult topic.

New Customer Satisfaction Index for the Japanese Market: From Net Promoter Score to Promoter Score Japan

Objective - This paper aims to examine the applicability of the Net Promoter Score (NPS) to Japanese consumers. The NPS measures the relationship between customer loyalty and corporate growth as an alternative to the conventional customer satisfaction index because of its simplicity and ease of understanding for managers. However, because of its simplicity, the NPS has been criticized for not considering the culture and consumer characteristics of the various countries in which it is used. In this study, we review Japanese consumers' response styles and examine a more adaptable measurement scale to the Japanese market. Methodology/Technique - The sales data and customer NPS ratings of four companies in four industries—a cosmetics brand, a fashion e-commerce company, a daily necessities manufacturer, and a credit card company—were collected, and the relationship between them was analyzed to examine the validity of NPS classifications for the Japanese market. Findings –The study yielded the following results. (1) Customers who give the highest ratings tend to exert a more positive impact on corporate earnings through sales (spending amount). (2) Median spending amount intersects with the recommendation score at around 6. (3) On an ordinal scale from 0 to 10, the recommendation frequency significantly increases at 5. Moreover, this study reveals that the NPS classification of consumers is inconsistent with Japanese customers' actual situation. Novelty - As an alternative to the Net Promoter Score (NPS), the Promoter Score Japan (PSJ) is proposed to measure customer satisfaction and produce more accurate recommendations in the Japanese market. Type of Paper: Empirical JEL Classification: M31, M39. Keywords: Net Promoter Score; customer satisfaction; customer recommendation; response bias; net promoter ratio; extreme response style; midpoint response style Reference to this paper should be made as follows: Kimura, T. (2022). New Customer Satisfaction Index for the Japanese Market: From Net Promoter Score to Promoter Score Japan, J. Mgt. Mkt. Review, 7(3), 129 – 137. https://doi.org/10.35609/jmmr.2022.7.3(5)

Effects of gender on income and family planning for diplomates of the American College of Zoological Medicine.

To evaluate income and family planning decisions of American College of Zoological Medicine (ACZM) diplomates. 98 ACZM diplomates. An online survey was sent to 201 ACZM diplomates. Participation was voluntary. 98 (49%) diplomates responded to the survey. The most commonly reported income categories were $90,000 to $94,999, $100,000 to $104,999, and $110,000 to $114,999. Overall, the mean of the salary-category midpoint responses was $105,357 but was $122,917 for those in academia and $94,508 for those working in zoos and aquaria. When incomes of males and females were matched (24 pairs matched for gender and age), no difference in income was observed. There were no significant differences in income between males and females with and without children. Diplomates who did not complete a residency had significantly higher incomes than diplomates who did. Sixteen of 21 (76%) females and 9 of 19 (47%) males reported delaying having children because of their career. Additionally, a higher percentage of females with children (13/20 [65%]) than males with children (3/19 [16%]) felt that having children had had a negative effect on their career. Thirty-five of 41 (85%) females without children and 4 of 9 (44%) males without children thought having children would have negatively affected their careers. Although substantial differences in income between female and male ACZM diplomates were not identified, differences in family planning and perceptions of the impact of having children on their careers did exist.

A psychometric model for respondent-level anchoring on self-report rating scale instruments.

Among the various forms of response bias that can emerge with self-report rating scale assessments are those related to anchoring, the tendency for respondents to select categories in close proximity to the rating category used for the immediately preceding item. In this study we propose a psychometric model based on a multidimensional nominal model for response style that also simultaneously accommodates a respondent-level anchoring tendency. The model is estimated using a fully Bayesian estimation procedure. By applying this model to a real test data set measuring extraversion, we explore a theory that both response styles and anchoring might be viewed as evidence of a lack of effortful responding. Empirical results show that there is a positive correlation between the strength of midpoint response style and the anchoring effect; further, responses indicative of either anchoring or response style both negatively correlate with response time, consistent with a theory that both phenomena reflect reduced respondent effort. The results support attending to both anchoring and midpoint response style as ways of assessing respondent engagement.

Measuring Response Style Stability Across Constructs With Item Response Trees.

Individual response style behaviors, unrelated to the latent trait of interest, may influence responses to ordinal survey items. Response style can introduce bias in the total score with respect to the trait of interest, threatening valid interpretation of scores. Despite claims of response style stability across scales, there has been little research into stability across multiple scales from the beneficial perspective of item response trees. This study examines an extension of the IRTree methodology to include mixed item formats, providing an empirical example of responses to three scales measuring perceptions of social media, climate change, and medical marijuana use. Results show extreme and midpoint response styles were not stable across scales within a single administration and 5-point Likert-type items elicited higher levels of extreme response style than the 4-point items. Latent trait of interest estimation varied, particularly at the lower end of the score distribution, across response style models, demonstrating as appropriate response style model is important for adequate trait estimation using Bayesian Markov chain Monte Carlo estimation.

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

BackgroundIn three recent studies, Maul demonstrated that sets of nonsense items can acquire excellent psychometric properties. Our aim was to find out why responses to nonsense items acquire a well-defined structure and high internal consistency.MethodWe designed two studies. In the first study, 610 participants responded to eight items where the central term (intelligence) was replaced by the term “gavagai”. In the second study, 548 participants responded to seven items whose content was totally invented. We asked the participants if they gave any meaning to “gavagai”, and conducted analyses aimed at uncovering the most suitable structure for modeling responses to meaningless items.ResultsIn the first study, 81.3% of the sample gave “gavagai” meaning, while 18.7% showed they had given it no interpretation. The factorial structures of the two groups were very different from each other. In the second study, the factorial model fitted almost perfectly. However, further analysis revealed that the structure of the data was not continuous but categorical with three unordered classes very similar to midpoint, disacquiescent, and random response styles.DiscussionApparently good psychometric properties on meaningless scales may be due to (a) respondents actually giving an interpretation to the item and responding according to that interpretation, or (b) a false positive because the statistical fit of the factorial model is not sensitive to cases where the actual structure of the data does not come from a common factor. In conclusion, the problem is not in factor analysis, but in the ability of the researcher to elaborate substantive hypotheses about the structure of the data, to employ analytical procedures congruent with those hypotheses, and to understand that a good fit in factor analysis does not have a univocal interpretation and is not sufficient evidence of either validity nor good psychometric properties.

Explaining Variability in Response Style Traits: A Covariate-Adjusted IRTree.

Contamination of responses due to extreme and midpoint response style can confound the interpretation of scores, threatening the validity of inferences made from survey responses. This study incorporated person-level covariates in the multidimensional item response tree model to explain heterogeneity in response style. We include an empirical example and two simulation studies to support the use and interpretation of the model: parameter recovery using Markov chain Monte Carlo (MCMC) estimation and performance of the model under conditions with and without response styles present. Item intercepts mean bias and root mean square error were small at all sample sizes. Item discrimination mean bias and root mean square error were also small but tended to be smaller when covariates were unrelated to, or had a weak relationship with, the latent traits. Item and regression parameters are estimated with sufficient accuracy when sample sizes are greater than approximately 1,000 and MCMC estimation with the Gibbs sampler is used. The empirical example uses the National Longitudinal Study of Adolescent to Adult Health's sexual knowledge scale. Meaningful predictors associated with high levels of extreme response latent trait included being non-White, being male, and having high levels of parental support and relationships. Meaningful predictors associated with high levels of the midpoint response latent trait included having low levels of parental support and relationships. Item-level covariates indicate the response style pseudo-items were less easy to endorse for self-oriented items, whereas the trait of interest pseudo-items were easier to endorse for self-oriented items.

A Comparison of Limited-Information Test Statistics for a Response Style MIRT Model

An increased use of models for measuring response styles is apparent in recent years with the multidimensional nominal response model (MNRM) as one prominent example. Inclusion of latent constructs representing extreme (ERS) or midpoint response style (MRS) often improves model fit according to information criteria. However, a test of absolute model fit is often not reported even though it could comprise an important piece of validity evidence. Limited information test statistics are candidates for this task, including the full (), ordinal (), and mixed () statistics, which differ in whether additional collapsing of univariate or bivariate contingency tables is conducted. Such collapsing makes sense when item categories are ordinal, which may not hold under the MNRM. More generally, limited information test statistics have gone unevaluated under nominal data and non-ordinal latent trait models. We present a simulation study evaluating the performance of and with the MNRM. Manipulated conditions included sample size, presence and type of response style, and strength of item slopes on substantive and style dimensions. We found that sometimes had inflated Type I error rates, always had little power, and lacked power under some conditions. and may provide complementary and valuable information regarding model fit.

Seeing the Forest and the Trees: Comparison of Two IRTree Models to Investigate the Impact of Full Versus Endpoint-Only Response Option Labeling

In this study, we examined the results and interpretations produced from two different IRTree models-one using paths consisting of only dichotomous decisions, and one using paths consisting of both dichotomous and polytomous decisions. We used data from two versions of an impulsivity measure. In the first version, all the response options had labels; in the second version, only the endpoints were labeled. Based on past research, we hypothesized that the endpoints would be selected more frequently in the endpoint-only labeled condition, and the midpoint response option would be selected more frequently in the fully labeled condition. Results from the two models (dichotomous and polytomous) were similar and indicated that our hypotheses were partially supported-specifically, there was no consistent pattern in terms of which condition saw a higher frequency of midpoint response selection. However, our hypotheses regarding extreme responding in the endpoint-only labeling condition were supported.

Do response styles affect the predictive validity of self-rated health among Chinese older men and women?

This study intended to examine whether correcting for response styles of the widely used self-rated health (SRH) question would affect its predictive validity in estimating respondents' mortality risks. We used data on a sample of 3319 men and 3250 women aged 60 and above from a nationally representative survey in China. Response styles were estimated with both ratings on two health anchoring vignettes and self-rated items in domains unrelated to health. Gender-stratified logistic regressions were used to estimate the effects of SRH on 4-year mortality risks. We compared the results before and after adjusting for response styles. In the unconditional model, the effects of the five-point SRH on 4-year mortality risks were significant in both older women and men, and stronger among Chinese older men based on discrete change in probabilities of mortality at means. Gender difference was observed in response styles. Men assigned lower ratings to the vignettes than women, reflecting their higher standards in defining good health. Women presented higher proportions of extreme responses and lower proportions of midpoint responses. Correcting for response styles had no effect on the predictive validity of SRH for Chinese older men, but had some positive effects for Chinese older women when no other covariates were included. Adjusting for response styles contributed to a small improvement in predicting mortality risks only among Chinese older women under restricted conditions, but not men.

Validity of Three IRT Models for Measuring and Controlling Extreme and Midpoint Response Styles.

Response styles, the general tendency to use certain categories of rating scales over others, are a threat to the reliability and validity of self-report measures. The mixed partial credit model, the multidimensional nominal response model, and the item response tree model are three widely used models for measuring extreme and midpoint response styles and correcting their effects. This research aimed to examine and compare their validity by fitting them to empirical data and correlating the content-related factors and the response style-related factors in these models to extraneous criteria. The results showed that the content factors yielded by these models were moderately related to the content criterion and not related to the response style criteria. The response style factors were moderately related to the response style criteria and weakly related to the content criterion. Simultaneous analysis of more than one scale could improve their validity for measuring response styles. These findings indicate that the three models could control and measure extreme and midpoint response styles, though the validity of the mPCM for measuring response styles was not good in some cases. Overall, the multidimensional nominal response model performed slightly better than the other two models.

The Dynamics of “Neither Agree Nor Disagree” Answers in Attitudinal Questions

Abstract Attitudinal questions are an integral part of surveys in the social sciences. Previous research based on cross-sectional data has shown that both respondents’ characteristics and questionnaire design can lead to higher use of midpoint responses when questions are operationalized on uneven rating scales. To further current understanding of this phenomenon, in this article we apply hybrid regression models to analyze differences between respondents but also developmental changes within respondents, allowing for a more profound interpretation of the dynamics behind midpoint responses, theoretically explained by satisficing behavior. For our midpoint analyses, we use a set of attitudinal item blocks asked in the British Household Panel Survey (BHPS) from 1991 to 2008. Respondents’ reports and interviewers’ judgments offer satisficing-related indicators regarding ability and motivation, enabling particularly accurate analyses of midpoint response behavior in terms of the “neither agree nor disagree” option. Results show that depending on respondents’ specific characteristics, higher use of the midpoint cannot automatically be equated with nonsubstantive response behavior. Furthermore, we demonstrate that specific indicators related to low ability and motivation do not uniformly increase midpoint responses. Application of the hybrid model reveals that changes in respondents’ characteristics do not affect midpoint response. We speculate that there are unobserved personality attributes that affect the propensity to midpoint response and conclude by reflecting on reasons for the increase in midpoint responses over the years.

IRTree models with ordinal and multidimensional decision nodes for response styles and trait-based rating responses.

IRTree models decompose observed rating responses into sequences of theory-based decision nodes, and they provide a flexible framework for analysing trait-related judgements and response styles. However, most previous applications of IRTree models have been limited to binary decision nodes that reflect qualitatively distinct and unidimensional judgement processes. The present research extends the family of IRTree models for the analysis of response styles to ordinal judgement processes for polytomous decisions and to multidimensional parametrizations of decision nodes. The integration of ordinal judgement processes overcomes the limitation to binary nodes, and it allows researchers to test whether decisions reflect qualitatively distinct response processes or gradual steps on a joint latent continuum. The extension to multidimensional node models enables researchers to specify multiple judgement processes that simultaneously affect the decision between competing response options. Empirical applications highlight the roles of extreme and midpoint response style in rating judgements and show that judgement processes are moderated by different response formats. Model applications with multidimensional decision nodes reveal that decisions among rating categories are jointly informed by trait-related processes and response styles.

調査の回答における中間選択 ―原因，影響とその対策―

調査の回答における中間選択 ―原因，影響とその対策―

Multidimensional Modeling of Traits and Response Styles

Abstract. Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

New Evidence on the Construct Validity of the ASI-3 and the Dimensional Conceptualization of Trait Anxiety Sensitivity From IRT Modeling

Abstract. Anxiety Sensitivity (AS) denotes the tendency to fear anxiety-related sensations. Trait AS is an established risk factor for anxiety pathology. The Anxiety Sensitivity Index-3 (ASI-3) is a widely used measure of AS and its three most robust dimensions with well-established construct validity. At present, the dimensional conceptualization of AS, and thus, the construct validity of the ASI-3 is challenged. A latent class structure with two distinct and qualitatively different forms, an adaptive form (normative AS) and a maladaptive form (AS taxon, predisposing for anxiety pathology) was postulated. Item Response Theory (IRT) models were applied to item-level data of the ASI-3 in an attempt to replicate previous findings in a large nonclinical sample (N = 2,603) and to examine possible interpretations for the latent discontinuity observed. Two latent classes with a pattern of distinct responses to ASI-3 items were found. However, classes were indicative of participant’s differential use of the response scale (midpoint and extreme response style) rather than differing in AS content (adaptive and maladaptive AS forms). A dimensional structure of AS and the construct validity of the ASI-3 was supported.

日本人の回答バイアス――レスポンス・スタイルの種別間・文化間比較――

This study attempts to identify response styles of Japanese students by cross-level and cross-national analyses. Tasaki and Ninomiya (2013) demonstrated the existence of an acquiescence response behavior of Japanese students based on the model by Billet and McClendon(2000), who measured latent response characteristics as “style factor.” In this study, we used a different response style measurement model proposed by Weijters, Schillewaert, & Geuens (2008) positing that response styles are psychological constructs and observed response style variables are summary indicators derived from sets of various, but independent, test items. Results of cross-national analyses indicated that Japanese and Korean participants were more likely to choose a mid-point response (MRS) and less likely to choose an acquiescence response (ARS) than American counterparts, but no significant difference was observed for an extreme response (ERS) among participants from the three nations. Within the Japanese response style model, cross-level analyses showed that ARS scores were significantly higher than ERS scores, indicating that Japanese participants had an acquiescence response style, consistent with the findings from Tasaki & Ninomiya (2013).

The Person-Fit Statistic in an Unfolding Model Context.

Although person-fit analysis has a long-standing tradition within item response theory, it has been applied in combination with dominance response models almost exclusively. In this article, a popular log likelihood-based parametric person-fit statistic under the framework of the generalized graded unfolding model is used. Results from a simulation study indicate that the person-fit statistic performed relatively well in detecting midpoint response style patterns and not so well in detecting extreme response style patterns.