Question Order Effects Research Articles

On the social signal function of emotional crying: Broadening the perspective to social interactions in daily life.

Emotional crying is a potent signal that can influence how social interactions unfold. Although the social signal function of crying has been mostly studied in experimental approaches involving hypothetical scenarios with strangers as targets, we propose that daily social interactions should be considered to address aspects of external validity. We conducted three retrospective studies (total N = 2,277; convenience samples; Studies 2 and 3 were preregistered; data were collected in 2022 and 2023) and found that individuals primarily observed close others crying in their daily lives; a boundary condition that was hardly reflected in former studies. Crying episodes were typically characterized by tears and changes in facial expression, vocalizations, and gestures, whereas isolated emotional tears, were rarely reported. Participants indicated that they frequently helped close others, but that helping strangers was the exception. The level of familiarity with the target also influenced the form of help provided. Furthermore, the crying person was rated as less warm and less competent when crying (vs. in general or in a neutral situation). Additional analyses suggest that question order effects and recall biases are no plausible alternative explanations of our findings. Taken together, these results call into question the validity of the experimental approaches used in this line of research. We discuss strategies for future study of the social signal function of crying in nonhypothetical scenarios with reference to the "generalizability crisis" of psychological science. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Question-Order Effect in the Study of Satisfaction with Democracy: Lessons from Three Split-Ballot Experiments

This study examines question-order effects in measuring satisfaction with democracy (SWD). Particularly, the authors are interested in whether the relative position of the question regarding satisfaction with the state of the economy (SWE) in the questionnaire affects responses to the SWD item. The authors conducted three independent split-ballot experiments in Hungary between March 2021 and May 2022. They report a significant and substantial negative priming effect that possibly leads to a systematic underestimation of SWD. Importantly, the authors find no question-order effect in the measurement of SWE. The analysis further reveals a contrast effect: when the SWD question is primed, the difference between SWE and SWD means increases. The authors’ final recommendation is that researchers either put the SWD question before the SWE item to avoid question-order bias or randomize question order. These findings should assist future data collection efforts (comparative or single-country studies) in developing and integrating a battery of satisfaction items into questionnaires and help users assess data quality.

Bias in job analysis survey ratings attributed to order effects

AbstractWhen respondents complete long surveys, evidence suggests that they may change their response behavior as they become fatigued. The present study examines question order effects in a block‐randomized job analysis survey that collected task ratings (importance, frequency, and needed at certification) for a certification program. Analyses showed that average task ratings were lower for tasks presented later in the survey for both complete and partial responses, though the effect of question ordering was stronger for non‐completers in two rating categories (importance and needed at certification). Additionally, rater variance was lower for questions that appeared later in the survey. The results highlight the value of using randomization to mitigate order effects on survey response quality.

Future Directions: The Phenomenology of Irritable Mood and Outbursts: Hang Together or Hang Separately1

ABSTRACT Recognition of the importance of irritable mood and outbursts has been increasing over the past several decades. This “Future Directions” aims to develop a set of recommendations for future research emphasizing that irritable mood and outbursts “hang together,” but have important distinctions and thus also need to “hang separately.” Outbursts that are the outcome of irritable mood may be quite different from outbursts that are the trigger or driving force that make youth and his/her environment miserable. What, then, is the relation between irritable mood and outbursts? As the field currently stands, we not only cannot answer this question, but we may also lack the tools to effectively do so. Here, we will propose recommendations for understanding the phenomenology of irritable mood and outbursts so that more directed and clinically useful assessment tools can be designed. We discuss the transdiagnostic and treatment implications that relate to improvements in measurement. We describe the need to do more than repurpose our current assessment tools, specifically interviews and rating scales, which were designed for different purposes. The future directions of the study and treatment of irritable mood and outbursts will require, among others, using universally accepted nomenclature, supporting the development of tools to measure the characteristics of each irritable mood and outbursts, understanding the effects of question order, informant, development and longitudinal course, and studying the ways in which outbursts and irritable mood respond to treatment.

Question order effects: how robust are survey measures on political solidarities with reference to Germany and Europe?

ABSTRACT The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people’s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people’s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.

A quantum-like information processing model with memory noise for question order effect

This paper aims to better explain a typical quantum-like phenomenon in human uncertain decision-making, i.e., question order effect (QOE), by characterizing the evolution of human preferences for decision-making during information processing. It is achieved mainly by introducing two mechanisms into the information processing model with quantum mathematical formalism: immediate feedback and noise disturbance. The immediate feedback mechanism is based on the projection postulate and non-commutative operators, providing the model with the basic ability to explain QOE. This is essentially consistent with one existing well-known quantum-like model. Moreover, the noise disturbance mechanism from memory is proposed for the first time, which formalizes the evolution of human preferences from the unstable state formed by immediate feedback to the stable preferences represented by long-term memory. This mechanism can weaken QOE by partially offsetting the immediate feedback. In applying to five datasets, the model performs better in explaining QOE than three types of existing quantum-like models. Besides, the response replicability effect, which typically contradicts the interpretation of QOE in other quantum-like models, can be explained by the proposed model owing to the noise disturbance. The quantum-like information processing model considering memory noise provides an innovative and noteworthy insight into the evolution of human preferences in information processing, especially the quantum-like phenomena in uncertain decision-making.

Nondistributivity of human logic and violation of response replicability effect in cognitive psychology

The aim of this paper is to promote quantum logic as one of the basic tools for analyzing human reasoning. We compare it with classical (Boolean) logic and highlight the role of violation of the distributive law for conjunction and disjunction. It is well known that nondistributivity is equivalent to incompatibility of logical variables — the impossibility to assign jointly the two-valued truth values to these variables. A natural question arises as to whether quantum logical nondistributivity in human logic can be tested experimentally. We show that testing the response replicability effect (RRE) in cognitive psychology is equivalent to testing nondistributivity — under the prevailing conjecture that the mental state update generated by observation is described as orthogonal projection of the mental state vector (the projective update conjecture of Wang and Busemeyer). A simple test of RRE is suggested. In contrast to the previous works in quantum-like modeling, we proceed in the state-dependent framework; in particular, distributivity, compatibility, and RRE are considered in a fixed mental state. In this framework, we improve the previous result on the impossibility to combine question order and response replicability effects by using (von Neumann–Lüders) projective measurements.

A comparison of question order effects on item-by-item and grid formats: visual layout matters

Question order effect refers to the phenomenon that previous questions may affect the cognitive response process and respondents’ answers. Previous questions generate a context or frame in which questions are interpreted. At the same time, in online surveys, the visual design may also shift responses. Past empirical research has yielded considerable evidence supporting the impact of question order on measurement, but few studies have investigated how question order effects vary with the visual design. Our main research question was whether question order effects are different on item-by-item formats compared to grid formats. The study uses data from an online survey experiment conducted on a non-probability-based online panel in Hungary, in 2019. We used the welfare-related questions of the 8’th wave of ESS. We manipulated the questionnaire by changing the position of a question that calls forth negative stereotypes about such social benefits and services. We further varied the visual design by presenting the questions in separate pages (item-by-item) or one grid. The results show that placing the priming questions right before the target item significantly changed respondents’ attitudes in a negative way, but the effect was significant only when questions were presented on separate pages. A possible reason behind this finding may be that respondents engage in a deeper cognition when questions are presented separately. On the other hand, the grid format was robust against question order, in addition, we found little evidence of stronger satisficing on grids. The findings highlight that mixing item-by-item and grids formats in online surveys may introduce measurement inequivalence, especially when question order effects are expected.

More Causes Less Effect: Destructive Interference in Decision Making.

We present a new experiment demonstrating destructive interference in customers’ estimates of conditional probabilities of product failure. We take the perspective of a manufacturer of consumer products and consider two situations of cause and effect. Whereas, individually, the effect of the causes is similar, it is observed that when combined, the two causes produce the opposite effect. Such negative interference of two or more product features may be exploited for better modeling of the cognitive processes taking place in customers’ minds. Doing so can enhance the likelihood that a manufacturer will be able to design a better product, or a feature within it. Quantum probability has been used to explain some commonly observed “non-classical” effects, such as the disjunction effect, question order effect, violation of the sure-thing principle, and the Machina and Ellsberg paradoxes. In this work, we present results from a survey on the impact of multiple observed symptoms on the drivability of a vehicle. The symptoms are assumed to be conditionally independent. We demonstrate that the response statistics cannot be directly explained using classical probability, but quantum formulation easily models it, as it allows for both positive and negative “interference” between events. Since quantum formalism also accounts for classical probability’s predictions, it serves as a richer paradigm for modeling decision making behavior in engineering design and behavioral economics.

Revealing Gender Double Standards in the Parenthood Norm Depends on Question Order

Becoming a parent has been described as a dominant social norm, especially for women. Though some research has indicated changes toward more flexible gendered parenthood norms, methodological issues may be masking the continued presence of a gender double standard. In line with the condition for activation of double standards, we postulated that endorsement of the parenthood norm would vary depending on the response context. Our aim was to analyze the parenthood norm for women and for men taking into account the response context in a quantitative survey. In a French nationally-representative sample, more than 4,000 female and male adults were asked whether a woman/man can have a fulfilled life without having children in two questions presented in a random order. Based on the literature on question-order effects, the answer to the first question should be influenced by the participant’s personal background (e.g., gender, parental status), i.e., the personal background context, whereas the question asked second should be influenced by the comparison with the first question, i.e. the social relational context. In the personal background context, the own-gender parenthood norm was endorsed more strongly than the other-gender parenthood norm by both female and male participants. In contrast, in the social relational context, the parenthood norm for women was endorsed more strongly than the parenthood norm for men by both female and male participants. Our results showed a strong gender double standard observed only in the comparative context and illustrates the need to use appropriate survey methodology to examine the presence of gendered social norms.

Quantum Explanations for Interference Effects in Engineering Decision Making

<div class="section abstract"><div class="htmlview paragraph">Engineering practice routinely involves decision making under uncertainty. Much of this decision making entails reconciling multiple pieces of information to form a suitable model of uncertainty. As more information is collected, one expectedly makes better and better decisions. However, conditional probability assessments made by human decision makers, as new information arrives does not always follow expected trends and instead exhibits inconsistencies. Understanding them is necessary for a better modeling of the cognitive processes taking place in their mind, whether it be the designer or the end-user. Doing so can result in better products and product features. Quantum probability has been used in the literature to explain many commonly observed deviations from the classical probability such as: question order effect, response replicability effect, Machina and Ellsberg paradoxes and the effect of positive and negative interference between events. In this work, we present results from a survey demonstrating responses that while difficult to explain using classical probability, can be explained using a quantum formulation - highlighting its potential in engineering applications. Since quantum formulism is more general and can also match the predictions of classical probability, it serves as a richer paradigm for modeling decision making behavior in engineering practice.</div></div>

1431. Evaluating Physician Decision Making in Inpatient Antibiotic Prescription for Suspected Urinary Tract Infection

BackgroundPhysicians are constantly asked to evaluate inpatients for possible antibiotic treatment. As part of antibiotic stewardship it is imperative to understand the decision-making process behind a physician’s choice to prescribe antibiotics appropriately in an inpatient setting. Fuzzy Trace Theory (FTT) suggests that physicians use one of two methods in medical decision making; verbatim, employing a comprehensive risk benefit analysis, and gist, considering a bottom line analysis. MethodsSeventy-six hospitalists at Weill Cornell Medicine in Manhattan, New York received a survey with two reminders to evaluate their decision-making process. Five basic demographic questions regarding participant gender, race, background, age, and years in practice were asked. A clinical vignette describing an inpatient with a possible urinary tract infection (UTI) was followed with statements framing hypothetical antibiotic prescription. A seven point Likert scale with response choices from Strongly Disagree scored as one to Strongly Agree scored as seven was used to assess degree of participant agreement with each statement. Questions were presented in a random order to eliminate possible effects of questions clusters or question order. ResultsTwenty-six hospitalists completed the survey. Consistent with previous literature, the hospitalists surveyed displayed a gist interpretation of the risks and benefits of antibiotics, with a mean Likert scale score of 5.54 agreeing that there are benefits to antibiotic prescription, and a mean Likert scale score of 6.04, agreeing that there are risks to antibiotic prescription. . However, the clinicians surveyed ultimately found antibiotics to be a necessary risk given the possible benefit of improving patient health. The hospitalists surveyed also did not view antibiotic prescription to be a product of pressure from patient families, agreeing by a mean Likert scale score of 5.08 that the patient’s family will trust their physician to prescribe antibiotics if needed. ConclusionThese findings suggest that physician education to reduce overprescribing of antibiotics should underscore possible antibiotic risk, despite potential benefit. Disclosures Marshall J. Glesby, MD, Enzychem (Consultant)Gilead (Grant/Research Support)ReAlta Life Sciences (Consultant)Regeneron (Consultant, Grant/Research Support)Sobi (Consultant)Springer (Other Financial or Material Support, Royalties)UpToDate (Other Financial or Material Support, Royalties) Justin J. Choi, MD, Allergan (Consultant, Grant/Research Support)Roche (Consultant, Grant/Research Support)

The Ordering of Gambling Severity and Harm Scales: A Cautionary Tale

Question-order effects are known to occur in surveys, particularly those that measure subjective experiences. The presence of context effects will impact the comparability of results if questions have not been presented in a consistent manner. In this study, we examined the influence of question order on how people responded to two gambling scales in the Australian Capital Territory Gambling Prevalence Survey: The Problem Gambling Severity Index and the Short Gambling Harm Screen. The application of these scales in gambling surveys is continuing to grow, the results being compared across time and between jurisdictions, countries, and populations. Here we outline a survey experiment that randomized the question ordering of these two scales. The results show that question-order effects are present for these scales, demonstrating that results from them may not be comparable across jurisdictions if the scales have not been presented consistently across surveys. These findings highlight the importance of testing for the presence of question-order effects, particularly for those scales that measure subjective experiences, and correcting for such effects where they exist by randomizing scale order.

Quantum Cognition.

Uncertainty is an intrinsic part of life; most events, affairs, and questions are uncertain. A key problem in behavioral sciences is how the mind copes with uncertain information. Quantum probability theory offers a set of principles for inference, which align well with intuition about psychological processes in certain cases: cases when it appears that inference is contextual, the mental state changes as a result of previous judgments, or there is interference between different possibilities. We motivate the use of quantum theory in cognition and its key characteristics. For each of these characteristics, we review relevant quantum cognitive models and empirical support. The scope of quantum cognitive models encompasses fallacies in decision-making (such as the conjunction fallacy or the disjunction effect), question order effects, conceptual combination, evidence accumulation, perception, over-/underdistribution effects in memory, and more. Quantum models often formalize psychological ideas previously expressed in heuristic terms, allow unified explanations of previously disparate findings, and have led to several surprising, novel predictions. We also cast a critical eye on quantum models and consider some of their shortcomings and issues regarding their further development.

Results of a Global Survey of Experts to Categorize the Suitability of Interventions for Inclusion in School Health Services

PurposeThis global survey of experts assessed the suitability of different health-related interventions for inclusion in school health services (SHSs) to inform development of the World Health Organization global guideline on SHSs. MethodsA review of 138 global World Health Organization publications identified 406 health service interventions for 5- to 19-year-old individuals. These were consolidated, pretested, and pilot-tested in a questionnaire as 86 promotion, prevention, care, or treatment interventions. A total of 1,293 experts were identified through purposive sampling of journal databases and professional networks. In July 2019, experts were invited to complete the questionnaire online in Arabic, Chinese, English, French, Russian, or Spanish. Respondents categorized each intervention as essential, highly suitable, suitable, or unsuitable in SHSs (everywhere or in certain geographic areas only). They could also suggest interventions. ResultsInterventions categorized most often as “Essential in SHSs everywhere” (70%–80%) are related to health promotion and health education. Clinical interventions categorized most often in this way (60%–68%) are related to immunization, screening, assessment, and general care. Interventions categorized most often as “Essential in SHSs in certain geographic areas only” (27%–49%) are related to immunization, mass drug administration, and health promotion. Interventions categorized most often as “Unsuitable in SHSs anywhere” (12%–14%) are related to screening of noncommunicable conditions. There were no important regional differences. Of 439 respondents from 81 countries, 188 suggested 378 additional interventions. Question order effect and/or purposive sampling biases may have influenced both quantitative and qualitative results for different types of intervention. ConclusionsFavorable responses to almost all interventions supported their World Health Organization guideline inclusion but provided little guidance for intervention prioritization.

Question Order Effects in Cross-Cultural Web Probing: Pretesting Behavior and Attitude Questions

Cognitive pretesting is an essential method of piloting questionnaires and ensuring quality of survey data. Web probing has emerged as an innovative method of cognitive pretesting, especially for cross-cultural and web surveys. The order of presenting questions in cognitive pretesting can differ from the order of presentation in the later survey. Yet empirical evidence is missing whether the order of presenting survey questions influences the answers to open-ended probing questions. The present study examines the effect of question order on web probing in the United States and Germany. Results indicate that probe responses are not strongly impacted by question order. However, both content and consistency of probe responses may differ cross-culturally. Implications for cognitive pretesting are discussed.

Message framing, environmental behaviour and support for carbon capture and storage in Australia

In Australia, the polarised debate over the pace and scale of change necessary to mitigate climate change has been exacerbated by inconsistent federal government policies which are seen to hinder the transition to a low carbon energy future. This includes the way in which particular energy technologies have been framed within the debate, with potential effects on public support for a given technology. This paper presents the results of a study that examined question order framing effects on support for a range of energy technologies across a nationally representative survey of the Australian public. It was hypothesised that this experimental manipulation would allow us to better understand how the salience of different frames may lead to shifts in individuals’ preferences for various energy technologies. Four different experimental frames were developed covering the following topics: knowledge; cost and reliability; climate change: and energy behaviours. Overall, our results demonstrated that such question order effects can impact support for the range of different options. The energy behaviours frame demonstrated an increase in overall levels of support for nearly all energy technologies and was the only frame that significantly increased overall levels of support for CCS. Consistent with prior research, overall support for renewable energy technologies remained highest. However, the frames led to unexpected outcomes in their impacts on expressed support. While the reasons for these responses are not clear, this study highlights an important consideration for ongoing survey research into the effects of question sequencing and priming around topic salience.

Modeling combination of question order effect, response replicability effect, and QQ-equality with quantum instruments

We continue to analyze basic constraints on the human decision making from the viewpoint of quantum measurement theory (QMT). As it has been found, the conventional QMT based on the projection postulate cannot account for the combination of the question order effect (QOE) and the response replicability effect (RRE). This was an alarming finding for quantum-like modeling of decision making. Recently, it was shown that this difficulty can be resolved by using of the general QMT based on quantum instruments. In the present paper we analyze the problem of the combination of QOE, RRE, and the well-known QQ-equality (QQE). This equality was derived by Busemeyer and Wang, and it was shown (in a joint paper with Solloway and Shiffrin) that statistical data from many social opinion polls satisfy it. Here we construct quantum instruments satisfying QOE, RRE and QQE. The general features of our approach are formalized with postulates that generalize (the Wang–Busemeyer) postulates for quantum-like modeling of decision making. Moreover, we show that our model closely reproduces the statistics of the well-known Clinton–Gore Poll data with a prior belief state independent of the question order. This model successfully corrects for the order effect in the data to determine the “genuine” distribution of the opinions in the Poll. The paper also provides an accessible introduction to the theory of quantum instruments — the most general mathematical framework for quantum measurements.

Effects of Question Type and Order When Measuring Peak Consumption of Risky Drinking Events.

There is new interest in measuring alcohol consumption during risky drinking events, but there is little guidance on how to best ask such questions. In this study, we contrast two different types of questions on peak consumption over a single heavy drinking occasion. We used a general question that ask respondents to recall the total amount consumed (total consumption question), and location-specific questions that ask respondents to recall consumption in each drinking location (location-specific peak consumption, LSPC). Heavy drinkers (≥11 Australian Standard Drinks (ASD) per occasion for males, ≥8 for females) from the second wave of a prospective cohort study were recruited via landline random digit dial from Melbourne in 2012. Respondents were randomly assigned to surveys of different question order, and either first received total consumption (n=127) or LSPC questions (n=147). T-tests compared peak consumption between categories stratified by sex and consumption tercile. Mean peak consumption was 12.5 ASD. Irrespective of question order, consumption amounts for total consumption and LSPC questions were not significantly different for both sexes. However, drinkers in the highest tercile asked LSPC questions first provided significantly higher consumption estimates in response to the total consumption question than in response to the LSPC questions. At a population level, LSPC and total consumption questions produce similar estimates of peak consumption for risky drinking events. Except for heavy drinkers, general consumption questions may be sufficient when asking about these drinking events in consumption surveys, without the greater response burden of longer LSPC questions.

Question order bias revisited: A split‐ballot experiment on satisfaction with public services among experienced and professional users

AbstractPublic decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.