Effect Of Response Mode Research Articles

Determining the Effects of Response Mode and Incentives on Survey Response Rates of School-based Agricultural Education Teachers: An Experimental Study

A decline in response rates was found for agricultural education research studies using survey research methods. The use of incentives and various response modes can affect survey response rates and were the focus of this experiment. The purpose of this study was to determine the effects of survey response mode and incentives on response rates when surveying SBAE teachers. Findings revealed a significant increase in response rates for groups receiving an incentive. No significant differences were found between groups using different survey response modes. When examining summated scale scores for each construct in the questionnaire, participants answering on paper scored more positively in four of the six constructs compared to web responses. Recommendations for practice included using mixed response modes when financially feasible and oversampling to achieve the desired sample size to represent a population. Further research on the effects of communication mode on response rates is needed. An examination of why differences in scale scores occur with different response modes is recommended.

Effects of survey response mode, purported topic, and incentives on response rates in human dimensions of fisheries and wildlife research

ABSTRACT Incorporating human values and behavior into natural resource management has been linked to more sustainable and efficient outcomes. Estimating these values often requires survey-based data collection. Declining response rates have amplified the need to more fully understand the influence of survey design features on response. In this study, we used an experiment to measure the effect of response mode (mail vs. web), purported topic (recreation vs. saltwater angling), and incentives ($0, $2, $5) on two outcomes of interest: screener response and extended questionnaire response. The context is a survey of saltwater anglers on the Pacific coast of the United States. We found increased probabilities of screener response with an incentive, with a stronger effect within the mail response mode. At the extended questionnaire stage, we found some effect of survey response mode that was moderated by topic and incentive. Our results help inform future survey efforts.

Measuring Subjective Probabilities: The Effect of Response Mode on the Use of Focal Responses, Validity, and Respondents’ Evaluations

Subjective probabilities are central to risk assessment, decision making, and risk communication efforts. Surveys measuring probability judgments have traditionally used open-ended response modes, asking participants to generate a response between 0% and 100%. A typical finding is the seemingly excessive use of 50%, perhaps as an expression of "I don't know." In an online survey with a nationally representative sample of the Dutch population, we examined the effect of response modes on the use of 50% and other focal responses, predictive validity, and respondents' survey evaluations. Respondents assessed the probability of dying, getting the flu, and experiencing other health-related events. They were randomly assigned to a traditional open-ended response mode, a visual linear scale ranging from 0% to 100%, or a version of that visual linear scale on which a magnifier emerged after clicking on it. We found that, compared to the open-ended response mode, the visual linear and magnifier scale each reduced the use of 50%, 0%, and 100% responses, especially among respondents with low numeracy. Responses given with each response mode were valid, in terms of significant correlations with health behavior and outcomes. Where differences emerged, the visual scales seemed to have slightly better validity than the open-ended response mode. Both high-numerate and low-numerate respondents' evaluations of the surveys were highest for the visual linear scale. Our results have implications for subjective probability elicitation and survey design.

Two Violations of Expected Utility Theory - An Alternative Solution

Two well-known violations of Expected Utility Theory (EUT) are the Allais Paradox and the preference reversal between gamble choices and gamble bids. The Allais Paradox undermines the independence axiom, according to which the preferences for two gambles should be unaffected by an independent third gamble. There have been several proposed solutions that generally involve some form of non-linear probabilities (Machina 1987). The preference-reversal violation has been more problematic as it pre-supposes either non-transitive preferences or alternatively, some response-mode effect (or effects), whereby preferences and valuations are driven by distinct processes and outside influences (Machina 1987). This paper outlines a single solution to the two problems using a new utility framework that incorporates reference points for the agent’s income and needs. Rather than using non-linear probabilities or non-transitive preferences, the new utility framework models a response-mode effect, whereby agents recalibrate their needs in response to each prospect. This recalibration changes the convexity of the agent’s utility curve and, by extension, the agent’s tolerance for risk, which gives rise to the observed EUT violations.

Quantifying Under- and Overreporting in Surveys through a Dual-Questioning-Technique Design

In recent years, marketing researchers have become increasingly interested in under- and overreporting. However, there are few suitable approaches to operationalize deviations from the truth, particularly in behavioral domains in which self-reports are usually the only viable method of choice to measure behavior or attitudes. An especially difficult situation arises if some people underreport while others overreport. This article proposes a Bayesian item response theory model to quantify under- and overreporting in surveys. The method utilizes within-person differences between answers obtained under direct questioning (no privacy protection) and randomized-response questioning (which ensures item-level privacy protection). This method has the important features of incorporating behavioral response-mode effects (e.g., privacy loss when switching from direct to randomized-response questioning, response-mode inertia effects) and allowing the direction of bias to differ across respondents. The authors provide an empirical application for excessive alcohol consumption involving 1,408 respondents from a commercial web panel. The results show that respondents are averse to decreases in privacy and that randomized response is less effective if respondents provide biased responses to earlier direct questions.

Influence of Response Mode on Order Effects in the Interview

Two experiments involving students (N = 142 and N = 174, respectively) were conducted to investigate the effects of response mode and order on the evaluation of a hypothetical interviewee. Response mode (step-by-step or end-of-sequence) and order of positive and negative interview responses were manipulated in both studies. Competing hypotheses based on the belief adjustment model and predecisional distortion were tested. In both studies, the evaluation of the hypothetical candidate was more consistent with predictions of the belief adjustment model than with predecisional distortion. The results suggest that structured interviews are not immune to order effects. Furthermore, the belief adjustment model may be used to predict whether primacy or recency effects in interviews are likely to occur.

Why do alternative ways of responding improve children's performance on tests of strategic reasoning?

Changing the way in which children respond significantly improves performance across a variety of cognitive domains. However, the basis for this response-mode effect is unknown. To address this issue, the current study tested 78 preschool children on a measure of executive functioning (the windows task: Hala & Russell, ). The task requires children to point to an empty box to receive a box with a treat in it. There were four versions of the task, differing only in the way in which children made their responses. Children in the baseline condition, who pointed with their finger, performed poorly. However, children in three alternative response-mode conditions won the treat significantly more often. Strikingly, even children who pretended to be pointing with an arrow - but were in reality pointing with their finger - performed significantly better than baseline. We suggest that non-standard response modes encourage children to reflect on their actions and that this reduces the number of unreflective errors they make. The findings demonstrate that response-mode effects do not depend on the presence of a physical means of responding (such as an arrow), but can be achieved via a purely cognitive means (such as engaging in pretence).

A randomised trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors

BackgroundSurveys of doctors are an important data collection method in health services research. Ways to improve response rates, minimise survey response bias and item non-response, within a given budget, have not previously been addressed in the same study. The aim of this paper is to compare the effects and costs of three different modes of survey administration in a national survey of doctors.MethodsA stratified random sample of 4.9% (2,702/54,160) of doctors undertaking clinical practice was drawn from a national directory of all doctors in Australia. Stratification was by four doctor types: general practitioners, specialists, specialists-in-training, and hospital non-specialists, and by six rural/remote categories. A three-arm parallel trial design with equal randomisation across arms was used. Doctors were randomly allocated to: online questionnaire (902); simultaneous mixed mode (a paper questionnaire and login details sent together) (900); or, sequential mixed mode (online followed by a paper questionnaire with the reminder) (900). Analysis was by intention to treat, as within each primary mode, doctors could choose either paper or online. Primary outcome measures were response rate, survey response bias, item non-response, and cost.ResultsThe online mode had a response rate 12.95%, followed by the simultaneous mixed mode with 19.7%, and the sequential mixed mode with 20.7%. After adjusting for observed differences between the groups, the online mode had a 7 percentage point lower response rate compared to the simultaneous mixed mode, and a 7.7 percentage point lower response rate compared to sequential mixed mode. The difference in response rate between the sequential and simultaneous modes was not statistically significant. Both mixed modes showed evidence of response bias, whilst the characteristics of online respondents were similar to the population. However, the online mode had a higher rate of item non-response compared to both mixed modes. The total cost of the online survey was 38% lower than simultaneous mixed mode and 22% lower than sequential mixed mode. The cost of the sequential mixed mode was 14% lower than simultaneous mixed mode. Compared to the online mode, the sequential mixed mode was the most cost-effective, although exhibiting some evidence of response bias.ConclusionsDecisions on which survey mode to use depend on response rates, response bias, item non-response and costs. The sequential mixed mode appears to be the most cost-effective mode of survey administration for surveys of the population of doctors, if one is prepared to accept a degree of response bias. Online surveys are not yet suitable to be used exclusively for surveys of the doctor population.

Online shoppers' decision biases: The effect of response mode

The transaction under electronic commerce context is full of uncertainty where the information collected by buyers is limited (that is, they can only obtain the information such as product pictures but cannot touch the products). The way how the information was presented to the consumers will influence people’s decision. Framing effect and anchoring effect are both the human judgment decision biases induced by the presentation way of decision problems. In addition, how people respond to the decision problem is also one possible source in resulting in different decisions. There are three response modes discussed in the present study: choice, evaluation and bidding. This study is going to discuss the whether the influence of framing or anchoring messages on consumers’ decision depends upon the way they respond to the decision problem. The experimental results confirmed this phenomenon in which different response modes determined whether the framing and anchoring effect occurred. Specifically, when the consumers respond in choice mode, both attribute framing effect and anchoring effect were observed. In addition, significant attribute framing effect occurred when the subjects respond by using evaluation task. Finally, anchoring effect occurred in bidding mode where high and low anchoring messages resulted in significantly different bidding prices. Key words: E-commerce, framing effect, anchoring effect, experiment, response mode

Effects of Response Mode and Time Allotment on College Students' Writing

Written essays are a common feature of classroom and high stakes standardized tests at many age levels. However, little is known about how small alterations in the writing task affect students' writing, an issue made more important by the increasing use of task alterations for students with disabilities. In the present study, 140 college students completed a standardized assessment of writing ability under one of two timing conditions (10 minutes, 15 minutes) and with one of two response formats (handwritten, word-processed). Students who used a word processor wrote significantly more than did students who handwrote their essays. However, the extended time allotment only increased students' amount of writing in the word processor condition. Only small differences between the groups' essays were found in spelling, grammar, and related characteristics. Implications of these results for future research and writing assessment policies are discussed.

Text Messaging versus Talking on a Cell Phone: A Comparison of their Effects on Driving Performance

This study compared the effects of texting to other modes of responding on driving performance. While driving a simulator participants were instructed to categorize words appearing on billboards as a state (e.g., Maine), fruit (e.g., kiwi) or drink (e.g., Pepsi). The word categories were reported by texting, phoning in or identifying them aloud. There was significant effect of response mode on measures of driving performance. Drivers in the texting condition had significantly slower reaction times to peripheral letter targets, drove more slowly, exhibited greater variance in their lane position and took their eyes off of the road more often than in either the cell phone condition or the verbal condition. Drivers in the cell phone condition often performed more poorly than in the verbal response condition.

Text Messaging versus Talking on a Cell Phone: A Comparison of their Effects on Driving Performance

This study compared the effects of texting to other modes of responding on driving performance. While driving a simulator participants were instructed to categorize words appearing on billboards as a state (e.g., Maine), fruit (e.g., kiwi) or drink (e.g., Pepsi). The word categories were reported by texting, phoning in or identifying them aloud. There was significant effect of response mode on measures of driving performance. Drivers in the texting condition had significantly slower reaction times to peripheral letter targets, drove more slowly, exhibited greater variance in their lane position and took their eyes off of the road more often than in either the cell phone condition or the verbal condition. Drivers in the cell phone condition often performed more poorly than in the verbal response condition.

The instructions modulate the effect of response mode in the causal learning

El objetivo del presente trabajo fue demostrar el efecto de las instrucciones posentrenamiento en los modos de respuesta ensayo-ensayo y global, en una tarea de interferencia en aprendizaje causal. En esta situación, los modelos asociativos predicen efecto de recencia, en el cual se responde acorde con los últimos ensayos del entrenamiento, mientras que los modelos estadísticos predicen un efecto de integración de fases, es decir, se responde conforme a todos los ensayos de la tarea. Se presentan dos experimentos en donde se manipularon los modos de respuesta cada 5 ensayos y global, así como las instrucciones posentrenamiento. Los resultados demostraron que al emplear el modo de respuesta ensayo-ensayo se observó un efecto de recencia, mientras que al usar el modo de respuesta global se evidenció un efecto de integración de fases. Asimismo, las instrucciones posentrenamiento permiten modular los efectos obtenidos por el modo de respuesta, de manera que se puede observar integración de fases en el modo ensayo-ensayo y recencia en el modo global. Así, las instrucciones posentrenamiento y el modo de respuesta pueden ser variables que permiten observar la flexibilidad del aprendizaje causal.

Testing boundary conditions for the conjunction fallacy: Effects of response mode, conceptual focus, and problem type

Two experiments used within-subject designs to examine how conjunction errors depend on the use of (1) choice versus estimation tasks, (2) probability versus frequency language, and (3) conjunctions of two likely events versus conjunctions of likely and unlikely events. All problems included a three-option format verified to minimize misinterpretation of the base event. In both experiments, conjunction errors were reduced when likely events were conjoined. Conjunction errors were also reduced for estimations compared with choices, with this reduction greater for likely conjuncts, an interaction effect. Shifting conceptual focus from probabilities to frequencies did not affect conjunction error rates. Analyses of numerical estimates for a subset of the problems provided support for the use of three general models by participants for generating estimates. Strikingly, the order in which the two tasks were carried out did not affect the pattern of results, supporting the idea that the mode of responding strongly determines the mode of thinking about conjunctions and hence the occurrence of the conjunction fallacy. These findings were evaluated in terms of implications for rationality of human judgment and reasoning.

Disability assessments: Effects of response mode and experience

Purpose. To what extent response mode and experience affect the assessment of disability was investigated.Method. An experiment was conducted in which 34 medical doctors (17 inexperienced and 15 experienced) were required to assess disability of a videotaped client. Participants either gave a probability assessment after each piece of information or only after all information had been processed (step-by-step, SBS, or end-of-sequence, EOS). They were furthermore required to indicate how confident they were of their judgement and which information was most important for their judgement.Results. Neither response mode nor experience affected the assessment of disability. Only experienced doctors changed their judgement after seeing the video as compared to their judgement after reading the file. Even though all doctors became more confident after seeing the video, this increase was stronger for experienced than inexperienced doctors. Experienced doctors more often mentioned limitations as the basis for their judgements and, to a lesser degree, client's motivation to return to work than inexperienced doctors.Conclusions. The results suggest that assessments of disability are largely based on the initial representation that is formed after reading the file. The main pitfall is that the final representation is based on general beliefs rather than on actual client information. For training and support this would mean that doctors should be made aware of the extent to which their assessment is anchored in the case at hand.

Response mode and performance on the Rey 15-item test: A preliminary study

Objective: To investigate the effect of response mode and perceptual organization on performance on the Rey 15-item test.Method: The test was administered to 65 participants with a brain injury and 30 participants without a brain injury in a clinical setting.Results: The frequency with which participants appeared to perceive the test stimuli vertically instead of horizontally was 17% in the brain injury group and 10% in the general clinical group. In both groups, those who perceived the test stimuli vertically performed significantly worse than those that perceived the stimuli horizontally.Conclusion: This study identified a potential performance variable that may undermine the utility of the test. Strategies are suggested for increasing the rate of horizontal perception, which may improve test performance among clinical groups, thereby reducing the likelihood of genuine respondents obtaining a low score and being misclassified as malingerers.

On self-referencing differences in judgment and choice

Compared to judgment, choice is argued to elicit more self-referent processing and thereby produce greater subsequent recall of evaluated information. This response mode effect is shown to be dependent upon sufficient visualization to overcome the use of heuristic processing during choice. When visualizing prior to the task, choice leads to increased thinking about personal consumption occasions relative to judgment, leading to enhanced recall of vivid (vs. non-vivid) attributes. This proposed interaction of task and visualization was found in two experiments that assessed incidental recall following a choice or judgment task. In experiment 1, participants recalled more vivid product attribute information after choosing between options than after rating each option separately, but only when instructed to visualize during evaluation. To eliminate a comparison-based explanation of this effect, a second experiment was conducted that presented only one option in each category. Participants who evaluated their intention to purchase the option (a judgment equivalent of choice) demonstrated greater recall of vivid product attribute information than did participants who rated their liking for the option, and this recall difference was again moderated by instructions to visualize.

Comparing preference reversal for general lotteries and income distributions

This paper investigates preference reversals both for lotteries and income distributions as well as their interrelationship. In doing so, it offers several new features: Firstly, it provides a joint analysis of lotteries and income distributions. Secondly, the stimulus material used is considerably more general than used so far in the analysis of preference reversal. Instead of relying on binary lotteries and income distributions, we used ten multiple-outcome lotteries and ten n-dimensional income distributions which corresponded exactly to the lotteries. The lotteries and income distributions were chosen so as to model generalized P-bets (negatively skewed distributions) and generalized $-bets (positively skewed distributions), but we also used other shapes (uniform, symmetrical, and bimodal distributions). Thirdly, we applied material incentives to our subjects which were recruited from students of the Universities of Bari, Italy, and Castellón, Spain. Subjects were asked both to rate and evaluate the ten lotteries or the ten income distributions, respectively. We found heavy response-mode effects which confirm classical preference reversal, and revealed new patterns of preference reversal. Further, the transfer principle was largely violated.

The effect of response mode on lateralized lexical decision performance

We examined the effect of manipulations of response programming, i.e. post-lexical decision making requirements, on lateralized lexical decision. Although response hand manipulations tend to elicit weaker laterality effects than those involving visual field of presentation, the implementation of different lateralized response strategies remains relatively unexplored. Four different response conditions were compared in a between-subjects design: (1) unimanual, (2) bimanual, (3) congruent visual field/response hand, and (4) confounded response hand/target lexicality response. It was observed that hemispheric specialization and interaction effects during the lexical decision task remained unchanged despite the very different response requirements. However, a priori examination of each condition revealed that some manipulations yielded a reduced power to detect laterality effects. The consistent observation of left hemisphere specialization, and both left and right hemisphere lexicality priming effects (interhemispheric transfer), indicate that these effects are relatively robust and unaffected by late occurring processes in the lexical decision task. It appears that the lateralized response mode neither determines nor reflects the laterality of decision processes. In contrast, the target visual half-field is critical for determining the deciding hemisphere and is a sensitive index of hemispheric specialization, as well as of directional interhemispheric transfer.

Preference reversal in decision making: The attraction effect in choice and rejection 1The research was funded by a 60% grant of the University of Padua. The authors wish to thank Sandro Bettella for his invaluable technical assistance.

When people express a preference between two alternatives A and B in terms of a positive choice of one option, this can exceed in strength the same preference expressed as a rejection of the alternative. This effect of response mode has been interpreted by Shafir (1993) in terms of response compatibility theory, according to which decision makers display an influence of the compatibility between the type of response (choose/reject) and the positive/negative attributes of the options. In the present study we investigated the influence on response mode (choice/rejection) of the attraction effect, in which a decoy similar to one of two options, but lower in value, modifies the share of the option to which it is similar when added to the original set ( Huber, Payne & Puto, 1982 ; Simonson & Tversky, 1992 ). A decoy negative in value, but similar to one of the alternatives was added to a two-option set, one with a high variation in attributes (enriched) and one with a low variation (impoverished). We investigated whether both choice and rejections were influenced by the decoy’s presence, as compared to the baseline two-option condition, and whether, consequently, the pattern of dominance between choice and rejection was modified. We found a pattern of rejection dominance in the two-option condition (Experiment 1). In Experiment 2, where the attraction effect was investigated, there was an increase in the proportion of choices (and a correspondent decrease in rejections) of the similar alternative, as compared to the original two-option set, only for the option with low variation in the attributes. For the enriched option, rejection, but not choice, was influenced by the presence of the decoy.