Web Survey Design Research Articles

Using memes and emoji-scales in a web survey: experimental assessment of consequences for multimodal cognitive effort and data quality

ABSTRACT Decreasing web survey participation rates lead to higher importance of ways to increase respondents’ motivation to participate and gamification is seen as one of possible ways to do so. However, there is no clear understanding on how it affects respondents’ cognitive load. This study investigates whether gamification reduces cognitive load among 18–25-year-old survey respondents. A laboratory experiment was conducted with 128 randomly assigned students to either a non-gamified or gamified web survey design. Gamification was implemented using emojis for ordinal scales and motivating memes between sets of questions. Cognitive load was measured through paradata, subjective evaluation, and neurophysiological indicator (pupil diameter dynamics). Results show higher cognitive effort for the gamification group based on pupil diameter dynamics, but subjective evaluation, mouse movements, and completion times show increased cognitive effort for the non-gamified group. Attention was higher for the gamification group, and data quality did not differ. Therefore, gamification may increase cognitive load, but it also increases motivation and engagement, leading to similar data quality compared to non-gamified web surveys.

Examining final comment questions with requests for written and oral answers

Self-administered web surveys provide respondents only limited opportunities for feedback. Therefore, many web surveys include so-called “final comment questions” (FCQs) that allow respondents to elaborate on the survey in general and the survey questions in particular. Typing in open answers in text boxes is burdensome – especially via smartphones with virtual on-screen keypads – so that respondents frequently provide short or no answers at all. In this study, we make use of new technological advancements in web survey methodology and investigate FCQs with requests for written and oral answers. For this purpose, we conducted an experiment in a smartphone survey ( N = 1,001) in a German online panel. The results reveal that FCQs with a request for written and oral answers do not differ with respect to item-nonresponse. However, oral answers are substantially longer than written answers pointing to more in-depth information. The oral answer condition also results in more positive comments than the written answer condition. This study is a methodological showcase for innovative web survey design that contributes to the improvement of data quality.

Tourism and Impacts of COVID-19:

The search for natural environments is a valued alternative post-lockdown caused by Covid-19. This trend increases interest in the role of Conservation Units - CUs and the perspective of (re) connecting with nature. Thus, this study aims to understand the effects of the pandemic from the perspective of tourists who visit CUs in the state of Paraná. The research has an exploratory nature, adopting a mixed approach with a web survey and documentary research design. The results show that CUs underwent modifications with the outbreak of the pandemic. The use of technologies played a crucial role in keeping tourists informed, allowing them to explore CUs digitally or make reservations. The closure of CUs at various points during the pandemic was a significant hindrance to visitations, along with the fear of leaving home. Concerns such as changes in occupation/work, income, and difficulties in other routine activities also limited travel. However, the respondents understand that at the end of their visitation to the CUs, they tend to recover their well-being. In conclusion, natural spaces are deemed important for the society's mental health, offering the possibility of (re) connecting individuals with nature, highlighting the significance of CUs for mental well-being.

Lifestyle behaviors and stress are risk factors for overweight and obesity in healthcare workers: a cross-sectional survey

BackgroundOverweight and obesity have become major public health concerns worldwide. Persistent stress can activate the human hypothalamic‒pituitary‒adrenal axis (HPA) and increase the intake of “self-rewarding food”, thereby raising the incidence of obesity. Health care workers (HCWs) experience higher workloads and mental stress than workers in many other industries, which may put them at increased risk for overweight/obesity. However, few studies have been carried out on overweight and obesity among HCWs in China, and the overall scenario and behind-the-scenes factors of their overweight and obesity are unknown. The aim of this study is to understand the epidemic of overweight and obesity and risk factors among Chinese HCWs.MethodsBased on a cross-sectional web survey design, 23,234 HCWs from 100 health institutions in 5 provinces/autonomous regions/municipalities across China were sampled to answer a self-administered questionnaire that was purposely developed using a multi-staged clustered random-sampling method. Chi-square test and ANOVA were performed to compare variables between two or more groups. Univariate analyses were conducted to identify the influence of self-reported persistent stress and/or recurrent anxiety/depressed mood on lifestyle behaviors. A multivariate binary logistic regression model was used to analyse the risk factors of overweight/obesity.ResultsAmong the respondents, 34.26% were overweight, and 11.22% were obese. Most of the respondents had regular exercise habits (68.17%), had habitually stayed-up late (65.06%) and had been affected by persistent stress and/or recurrent anxiety/depressed mood (62.04%). A higher proportion of those with persistent stress and/or recurrent anxiety/depressed mood than those without habitually staying-up late (76.18%); consumed take-out food (54.92%), fried food (49.93%), snacks or desserts (50.51%); drank sugary drinks (46.57%); smoked (14.27%); and drank alcohol (23.34%). Gender (Female) (OR: 0.314, 95%CI: 0.292–0.336), age (OR: 1.742–2.334, 95%CI: 1.544–2.858), education (OR: 0.620–0.728, 95%CI: 0.445–0.973), living and working area (OR: 1.271, 95%CI: 1.192–1.355), breakfast (OR: 0.898, 95%CI: 0.839–0.960), fried food (OR: 1.133, 95%CI: 1.048–1.224), and alcohol consumption (OR: 1.111, 95%CI: 1.017–1.214) were factors for overweight/obesity. All of the aforementioned results were significant (P < 0.05).ConclusionsThe overweight/obesity rate of Chinese HCWs is rather high, which might be directly associated with lifestyle behaviors. However, these behaviors fundamentally originated from persistent stress and/or recurrent anxiety/depression, mediated by lifestyle behaviors. Substantial measures should be taken for stress reduction and mental health promotion for overweight/obesity prevention and control among HCWs.

A new two-dimensional question format in web survey design: Assimilation and contrast effects

Computer- and web-based survey technology has enabled the use of question formats and layouts that would be difficult or even impossible to implement with traditional paper questionnaires. The present article investigates context effects—specifically assimilation versus contrast effects—in connection with four web survey question designs. Three traditional types of matrix questions are experimentally tested against a new two-dimensional question format referred to as ZEF. This new question format allows survey practitioners to present two questions at the same time, the responses to which are entered on the x- and y-axes of a two-dimensional chart. The chart also displays a respondent’s answers to the preceding two-dimensional questions. This approach accentuates the simultaneous application of two dimensions of a question and encourages the comparison of pairwise items across the dimensions. The two-dimensional format generates a higher number of assimilation effects as compared with the three traditional matrix question formats. This suggests that instead of conceptual interconnection of items, context effect formation in ZEF is dominated by the visual proximity of the item scales and the common region of the two-dimensional chart.

Hows and Whys That Lead to Online Brand Engagement

This article contributes to digital branding and customer involvement via social media. To improve online brand engagement, this study examined brand awareness, social linkage, and online trust value. Effective web survey design acquired 317 empirical online sample answers for the paper. The current study uses structural equation modeling to evaluate and verify the postulated model. In this competitive internet era, social networking-supported marketing may raise brand engagement to increase online brand trust, positive brand attitude, and deeper customer emotional connection and brand likability. This study proposes a unique paradigm to improve online brand engagement by investigating brand awareness, social, and online trust value correlations.

Implementation of Flexible Work Arrangement among Healthcare Workers in Miri Hospital-Assessment of the Validity and Reliability of Flexible Work Arrangement Perceived Benefits and Barriers Scale, and the Exploratory Study.

Flexible work arrangements (FWAs) have been widely implemented during the COVID-19 pandemic. We aimed to assess the validity and reliability of the FWA perceived benefits and barriers (FWAPB) scale and subsequently, to determine the preference and perceived feasibility, perceived benefits and barriers, and readiness to implement FWA among healthcare workers. We conducted a cross-sectional study using a self-administered questionnaire in Miri Hospital. The questionnaire was administered via a web survey design (Google Forms). The convenience sampling method was applied to recruit respondents. All healthcare workers in Miri Hospital who could read and understand English were invited to participate in the study. Response process validation, exploratory factor analysis, reliability analyses and descriptive statistics were performed. A total of 339 respondents participated. All items had satisfactory response process indices. Exploratory factor analysis revealed a three-factor structure. Items of 'perceived benefits-workplace management', 'perceived benefits-family life balance' and 'perceived barriers' have high internal consistency reliability (Cronbach's alpha = 0.852-0.884) and factor loadings. Flextime is preferred and perceived to be the most feasible work arrangement. Most agreed that FWA helps in improving social distancing among colleagues (mean = 3.65, standard deviation [SD] = 0.99) and reduces their exposure to COVID-19 (mean = 3.60, SD = 1.06). A total of 44.0% of the respondents agreed Miri Hospital is ready to implement FWA. The FWAPB is valid and reliable. Almost half of the respondents were positive towards the implementation of FWA. These findings contribute to the understanding of FWA, and thus increase the readiness and acceptance of such an arrangement.

Social media, political efficacy and political participation in Nigeria

The role of social media in influencing political thinking, perception, and participation is becoming increasingly important, as evident in the volume of research the topic has attracted. This article presents findings on social media, political efficacy, and political participation in Nigeria. The objective is to evaluate the relationship between social media and political efficacy; social media and political participation; and political efficacy and political participation in Nigeria. To this end, the study adopted a web survey design for sample participants across the country, using links distributed through Facebook and Twitter. Snowball sampling was adopted to determine the total number of respondents (N = 3,407). The results of the descriptive statistics and Pearson Product-Moment Correlation Coefficient yielded evidence that the majority of social media users in Nigeria are male, young, educated and largely unemployed . It concludes that social media influences political efficacy and political participation in Nigeria.

Typographic Cueing Facilitates Survey Completion on Smartphones in Older Adults

As more people use mobile devices to complete Web surveys, it is critical to verify that current Web survey design practices carry over to mobile Web. One such practice is to bold the question to draw attention to the start of each question and to italicize instructions to facilitate filtering of potentially unneeded information. We conducted a laboratory experiment to investigate whether the common Web survey design practice of utilizing typographic cues to indicate different categories of question text is beneficial for surveys designed for smartphones. Thirty participants completed a 5-question survey on an iPhone. Each participant was randomly assigned to one of the four designs: (1) bolded questions and italicized instructions, (2) bolded questions and plain instructions, (3) plain questions and italicized instructions, (4) plain questions and plain instructions. Survey performance was measured with survey completion time, difficulty rating, and preference. We found that italicizing the instructions led to shorter completion times, while bolded questions had no effect on performance. All designs were rated easy to use. The designs with bolded questions were preferred by 83% of the participants. We argue that italicized instructions assist in visual filtering by providing a cue to respondents that they may not need to read the instructions in detail but may scan or skim the instructions instead. Scanning or skimming leads to shorter completion time without detriment to accuracy because respondents can still acquire the relevant instructions.

Proximity, contact and tolerance of homosexuals in Korea: evidence from an experimental survey

ABSTRACT Compared to most developed democracies, South Korea lags behind in terms of LGBT tolerance. This analysis suggests that relational proximity and knowing a Korean LGBT person influence this tolerance. Through an experimental web survey design, results show that the closer the relational proximity (coworker versus friend versus family member), the less one stated their comfort with being around a homosexual. The same pattern endures among those who knew a South Korean LGBT person, although rates of acceptance were much higher. In addition, this analysis finds of South Korea’s three main religions, only Protestant identification corresponded with a statistically significant decline in tolerance.

The Examination of non-pharmacologic treatment of pain after spinal cord injury

Background: One of the primary debilitating secondary health conditions experienced by individuals living with a spinal cord injury (SCI) is chronic pain. Approximately, over 30% of SCI patients endure chronic pain after sustaining the injury, and engage in opioid pharmacotherapy as the first form of treatment. The increase in use and misuse of prescribed opioids for chronic pain can lead to both physical and psychological health risks. This danger is exacerbated by the notion that a large percentage of the SCI population have a pre-existing condition of drug and alcohol abuse and addiction. This study will examine how pain is actually being addressed through non-pharmaceutical methods among the SCI adult population. &#x0D; Experimental Design: Employing a cross-sectional web survey design with a convenience sample of adults with a SCI. A Qualtrics survey is currently being administered to known individuals with SCIs via email. The survey questions will illuminate what pain management strategies are being implemented in the SCI population within the US. &#x0D; Anticipated Results: The results from the data will provide insight on what non-pharmacological interventions can be employed in place of or in combination with the pharmacological management. This will allow community-based rehabilitation therapies to incorporate appropriate pain management strategies. &#x0D; Potential Impact: Non-pharmacological interventions may hold the answer in treating chronic pain in the SCI population by limiting or eliminating the use of opioids. This study will help develop the most appropriate non-pharmacological intervention to test in quasi-experimental clinical trials, and ultimately minimize the use of opioids for chronic pain.

Problematic Gaming and Internet Use but Not Gambling May Be Overrepresented in Sexual Minorities - A Pilot Population Web Survey Study.

Background: Substance-related addictive disorders are known to be overrepresented in non-heterosexual individuals, but it is largely unknown whether this is also the case for behavioral addictions such as problem gaming and gambling. This study aimed, in a pilot web survey design, to assess whether problematic gambling, gaming and internet use may be more common in individuals with a non-heterosexual orientation.Methods: An online survey was distributed through media and social media, and answered by 605 individuals (51% women and 11% non-heterosexual). Problem gambling, problem gaming and problematic internet use were measured through structured screening instruments (the CLiP, the GAS and the PRIUSS, respectively).Results: Problem gaming and problematic internet use were significantly more prevalent in non-heterosexual subjects. Instead, problem gambling did not differ between heterosexual and non-heterosexual respondents. Psychological distress and social media use for more than 3 h daily were significantly more common in non-heterosexual respondents. In the overall sample, gaming and gambling were associated statistically.Conclusion: Based on the present pilot online survey, problematic gaming and internet use, but not problem gambling, may be more common in non-heterosexual populations. This area merits more and larger studies, and potentially preventive efforts aimed for non-heterosexual individuals in the population. Possible explanations and study limitations are discussed in the paper.

Changing identity through sport: The Paralympic sport club experience among adolescents with mobility impairments

BackgroundAdolescents with mobility impairments have fewer opportunities to challenge identity as an athlete through sport participation. Objective/HypothesisThis study examined athletic identity impact of adolescent athletes with mobility impairments who participate in Paralympic Sport Clubs. MethodsThis cross-sectional web-survey design examined demographics (gender, academic level, hours of time spent in sports participation, and preferred amount of time in sport participation) and athletic identity through the Athletic Identity Measurement Scale (AIMS). Forty-seven adolescents (male = 34, female = 13) between the ages of 13–18 with mobility impairments who were currently participating in a Paralympic Sport Club were recruited for the study. ResultsTwo factorial ANOVA examined the differences of AIMS scores by gender and academic level, and gender and hours of time spent in sports participation per week. Hours of time spent in sports participation by adolescent athletes with physical disabilities showed significant difference on athletes' identity. AIMS score increased along with augmented time spent in sport participation. However, gender and academic level made no significant difference on athletic identity. A one-way ANOVA test was applied to compare the effect of preferred amount of time in sport participation per week on scores for AIMS and showed significant group difference on AIMS. Participants who indicated more preferred amount of time in sport participation also scored higher on AIMS. ConclusionsThese findings support the notion that hours of time spent and preferred amount of time in sports participation influences athletic identity. Sport participation, when available can influence athletic identity with this population.

Device and Internet Use Among Spanish-Dominant Hispanics: Implications for Web Survey Design and Testing

This paper focuses on the lessons learned for design and administration of mobile Web surveys for the Spanish-dominant Hispanic population in the United States. The scholarship on completion of Web surveys has recently started to examine Web usage on mobile devices and Internet browsing patterns to better understand the behavior of mobile users. Yet, we still know little about how Spanish-dominant Hispanics in the United States use their electronic devices to answer mobile Web surveys. In this piece, we use findings from usability and cognitive interviews conducted in 2015. The interviews were part of U.S. Census Bureau tests that will inform design decisions for the 2020 Census. For this project, we recruited and interviewed Hispanic or Latino participants who reported not speaking English well, owned smartphones or tablets, reported accessing the Internet with those devices, and reported feeling somewhat or very comfortable using the Internet. We found that access to an electronic device such as those used for completing Web surveys (i.e., computers, smartphones, tablets) did not necessarily indicate full familiarity with the devices, with Internet activities (e.g., browsing, use of Internet applications) or with responding to mobile Web surveys. Moreover, we found evidence that most participants used a very limited number of Internet applications (e.g., only Facebook or Google). Future research should focus on these issues and their implications for the design of Web surveys. Advancing research on these issues is important for mobile-dependent populations who are still not familiar with answering Web surveys in general.

Profiling perpetrators of interpersonal violence against children in sport based on a victim survey

The current article reports on perpetrator characteristics gathered in the first large-scale prevalence study on interpersonal violence against children in sport in the Netherlands and Belgium. Using retrospective web survey design, 4043 adults answered questions on their experiences in youth sport. The study looks at the number of perpetrators as well as individual descriptive characteristics (sex, age, and role in the sport organization) of perpetrators of psychological, physical and sexual violence as reported retrospectively by victim-respondents. This information was then clustered to provide an overview of the most common perpetrator profiles. Results show that in all types of interpersonal violence in sport, perpetrators are predominantly male peer athletes who frequently operate together in (impromptu) groups. Several differences between the three types of interpersonal violence are highlighted. While incidents of physical violence perpetrated by coaches tend to be less severe compared to those by other perpetrators, acts of sexual violence committed by a coach are significantly more severe. The presented findings shed new light on perpetrators of interpersonal violence in sport, nuancing the predominant belief that the male coach is the main perpetrator while providing nuanced information that can be utilized to improve prevention and child protection measures and other safeguarding initiatives in sport.

Design Characteristics of Sensory Gardens in Norwegian Nursing Homes: A Cross-Sectional E-Mail Survey

ABSTRACTSensory gardens are increasingly popular in Norwegian nursing homes. This study examined the design characteristics of Norwegian sensory gardens using a cross-sectional web survey design addressing nursing home leaders (n = 121). The findings revealed that Norwegian sensory gardens were generally compliant with recommended design guidelines involving sensory experiences, stable walkways, landmarks, accessibility, visibility from indoors, seating features, and prosthetic supports. There is, however, a potential for further improvements concerning walkways with rails, access to alarm systems, clear signposts to toilets from the outdoors, lighting in the evenings, greenhouses, increased awareness of edible and nontoxic plants, and maintenance of the gardens.

Not to Be Considered Harmful

The number of respondents who access web surveys on a mobile device (smartphone or tablet) has been increasing rapidly over the last few years. Compared with desktop computers, mobile devices have smaller screens, different input options, and are used in a larger variety of locations and situations. The suspicion that the quality of data may suffer when online respondents use mobile devices has stimulated a growing body of research, which has mainly focused on paradata and web survey design. To investigate whether the respondents’ device affects the quality of web survey data, we examined the responses of 1,826 mobile-device and desktop participants in a political online survey that asked questions about the 2013 German federal election. To determine the reliability and validity of data submitted via mobile devices, we determined the consistency of the participants’ responses across questions and validated the responses against various internal and external criteria. Replicating previous findings, mobile-device respondents were younger and more likely to be female, and they produced higher dropout rates and longer completion times than desktop respondents. However, data produced by respondents using mobile devices were as consistent, reliable, and valid as data produced by respondents using desktop computers. These findings contradict the notion that mobile-device users compromise the reliability and validity of data collected online and suggest that researchers do not necessarily need to be afraid of the participation of mobile-device respondents in web surveys.

Beauty is more than screen deep: Improving the web survey respondent experience through socially-present and aesthetically-pleasing user interfaces

Web surveys are rapidly becoming standard issue in many researchers’ toolkits; however, measurement error has been shown to affect web surveys to a greater extent than paper-and-pencil surveys (Couper, 2000; Manfreda & Vehovar, 2002). Principles of aesthetic design and social presence have been applied to web surveys to reduce the prevalence of such error with promising results, which were further investigated in this research. A sample of 181 first-year psychology undergraduate students participated in this study. Participants were randomly allocated to view one of eight web survey interfaces, which varied by aesthetic quality and social presence. Exploratory structural equation modeling using the partial least squares method revealed that classical aesthetic quality and social presence were both positively related to perceived ease of use of the web survey interface and positive state affect; social presence and perceived ease of use were positively related to trust in the web survey researcher; classical aesthetic quality was negatively related to negative state affect; and, expressive aesthetic quality was negatively related to perceived ease of use and positively related to positive state affect. Interestingly, expressive aesthetic quality was also positively related to negative state affect. These relationships between aesthetic quality and social presence should inform best practice web survey design recommendations, and future empirical work should extend and test the generalizability of these findings.

Web-based surveys in educational research

Selcuk University School of Foreign Language, Konya, (TURKEY) E-mails: arifbakla@yahoo.com, ahmetcekic42@hotmail.com, onurkoksalmeb@hotmail.com DOI: 10.7813/2075-4124.2013/5-1/B.1 ABSTRACT Survey research is a challenging and time-consuming process, but current technology offers educational researchers ample opportunities to design and administer time-efficient surveys. With the help of ubiquitous web-based technologies, it is now possible to design and administer a full-length survey on an online platform at no cost at all. The use of such open-source software or free web services makes it possible to design web surveys which are virtually impossible for the non-expert to design and publish online through coding oriented web design software. This study aims to get to the basics of efficiently designed and scientifically validated surveys. It basically discusses various significant points related to survey methodology such as web-based survey design, sampling, data analysis, validation and ethical issues. Key words: Survey design/administration, web-based surveys, survey sampling and validation 1. INTRODUCTION Web-based surveys have traditionally been used in market research, but recently they are also seen in social and educational research. However, most educational researchers seem to have fallen behind technological developments in web survey design and administration because they have little or no information about how to construct and administer scientifically valid and reliable surveys by using web technologies. Still worse than that, they have no awareness of such opportunities. A brief overview of electronic survey methodology might help these researchers consider designing and administering electronic surveys to carry out educational research. Web-based surveys are highly effective if they are designed and administered in scientifically proper ways. Though it might be challenging for those who lack computer literacy to construct web-based surveys, collecting data from participants in different geographical locations is relatively easy when compared with traditional surveys and transferring the collected data into data analysis software is highly time efficient and error free. Though there are significant differences between web-based surveys and their traditional counterparts in terms of such issues as delivery, cost, integration of multimedia elements, etc., both traditional and web-based surveys make it necessary to pay attention to item construction, sampling, validation and administration processes. This paper aiming to explore some basic aspects of web-based surveys is organized into several sections beginning with the basis of survey methodology and electronic surveys. Building upon the background information, the researchers discuss some critical issues such as sampling, representativeness, validation and ethical considerations. 2. BACKGROUND TO THE STUDY Although some censuses were carried out in the ancient world by authorities to explore some aspects of economic and military life, more scientific-like attempts were made in the seventeenth and nineteenth centuries by Graunt (1666) and Laplace (1812). Basically calculating average numbers, English merchant John Graunt tried to count the number of people in London based on the data he gained from a specific area. Then, he generalized his findings to the whole population. The problem with his approach was that taking averages may not give the same results across time and space. A similar attempt made by Laplace (1812) was to find out the population of France. He used central limit theorem, but he made a sampling error since he failed to use random sampling; he instead used cluster sampling (cited in Bethelem, 2009, p. 6). At the end of the nineteenth century, a statistician named Anders Kiaer introduced his Representative Method, and later in 1906 Bowley put forward random sampling method, which enabled him to reach normal distribution through randomly selected samples (Bethelem, 2009, pp. 7-8). His idea formed the basis of scientific sampling methods in survey research. A survey is a solution to many of the problems of a census. In other words, it is a shortcut to the data gathered through a census which involves all members of a given population. Unlike censuses, surveys collect information about only a small part of the population to eliminate problems related to high cost and time (Bethelem, 2009). Bethelem also has the following to say about the seemingly magic side of survey research:

Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality

Due to recent scholarly attention to visual design, much is known about the effects that specific design features have on web surveys, but little attention has been paid to the effects of overall screen design. Applying Norman's (2004) emotional design concepts to web survey design, we propose and test hypotheses related to potential detrimental impacts of poor screen aesthetics. To do so, we compare data collected from two versions of the same Student Experience Survey, an experimental design representing poor aesthetics and a control design representing good emotional design. By triggering negative visceral responses and, thus, emotional reactions, the experimental design is predicted to reduce data quality along four avenues: respondent cooperation, substantive response patterns, respondent commitment, and subjective survey experience. Sixteen of 30 total comparisons show reduced data quality on the experimental forms, and all avenues to quality data show some impact. These findings suggest that screen design aesthetics and emotional reactions can impact data quality, independent of survey content. However, the inconsistent results according to some indicators suggest the need for further research on screen design effects to hone our understanding.