Survey Design Strategies Research Articles

Can We Get Rid of Bias? Mitigating Systematic Error in Data Donation Studies through Survey Design Strategies

Can We Get Rid of Bias? Mitigating Systematic Error in Data Donation Studies through Survey Design Strategies

Does Providing an Example Social Network Map Increase Referrals or Affect Types of Ties Reported in an Online Survey?

Social network research often depends on the willingness of respondents to provide personal information about themselves and alters. Survey design strategies that increase willingness to share this information are necessary for social network research to be feasible, especially when name generators are used for sampling because rosters are unavailable. We conducted an experiment in which one group of respondents ( n = 94) received an online survey that included an example network map and the other group ( n = 100) received one that did not. Results show the map did not increase or decrease provision of network contacts nor influence the types of ties reported. Furthermore, respondents were reluctant to provide names and contact information of alters. Our study demonstrates the difficulty of collecting network information without a previously defined roster and an attempt to improve data collection through strategic survey design.

Kelp Patch-Specific Characteristics Limit Detection Capability of Rapid Survey Method for Determining Canopy Biomass Using an Unmanned Aerial Vehicle

Kelp forests dominate autotrophic biomass and primary productivity of approximately 30,000 to 60,000 km of shallow temperate and Arctic rocky reef coastline globally and contribute significantly to carbon cycling in the coastal ocean. Rapid biomass turnover is driven by very high growth rates and seasonal environmental drivers. As a result, kelp biomass varies greatly with time, space, and by species. In the northeast Pacific region, bull kelp (Nereocystis leutkeana) and giant kelp (Macrocystis pyrifera) form extensive floating surface canopies with a distinct spectral signature compared to the surrounding water. Studies have shown that remote sensing is advantageous for deriving large-scale estimates of floating surface canopy biomass, which comprises more than 90% of bull and giant kelp standing stock. However, development and validation of remotely derived kelp canopy biomass is lacking because existing approaches are time intensive and costly. This study attempted to close that gap by developing a rapid survey design utilizing diver and unmanned aerial vehicle (UAV) imagery across six sites in northern and central California. Kelp sporophytes were collected and measured for morphometric characteristics and genera-specific allometry to canopy biomass. Kelp density was measured using in situ diver surveys and coupled with UAV imagery to quantify kelp canopy biomass at a range of ground sampling distances. We successfully estimated kelp canopy biomass from UAV imagery at 33% (2/6) of the survey sites, but consistently determining canopy biomass via this approach was challenged by both survey design and kelp patch-specific spatial characteristics. The morphologies of bull kelp in Monterey were significantly different than other regions measured, but further work should be conducted to fully characterize differences in canopy biomass at the regional and sub-regional scale. We use this opportunity to suggest survey design strategies that will increase the success of future methodological development of UAV biomass retrieval. We also recommend developing long-term, annual genera-specific monitoring programs across the northeast Pacific region and beyond to validate remote sensing derived biomass estimates beyond the small number of existing well-characterized sites.

Evaluating species-specific responses to camera-trap survey designs

Camera traps are widely used to collect information on the distribution and abundance of multiple species simultaneously. However, we still lack important guidance for designing camera-trap surveys to monitor multiple species, and the consequences of species-specific responses to survey design strategies are often overlooked. Using camera-trap data collected on ten medium-to-large North-American carnivores in northern Minnesota, USA, between 2016 and 2018 (23 337 active trap-days), we evaluated: 1) two different survey-design frameworks (random- versus road-based), 2) two different lure types (salmon oil versus fatty acid scent oil), 3) two different placement strategies (completely random versus randomly-selected sites with feature-based placement), 4) survey timing (spring versus fall) and 5) temporal trends in daily encounter probabilities. Using generalized linear mixed models, we found evidence of differential responses to all of these design strategies. For 9 out of 10 species, we found strong responses to survey design frameworks: red foxes Vulpes vulpes, coyotes Canis latrans, bobcats, Lynx rufus, striped skunks Mephitis mephitis, wolves C. lupus and gray foxes Urocyon cinereoargenteus, had estimated encounter frequencies that were 9- to 106-fold higher at unlured sites along secondary roads; black bears Ursus americanus, martens Martes americana and fishers Pekania pennanti had estimated encounter frequencies that were 15- to > 3600-fold higher at lured, randomly selected sites. For six species, salmon oil provided 2- to 4-fold more encounters than fatty acid scent oil, but feature-basedplacement only improved detections of fishers. Daily encounter probabilities differed between spring and fall for all species, and usually decreased slightly within each sampling period Our study confirms that even similar-sized or closely-related species respond differently to survey-design choices. To maximize encounter frequencies, we recommend that multi-species camera-trap studies use a mix of survey-design strategies and include these design features during statistical analysis.

Automated survey design for blended acquisition with irregular spatial sampling via the integration of a metaheuristic and deep learning

Blended acquisition along with efficient detector and source geometries allows for a cost-effective operation. The outcome of subsequent deblending and data reconstruction is of primary importance in determining the technical success of this manner of data acquisition. Despite its advantages over conventional seismic surveys, finding optimum survey parameters is a difficult task. Although incorporating irregularity into spatial sampling and blending schemes leads to effective deblending and data reconstruction, it inherently provides a significantly large problem space. We have developed a survey-design workflow to provide acquisition parameters that account for the source blending as well as the spatial sampling of detectors and sources in an automated manner. Our method involves an iterative scheme to derive the survey parameters that lead to optimum deblending and data reconstruction quality. The approach deals jointly with deblending and data-reconstruction via a sparse inversion in the frequency-wavenumber domain coupled with constraints based on causality and coherency. The residue from this process is subsequently used to update the survey parameters by integrating a genetic algorithm and a convolutional neural network (CNN). Bioinspired operators enable the simultaneous update of the blending and sampling operators. To relate the choice of survey parameters to the performance of deblending and data reconstruction, we have used a CNN. The applied network architecture successfully rejects suboptimal solutions among newly generated ones from genetic operators. Consequently, only optimal ones are fed into the subsequent step, making our approach computationally affordable. We apply our workflow to design a seismic survey that incorporates the dispersed source array concept. A comparison among different survey-design strategies highlights the ability of the method to effectively derive optimum solutions. The resultant acquisition scenario derived from our approach yields a notable enhancement of deblending and data reconstruction quality attributed solely to the choice of survey parameters.

How Much Time Do You Spend Online? Understanding and Improving the Accuracy of Self-Reported Measures of Internet Use

ABSTRACTGiven the importance of survey measures of online media use for communication research, it is crucial to assess and improve their quality, in particular because the increasingly fragmented and ubiquitous usage of internet complicates the accuracy of self-reported measures. This study contributes to the discussion regarding the accuracy of self-reported internet use by presenting relevant factors potentially affecting biases of self-reports and testing survey design strategies to improve accuracy. Combining automatic tracking data and survey data from the same participants (N = 690) confirmed low levels of accuracy and tendencies of over-reporting. The analysis revealed biases due to a range of factors associated with the intensity of (actual) internet usage, propensity to multitask, day of reference, and the usage of mobile devices. An anchoring technique could not be proved to reduce inaccuracies of reporting behavior. Several recommendations for research practice follow from these findings.

Choice of rating scale labels: implication for minimizing patient satisfaction response ceiling effect in telemedicine surveys.

Lack of response variability is problematic in surveys because of its detrimental effects on sensitivity and consequently reliability of the responses. In satisfaction surveys, this problem is caused by the ceiling effect resulting from high satisfaction ratings. A potential solution strategy is to manipulate the labels of the rating scale to create greater discrimination of responses on the high end of the response continuum. This study examined the effects of a positive-centered scale on the distribution and reliability of telemedicine satisfaction responses in a highly positive respondent population. In total, 216 telemedicine participants were randomly assigned to one of three experimental conditions as defined by the form of Likert scale: (1) 5-point Balanced Equal-Interval, (2) 5-point Positive-Packed, and (3) 5-point Positive-Centered Equal-Interval. Although the study findings were not statistically significant, partially because of sample size, the distribution and internal consistency reliability of responses occurred in the direction hypothesized. Loading the rating scale with more positive labels appears to be a useful strategy for reducing the ceiling effect and increases the discrimination ability of survey responses. The current research provides a survey design strategy to minimize ceiling effects. Although the findings provide some evidence suggesting the benefit of using rating scales loaded with positive labels, more research is needed to confirm this, as well as extend it to examine other types of rating scales and the interaction between rating scale formats and respondent characteristics.

Lunar ground penetrating radar: Minimizing potential data artifacts caused by signal interaction with a rover body

Ground-penetrating radar (GPR) is the leading geophysical candidate technology for future lunar missions aimed at mapping shallow stratigraphy (<5m). The instrument’s exploration depth and resolution capabilities in lunar materials, as well as its small size and lightweight components, make it a very attractive option from both a scientific and engineering perspective. However, the interaction between a GPR signal and the rover body is poorly understood and must be investigated prior to a space mission. In doing so, engineering and survey design strategies should be developed to enhance GPR performance in the context of the scientific question being asked. This paper explores the effects of a rover (simulated with a vertical metal plate) on GPR results for a range of heights above the surface and antenna configurations at two sites: (i) a standard GPR testing site with targets of known position, size, and material properties, and; (ii) a frozen lake for surface reflectivity experiments. Our results demonstrate that the GPR antenna configuration is a key variable dictating instrument design, with the XX polarization considered optimal for minimizing data artifact generation. These findings could thus be used to help guide design requirements for an eventual flight instrument.

Cognitive Probes in Web Surveys

In this study, we explore to what extent the visual presentation of open-ended probes, in connection with different prior probing exposure, impacts on response quality. We experiment with two text box sizes for a specific immigrant probe ( Which type of immigrants were you thinking of when you answered the question?). On the one hand, we use a standard size equal to the other text box sizes in the survey but oversized for the specific response task. On the other hand, we use a smaller text box which fits the response task. The other probes in the survey that use the standard text box are mainly category-selection probes that ask for a reasoning for the chosen answer value. Due to randomized rotation of questions, respondents receive different numbers of category-selection probes prior to the immigrant probe, resulting in different degrees of exposure to category-selection probing prior to the immigrant probe. For the immigrant probe, we find that respondents who receive the standard text box and who have had a high exposure to category-selection probing are more likely to provide mismatching answers: The mismatch consists of not answering the specific immigrant probe but rather providing a reasoning answer as typically expected for a category-selection probe. Thus, previous experience with probing in the questionnaire can override the actual probe wording. This problem can be minimized by considering possible carryover effects of prior probes and using an appropriate survey design strategy.

The NLAAS story: Some reflections, some insights.

The General Social Survey (GSS), a national survey that queries U.S. residents about their behaviors and attitudes on social issues, shows that many Americans do not have as much familiarity with Asian Americans as they do with other racial and ethnic groups (Gee, Ro, Shariff-Marco & Chae, 2009). Thirty-four percent of the GSS sample stated that they had less in common with Asian Americans compared to African Americans, Latinos, Whites, and Jews. African Americans received the next highest percentage (17%). When the question was reversed, “Which group do you have the most in common with?”, a small percentage of the GSS respondents cited Asian Americans (7.5%) and African Americans (7.7%). What accounts for this lack of connection with Asian Americans? There are several likely reasons, including past and current stereotypes of Asian Americans, the geographic and residential distribution of racial groups across the country, and the high proportion of immigrants among Asian Americans that make them seemingly dissimilar to U.S.-born residents. It is also likely that there is relatively limited scientific data available about Asian Americans which make it difficult to adequately inform the general public about Asian American experiences on a wide range of dimensions. It is within this context that the National Latino and Asian American Study (NLAAS) was conceived, designed, implemented, and analyzed. NLAAS is the first national epidemiological study of Asian Americans that used state-of-the-art sampling and survey design strategies, an interview guide that was translated into multiple languages and dialects, and includes data that can be compared across Black American, Latino and White samples see Pennel). Its findings have been disseminated to a broad range of public and scientific audiences and the data are still being analyzed. NLAAS has received special recognition for its contribution to the scientific enterprise and it became one of the most downloaded datasets when it became publically-available. We are gratified and honored that this special issue of the journal has focused on analyses derived from the NLAAS. This paper serves to provide some of the initial thinking behind NLAAS and discuss some of the themes that we find in the papers included in the two special issues.

Travel using managed lanes: An application of a stated choice model for Houston, Texas

Managed lane (ML) travel adds flexibility, but also complexity, to travel choices. Stated choice models (SCMs) are often used for modeling complex transportation choices such as these in an effort to predict demand for these travel options. The design methods for SCMs have evolved from simple orthogonal designs to more sophisticated designs such as D-efficient design that can increase efficiency in estimation. We used three different survey design strategies to produce the stated preference portion of surveys, which were used to elicit travel choices for a sample of Houston travelers. Apart from the D-efficient design we also used random and adaptive random designs to generate attribute levels. There were observable differences in choice behavior depending on what design strategy was used. These differences appear to influence estimates of the value of travel time savings (VTTS) obtained from the random parameter logit (RPL) models estimated using these data. This, in turn, would greatly impact the percentage of travelers predicted to use the MLs. The adaptive random strategy was superior to the other design methods in several categories, and it had similar efficiency to the D-efficient design. However, the mean of VTTS estimate obtained from a D-efficient design was closer to what is typically found in the literature. The difference was considerable and could greatly influence traffic and revenue estimates for the MLs, illustrating the importance of the survey design strategy.

Measuring revisions to subjective expectations

This paper develops a new metric to measure revisions to subjective expectations and proposes a survey design strategy that enables the estimation of the metric. As an application, I analyze how women update their ex pectations about the effectiveness of contraception methods. The women interviewed exhibit substantial heterogeneity in the way they revise their expectations with receipt of the same information. When relating the heterogeneity in the updating process to observable characteristics, I find that schooling, having a regular sexual partner and knowledge and use of birth control methods have a large impact on the revision process about the effectiveness of contraceptives.

The Need to Adapt Standardized Outcomes Measures for Community Mental Health

SUMMARY In order to determine whether evidence-based practices are effective in community settings, the use of measurement tools that accurately capture symptoms and functioning in the community context is imperative. Without accurate measurement, researchers face the troubling possibility that tests of effective treatments may not attain positive results because of measurement error. The authors have experienced apparently serious problems with measurement validity in community mental health practice settings in which we are conducting diffusion research. Features of survey design that strengthen measurement in the university setting (where survey instruments are designed and treatments are usually tested) appear to work against us in community settings. Problems are compounded when it is necessary to measure across several domains, and thus, to employ multiple questionnaires. Here, we discuss the problems we have experienced with three common survey design strategies: (1) the use of multiple, similar items; (2) the use of reverse-coded items; and (3) the use of graded response-items. These strategies result in measures whose length and cognitive complexity may compromise their validity in community settings. Race is an especially important issue in diffusion research, as we lack even basic information concerning efficacy and effectiveness for some important evidence-based treatments. Invalid measures could seriously delay transfer and development of appropriate treatments. Careful psychometric work is badly needed, but such work is time intensive and costly. In the meantime, we encourage researchers to consider the modifications suggested here.

Population coverage and nonresponse bias in a large-scale human exposure study

We used estimates derived from screener variables of the National Human Exposure Assessment Survey (NHEXAS) Phase I field study in EPA Region V (one of three NHEXAS Phase I field studies) to examine biases resulting from survey nonresponse and/or incomplete population coverage inherent in the study design. For variables with population values obtainable from Census projections, the combined effect of nonresponse and coverage bias was tested for after each stage of nonresponse using design-based weights. For variables where population values were not available as Census projections, nonresponse bias was tested for after the screener stage of nonresponse using weights adjusted for screener nonresponse. Additional tests for bias were performed using final survey weights to evaluate the performance of survey weight adjustments in reducing observed bias. Comparison of biases estimated using both design-based and adjusted weights was used to identify potentially important weight adjustment variables for future exposure studies, identify possible weaknesses in survey design strategies, and support the use of nonresponse and poststratification weight adjustments to reduce bias in future survey studies.

An empirical approach to inversion of an unconventional helicopter electromagnetic dataset

A helicopter electromagnetic (HEM) survey acquired at the U.S. Idaho National Engineering and Environmental Laboratory (INEEL) used a modification of a traditional mining airborne method flown at low levels for detailed characterization of shallow waste sites. The low sensor height, used to increase resolution, invalidates standard assumptions used in processing HEM data. Although the survey design strategy was sound, traditional interpretation techniques, routinely used in industry, proved ineffective. Processed data and apparent resistivity maps were severely distorted, and hence unusable, due to low flight height effects, high magnetic permeability of the basalt host, and the conductive, three-dimensional nature of the waste site targets. To accommodate these interpretation challenges, we modified a one-dimensional inversion routine to include a linear term in the objective function that allows for the magnetic and three-dimensional electromagnetic responses in the in-phase data. Although somewhat ad hoc, the use of this term in the inverse routine, referred to as the shift factor, was successful in defining the waste sites and reducing noise due to the low flight height and magnetic characteristics of the host rock. Many inversion scenarios were applied to the data and careful analysis was necessary to determine the parameters appropriate for interpretation, hence the approach was empirical. Data from three areas were processed with this scheme to highlight different interpretational aspects of the method. Wastes sites were delineated with the shift terms in two of the areas, allowing for separation of the anthropomorphic targets from the natural one-dimensional host. In the third area, the estimated resistivity and the shift factor were used for geological mapping. The high magnetic content of the native soil enabled the mapping of disturbed soil with the shift term.

Role of Response Behavior Theory in Survey Research: A Cross-National Study

This study examines four theoretical frameworks for explaining survey response behavior and their role in survey research. The results of a survey of 282 research practitioners in Asia-Pacific, North America, and Western Europe show that research practitioners in general are aware and do make use of the theories of cognitive dissonance, commitment and involvement, social exchange, and self-perception. Although the literature indicates that commitment and involvement have been used very little to explain methodological effects, the present study provides evidence to the contrary. A comparison of the results obtained from the three sample groups reveals some significant differences in the research practitioners’ perceptions of why people participate in surveys as well as in the survey design strategies they adopt. There also is evidence that survey design practices are associated with, and perhaps influenced by, the research practitioners’ beliefs about why people participate in surveys.