Doing understanding differently: Rethinking our relationship with metrics – and the people behind the data

Survey Research: A Primer for Hand Surgery

We examined features of citizen science that influence data quality, inferential power, and usefulness in ecology. As background context for our examination, we considered topics such as ecological sampling (probability based, purposive, opportunistic), linkage between sampling technique and statistical inference (design based, model based), and scientific paradigms (confirmatory, exploratory). We distinguished several types of citizen science investigations, from intensive research with rigorous protocols targeting clearly articulated questions to mass‐participation internet‐based projects with opportunistic data collection lacking sampling design, and examined overarching objectives, design, analysis, volunteer training, and performance. We identified key features that influence data quality: project objectives, design and analysis, and volunteer training and performance. Projects with good designs, trained volunteers, and professional oversight can meet statistical criteria to produce high‐quality data with strong inferential power and therefore are well suited for ecological research objectives. Projects with opportunistic data collection, little or no sampling design, and minimal volunteer training are better suited for general objectives related to public education or data exploration because reliable statistical estimation can be difficult or impossible. In some cases, statistically robust analytical methods, external data, or both may increase the inferential power of certain opportunistically collected data. Ecological management, especially by government agencies, frequently requires data suitable for reliable inference. With standardized protocols, state‐of‐the‐art analytical methods, and well‐supervised programs, citizen science can make valuable contributions to conservation by increasing the scope of species monitoring efforts. Data quality can be improved by adhering to basic principles of data collection and analysis, designing studies to provide the data quality required, and including suitable statistical expertise, thereby strengthening the science aspect of citizen science and enhancing acceptance by the scientific community and decision makers.

ObjectivesThe inherent nature of the Korean National Health and Nutrition Examination Survey (KNHANES) design requires special analysis by incorporating sample weights, stratification, and clustering not used in ordinary statistical procedures.MethodsThis study investigated the proportion of research papers that have used an appropriate statistical methodology out of the research papers analyzing the KNHANES cited in the PubMed online system from 2007 to 2012. We also compared differences in mean and regression estimates between the ordinary statistical data analyses without sampling weight and design-based data analyses using the KNHANES 2008 to 2010.ResultsOf the 247 research articles cited in PubMed, only 19.8% of all articles used survey design analysis, compared with 80.2% of articles that used ordinary statistical analysis, treating KNHANES data as if it were collected using a simple random sampling method. Means and standard errors differed between the ordinary statistical data analyses and design-based analyses, and the standard errors in the design-based analyses tended to be larger than those in the ordinary statistical data analyses.ConclusionsIgnoring complex survey design can result in biased estimates and overstated significance levels. Sample weights, stratification, and clustering of the design must be incorporated into analyses to ensure the development of appropriate estimates and standard errors of these estimates.

The MaNGA Survey (Mapping Nearby Galaxies at Apache Point Observatory) is one of three core programs in the Sloan Digital Sky Survey IV. It is obtaining integral field spectroscopy for 10,000 nearby galaxies at a spectral resolution of R ∼ 2000 from 3622 to 10354 Å. The design of the survey is driven by a set of science requirements on the precision of estimates of the following properties: star formation rate surface density, gas metallicity, stellar population age, metallicity, and abundance ratio, and their gradients; stellar and gas kinematics; and enclosed gravitational mass as a function of radius. We describe how these science requirements set the depth of the observations and dictate sample selection. The majority of targeted galaxies are selected to ensure uniform spatial coverage in units of effective radius (R e ) while maximizing spatial resolution. About two-thirds of the sample is covered out to 1.5R e (Primary sample), and one-third of the sample is covered to 2.5R e (Secondary sample). We describe the survey execution with details that would be useful in the design of similar future surveys. We also present statistics on the achieved data quality, specifically the point-spread function, sampling uniformity, spectral resolution, sky subtraction, and flux calibration. For our Primary sample, the median r-band signal-to-noise ratio is ∼70 per 1.4 Å pixel for spectra stacked between 1R e and 1.5R e . Measurements of various galaxy properties from the first-year data show that we are meeting or exceeding the defined requirements for the majority of our science goals.

In ecological field surveys, observations are gathered at different spatial locations. The purpose may be to relate biological response variables (e.g., species abundances) to explanatory environmental variables (e.g., soil characteristics). In the absence of prior knowledge, ecologists have been taught to rely on systematic or random sampling designs. If there is prior knowledge about the spatial patterning of the explanatory variables, obtained from either previous surveys or a pilot study, can we use this information to optimize the sampling design in order to maximize our ability to detect the relationships between the response and explanatory variables? The specific questions addressed in this paper are: a) What is the effect (type I error) of spatial autocorrelation on the statistical tests commonly used by ecologists to analyse field survey data? b) Can we eliminate, or at least minimize, the effect of spatial autocorrelation by the design of the survey? Are there designs that provide greater power for surveys, at least under certain circumstances? c) Can we eliminate or control for the effect of spatial autocorrelation during the analysis? To answer the last question, we compared regular regression analysis to a modified t‐test developed by Dutilleul for correlation coefficients in the presence of spatial autocorrelation. Replicated surfaces (typically, 1000 of them) were simulated using different spatial parameters, and these surfaces were subjected to different sampling designs and methods of statistical analysis. The simulated surfaces may represent, for example, vegetation response to underlying environmental variation. This allowed us 1) to measure the frequency of type I error (the failure to reject the null hypothesis when in fact there is no effect of the environment on the response variable) and 2) to estimate the power of the different combinations of sampling designs and methods of statistical analysis (power is measured by the rate of rejection of the null hypothesis when an effect of the environment on the response variable has been created). Our results indicate that: 1) Spatial autocorrelation in both the response and environmental variables affects the classical tests of significance of correlation or regression coefficients. Spatial autocorrelation in only one of the two variables does not affect the test of significance. 2) A broad‐scale spatial structure present in data has the same effect on the tests as spatial autocorrelation. When such a structure is present in one of the variables and autocorrelation is found in the other, or in both, the tests of significance have inflated rates of type I error. 3) Dutilleul's modified t‐test for the correlation coefficient, corrected for spatial autocorrelation, effectively corrects for spatial autocorrelation in the data. It also effectively corrects for the presence of deterministic structures, with or without spatial autocorrelation. The presence of a broad‐scale deterministic structure may, in some cases, reduce the power of the modified t‐test.

The health literacy refers to the ability of individuals to acquire and understand basic health information and services and use them to make the right decisions to maintain and promote their health. Health literacy data focusing on cancer prevention and control was limited in China. In order to understand the health literacy and awareness of cancer risk factors and the cancer screening, early diagnosis and treatment in Chinese urban residents and compare the effect of different stages of the cancer intervention, the Cancer Screening Program in Urban China (CanSPUC) program, supported by the National Key Public Health Program, conducted a survey on the health literacy of the cancer prevention and treatment among urban residents in 16 provinces nationwide from 2015 to 2017. Four subgroups were designed in this survey, including (1) general population who have never participated in any cancer screening programs at a community-level; (2) individuals who have previously attended the CanSPUC program for cancer risk assessment or screening intervention; (3) cancer patients who were receiving treatment in local hospitals; (4) a special group from employees of government and public institutions (non-health system), state-owned enterprises and private enterprises (to have better understand on the impact of socioeconomic factors). The self-designed questionnaire covered six parts, including basic information, consciousness of common risk factors to cancer, awareness of early detection, awareness of early diagnosis, awareness of early treatment, and the needs and approaches for knowledge of cancer prevention and treatment. A total of 32 257 individuals were included in the final analyses. This paper landscaped the overall design of the survey, including participants, domains of the instrument, quality control, basic characteristics of the included individuals. These descriptions are applicable to each individual report of the current special issue of "Health Literacy of Cancer Control in Urban China" and future reports, in which more detailed results are and will be reported. The findings of this survey could provide some useful implications for similar researches in the future.

Beyond the limits of memory? The reliability of retrospective data in travel research

Collecting, Managing, and Assessing Data Using Sample Surveys provides a thorough, step-by-step guide to the design and implementation of surveys. Beginning with a primer on basic statistics, the first half of the book takes readers on a comprehensive tour through the basics of survey design. Topics covered include the ethics of surveys, the design of survey procedures, the design of the survey instrument, how to write questions and how to draw representative samples. Having shown readers how to design surveys, the second half of the book discusses a number of issues surrounding their implementation, including repetitive surveys, the economics of surveys, web-based surveys, coding and data entry, data expansion and weighting, the issue of non-response, and the documenting and archiving of survey data. The book is an excellent introduction to the use of surveys for graduate students as well as a useful reference work for scholars and professionals.

The linguistic survey have been particularly well-considered by researchers in the field of social linguistics, for being a working field tool enabling the researcher in many fields, such as dialects, atastics and language survey projects to monitor and investigate the sociolsani reality of any region. The survey paper is no less than determining the appropriate sample of the study for research and investigation. In terms of stability of extrapolation, it is very important to focus on the formulation and design of the survey, the knowledge of samples and their quality and the conditions of the good sample that ensure a genuine representation of society away from bias, and a further accurate estimates resulting in more efficient research findings.
 This paper is intended to address the rules of the construction and design of the linguistic survey and to present its relevance, function, types, characteristics, components, sources and stages of preparation, in order to reach a systematic guide for the construction and utilization of the linguistic survey and the development of field-based researchers' capabilities and techniques of interview and investigation.

SummaryNon‐response is a common source of error in many surveys. Because surveys often are costly instruments, quality‐cost trade‐offs play a continuing role in the design and analysis of surveys. The advances of telephone, computers, and Internet all had and still have considerable impact on the design of surveys. Recently, a strong focus on methods for survey data collection monitoring and tailoring has emerged as a new paradigm to efficiently reduce non‐response error. Paradata and adaptive survey designs are key words in these new developments. Prerequisites to evaluating, comparing, monitoring, and improving quality of survey response are a conceptual framework for representative survey response, indicators to measure deviations thereof, and indicators to identify subpopulations that need increased effort. In this paper, we present an overview of representativeness indicators or R‐indicators that are fit for these purposes. We give several examples and provide guidelines for their use in practice.

Surveys rely on structured questions used to map out reality, using sample observations from a population frame, into data that can be statistically analyzed. This paper focuses on the planning and design of surveys, making a distinction between individual surveys, household surveys and establishment surveys. Knowledge from cognitive science is used to provide guidelines on questionnaire design. Non-standard, but simple, statistical methods are described for analyzing survey results. The paper is based on experience gained by conducting over 150 customer satisfaction surveys in Europe, America and the Far East.

Lies, Damned Lies, and Surveys.

Vegetation maps are critical biodiversity planning instruments, but the classification of vegetation for mapping can be strongly biased by survey design. Standardization of survey design across different vegetation types is therefore increasingly recommended for vegetation mapping programs. However, some vegetation types have complex small‐scale vegetation patterns that are important in characterizing these vegetation types, and standard designs will often not capture these patterns. The objective of this paper was to investigate the magnitude of potential map bias that results from survey design standardization and recommend approaches to deal with this bias. We surveyed upland swamps of the Greater Blue Mountains World Heritage Area Australia using two contrasting survey designs, including the standard 400 m2 single quadrat design recommended and used by authorities. We then derived a classification for these swamps and tested the effect of survey design on this classification, species richness and the type of species detected (obligate or facultative swamp species). Species richness and species type were not significantly different among survey techniques. However, more than 40% of swamps clustered differently among survey designs. Thus, one of the 10 derived communities (which is floristically consistent with a previously mapped endangered community) was indistinct, and some individual swamps misclassified using the standard survey design. An effect of landscape position on swamp floristic patterns and a significant trend for high similarity scores among swamps surveyed with multiple small quadrats compared to the standard survey design was also determined. Australian upland swamps are classified at the global scale as shrub‐dominated wetlands, and complex floristic patterns have been recorded in shrub‐dominated wetlands in both northern and southern hemispheres. We therefore advocate either multiple survey designs or different survey standards for upland swamp communities and other vegetation types that have complex floristic patterns at small scales.

Context Sampling methods to estimate animal density require good survey design to ensure assumptions are met and sampling is representative of the survey area. Management decisions are often made based on these estimates. However, without knowledge of true population size it is not possible for wildlife biologists to evaluate how biased the estimates can be if survey design is compromised. Aims Our aims were to use distance sampling to estimate population size for domestic sheep free-ranging within large enclosed areas of hill country and, by comparing estimates against actual numbers, examine how bias and precision are impaired when survey design is compromised. Methods We used both line and point transect sampling to derive estimates of density for sheep on four farms in upland England. In Stage I we used limited effort and different transect types to compromise survey design. In Stage II we increased effort in an attempt to improve on the Stage I estimates. We also examined the influence of a walking observer on sheep behaviour to assess compliance with distance sampling assumptions and to improve the fit of models to the data. Key results Our results show that distance sampling can lead to biased and imprecise density estimates if survey design is poor, particularly when sampling high density and mobile species that respond to observer presence. In Stage I, walked line transects were least biased; point transects were most biased. Increased effort in Stage II reduced the bias in walked line transect estimates. For all estimates, the actual density was within the derived 95% confidence intervals, but some of these spanned a range of over 100 sheep per km2. Conclusions Using a population of known size, we showed that survey design is vitally important in achieving unbiased and precise density estimation using distance sampling. Adequate transect replication reduced the bias considerably within a compromised survey design. Implications Management decisions based on poorly designed surveys must be made with an appropriate understanding of estimate uncertainty. Failure to do this may lead to ineffective management.

Surveys are frequently used to collect data in graduate medical education (GME) settings.1 However, if a GME survey is not rigorously designed, the quality of the results is likely to be lower than desirable. In a recent editorial we introduced a framework for developing survey instruments.1 This systematic approach is intended to improve the quality of GME surveys and increase the likelihood of collecting survey data with evidence of reliability and validity. In this article we illustrate how researchers in medical education may operationalize this framework with examples from a survey we developed during the recent integration of 2 independent internal medicine (IM) residency programs.