Survey Error Research Articles

Galaxy blending effects in deep imaging cosmic shear probes of cosmology

ABSTRACT Upcoming deep imaging surveys such as the Vera C. Rubin Observatory Legacy Survey of Space and Time will be confronted with challenges that come with increased depth. One of the leading systematic errors in deep surveys is the blending of objects due to higher surface density in the more crowded images; a considerable fraction of the galaxies which we hope to use for cosmology analyses will overlap each other on the observed sky. In order to investigate these challenges, we emulate blending in a mock catalogue consisting of galaxies at a depth equivalent to 1.3 yr of the full 10-yr Rubin Observatory that includes effects due to weak lensing, ground-based seeing, and the uncertainties due to extraction of catalogues from imaging data. The emulated catalogue indicates that approximately 12 per cent of the observed galaxies are ‘unrecognized’ blends that contain two or more objects but are detected as one. Using the positions and shears of half a billion distant galaxies, we compute shear–shear correlation functions after selecting tomographic samples in terms of both spectroscopic and photometric redshift bins. We examine the sensitivity of the cosmological parameter estimation to unrecognized blending employing both jackknife and analytical Gaussian covariance estimators. An ∼0.025 decrease in the derived structure growth parameter S8 = σ8(Ωm/0.3)0.5 is seen due to unrecognized blending in both tomographies with a slight additional bias for the photo-z-based tomography. This bias is greater than the 2σ statistical error in measuring S8.

Just you wait… and fill out this survey. Discussion of the methodological aspects of waiting room surveys

A method commonly used in health care research is the waiting room (WR) survey. While patients are waiting for their appointment, they are asked to complete a questionnaire measuring their attitudes, behaviors and other characteristics. In this paper we synthesize practical guidelines for WR surveys by comparing the method with two similar approaches: public intercept (PI) surveys and drop-off-pick-up (DOPU) surveys. In this comparison we use the Total Survey Error approach Groves (Survey Methodology, Wiley, New York, 2004); (Groves in Public Opinion Quarterly 74(5): 849-879, 2010) and apply it to three case examples in which one of the three surveys is used. We take into account measurement (validity, measurement- and processing error) and representation (coverage-, sampling- and nonresponse error). From our review, we conclude that waiting room surveys, though limited to patients and their caregivers, can provide useful information on patients’ perspective on health care. Response rates in waiting rooms are usually high, but often not even reported. We recommend adjustment for sampling bias by taking into account the number of visits to the hospital per respondent and sample times proportionate to the number of sample members expected on a particular time. These surveys also allow for collection of para-data; i.e., relevant information in the circumstances of a request to participate in survey research, and behavior of surveyors can easily be controlled, or investigated in an experimental design.

Non-sampling errors in questionnaire surveys: findings from a National Fertility Survey

Non-sampling errors can generally be divided into three types: sampling frame errors, non-response errors and measurement errors. Missing target units in the sampling frame, improper handling of non-responses, and misreporting or underreporting of key variables in the questionnaire can all cause deviations in a survey’s results. The widespread application of Computer-Assisted Personal Interviewing (CAPI) systems and the inclusion of administrative records from government sources in surveys has strengthened the ability to control non-sampling errors. Taking a national fertility sampling survey as an example, this study summarizes the sources of various non-sampling errors and explains how to harness big data resources such as administrative records to control non-sampling errors throughout the survey. The study analyzes the impact of three types of non-sampling errors on the results of the fertility survey and examines the strategies used to address the problems caused by these non-sampling errors. The findings indicate that non-sampling errors were the main source of total error in the survey, and that the errors found came mainly from sampling frame errors; non-response errors and measurement errors were controlled and had little impact on the survey results.

Addressing Safety concerns in Hybrid Electric Aircrafts: In-Flight Icing Detection, Moisture Detection in Fuselage and Electrical Wiring and Interconnect System (EWIS)

Moisture - in the form of water or ice – has adverse consequences on the performance of an aircraft including wing stall, tail stall, loss of thrust, engine flameout, and material damage arising from corrosion and electrical shortcuts. During flight, a large flux of supercooled water droplets on the surface of a hybrid electric aircraft per unit time could result into icing. A prototype to detect ice accretion on supercooled aircraft surface using acoustic emissions is developed. Moreover, during descend, a large portion of frost/ice melts rapidly due to rise in temperature. If not controlled, water may seep inside a hybrid electric aircraft body through gaps in joints. Furthermore, for the problem of water ingression in the wiring of hybrid aircrafts, titanium carbonitride-polyvinyl alcohol (TiCN-PVA) composite material is placed in direct water contact to work as grid health monitoring sensor. For the problem of water ingression inside structural parts (floor, bilge, etc.), an electrically conductive epoxy – carbon black composite is fabricated, characterized using SEM, and tested for DC conductivity under dry and wet conditions. Effect of different mixing intensities on morphology of carbon black in epoxy has also been explored. Through literature survey and trial and error, an optimized method at laboratory scale for preparation of carbon black-epoxy composite material has been presented.

СТАБИЛИЗАЦИЯ ПРОСАДОК ЗДАНИЙ СОВРЕМЕННЫХ МЕДИЦИНСКИХ ЦЕНТРОВ

the group of deformed structures includes buildings that have received unacceptable subsidence and deformation during the period of their construction and especially operation, which, however, do not interfere with the perfor-mance of their main functions, but may eventually collapse. Their causes are errors in engineering and geological surveys and design; violation of the rules for performing construction work and operation of buildings and struc-tures. Long-term geodetic observations of the precipitation of the foundations of buildings on pile foundations have shown that both absolute and relative stabilized values of subsidence in the vast majority of cases are less than them and the normative limit values are calculated. Therefore, the group of deformed buildings on pile foun-dations includes somewhat less often similar objects with shallow foundations. The reasons for excessive subsid-ence of the foundations of pile foundations of buildings (and as a result, the occurrence and development of cracks and other deformations in load – bearing structures), in addition to these, are most often: unjustified use of increasing correction coefficients for the results of compression tests of highly acidic soils; the lower ends of the piles falling into layers of weak soil; the tip of the piles sinking from the design mark; overestimation of the bear-ing capacity of the piles due to non-compliance with the optimal time of their "rest" after immersion or erroneous interpretation of the graphs "load-pile sediment"; excessively close placement of neighboring piles in the plan, which when they are immersed, especially in the sand, leads to "pushing" up previously submerged; uneven load-ing of piles as part of the grillage; deformation of existing buildings and structures when driving piles near and tongue-and-groove, the development of pits, etc.

Well Trajectory Measurement Error Based on Gyro Survey

Gyroscopic survey technology is one of the key technologies to promote the progress of directional drilling technology. It is of great practical significance to study the error of well trajectory measurement based on gyro surveys and identify the main error sources. This paper introduces the error sources which lead to the uncertainty of borehole position in gyroscopic survey, six gyroscopic survey error models, and the calculation model of the contribution rate of the error sources to the error ellipse. The differences between gyro and MWD survey errors and the main error sources of gyro surveys are investigated with an example. The results show that gyroscopic surveys cannot reduce the uncertainty of borehole position compared to MWD surveys in the environment without magnetic interference. The static measurement by a three-axis gyro inclinometer is more beneficial for the improvement of the measurement accuracy of the borehole position. The error sources that contribute to the measurement error of the well trajectory are also different in the vertical section, build-up section, and horizontal section. Unlike the main error source of the MWD survey, the g-dependent error sources of the gyro sensor significantly contribute to the measurement error of the borehole trajectory. Therefore, the measurement error of the well trajectory can be significantly reduced by improving the gyroscope survey tool and reducing the influence of spin mass unbalance.

A response to Hesford & MacLeod (2022): Rejection of a model estimating high densities of mountain hares in the Peak District, England.

A recent Game and Wildlife Conservation Trust (GWCT) report (Hesford & MacLeod 2022) suggested densities of mountain hares (Lepus timidus) may reach 52 - 125/km2 in parts of the Peak District, England. These are notably higher than previous and current estimates of 5 - 33 hares/km2 (Matthews et al. 2018; Bedson et al. 2022). We review Hesford & MacLeod (2022) who based their methods on those used in a review of mountain hare survey methods in Scotland (Newey et al. 2018). This review demonstrated a weak, non-significant relationship between hare encounter rates using spotlight surveys of walked transects at night and estimated densities derived from spatial capture-recapture (SCR) methods on managed heather moorland (p=0.08). Newey et al. (2018) recommended that this relationship should not be used to estimate hare densities. We reproduce the Newey et al. (2018) linear model and confirm its poor predictive ability and show that removal of one outlier reduces an already marginal relationship to a near-flat line (p=0.80). Hesford & MacLeod (2022), nonetheless, used this relationship to estimate hare densities along non-randomly placed transects. We conclude that reportedly high mountain hare densities estimated by Hesford & MacLeod (2022) are biased and based on a model with little predictive power; more recent Distance Sampling estimates are from 37 - 96% lower (Bedson et al. 2022). It is important that wildlife monitoring methods robustly account for survey bias and error, detection probability and variation between habitats, especially if results are to inform potential population management interventions.

Measuring contraceptive use in a displacement-affected population using the Multiple Indicator Cluster Survey: The case of Iraq

Access to safe, effective, affordable, and acceptable contraceptive methods of choice is a basic right for displaced people. Yet displaced people are typically invisible in national sample surveys on population health, and quantitative evidence on their reproductive health outcomes is limited. This study focuses on the case of Iraq, a country with widespread displacement and where contraceptive use is a government policy priority. Using displacement screening questions in the Iraq 2018 Multiple Indicator Cluster Survey questionnaires, I construct two displacement-related indicators based on reason for last move and previous household residence. Descriptive statistics and binary logistic regression are used to test associations between modern contraceptive use and displacement, demographic, and socioeconomic factors. Controlling for the variables in the models, factors significantly associated with modern contraceptive use among married women aged 15–49 in Iraq are Federal Iraq region (reference Kurdish Region of Iraq, OR 1.78), upper secondary and primary education (reference pre-primary or no education, OR 1.50 and 1.20, respectively), parity, age, and exposure to television. The association between displacement (reason for last move) and modern contraceptive use significantly depends on a woman's level of education and whether they live in an urban or rural area. Women who previously lived in a camp are almost half as likely to use modern contraception compared to other previous residence types. This paper highlights the methodological potential and substantive value of using national household surveys to analyse reproductive health outcomes through a displacement lens. It also critically examines the limitations of these data and measures, drawing on total survey error and feminist theory.

Using Visual Grouping to Improve Navigation of Skip Patterns in Mail Surveys: An Experiment

AbstractSkip patterns, in which certain questions should only be answered by subsets of respondents, are particularly challenging for mail surveys because respondents must navigate them themselves with no assistance from a computer or interviewer. Incorrect navigation of skip patterns can lead to (1) errors of omission, which is when respondents fail to answer follow-up questions they should answer; (2) errors of commission, which is when respondents answer follow-up questions they should skip; or (3) filter item nonresponse, which is when respondents fail to answer the initial filter question. Using experimental data from two general population surveys of Nebraskans, this paper examines three visual design elements—common region, indentation, and subnumbering—aimed at decreasing skip errors in mail surveys. Each design element is intended to create stronger grouping and subgrouping among items within skip patterns, thus clarifying the navigational path and decreasing skip errors. We compare error rates across treatments overall and by respondent age, education, and literacy to assess whether visual cues can provide additional aid to those who might struggle with reading text cues. We find that using separate enclosures or subnumbering of the follow-up questions significantly reduces commission error rates and using indentation significantly increases omission error rates. In addition, none of the tested visual designs were more effective for older, less educated, or lower literacy respondents than for their counterparts.

Estimating unmet need for family planning: A new look at the Bradley, Croft, Fishel and Westoff (2012) Algorithm.

Every year, hundreds of millions of women face the painful experience of unwanted pregnancies. To address this situation, access to and use of reproductive health services, including for family planning purposes, has been considered a Sustainable Development Goal (SDG). However, many women cannot access family planning services due to a common methodological error in surveys used to estimate unmet needs. In this article, I have discussed this historical error that has been widely accepted by the scientific community as well as perspectives that could reduce the incidence of unwanted pregnancies around the world.

Identifying and Minimizing Errors in the Measurement of Early Childhood Development: Lessons Learned from the Cognitive Testing of the ECDI2030.

Challenges in measuring early childhood development (ECD) at scale have been documented, yet little is known about the specific difficulties related to questionnaire design and question interpretation. The purpose of this paper is to discuss the challenges of measuring ECD at scale in the context of household surveys and to show how to overcome them. The paper uses examples from the cognitive interviewing exercises that were conducted as part of the methodological work to develop a measure of ECD outcomes, the ECDI2030. It describes the methodological work carried out to inform the selection and improvement of question items and survey implementation tools as a fundamental step to reduce and mitigate systematic measurement error and improve data quality. The project consisted of a total of five rounds of testing, comprising 191 one-on-one, in-depth cognitive interviews across six countries (Bulgaria, India, Jamaica, Mexico, Uganda, and the USA). Qualitative data analysis methods were used to determine matches and mismatches between intention of items and false positives or false negative answers among subgroups of respondents. Key themes emerged that could potentially lead to systematic measurement error in population-based surveys on ECD: (1) willingness of child to perform task versus ability of child to perform task; (2) performing task versus performing task correctly; (3) identifying letters or numbers versus recognizing letters or numbers; (4) consistently performing task versus correctly performing task; (5) applicability of skills being asked versus observability of skills being asked; and (6) language production versus language comprehension. Through an iterative process of testing and subsequent revision, improvements were made to item wording, response options, and interviewer training instructions. Given the difficulties inherent in population-level data collection in the context of global monitoring, this study’s findings confirm the importance of cognitive testing as a crucial step in careful, culturally relevant, and sensitive questionnaire design and as a means to reduce response bias in cross-cultural contexts.

Estimating the Problem of Non-Response and Measurement Error in Sample Survey

Estimating the Problem of Non-Response and Measurement Error in Sample Survey

Deriving Priors for Bayesian Prediction of Daily Response Propensity in Responsive Survey Design: Historical Data Analysis Versus Literature Review.

Responsive survey design (RSD) aims to increase the efficiency of survey data collection via live monitoring of paradata and the introduction of protocol changes when survey errors and increased costs seem imminent. Daily predictions of response propensity for all active sampled cases are among the most important quantities for live monitoring of data collection outcomes, making sound predictions of these propensities essential for the success of RSD. Because it relies on real-time updates of prior beliefs about key design quantities, such as predicted response propensities, RSD stands to benefit from Bayesian approaches. However, empirical evidence of the merits of these approaches is lacking in the literature, and the derivation of informative prior distributions is required for these approaches to be effective. In this paper, we evaluate the ability of two approaches to deriving prior distributions for the coefficients defining daily response propensity models to improve predictions of daily response propensity in a real data collection employing RSD. The first approach involves analyses of historical data from the same survey, and the second approach involves literature review. We find that Bayesian methods based on these two approaches result in higher-quality predictions of response propensity than more standard approaches ignoring prior information. This is especially true during the early-to-middle periods of data collection, when survey managers using RSD often consider interventions.

Two-stage sampling exponential and chain ratio type estimators in the presence of measurement error

Currently one of the most important topics that are discussed in survey methodology is the accuracy of statistics or survey errors that may occur in the parameters estimation process and measurement error is one of the most important non-sampling error. In the present study general class of exponential and chain ratio type estimators are considered under two stage sampling for the estimation of population mean by using two auxiliary variables in the presence of measurement error. The bias and mean square error of these estimator has been obtained. Moreover, two artificial populations are generated to check the properties of proposed estimator among other estimator. The relative efficiency is also computed and the results revealed that proposed estimators are more efficient than the ratio estimator and by increasing sample size efficiency of suggested estimators are improved.

Survey Methods: How Planning Practitioners Use Them, and the Implications for Planning Education

This article is concerned with two aspects of how planning practitioners use survey-derived data; how planners integrate the limitations of survey questionnaires into practice, and the prevalence of such data within planning. Using a web survey ( n = 201) and interviews ( n = 18) of Canadian municipal planners, I find that survey data are heavily relied on, but many planners do not seem to be aware of cognitive biases when designing surveys, and those that are, have little knowledge of how they ought to mitigate them. To develop planners’ understanding of these biases and improve the survey data they collect, quantitative methods courses within planning curricula could respond by expanding beyond statistical analysis to incorporate survey design and “the total survey error approach” of survey methodology.

A Total Error Framework for Digital Traces of Human Behavior on Online Platforms

AbstractPeople’s activities and opinions recorded as digital traces online, especially on social media and other web-based platforms, offer increasingly informative pictures of the public. They promise to allow inferences about populations beyond the users of the platforms on which the traces are recorded, representing real potential for the social sciences and a complement to survey-based research. But the use of digital traces brings its own complexities and new error sources to the research enterprise. Recently, researchers have begun to discuss the errors that can occur when digital traces are used to learn about humans and social phenomena. This article synthesizes this discussion and proposes a systematic way to categorize potential errors, inspired by the Total Survey Error (TSE) framework developed for survey methodology. We introduce a conceptual framework to diagnose, understand, and document errors that may occur in studies based on such digital traces. While there are clear parallels to the well-known error sources in the TSE framework, the new “Total Error Framework for Digital Traces of Human Behavior on Online Platforms” (TED-On) identifies several types of error that are specific to the use of digital traces. By providing a standard vocabulary to describe these errors, the proposed framework is intended to advance communication and research about using digital traces in scientific social research.

Dependent interviewing: a remedy or a curse for measurement error in surveys?

Longitudinal surveys often rely on dependent interviewing (DI) to lower the levels of random measurement error in survey data and reduce the incidence of spurious change. DI refers to a data collection technique that incorporates information from prior interview rounds into subsequent waves. While this method is considered an effective remedy for random measurement error, it can also introduce more systematic errors, in particular when respondents are first reminded of their previously provided answer and then asked about their current status. The aim of this paper is to assess the impact of DI on measurement error in employment mobility. We take advantage of a unique experimental situation that was created by the roll-out of dependent interviewing in the Dutch Labour Force Survey (LFS). We apply Hidden Markov Modeling (HMM) to linked LFS and Employment Register (ER) data that cover a period before and after dependent interviewing was abolished, which in turn enables the modeling of systematic errors in the LFS data. Our results indicate that DI lowered the probability of obtaining random measurement error but had no significant effect on the systematic component of the error. The lack of a significant effect might be partially due to the fact that the probability of repeating the same error was extremely high at baseline (i.e when using standard, independent interviewing); therefore the use of DI could not increase this probability any further.

Asset Pricing with Fading Memory

Abstract Building on evidence that lifetime experiences shape individuals’ macroeconomic expectations, we study asset prices in an economy in which a representative agent learns with fading memory about unconditional mean endowment growth. With IID fundamentals, constant risk aversion, and memory decay calibrated to microdata, the model generates a high and strongly countercyclical objective equity premium, while the subjective equity premium is virtually constant. Consistent with this theory, experienced payout growth (a weighted average of past growth rates) is negatively related to future stock market excess returns and subjective expectations errors in surveys, and positively to analysts’ forecasts of long-run earnings growth.

Probabilistic calibration and short‐term prediction of the prevalence herpes simplex type 2: A transmission dynamics modelling approach

An epidemiological model is proposed to study the transmission dynamics of the herpes virus type 2, a sexually transmitted infectious disease. This model considers two states, susceptible and infected, divides the population into sexes, assumes only heterosexual contacts and includes different transmission rates depending on whether the transmission is woman–man or man–woman. Reported and prevalence series data are retrieved from several sources. We consider the inherent data survey errors and the sensitivity of the diagnosis tests (data uncertainty). To calibrate the model to the available data and their uncertainty, a novel technique is proposed in two steps: (1) the application of the estimation of distribution algorithm (EDA) to find sets of model parameter values close to the data uncertainty and (2) the application of a selection algorithm to get a reduced number of model parameter values whose model outputs capture accurately the data uncertainty. Then, we check its robustness, and we provide a prediction of the evolution of the infected over the next 4 years. From the technical point of view, we conclude that the proposed technique to calibrate probabilistically the model is reliable and robust. Also, it is able to provide confidence intervals for the model parameter values and the predictions. From the medical point of view, the model returns that the transmission woman–man is higher than the man–woman, according to recent literature, and there is a mild increasing trend in the number of infected people over the next years.

Survey Data Quality in Analyzing Harmonized Indicators of Protest Behavior: A Survey Data Recycling Approach

This article proposes a new approach to analyze protest participation measured in surveys of uneven quality. Because single international survey projects cover only a fraction of the world’s nations in specific periods, researchers increasingly turn to ex-post harmonization of different survey data sets not a priori designed as comparable. However, very few scholars systematically examine the impact of the survey data quality on substantive results. We argue that the variation in source data, especially deviations from standards of survey documentation, data processing, and computer files—proposed by methodologists of Total Survey Error, Survey Quality Monitoring, and Fitness for Intended Use—is important for analyzing protest behavior. In particular, we apply the Survey Data Recycling framework to investigate the extent to which indicators of attending demonstrations and signing petitions in 1,184 national survey projects are associated with measures of data quality, controlling for variability in the questionnaire items. We demonstrate that the null hypothesis of no impact of measures of survey quality on indicators of protest participation must be rejected. Measures of survey documentation, data processing, and computer records, taken together, explain over 5% of the intersurvey variance in the proportions of the populations attending demonstrations or signing petitions.