Bias In Surveys Research Articles

Revisiting Sample Bias in the UK's Annual Survey of Hours and Earnings, with Implications for Estimates of Low Pay and the Bite of the National Living Wage

Revisiting Sample Bias in the UK's Annual Survey of Hours and Earnings, with Implications for Estimates of Low Pay and the Bite of the National Living Wage

Genuine or impression management? Uncovering biases in how people respond to the Black Lives Matter movement in sport

PurposeNational surveys reveal that sports fans exhibit greater support for athletes, sports teams and leagues endorsing social justice initiatives compared to the general population, highlighting the potential of sports for positive social impact. This study investigates whether such responses are influenced by systematic biases.Design/methodology/approachReplicating a Nielsen national survey, two experiments explore whether biases affect support for athletes' participation in the Black Lives Matter (BLM) movement. The study also examines partisan motivated reasoning as a factor driving sports fans' support for BLM.FindingsWhile avid fans display stronger endorsement of BLM compared to causal/non-sports fans, evidence suggests that systematic biases distort these responses. When sport identity becomes salient, reported support for the BLM movement becomes inflated.Research limitations/implicationsResearchers often employ self-report surveys to gauge audience perceptions of athlete activism or cause-related initiatives, particularly when assessing their impact. This study's findings indicate that this context is susceptible to SDB.Originality/valueThe study underscores the role of systematic biases in self-report surveys, particularly in socially desirable contexts. People tend to over-report “positive behavior,” leading survey participants to respond more favorably to questions that are socially desirable. Therefore, interpreting survey results with caution becomes essential when the research context is deemed socially (un)desirable. It is crucial for researchers to apply appropriate measures to identify and mitigate systematic response biases. The authors recommend that researchers adopt both procedural and statistical remedies to detect and reduce social desirability biases.

Measuring participation in undeclared work in Europe using survey data: A method for resolving social desirability bias

Measuring participation in undeclared work using surveys has been criticized for under-estimating the level of engagement due to social desirability bias that leads to an under-reporting of “bad” behavior. Until now, few studies have sought to quantify the amplitude of this bias in surveys of undeclared work. The aim of this paper is to fill this gap by using the most appropriate methodologies for estimating the probability of misleading responses in such surveys. Reporting data from special Eurobarometer survey no. 498 conducted in 2019 and involving 27,565 respondents in EU-27 countries and the UK, only 3.5% openly admitted to participating in undeclared work. The results of a Probit model with correction for misclassified cases (i.e., those undertaking undeclared work but declaring that they do not) reveals that nearly a quarter (23.3%) of the respondents undertaking undeclared work refused to openly admit this during the survey, due to the social desirability bias. The estimated overall proportion of undeclared workers is 17.3%. We obtained this value by correcting for both misclassification and the additional source of negative bias due to the large imbalance in the data (i.e., observations in one class are much lower than the other). The outcome of this new advanced approach in analysing undeclared work is that survey estimates can now report its size and determinants in a more accurate manner than has been previously the case.

Reweighting in panel surveys: machine learning techniques for the Health Care and Social Survey

Abstract Background Healthcare statistical services have used probability surveys to respond to such information needs. The Health Care and Social Survey (ESSOC) research project arises from the need to provide data on the evolution of the COVID-19 impact. This survey has an overlapping panel survey design with 4 measurements throughout 1 year with random samples. Objective To develop a new reweighting method for overlapping panel surveys affected by non-response. Methods Each ESSOC measurement is composed of two samples: a longitudinal sample from previous measurements and a new sample at each measurement. In each measurement, missing units are substituted by new surveyed units, allowing the obtention of cross-sectional and longitudinal estimates. The advantage of this design is that, in addition to being able to obtain longitudinal estimates, cross-sectional estimates are more accurate because of the larger sample size. However, the problem of non-response is particularly aggravated in the case of panel surveys, due to the fatigue of the population to be repeatedly surveyed. Results Considering the design, timing and objectives of this survey, our reweighting method produces suitable estimators for both cross-sectional and longitudinal samples. The weights are the result of a two-step process: the original sampling design weights are corrected during a 1st phase by modelling the non-response with respect to the longitudinal sample obtained in a previous measurement using machine learning techniques. Then, during a 2nd phase, they are calibrated using the auxiliary information available at the population level. The proposed method is applied to the estimation of totals, proportions, ratios, and differences between measurements as well as gender gaps. Conclusions For addressing future health crises like COVID-19, it is therefore necessary to reduce potential coverage and non-response biases in surveys by means of utilising reweighting techniques as proposed in this study. Key messages • It is therefore necessary to reduce potential coverage and non-response biases in surveys. • This study proposes the utilisation of reweighting techniques based on calibration and XGBoost.

Respondent biases in agricultural household surveys

Two sources of respondent bias introduce measurement error into household statistics: asymmetric information between the proxy respondent and the individual on whom they report; and aggregation bias when a proxy respondent reports on a household-level outcome across multiple individuals. We estimate the effects of respondent biases in a survey experiment in Burkina Faso by varying who reports on the agricultural production of household members. We find respondent biases are not solely attributable to asymmetric information. Choosing a household head proxy lowers aggregation biases, but results in both over and under-estimates of agricultural variables relative to random proxies. Random proxies systematically under report agricultural statistics. Self-reporting protocols increase enumerator work days by only 5% indicating a high bias-cost tradeoff in choosing proxy response over self-reports. Survey designers should weight whether proxy bias magnitude or direction of bias are more significant threats to parameter estimation when determining their proxy response protocol.

Hybrid mark–resight approaches account for heterogeneous detection bias in aerial surveys

Abstract Population abundance estimation is a critical yet challenging task for wildlife management decision‐making. For aerial surveys of terrestrial mammals in highly heterogenous landscapes, estimation models that address multiple sources of detection bias are needed. A typical hybrid mark–resight model includes an additive parameter that accounts for detection probabilities derived from marked animals. While this additive hybrid method improves abundance estimates, there remains a need to account for heterogeneity due to the conditional dependence in detection probability inherent in aerial surveys with multiple observers. We propose a modified approach, referred to as a conditional hybrid model, to address this heterogeneity directly. The conditional hybrid model applies a mark–resight approach typically used to account for tag loss in mark–recapture studies to aerial surveys with a subset of the population marked with GPS collars. We conduct a structured comparison between the additive and conditional hybrid estimation models and fit these two hybrid models to data from a 2020 aerial survey of mule deer in southern California. Both the additive and conditional hybrid models had general agreement in overall model structure and variable importance, however, the conditional model outperformed the additive model in capturing sources of detection heterogeneity and overall abundance estimates differed by 9%. Our top‐ranked models suggest that deer movement, larger group sizes, and rougher topography increased detection probability, whereas high cloud cover, obscuring vegetation and high observer fatigue and helicopter height decreased detection probability. Our findings illustrate the need to assess multiple correction methods when estimating abundance from aerial survey data to ensure accurate abundance estimates are being used to inform wildlife management strategies, with the caveat that specific survey conditions may dictate which method is the most appropriate for achieving robust estimates. These approaches represent the cutting‐edge science on wildlife abundance estimation from aerial survey data, a critical ingredient for effective wildlife management. By testing innovative hybrid methods, we present the first formal comparison of the strengths and weaknesses of each method within a highly heterogeneous landscape, providing practitioners with necessary information for future application.

Ensuring cross-cultural data comparability by means of anchoring vignettes in heterogeneous refugee samples

BackgroundConfigural, metric, and scalar measurement invariance have been indicators of bias-free statistical cross-group comparisons, although they are difficult to verify in the data. Low comparability of translated questionnaires or the different understanding of response formats by respondents might lead to rejection of measurement invariance and point to comparability bias in multi-language surveys. Anchoring vignettes have been proposed as a method to control for the different understanding of response categories by respondents (the latter is referred to as differential item functioning related to response categories or rating scales: RC-DIF). We evaluate the question whether the cross-cultural comparability of data can be assured by means of anchoring vignettes or by considering socio-demographic heterogeneity as an alternative approach.MethodsWe used the Health System Responsiveness (HSR) questionnaire and collected survey data in English (n = 183) and Arabic (n = 121) in a random sample of refugees in the third largest German federal state. We conducted multiple-group Confirmatory Factor Analyses (MGCFA) to analyse measurement invariance and compared the results when 1) using rescaled data on the basis of anchoring vignettes (non-parametric approach), 2) including information on RC-DIF from the analyses with anchoring vignettes as covariates (parametric approach) and 3) including socio-demographic covariates.ResultsFor the HSR, every level of measurement invariance between the Arabic and English languages was rejected. Implementing rescaling or modelling on the basis of anchoring vignettes provided superior results over the initial MGCFA analysis, since configural, metric and – for ordered categorical analyses—scalar invariance could not be rejected. A consideration of socio-demographic variables did not show such an improvement.ConclusionsSurveys may consider anchoring vignettes as a method to assess cross-cultural comparability of data, whereas socio-demographic variables cannot be used to improve data comparability as a standalone method. More research on the efficient implementation of anchoring vignettes and further development of methods to incorporate them when modelling measurement invariance is needed.

Precise self-calibration of interloper bias in spectroscopic surveys

ABSTRACT Interloper contamination due to line misidentification is an important issue in the future low-resolution spectroscopic surveys. We realize that the algorithm previously used for photometric redshift self-calibration, with minor modifications, can be particularly applicable to calibrate the interloper bias. In order to explore the robustness of the modified self-calibration algorithm, we construct the mock catalogues based on China Space Station Telescope (CSST), taking two main target emission lines, Hα and [O iii]. The self-calibration algorithm is tested in cases with different interloper fractions at 1 per cent, 5 per cent, and 10 per cent. We find that the interloper fraction and mean redshift in each redshift bin can be successfully reconstructed at the level of ∼ 0.002 and ∼ 0.001(1 + z), respectively. We also find the impact of the cosmic magnification can be significant, which is usually ignored in previous works, and therefore propose a convenient and efficient method to eliminate it. Using the elimination method, we show that the calibration accuracy can be effectively recovered with slightly larger uncertainty.

Lobbying in social media as a new source of survey bias

Questionnaires are widely-used tools in empirical social science, including tourism research. The key premise of each quantitative survey is the representativeness of the investigated sample. Yet, the rapid development of information and communication technology forces social scientists to take new sources of biases into account. This paper describes the problem of lobbying in social media, initiated by survey participants themselves. We report a collective action of online communities which resulted in over-representation of the collective opinions of certain interest groups. The bias has been observed when analyzing data collected during a visitor monitoring study in a popular forest recreation destination: the Tricity Landscape Park in Poland. Two different methods: on-site survey (n = 994) and online survey (n = 1023) were used to collect data on local recreational preferences. A significant effect of lobbying in social media on online survey responses has been identified in the case of questions related to the forest management. Management implicationsOur study confirms significant differences between results obtained via on-site and online surveys.Cost-efficient distribution of online survey link via social media may cause biased results.Especially, distribution channels of online surveys related to problematic management issues need to be carefully chosen.In such cases, social media activity (link sharing, post assessment (e.g. likes or negative reactions) accompanying the discussion) need to be additionally monitored in order to detect uncontrolled lobbying of certain users’ opinions.

The yea-paradox: Cognitive bias in technology acceptance surveys

Technology acceptance is widely regarded as one of the most established, mature, and continuously relevant streams of information systems research. Its fundamental assumption is that technology acceptance scores accurately reflect users’ intentions, needs, and emotions toward the technology of interest. However, many studies (including this one) present evidence that challenges this assumption, suggesting that the link between reported scores and actual perception is more complex than often assumed. We aim to find out how cognitive biases influence self-reported technology acceptance scores. We draw on dialogical iterations between an exploratory case study at Cloud Corp, a multinational Software-as-a-Service provider and related literature on technology acceptance and cognitive bias. Our study reveals a paradoxical relationship between reported acceptance scores and actual perceptions: Despite providing high acceptance scores, users did not actually use the technology. Conversely, users who complained a lot by filing many support tickets were more engaged with the technology When yea-saying users meet yea-hearing designers, technology acceptance scores may be spuriously inflated, providing an exaggerated picture of users’ attitudes toward a technology. We call this the ‘yea-paradox’ because of its roots in people's tendency to agree half-heartedly (‘yea’), even when they privately disagree, to avoid more complex and potentially difficult conversations.

Using increasing hypothetical chronic wasting disease prevalence rates to explore hunter behavioral intention and hypothetical bias in surveys

ABSTRACT Deer hunter surveys commonly employ hypothetical chronic wasting disease (CWD) prevalence scenarios to measure behavioral intentions. These surveys may suffer from hypothetical bias (i.e. respondents describe unrealistic behaviors in hypothetical situations) and it is unclear how hypothetical bias affects results. We used an online-based survey of Arkansas deer hunters to (1) investigate how behavioral intentions change as CWD prevalence hypothetically varies, and (2) explore the presence of hypothetical bias in behavioral intention by comparing responses across hypothetical prevalence rates from those whose primary hunting counties had CWD detected versus not. We found undesired behaviors (e.g. hunting less often, hunting cessation) were unlikely to change substantially as prevalence increased, and assistance with some CWD monitoring and management behaviors may increase. We found some evidence of hypothetical bias in anticipated behavior change. Agencies should be aware that survey respondents may overstate behavioral intentions, and should consider this when making CWD management decisions.

Fish avoidance of ships during acoustic surveys tested with quiet uncrewed surface vessels

Acoustic surveys to estimate fish biomass and abundance are a major component of many fisheries monitoring programs. An important bias in acoustic surveys is that fish may avoid the survey vessel. Here, we utilized quiet uncrewed surface vessels (USVs) equipped with 120 kHz split beam echosounders to evaluate fish responses to motorized survey vessels. Two of these USVs were deployed in Lakes Michigan and Huron in summer 2021 and compared against three motorized vessels used in conventional fisheries acoustic surveys. Paired comparisons employed vessel-drone passes that provided the opportunity to observe fish response as vessels approached and then overtook their quiet USV counterparts. Sound originating from the vessels was primarily in the 10–1000 Hz range. Overall received sound pressure levels for ships at the closest pass were 100 dB (re 1 µPa @ 1 m) for one and 133–134 dB for the other two vessels. In contrast, sound originating from the USV was not detectable over ambient noise. We examined acoustic data from the USVs for potential changes in total acoustic backscatter, average target depth, and average in situ target strength as vessels approached. We observed weak evidence of an avoidance response by fish to the vessel with the loudest noise profile and highest survey speed but not for the other two vessels. We also compared acoustic data from 33 2-km transects surveyed by both vessels and the USVs, finding few differences between vessel and USV data for water depths between 5 and 80 m. Results from this work suggest that acoustics estimates of fishes in Lakes Michigan and Huron (primarily alewife, rainbow smelt, and bloater) are largely consistent among the vessels used in these two lakes for standard acoustic surveys and that fish avoidance is minimal in water depths > 5 meters.

Nonresponse trends in establishment panel surveys: findings from the 2001–2017 IAB establishment panel

Many household panel surveys have experienced decreasing response rates and increasing risk of nonresponse bias in recent decades, but trends in response rates and nonresponse bias in business or establishment panel surveys are largely understudied. This article examines both panel response rates and nonresponse bias in one of the largest and longest-running establishment panels, the IAB Establishment Panel. Response rate trends are reported over a 17-year period for each annual cohort and rich administrative data are used to evaluate changes in nonresponse bias and test hypotheses regarding short-term and long-term panel participation. The findings show that while cumulative panel response rates have declined over time, wave-to-wave reinterview rates have remained largely stable. Reinterview nonresponse bias has also remained stable, while cumulative nonresponse bias has consistently increased within all cohorts. Larger establishments and those that experienced an interviewer change or did not answer all survey questions (item nonresponse) in a previous wave were less likely to continue participating in the panel. These findings and their practical implications are discussed in conclusion.

Surveyor and analyst biases in forest density estimation from United States Public Land Surveys

AbstractAccurate forest density estimates based on United States Public Land Surveys have long been questioned because of doubts about randomness of both the surveyors' selection of witness trees and the underlying tree dispersion. This study analyzes the surveyor sampling of witness trees in six Midwestern states in the mid‐1800s. It develops universal methods for identification, quantification, and correction of bias, and then calculation of unbiased density. Applying these techniques produces unbiased site‐specific densities before Euro‐American settlement, which are the essential baseline for determining historic changes in forest structure. Previous analysts used untested assumptions, inaccurate estimators, unknown or unrealistic sampling designs, and omitted or poorly corrected surveyor bias, resulting in hundreds of unreliable density estimates. The surveyors' recording of the empirical distance, bearing, and diameter of witness trees documented the exact sampling design. The intended design and deviation from it are investigated with a combination of descriptive statistics, probability theory, computer simulations, analogue geometric models, and modern stand conditions. Herein, analyst bias is eliminated using the robust Morisita II density estimator matching the predominant sampling design of two trees in opposing semicircles. Six widespread surveyor biases deviating from the nearest tree to the corner are evaluated. Quadrant bias and diameter bias for medium‐sized trees are subsumed under newly framed design and small tree biases. Two novel surveyor positional biases (pair angle and near‐post) are introduced here. Previously recognized azimuthal and species biases are analyzed with new techniques. Widely postulated surveyor bias for certain species was found to be minimal. Bias correction and density estimation are applied in detail over 68 townships in northern Wisconsin. The estimated historical forest density in northern Wisconsin, corrected for bias and small tree truncation, averaged 323 trees/ha ≥20 cm. Over 80 Midwestern subregions, surveyors bypassed an estimated 17% of the nearest trees due to their position, resulting in an average bias correction of +24% over the base density. If censored trees below a 20‐cm “veil‐line” are considered, the surveyors bypassed 48% of the nearest trees >12.7 cm in diameter. This study resolves a 70‐year‐old conundrum of surveyor and analyst biases in historical density estimation.

OSSOS. XXIX. The Population and Perihelion Distribution of the Detached Kuiper Belt

The detached trans-Neptunian objects (TNOs) are those with semimajor axes beyond the 2:1 resonance with Neptune that are neither resonant nor scattering. Using the detached sample from the Outer Solar System Origins Survey (OSSOS) telescopic survey, we produce the first studies of their orbital distribution based on matching the orbits and numbers of the known TNOs after accounting for survey biases. We show that the detached TNO perihelion (q) distribution cannot be uniform but is instead better matched by two uniform components with a break near q ≈ 40 au. We produce parametric two-component models that are not rejectable by the OSSOS data set and estimate that there are 36,000 detached TNOs with absolute magnitudes H r < 8.66 (D ≳ 100 km) and semimajor axes 48 au < a < 250 au (95% confidence limits). Although we believe that these heuristic two-parameter models yield a correct population estimate, we then use the same methods to show that the perihelion distribution of a detached disk created by a simulated rogue planet matches the q distribution even better, suggesting that the temporary presence of other planets in the early solar system is a promising model to create today’s large semimajor axis TNO population. This cosmogonic simulation results in a detached TNO population estimate of 48,000. Because this illustrates how difficult-to-detect q > 50 au objects are likely present, we conclude that there are (5 ± 2) × 104 dynamically detached TNOs, roughly twice as many as in the entire trans-Neptunian hot main belt.

Reducing vertical bias and error in tidal marsh digital elevation models with machine learning and LiDAR derivatives

The vertical bias and uncertainty of LiDAR-derived elevation data in tidal marshes is a major challenge for researchers in these critical ecosystems. Small positive bias errors in land surface elevation can lead to large underestimates of coastal inundation. Previous studies have shown that the bias and uncertainty of elevation data can be reduced using a variety of statistical techniques. However, many studies cover a relatively small extent, cover a large extent at coarse resolution, or rely on local data products that may not be widely available or are of unknown quality. This study aimed to develop and compare bias reduction approaches for digital elevation models (DEMs) using LiDAR derivatives in tidal marshes along Delaware Bay, Delaware, USA, a region that experiences high rates of relative sea-level rise and frequent coastal flooding. In this study, several statistical and machine-learning approaches were evaluated based on their ability to reduce vertical DEM error using field GPS survey data and LiDAR and terrain derivatives as inputs. The evaluated approaches for reducing DEM error included ensembles of multiple linear regressions, kernel-K Nearest Neighbors, gradient boosted regression trees, random forests, and deep neural networks (DNNs), which were compared based on statistical performance metrics. An ensemble of deep neural networks performed best at removing vertical DEM bias and reducing DEM uncertainty, though other approaches also performed well. GPS survey data indicated a mean absolute error, mean bias, and root mean square error in GPS surveys of 11.9 cm, 10.9, and 14.8 cm, respectively, in the original DEM. These error metrics were reduced to 3.5 cm, −0.2 cm, and 5.1 cm in the DNN ensemble-corrected DEM. The DNN ensemble was then applied to all tidal marshes throughout the study area. This corrected DEM clearly showed how interpretations of marsh platform inundation based on uncorrected DEM surfaces could be misguided. The approach used in this study only requires LiDAR-derived products and field surveys, so it may be applied readily in other regions.

The second set of pulsar discoveries by CHIME/FRB/Pulsar: 14 rotating radio transients and 7 pulsars

ABSTRACT The Canadian Hydrogen Mapping Experiment (CHIME) is a radio telescope located in British Columbia, Canada. The large field of view allows CHIME/FRB to be an exceptional pulsar and rotating radio transient (RRAT) finding machine, despite saving only the metadata of incoming Galactic events. We have developed a pipeline to search for pulsar/RRAT candidates using density-based spatial clustering of applications with noise (dbscan), a clustering algorithm. Follow-up observations are then scheduled with the more sensitive CHIME/Pulsar instrument capable of near-daily high-time resolution spectra observations. We have developed the CHIME/Pulsar Single Pulse Pipeline to automate the processing of CHIME/Pulsar search-mode data. We report the discovery of 21 new Galactic sources, with 14 RRATs, 6 isolated long-period pulsars, and 1 binary system. Owing to CHIME/Pulsar’s observations, we have obtained timing solutions for 8 of the 14 RRATs along with all the regular pulsars and the binary system. Notably, we report that the binary system is in a long orbit of 412 d with a minimum companion mass of 0.1303 solar masses and no evidence of an optical companion within 10″ of the pulsar position. This highlights that working synergistically with CHIME/FRB’s large survey volume CHIME/Pulsar can obtain arc second localizations for low-burst rate RRATs through pulsar timing. We find that the properties of our newly discovered RRATs are consistent with those of the presently known population. They tend to have lower burst rates than those found in previous surveys, which is likely due to survey bias rather than the underlying population.

Gender and racial biases in Press Ganey patient satisfaction surveys

Gender and racial biases in Press Ganey patient satisfaction surveys

DP01 Survey on the management of dysplastic naevus

Abstract Although dysplastic naevus (DN) is a frequent diagnosis, there is significant controversy on its definition and gradation and its relationship to melanoma risk. Therefore, there is no consensus over optimal management, and surveys from Australia, Canada and the USA have shown variation in clinical practice. Dysplastic naevus with involved histological margins has not been shown to progress to melanoma. To assess UK management of DN depending on the grade of atypia and reported excision margin, we circulated a questionnaire via email to dermatologists in Scotland, north Wales, Liverpool and Manchester. Seventy-two responded, of whom 46 (64%) were consultants. The cohort was generally highly experienced, with 10 (14%) responders with 1–5 years’ dermatology experience, 15 (21%) with 6–10 years, 27 (37%) with 11–20 years and 20 (28%) with &amp;gt; 20 years’ experience. Management of DN varied greatly. For mild DN excised with &amp;lt; 1-mm margin, 59 (82%) agreed that no further management was required. Opinions were divided when margins were involved in mild DN, with 14 (19%) agreeing that no further management was required, 42 (58%) choosing to re-excise and six (8%) to re-excising only any residual pigmentation. Most dermatologists reassured patients with moderately DN with margins &amp;lt; 1 mm (49, 68%); 18 (25%) preferred to re-excise. For moderate DN with involved margins, six respondents (8%) felt that no further management was necessary and 54 (75%) would re-excise. For severe DN with margins &amp;gt; 1 mm, 16 (22%) would still re-excise and six (8%) would agree management in the multidisciplinary team meeting (MDT). For severe DN with margins &amp;lt; 1 mm, 41 (57%) would re-excise and seven (10%) would agree management in the MDT. Lastly, for severe DN with involved margins, 62 (86%) would re-excise and seven (10%) would agree management in the MDT. This study has shown comparable results with those from Australia and the USA, whereby most dermatologists would re-excise moderate and severe DN with involved margins. Significant numbers indicated that they would re-excise in the case of close histological margins: this is unnecessary, and more standardized management may reduce expenditure. Limitations of the study include the usual potential sources of bias in a questionnaire survey; it should be noted that reported behaviour may differ from actual practice. However, this is the first UK study on this common clinical scenario. The move to report DN as low- or high-grade may help to reduce variation, but our survey suggests that some patients are currently undergoing unnecessary procedures.

Data weighting: An iterative process linking surveys, data synthesis, and population models to evaluate mis-specification

Integrated stock assessments specify a distribution for multiple data types, and these distributions control the relative leverage assigned to each datum. A decade of research has demonstrated that (1) proper data weighting is necessary to avoid bias resulting from overweighting noisy age- and length-composition data; (2) sampling data can be pre-processed to estimate the likely sampling variance for composition data; and (3) using random effects to estimate time-varying parameters can improve the fit to data while also changing statistical leverage, and thereby serve a similar role to reweighting data. However, there are also unresolved questions including: (A) Is it more appropriate to model age and length data as proportions-at-age and as an index for the total, or as a series of indices-at-age? (B) Are correlated residuals appropriately addressed via data weighting or do they require additional model changes (i.e., time-varying parameters)? (C) How to efficiently communicate information about sampling imprecision and model errors between sampling and stock-assessment teams? (D) How does model-based expansion of sampling data affect data weighting? and (E) How to address alternative hypotheses about factors driving poor fit to data? Here, we argue that stock assessment errors can be classified using four categories: sampling bias (e.g., changes in survey coverage), sampling imprecision (e.g., finite sample sizes), assessment model bias (e.g., incorrect demographic assumptions) and assessment model imprecision (e.g., random effects). This categorization has several implications with resulting practical recommendations. For example, we define Percent Excess Variance (PEV) from the ratio of input sample size (the measured variance of sampling imprecision) and effective sample sizes (the variance of assessment-model residuals). We propose calculating PEV as standardized diagnostic measuring the net effect of survey bias and assessment model bias and imprecision. We demonstrate PEV in a simulation experiment fitted using the Woods Hole Assessment Model (WHAM) conditioned upon Gulf of Alaska walleye pollock, where unacknowledged variation in fishery selectivity results in a PEV of 77 % and this is eliminated when correctly specifying a time-varying estimation model. We also argue that model-based expansion of data inputs using auxiliary information can mitigate sampling bias, while also measuring sampling imprecision for spatially unrepresentative surveys. Similarly, including random effects can mitigate model bias while increasing model imprecision when the demographic model has little explanatory power. Finally, we observe that down-weighting compositional data for a given fleet fails to propagate information about model residuals when interpreting abundance indices or reference points for that same fleet. When PEV is large for important fleets, we therefore encourage focused research to explain the sources of these errors rather than simply downweighting without propagating information about residuals. However, we acknowledge a continuing role for automated data weighting for less important fleets, although we recommend explicit hypotheses about potential sources of errors in those cases.