Occupancy models for data with false positive and false negative errors and heterogeneity across sites and surveys

Context In metapopulations, colonisation is the result of dispersal from neighbouring occupied patches, typically juveniles dispersing from natal to breeding sites. When occupancy dynamics are dispersal driven, occupancy should refer to the presence of established, breeding populations. The detection of transient individuals at sites that are, by definition, unoccupied (i.e. false positive detections), may result in misleading conclusions about metapopulation dynamics. Until recently, the issue of false positives has been considered negligible and current efforts to account for such error have been restricted to the context of species misidentification. However, the detection of transient individuals visiting multiple sites while dispersing is a distinct source of false positives that can bias estimates of occupancy because visited sites do not contribute to metapopulation dynamics in the same way as do sites occupied by established, reproducing populations. Although transient-induced false positive error presents a challenge to occupancy studies aiming to account for all sources of detection error and estimate occupancy without bias, accounting for it has received little attention. Aims Using a novel application of an existing occupancy model, we sought to account for false positives that result from transient individuals being observed at truly unoccupied sites (i.e. where no establishment has occurred). Methods We applied a Bayesian multi-season occupancy model correcting for false negative and false positive errors, to 3 years of detection or non-detection data from a metapopulation of water voles, Arvicola amphibious, in which both types of patch-state misclassification are suspected. Key results We provide evidence that transient individuals can cause false positive detection errors. We then demonstrate the flexibility of the occupancy model to account for both false negative and false positive detection errors beyond the typical application to species misidentification. Accounting for both types of observation error reduces the bias in estimates of occupancy and avoids misleading conclusions about the status of (meta) populations by allowing for the distinction to be made between resident and transient occupancy. Conclusion In many species, transience may result in patch-state misclassification which needs to be accounted for so as to draw correct inference about metapopulation status. Making the distinction between occupancy by established populations and visitation by transients will influence how we interpret patch occupancy dynamics, with important implications for the management of wildlife. Implications The ability to estimate occupancy free of bias induced by false positive detections can help ensure that downward trends in occupancy are detected despite such declines being accompanied by increasing frequency of transients associated with, for example, reductions in mate availability or failure to establish. Our approach can be applied to any occupancy study in which false positive detections are suspected because of the behaviour of the focal species.

The recent surge in the development and application of species occurrence models has been associated with an acknowledgment among ecologists that species are detected imperfectly due to observation error. Standard models now allow unbiased estimation of occupancy probability when false negative detections occur, but this is conditional on no false positive detections and sufficient incorporation of explanatory variables for the false negative detection process. These assumptions are likely reasonable in many circumstances, but there is mounting evidence that false positive errors and detection probability heterogeneity may be much more prevalent in studies relying on auditory cues for species detection (e.g., songbird or calling amphibian surveys). We used field survey data from a simulated calling anuran system of known occupancy state to investigate the biases induced by these errors in dynamic models of species occurrence. Despite the participation of expert observers in simplified field conditions, both false positive errors and site detection probability heterogeneity were extensive for most species in the survey. We found that even low levels of false positive errors, constituting as little as 1% of all detections, can cause severe overestimation of site occupancy, colonization, and local extinction probabilities. Further, unmodeled detection probability heterogeneity induced substantial underestimation of occupancy and overestimation of colonization and local extinction probabilities. Completely spurious relationships between species occurrence and explanatory variables were also found. Such misleading inferences would likely have deleterious implications for conservation and management programs. We contend that all forms of observation error, including false positive errors and heterogeneous detection probabilities, must be incorporated into the estimation framework to facilitate reliable inferences about occupancy and its associated vital rate parameters.

Objective. High frequency oscillations (HFOs) are a promising biomarker of tissue that instigates seizures. However, ambiguous data and random background fluctuations can cause any HFO detector (human or automated) to falsely label non-HFO data as an HFO (a false positive detection). The objective of this paper was to identify quantitative features of HFOs that distinguish between true and false positive detections. Approach. Feature selection was performed using background data in multi-day, interictal intracranial recordings from ten patients. We selected the feature most similar between randomly selected segments of background data and HFOs detected in surrogate background data (false positive detections by construction). We then compared these results with fuzzy clustering of detected HFOs in clinical data to verify the feature’s applicability. We validated the feature is sensitive to false versus true positive HFO detections by using an independent data set (six subjects) scored for HFOs by three human reviewers. Lastly, we compared the effect of redacting putative false positive HFO detections on the distribution of HFOs across channels and their association with seizure onset zone (SOZ) and resected volume (RV). Main results. Of the 15 analyzed features, the analysis selected only skewness of the curvature (skewCurve). The feature was validated in human scored data to be associated with distinguishing true and false positive HFO detections. Automated HFO detections with higher skewCurve were more focal based on entropy measures and had increased localization to both the SOZ and RV. Significance. We identified a quantitative feature of HFOs which helps distinguish between true and false positive detections. Redacting putative false positive HFO detections improves the specificity of HFOs as a biomarker of epileptic tissue.

Detection/nondetection data are widely collected by ecologists interested in estimating species distributions, abundances, and phenology, and are often imperfect. Recent model development has focused on accounting for both false-positive and false-negative errors given evidence that misclassification is common across many sampling protocols. To date, however, model-based solutions to false-positive error have largely addressed occupancy estimation. We describe a generalized model structure that allows investigators to account for false-positive error in detection/nondetection data across a broad range of ecological parameters and model classes, and demonstrate that previously developed model-based solutions are special cases of the generalized model. Simulation results demonstrate that estimators for abundance and migratory arrival time ignoring false-positive error exhibit severe (20-70%) relative bias even when only 5-10% of detections are false positives. Bias increased when false-positive detections were more likely to occur at sites or within occasions in which true positive detections were unlikely to occur. Models accounting for false-positive error following the site-confirmation or observation-confirmation designs generally reduced bias substantially, even when few detections were confirmed as true or false positives or when the process model for false-positive error was misspecified. Results from an empirical example focusing on gray fox (Urocyon cinereoargenteus) abundance in Wisconsin, USA reinforce concerns that biases induced by false-positive error can also distort spatial predictions often used to guide decision making. Model sensitivity to false-positive error extends well beyond occupancy estimation, but encouragingly, model-based solutions developed for occupancy estimators are generalizable and effective across a range of models widely used in ecological research.

Occupancy models are commonly used with motion‐sensitive camera data to estimate patterns of species occurrence while accounting for false negative detection error (i.e., the species is present but not detected). False positive detection error (i.e., the species is not present but is detected) is present in camera data sets, especially when morphologically similar species co‐occur. Researchers use different approaches to address this problem: ignore the potential for false positive detections, remove all ambiguous detections and treat them as non‐detections, or model false positive detection error by dividing detections into ambiguous detections (could be true or false positives) and unambiguous detections (true positives). We performed a simulation study to compare these 3 strategies. To implement these modeling strategies, detections must be classified as ambiguous or unambiguous, or all ambiguous detections must be re‐classified as non‐detections. We also performed a simulation study to assess the impact of researcher confidence in the designation of ambiguous and unambiguous detections. Ignoring false positive detection error resulted in biased parameter estimates, whereas removing ambiguous detections and modeling false positive detections resulted in similar estimates of occupancy probability (ψ) in most situations. Researcher over‐confidence (i.e., the tendency for observers to overestimate their own ability) positively biased estimates of ψ. Moderate under‐confidence did not increase bias or decrease precision in estimates of ψ. Consistent with the patterns observed in simulations, analysis of example data from a chipmunk (Neotamias minimus atristriatus) population in the Sacramento Mountains of south‐central New Mexico during 2019 indicated that removing ambiguous detections and modeling false positives resulted in similar estimates of ψ and that over‐confidence biased estimates of ψ. Our results expand on previous literature, suggesting that removing ambiguous detections provides similar estimates of occupancy compared to modeling false positives in many scenarios, and emphasizing the importance of the designation of ambiguous and unambiguous detections. We provide guidance on simple methods to define ambiguous and unambiguous detections, thus mitigating the chances for erroneous inferences.

Background: Next-generation sequencing (NGS) has emerged as a vital diagnostic tool for various diseases; however, errors associated with NGS present significant challenges to its clinical implementation. Specifically, in cancer diagnostics, the analysis of low-level mutations is complicated by contamination from normal cells and tumor heterogeneity. Results: In targeted NGS (T-NGS) analyses involving reference-standard samples consisting of mixtures of homozygous Hydatidiform mole DNA and blood genomic DNA at varying ratios from four certified NGS service providers, we observed considerable variability in both the lower detection limits of variants (16.3-fold difference, ranging from 1.51% to 24.66%) and false positive (FP) error rates (4280-fold difference, ranging from 5.814 × 10^-4 to 1.359 × 10^-7). The commercially available Dragen system for bioinformatics analysis reduced FP errors in results from companies BB and CC; however, inherent errors from the raw NGS data remained. Adjustments to bioinformatics parameters aimed at increasing sensitivity (by less than two times) substantially elevated FP error rates (by 610 to 8200 times). Additionally, we identified issues such as biased base calling during bioinformatics analysis and a high frequency of false negative (FN) errors in HLA regions. Conclusion: T-NGS results from certified NGS service providers reveal considerable variability in sensitivity and FP error rates, underscoring the necessity for enhanced quality control measures in the clinical application of T-NGS. Furthermore, our study suggests that mixtures of homozygous and heterozygous DNA can serve as effective reference-standard materials for T-NGS quality control. Citation Format: Chunghwan Hong, Youngbeen Moon, Hye Won Choi, Eun-Kyung Kang, Jong-Kwang Kim, Young-Ho Kim, Dong-eun Lee, Tae-Min Kim, Seong Gu Heo, Namshik Han, Kyeong-Man Hong. Evaluation of false positive and false negative errors in targeted next generation sequencing [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5058.

Environmental DNA (eDNA) sampling is prone to both false-positive and false-negative errors. We review statistical methods to account for such errors in the analysis of eDNA data and use simulations to compare the performance of different modelling approaches. Our simulations illustrate that even low false-positive rates can produce biased estimates of occupancy and detectability. We further show that removing or classifying single PCR detections in an ad hoc manner under the suspicion that such records represent false positives, as sometimes advocated in the eDNA literature, also results in biased estimation of occupancy, detectability and false-positive rates. We advocate alternative approaches to account for false-positive errors that rely on prior information, or the collection of ancillary detection data at a subset of sites using a sampling method that is not prone to false-positive errors. We illustrate the advantages of these approaches over ad hoc classifications of detections and provide practical advice and code for fitting these models in maximum likelihood and Bayesian frameworks. Given the severe bias induced by false-negative and false-positive errors, the methods presented here should be more routinely adopted in eDNA studies.

Existing approaches to trading off false positive versus false negative errors in input recognition are based on imprecise ideas of how these errors affect user experience that are unlikely to hold for all situations. To inform dynamic approaches to setting such a tradeoff, two user studies were conducted on how relative preference for false positive versus false negative errors is influenced by differences in the temporal cost of error recovery, and high-level task factors (time pressure, multi-tasking). Participants completed a tile selection task in which false positive and false negative errors were injected at a fixed rate, and the temporal cost to recover from each of the two types of error was varied, and then indicated a preference for one error type or the other, and a frustration rating for the task. Responses indicate that the temporal costs of error recovery can drive both frustration and relative error type preference, and that participants exhibit a bias against false positive errors, equivalent to ∼1.5 seconds or more of added temporal recovery time. Several explanations for this bias were revealed, including that false positive errors impose a greater attentional demand on the user, and that recovering from false positive errors imposes a task switching cost.

A model was developed to understand the effects of spatial resolution and Signal to Noise ratio on the detection and tracking performance of wide-field, diffraction-limited electro-optic and infrared motion imagery systems. False positive detection probability and false positive rate per frame were calculated as a function of target-to-background contrast and object size. Results showed that moving objects are fundamentally more difficult to detect than stationary objects because SNR for fixed objects increases and false positive probability detection rates diminish rapidly with successive frames whereas for moving objects the false detection rate remains constant or increases with successive frames. The model specifies that the desired performance of a detection system, measured by the false positive detection rate, can be achieved by image system designs with different combinations of SNR and spatial resolution, usually requiring several pixels resolving the object; this capability to tradeoff resolution and SNR enables system design trades and cost optimization. For operational use, detection thresholds required to achieve a particular false detection rate can be calculated. Interestingly, for moderate size images the model converges to the Johnson Criteria. Johnson found that an imaging system with an SNR >3.5 has a probability of detection >50% when the resolution on the object is 4 pixels or more. Under these conditions our model finds the false positive rate is less than one per hundred image frames, and the ratio of the probability of object detection to false positive detection is much greater than one. The model was programmed into Matlab to generate simulated images frames for visualization.

BackgroundNext-generation sequencing (NGS) has become an indispensable diagnostic tool across various diseases. However, sequencing and analysis errors remain major barriers to clinical implementation. In cancer diagnostics, detecting low-level somatic variants is particularly challenging due to tumor heterogeneity and contamination from normal cells.ResultsWe assess targeted next-generation sequencing (T-NGS) performance using reference-standard DNA mixtures of homozygote hydatidiform mole and heterozygote blood DNA at varying ratios, analyzed by certified NGS providers. Analytical sensitivity differs by up to 13.9-fold, and false positive (FP) error rates vary up to 615-fold, depending on provider and pipeline. For identical raw data, DRAGEN and the in-house pipeline differ by up to 36.3-fold in FP error rates. Moderately recurrent FP-prone alleles, although representing only 5.37% of all FP sites, contribute to 36.7% of total FP errors in the Geninus in-house result. Among 22 discordant variant calls between DRAGEN and in-house analyses, more than half of them are not confirmed by single base extension assays, indicating likely false positives. Compared to DRAGEN, a conventional BWA + GATK Mutect2 pipeline maintains equivalent sensitivity but produces a 4-fold increase in FP errors, along with a notable enrichment of recurrent FP-prone alleles.ConclusionsT-NGS results from certified providers exhibit substantial variability in both sensitivity and FP error rates. Conventional pipelines not only increase FP errors but also accumulate recurrent FP-prone alleles. These findings underscore the urgent need for standardized pipelines and rigorous quality control measures to ensure the reliability of T-NGS in clinical diagnostics.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13059-025-03882-2.

The neural network models for IDS based on the asymmetric costs of false negative errors and false positive errors

False positive errors are a significant component of many ecological data sets, which in combination with false negative errors, can lead to severe biases in conclusions about ecological systems. We present results of a field experiment where observers recorded observations for known combinations of electronically broadcast calling anurans under conditions mimicking field surveys to determine species occurrence. Our objectives were to characterize false positive error probabilities for auditory methods based on a large number of observers, to determine if targeted instruction could be used to reduce false positive error rates, and to establish useful predictors of among-observer and among-species differences in error rates. We recruited 31 observers, ranging in abilities from novice to expert, who recorded detections for 12 species during 180 calling trials (66,960 total observations). All observers made multiple false positive errors, and on average 8.1% of recorded detections in the experiment were false positive errors. Additional instruction had only minor effects on error rates. After instruction, false positive error probabilities decreased by 16% for treatment individuals compared to controls with broad confidence interval overlap of 0 (95% CI:--46 to 30%). This coincided with an increase in false negative errors due to the treatment (26%;--3 to 61%). Differences among observers in false positive and in false negative error rates were best predicted by scores from an online test and a self-assessment of observer ability completed prior to the field experiment. In contrast, years of experience conducting call surveys was a weak predictor of error rates. False positive errors were also more common for species that were played more frequently but were not related to the dominant spectral frequency of the call. Our results corroborate other work that demonstrates false positives are a significant component of species occurrence data collected by auditory methods. Instructing observers to only report detections they are completely certain are correct is not sufficient to eliminate errors. As a result, analytical methods that account for false positive errors will be needed, and independent testing of observer ability is a useful predictor for among-observer variation in observation error rates.

Purpose: We are developing a model‐based framework for the detection of spiculated masses on mammography, the current performance of which is 88% sensitivity with 2.7 false positives per image. The goal of this study is to identify features that uniquely characterize the true positive (TP) and false positive (FP) detections from this system. Method and Materials: A two alternative‐forced‐choice observer experiment was used. For each of the 47 cases of spiculated masses, the true lesion location and the highest‐ranked FP from the CAD were shown to the observer (radiologist). The observer was not told if he/she was looking at the true lesion location or FP and the order in which these were shown was random. The radiologist visually inspected these images to pick the one that corresponds to the true lesion location. We then compared this decision to the ground truth to analyze the TP and FP detections that were incorrectly determined by the radiologist. Results: The radiologist correctly identified the true lesion location and false positive for all 47 cases. These data suggest that the radiologist could easily dismiss the false positives marked by the CAD algorithm. Conclusion: While false positive detections remain a challenge with the current version of our model‐based framework for the detection of spiculated masses on mammography, this study implies that those false positives may be recognizable as such by the radiologists and suggests future directions for reducing the number of false positive marks.

To measure the Medicaid undercount and analyze response error in the 2007-2011 Current Population Survey Annual Social and Economic Supplement (CPS ASEC). Medicaid Statistical Information System (MSIS) 2006-2010 enrollment data linked to the 2007-2011 CPS ASEC person records. By linking individuals across datasets, we analyze false-negative error and false-positive error in reports of Medicaid enrollment. We use regression analysis to identify factors associated with response error in the 2011 CPS ASEC. We find that the Medicaid undercount in the CPS ASEC ranged between 22 and 31% from 2007 to 2011. In 2011, the false-negative rate was 40%, and 27% of Medicaid reports in CPS ASEC were false positives. False-negative error is associated with the duration of enrollment in Medicaid, enrollment in Medicare and private insurance, and Medicaid enrollment in the survey year. False-positive error is associated with enrollment in Medicare and shared Medicaid coverage in the household. Survey estimates of Medicaid enrollment and estimates of the uninsured population are affected by both false-positive response error and false-negative response error, and these response errors are non-random.

Surveillance programmes are essential for detecting emerging pathogens and often rely on molecular methods to make inference about the presence of a target disease agent. However, molecular methods rarely detect target DNA perfectly. For example, molecular pathogen detection methods can result in misclassification (i.e. false positives and false negatives) or partial detection errors (i.e. detections with ‘ambiguous’, ‘uncertain’ or ‘equivocal’ results). Then, when data are to be analysed, these partial observations are either discarded or censored; this, however, disregards information that could be used to make inference about the true state of the system. There is a critical need for more direction and guidance related to how many samples are enough to declare a unit of interest ‘pathogen free’. Here, we develop a Bayesian hierarchal framework that accommodates false negative, false positive and uncertain detections to improve inference related to the occupancy of a pathogen. We apply our modelling framework to a case study of the fungal pathogen Pseudogymnoascus destructans (Pd) identified in Texas bats at the invasion front of white‐nose syndrome. To improve future surveillance programmes, we provide guidance on sample sizes required to be 95% certain a target organism is absent from a site. We found that the presence of uncertain detections increased the variability of resulting posterior probability distributions of pathogen occurrence, and that our estimates of required sample size were very sensitive to prior information about pathogen occupancy, pathogen prevalence and diagnostic test specificity. In the Pd case study, we found that the posterior probability of occupancy was very low in 2018, but occupancy probability approached 1 in 2020, reflecting increasing prior probabilities of occupancy and prevalence elicited from the site manager. Our modelling framework provides the user a posterior probability distribution of pathogen occurrence, which allows for subjective interpretation by the decision‐maker. To help readers apply and use the methods we developed, we provide an interactive RShiny app that generates target species occupancy estimation and sample size estimates to make these methods more accessible to the scientific community (https://rmummah.shinyapps.io/ambigDetect_sampleSize). This modelling framework and sample size guide may be useful for improving inferences from molecular surveillance data about emerging pathogens, non‐native invasive species and endangered species where misclassifications and ambiguous detections occur.