Dealing with false positive risk as an indicator of misperceived effectiveness of conservation interventions.

Ji-Zhong Wan,Igor Khorozyan

doi:10.1371/journal.pone.0255784

Abstract

As human pressures on the environment continue to spread and intensify, effective conservation interventions are direly needed to prevent threats, reduce conflicts, and recover populations and landscapes in a liaison between science and conservation. It is practically important to discriminate between true and false (or misperceived) effectiveness of interventions as false perceptions may shape a wrong conservation agenda and lead to inappropriate decisions and management actions. This study used the false positive risk (FPR) to estimate the rates of misperceived effectiveness of electric fences (overstated if reported as effective but actually ineffective based on FPR; understated otherwise), explain their causes and propose recommendations on how to improve the representation of true effectiveness. Electric fences are widely applied to reduce damage to fenced assets, such as livestock and beehives, or increase survival of fenced populations. The analysis of 109 cases from 50 publications has shown that the effectiveness of electric fences was overstated in at least one-third of cases, from 31.8% at FPR = 0.2 (20% risk) to 51.1% at FPR = 0.05 (5% risk, true effectiveness). In contrast, understatement reduced from 23.8% to 9.5% at these thresholds of FPR. This means that truly effective applications of electric fences were only 48.9% of all cases reported as effective, but truly ineffective cases were 90.5%, implying that the effectiveness of electric fences was heavily overstated. The main reasons of this bias were the lack of statistical testing or improper reporting of test results (63.3% of cases) and interpretation of marginally significant results (p < 0.05, p < 0.1 and p around 0.05) as indicators of effectiveness (10.1%). In conclusion, FPR is an important tool for estimating true effectiveness of conservation interventions and its application is highly recommended to disentangle true and false effectiveness for planning appropriate conservation actions. Researchers are encouraged to calculate FPR, publish its constituent statistics (especially treatment and control sample sizes) and explicitly provide test results with p values. It is suggested to call the effectiveness “true” if FPR < 0.05, “suggestive” if 0.05 ≤ FPR < 0.2 and “false” if FPR ≥ 0.2.

Highlights

Large-scale and ever accelerating pressures of human activities on the environment urge for the implementation of practical, socially acceptable and effective interventions in biodiversity conservation [1,2]
Electric fences were reported to be more effective in reduction-aimed cases and less effective in addition-aimed cases (χ2 = 9.751, df = 1, p = 0.002), but the effect size of this relationship was low to medium (Cramer’s V = 0.299)
This study has demonstrated that the effectiveness of electric fences was overstated in at least one-third of cases described in the scientific literature (Table 1; Fig 1), regardless of the species and purposes of electric fencing

Summary

Introduction

Large-scale and ever accelerating pressures of human activities on the environment urge for the implementation of practical, socially acceptable and effective interventions in biodiversity conservation [1,2]. Translocation of conflict-causing predators to remote areas has been perceived and widely used as an effective intervention even though in practice it can be costly, cause high mortality of captured animals, or trigger more conflict [14,15] As another example, underestimation of adverse impacts of invasive species on native biodiversity may hinder ecological research, bias knowledge and applications, and delay restoration actions [16,17]. Misperceived effectiveness can be overstated when an intervention is reported as effective but statistically it is not, or understated when the scenario is opposite This kind of uncertainty is incorporated in statistical estimates [18,19], but is often ignored by conservation scientists yet the concept of false positives and negatives is rather common in species identification, genetics, distribution and monitoring [20,21,22]. False positive risk provides a clue to the understanding of which intervention applications are truly effective and which are ineffective

Objectives

Methods

Results

Conclusion