Immunoassays are not immune to errors: Examples from two studies of steroid output from freshwater trout farms

Abstract

A “reproducibility crisis” is widespread across scientific disciplines, where results and conclusions of studies are not supported by subsequent investigation. Here we provide a steroid immunoassay example where human errors generated unreproducible results and conclusions. Our study was triggered by a scientific report citing abnormally high concentrations (means of 4–79 ng L−1) of three natural sex steroids [11-ketotestosterone (11-KT), testosterone (T) and oestradiol (E2)] in water samples collected from two UK rivers over 4 years (2002–6). Furthermore, the data suggested that trout farms were a major source because reported steroid concentrations were 1.3–6 times higher downstream than upstream. We hypothesised that the reported levels were erroneous due to substances co-extracted from the water causing matrix effects (i.e. “false positives”) during measurement by enzyme-linked immunoassay (EIA). Thus, in collaboration with three other groups (including the one that had conducted the 2002–6 study), we carried out field sampling and assaying to examine this hypothesis. Water samples were collected in 2010 from the same sites and prepared for assay using an analogous method [C18 solid phase extraction (SPE) followed by extract clean-up with aminopropyl SPE]. Additional quality control (“spiked” and “blank”) samples were processed. Water extracts were assayed for steroids using radioimmunoassay (RIA) as well as EIA. Although there were statistically significant differences between EIA and RIA (and laboratories), there was no indication of matrix effects in the EIAs. Both the EIAs and RIAs (uncorrected for recovery) measured all three natural steroids at <0.6 ng L−1 in all river water samples, indicating that the trout farms were not a significant source of natural steroids. The differences between the two studies were considerable: E2 and T concentrations were ca. 100-fold lower and 11-KT ca. 1000-fold lower than those reported in the 2002–6 study. In the absence of evidence for any marked changes in husbandry practice (e.g. stock, diet) or environmental conditions (e.g. water flow rate) between the study periods, we concluded that calculation errors were probably made in the first (2002–6) study associated with confusion between extract and water sample concentrations. The second (2010) study also had several identified examples of calculation error (use of an incorrect standard curve; extrapolation below the minimum standard; confusion of assay dilutions during result work-up; failure to correct for loss during extraction) and an example of sample contamination. Similar and further errors have been noted in other studies. It must be recognised that assays do not provide absolute measurements and are prone to a variety of errors, so published steroid levels should be viewed with caution until independently confirmed.