Meta‐analysis results are unlikely to be biased by differences in variance and replication between ecological lab and field studies

Abstract

A recurrent question in meta‐analyses is the validity of including both field and laboratory studies in a single analysis, given the differences in characteristics such as the duration or size of these experiments. In particular, are field studies consistently more variable, longer in duration, or do they differ from laboratory studies in other important ways? We examined these questions using an unusually large and comprehensive data set on grazing effects on benthic microalgae, consisting of 865 experiments including both field and lab studies and using marine, lotic or lentic communities of small benthic primary producers. We found there was greater variation in field than laboratory studies, but that this difference explained only 0.5% of the variance in the coefficient of variation. Field studies were longer, on average, than laboratory studies, but experiments in the two systems did not differ consistently in other ecological or design parameters. Thus, the common assumption that field studies have higher variances than laboratory studies, while true, explained only a tiny proportion of the heterogeneity in variation. Therefore, perhaps surprisingly, any conclusions from research syntheses would be unlikely to be affected by consistent differences in variance (or other parameters we examined) between the laboratory and field studies.