A general framework for determining cutoff values to select pollen analogs with dissimilarity metrics in the modern analog technique

Abstract

The modern analog technique (MAT) is a quantitative calibration tool for using modern pollen assemblages to interpret fossil pollen assemblages for vegetation and climate reconstruction. When the MAT is applied using multivariate distance metrics, a cutoff value for the metric is often used to determine the presence/absence of analogs in a modern pollen reference set. Two kinds of error arise when a cutoff value is used: (1) false positive error, which occurs when analogy is falsely determined to exist between the vegetation (or other parameter) of a sample of interest and that of a sample in the reference set; and (2) false negative error, which occurs when analogy is falsely determined not to exist. The existing literature focuses primarily on examining cutoff thresholds from the perspective of reducing false positive error, with relatively little attention paid to false negative error and to the inherent trade-off between the two errors. This paper sets forth a general analytical framework for determining cutoff thresholds that minimize the joint occurrence of the two errors, and employs the squared chord distance metric with a newly developed reference set of modern pollen surface samples from southern California, USA, as a demonstration case. It also examines the nature of the tradeoffs that occur if an analyst decides to accept increased risk (beyond the joint minimum) of one of the kinds of error for additional reduction of the other. An asymmetric tradeoff in these risks above and below the joint error minimizing cutoff(s) is described (a more rapid proportionate increase of false negatives at cutoffs below the joint minimum in relation to the proportionate increase of false positives at cutoffs above it), which is controlled by the relative variances of the distributions of like- and non-like-vegetation sample comparisons in terms of the distance metric. This asymmetry is found to be general among sample sets reported using the squared chord distance, but is not general across other distance metrics.