Systematic Underestimation of Tree Branch Lengths by Lake's Operator Metrics: An Effect of Position-dependent Substitution Rates

Abstract

Lake has proposed a method, which he calls “operator metrics,” for estimating branch lengths (as transversion differences) on an unrooted phylogenetic tree of nucleotide sequences. Examination of published applications of operator metrics reveals that it often assigns branch lengths that sum to less than the observed sequence differences, indicating a substantial systematic bias. The problem occurs in all sets of sequence data examined. It is shown analytically that operator metrics does not yield the claimed estimate of transversion sequence differences when sequence positions differ in their nucleotide substitution rate, in which case the method underestimates tree branch lengths. The site-to-site variations in substitution rate that have been characterized by previous authors are of sufficient magnitude to explain the problems observed in the operator-metrics branch length estimates. Transversion substitutions estimated using Kimura’s two-parameter (transition / transversion) model are less subject to this problem and are more consistent with directly observed differences.