Choosing the best similarity index when performing fuzzy set ordination on binary data

Abstract

Abstract. Fuzzy set ordination (FSO) may be used with either abundance data or binary (presence/absence) data. FSO requires a similarity index that returns values between 0 and 1. Many indices will do so, but their suitability for FSO has not been tested. Nine binary indices were evaluated in this study. Simulated plant community data sets were generated with COMPAS; they contained five levels of β‐diversity, two levels of qualitative noise, and two sampling arrangements (regular or random) along one gradient. Indices were evaluated with rank and linear correlations between the apparent ecological gradient positions generated by FSO and actual gradient positions; the abilities of the best‐performing indices to minimize the curlover effect were also compared. All indices performed best at intermediate levels of β‐diversity and with regular sampling. Five indices had consistently higher rank and linear correlations (Baroni‐Urbani & Buser, Jaccard, Kulczynski, Ochiai and Sørensen), whereas four were consistently lower (Faith, Russell & Rao, Rogers & Tanimoto and Simple Matching). There were no significant differences in curlover among the five best indices. A step‐across algorithm, a flexible shortest path adjustment, improved correlations and reduced curlover for the five best indices at higher β‐diversity levels. We recommend that one of the five best‐performing similarity indices be used with FSO on binary data; a flexible shortest path adjustment should also be employed at higher β‐diversities when possible.