No empirical evidence to support the hypothesis that daily climate variation has an effect on species’ elevational range size: Reply to Chan et al.

Richard Field,Hong Qian

doi:10.1111/jbi.13372

Abstract

AbstractFollowing our critique, Chan et al. defend the approach used in their original paper. They reveal their “iterative strategy of SEM” (structural equation modelling), which they claim is “standard” (we show otherwise) and “required for the proper and most effective use of SEM” for hypothesis testing. However, publishing their detailed procedure exposes fundamental flaws: capitalizing on chance and violating important assumptions and principles of SEM. They used the same data to first explore numerous correlations, then fit 29 candidate models (all failed) using the best correlates, then fix model parameters to gain degrees of freedom, then evaluate the “best” model. In producing the “best” model, they fixed five parameters using estimates from regression on the same dataset. They further argue that their stationary bootstrap cures the problems of bias and pseudoreplication; we disagree. At best, Chan et al. developed a hypothesis; they did not perform a valid test of one.

Highlights

Following our critique, Chan et al defend the approach used in their original paper
We did not understand their modelling approach because they failed to even mention the ‘model modification’ they performed, let alone provide any details. They admit that they did not specify the details of their analyses in their original paper, and express regret about it
We challenge anyone to read the original paper (Chan et al, 2016) without the new Additional Supplementary Materials that have been added to it and be able to repeat the analyses with no further information! Certainly none of the people we have asked about it have even got close to guessing what they did

Summary

Introduction

Chan et al defend the approach used in their original paper. They reveal their “iterative strategy of SEM” (structural equation modelling), which they claim is “standard” (we show otherwise) and “required for the proper and most effective use of SEM” for hypothesis testing. We showed that key results they reported could not be repeated using their data and the methods described in their paper; their model should have been rejected.

Objectives

Results

Conclusion