Measuring the Agreement of Mathematical Peer Reviewers

Abstract

We investigate the possibility of arguing for or against the philosophical position that mathematics is an epistemic exception on the basis of agreement data from the mathematical peer review process and argue that Cohen’s kappa , the standard agreement measure used for inter-rater agreement, is unable to detect epistemic exceptionality from peer review data.