Of Dinosaurs and Barbecue Sauce: A Comment on King and Zeng

Abstract

When I first encountered this article as a reviewer, I was reasonably enthusiastic about it. The methods are creative and, if one accepts the assertions of the authors, apply to a very broad class of models. However, as I looked more closely, I began to have doubts about both the scope and novelty of the work. While I still believe there is a small technical contribution here, I am now substantially less confident about the overall value in light of the exaggerated claims of applicability. The comic strip Dilbert has a character named “Topper” who insists on always being better than any of his coworkers. In a one such strip, the boss is telling a fish story, and Topper comes along to say “I once caught a dinosaur using only dental floss and a bent paper clip.” When the boss asks to see the dinosaur, Topper replies, “I can't show it to you because I also make the world's greatest barbecue sauce.” So it is with this article: it promises to deliver a whole lot more than it actually does, making claims for uniqueness and generality that simply do not hold. The authors confidently assert that their methods apply “for the class of nearly all models, whether or not they are formalized, enumerated, and run, and for the class of all possible dependent variables,” (p. 185). In fact, one can readily come up with counter-examples where the methods provide little if any comparative advantage over techniques that have been known for decades. There is nothing magic about the convex hull, and the paper is presenting “best possible case” examples. If the convex hull is fairly compact, and the models are well behaved inside them, as presented in the figures, then you get the results the authors claim. But it is …