The Foundations of Statistics with Black Swans

Abstract

We extend the foundation of statistics to integrate rare events that are potentially catastrophic, called black swans.These include natural hazards, regime change in complex systems, market crashes, catastrophic climate change and major episodes of species extinction. Classic statistics and physics treat such events as ‘outliers’ and often disregard them. We propose a new axiomatization of subjective probability requiring equal treatment for rare and frequent events, and characterize the likelihoods or subjective probabilities that the axioms imply.These coincide with countably additive measures and yield normal distributions when the sample has no black swans. When the sample includes black swans, the new likelihoods are represented by a combination of countable and finitely additive measures with both parts present. The axioms were introduced in Chichilnisky (2000, 2002); they extend the axiomatic foundations of Savage (1954), Villegas(1964) and Arrow (1971) and they are valid for bounded and unbounded samples (Chichilnisky, 1996b). The finitely additive measures assign more weight to rare events than do standard distributions and in that sense explain the persistent observation of power laws and ‘heavy tails’ that eludes classic theory.