Abstract
The current literature views Simpson’s paradox as a probabilistic conundrum by taking the premises (probabilities/parameters/ frequencies) as known. In such a context, it is shown that the paradox arises within a very small subset of the relevant parameter space, rendering the paradox unlikely to occur in real data. The problem, however, is that the probabilistic perspective, ignores certain crucial empirical (data, statistical) issues raised by the original Pearson and Yule papers on ‘spurious’ association reversals. Placing the paradox in a broader empirical framework that begins with the raw data $${\mathbf {z}}_{0}$$ and an appropriately selected statistical model $${\mathcal {M}}_{{\varvec{{\theta }}}}({\mathbf {x}})$$ , the discussion elucidates the original Yule–Pearson conundrum by formalizing its notion of ‘spurious or fictitious associations’ into ‘statistically untrustworthy associations’ stemming from a misspecified $${\mathcal {M}}_{{\varvec{{\theta }}}}( {\mathbf {x}})$$ ; invalid probabilistic assumptions imposed on $${\mathbf {z}}_{0}$$ . It is shown that several empirical examples used to illustrate Simpson’s paradox in the current literature constitute examples of the Yule–Pearson untrustworthy association reversals. The empirical perspective is used to revisit the causal explanation of the paradox and make a case that several widely accepted causal claims are questionable on statistical adequacy grounds. It is also used to propose a procedure to detect and account for the ‘third entity’ in the paradox, as well as (reliably) select among different potential causal explanations, such as collider, mediator or confounder, on empirical grounds.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.