Post Hoc Analysis Decisions Drive the Reported Reading Time Effects in Hackl, Koster-Hale & Varvoutis (2012)

Abstract

Abstract Hackl, Koster-Hale & Varvoutis (2012; hereafter HKV) provide data that suggest that in a null context, antecedent-contained deletion (ACD) relative clause structures modifying a quantified object noun phrase (NP) are easier to process than those modifying a definite object NP. HKV argue that this pattern of results supports a quantifier-raising (QR) analysis of both ACD structures and quantified NPs in object position: under the account they advocate, both ACD resolution and quantified NPs in object position require movement of the object NP to a higher syntactic position. The processing advantage for quantified object NPs in ACD is hypothesized to derive from the fact that—at the point where ACD resolution must take place—the quantified NP has already undergone QR, whereas this is not the case for definite NPs. Here, we question these conclusions. In particular, our analyses of HKV’s reading time data reveal several unreported choice points, errors and concerns regarding multiple comparisons in the original HKV data analysis. Importantly, most other plausible ways to analyze these data that we describe here result in the crucial interaction being non-significant. Putting this observation together with the failure to observe the crucial interaction in Gibson & Levy (2016), we conclude that the experiments reported by HKV should not be viewed as providing evidence for the ACD quantifier-raising processing effect.