Acceptability Judgment Experiments Research Articles

Different Ways to Test Acceptability Judgments

The testing of acceptability judgments has recently attracted some attention in the study of syntax. This is because many linguists have come to realize that the extensive measuring the correlations between native speakers’ intuitions provides a reliable way to investigate the genuine nature of human language. The growing research interest notwithstanding, the methodology of acceptability judgment experiments has not been completely validated. Since experimental syntax is an empirical science, it is important to develop a strong research methodology for a large-scale and in-depth quantitative analysis. In this context, the research question this paper raises is whether or not different methodologies of acceptability judgment testing yield different conclusions. The present study conducts two experiments that share exactly the same test items but are controlled and analyzed in different ways. Looking into the convergence rate using different methods of experimental control and statistical analysis, the present study provides four findings which can lead to a better methodology for observing native speakers’ intuitions: First, randomization is a mandatory process in acceptability judgment testing. Second, the mean in itself does not provide good statistical power in acceptability judgment testing. Third, sociological variables can affect acceptability judgments. Fourth, the size of sample in acceptability judgment testing should be more than 40 in order to yield a trustworthy conclusion.

Against the Small Clause Hypothesis: Evidence from Swedish relative clause extractions

On the basis of data from Swedish, this paper examines the Small Clause Hypothesis (Kush & Lindahl 2011, Kush, Omaki & Hornstein 2013) proposed to account for relative clause (RC) extractions in Mainland Scandinavian. The hypothesis predicts that extraction possibilities differ for relative clauses in the complement of verbs which select and verbs which do not select a small clause (SC), and that the possibility of RC extraction hinges on the ability of the matrix verb to select SCs involving the predicational operatorsom. I report results from an acceptability judgment experiment on RC extraction in Swedish manipulating three conditions: (a) SC-selecting verbs compatible withsom, (b) SC-selecting verbs incompatible withsom, and (c) verbs that are incompatible with SCs. The results show no significant difference between these conditions, thus offer no support in favor of the Small Clause Hypothesis. Additional problems are posed by the possibility of extraction from object RCs and by extraction possibilities in the absence ofsom.

The role of givenness, presupposition, and prosody in Czech word order: An experimental study

We present evidence from acceptability judgment experiments that there is systematic prosodic givenness marking in Czech in that discourse-salient elements avoid sentence stress, contra the claim in Kučerová (2007, 2012) that givenness is marked only syntactically---by establishing a word order in which all given elements precede all new ones---and not prosodically in Czech. We argue that the syntactic movement of given elements results from the need to avoid the rightmost position where sentence stress falls, and not from a syntactic ordering requirement. This is supported by the empirical finding that given objects need not scramble if they are not in sentence-final position, even if they are preceded by new elements (experiment 2). We also argue against Kučerová's claim that given elements are marked only if definite/presupposed in Czech by showing that irrespective of this property, all given objects tend to avoid the sentence-final position (experiment 1). Finally, our results reveal an interaction between presupposition and word order, in the sense of an acceptability penalty for utterances in which non-presupposed expressions precede presupposed ones (experiment 1 and 2). We propose to model the results by a set of weighted constraints that apply post-derivationally. http://dx.doi.org/10.3765/sp.8.3 BibTeX info

The Distribution of the Hebrew Possessive Dative Construction: Guided by Unaccusativity or Prominence?

This article challenges the empirical generalization that the possessed element in the Hebrew possessive dative (PD) construction cannot be an external argument ( Borer and Grodzinsky 1986 ), a generalization that has been leveraged in the use of the PD construction as an unaccusativity diagnostic. An acceptability judgment experiment shows that the PD construction is dispreferred when the possessor is less prominent than the possessee on the animacy or definiteness scale; however, it shows no effect of unaccusativity, raising serious doubts regarding the construction’s validity as an unaccusativity diagnostic.

Escape from the Island: Grammaticality and (Reduced) Acceptability of wh-island Violations in Danish

In the syntax literature, it is commonly assumed that a constraint on linguistic competence blocks extraction of wh-expressions (e.g. what or which book) from embedded questions, referred to as wh-islands. Furthermore, it is assumed that there is an argument/adjunct asymmetry in extraction from wh-islands. We report results from two acceptability judgment experiments on long and short wh-movement and wh-extraction from wh-islands in Danish. The results revealed four main findings: (1) No adjunct/argument asymmetry in extraction from wh-islands. (2) Long adjunct wh-movement is less acceptable than long argument wh-movement, and this difference is attributable to matrix verb compatibility and factivity, not D-linking. (3) Long movement reduces acceptability, but is more acceptable than island violations. (4) Training effects reveal that island violations, though degraded, are grammatical in Danish. Since the standard assumptions cannot account for the range of results, we argue in favor of a processing account referring to locality (processing domains) and working memory.

A quantitative investigation of the imperative-and-declarative construction in English

The imperative-and-declarative (IaD) construction in English (e.g. Study hard and you will pass the class ) has two distinct readings: one that has the semantics of a conditional and additionally the meaning of an imperative, and one that has only the semantics of a conditional, with no imperative meaning. There are two general kinds of syntactic approaches in the literature for analyzing this construction: one that treats the two interpretations as underlyingly syntactically the same, and one that treats them as two distinct syntactic constructions. This short report presents the results of an acceptability-judgment experiment that was designed to inform the debate between the two kinds of approaches. The two types of IaDs were observed to behave differently with respect to two phenomena we evaluated, suggesting either that they should be treated as grammatically distinct, or that a theory that treats them as grammatically the same must give a pragmatic account of the differences. Furthermore, because the pattern of data that we observed was a statistical interaction between two factors—a pattern of data that is not detectable without quantitative measurements—the results provide compelling evidence for the need for quantitative evaluations of linguistic hypotheses.

A Test of the Cognitive Assumptions of Magnitude Estimation: Commutativity does not Hold for Acceptability Judgments

The introduction of the psychophysical technique of magnitude estimation to the study of acceptability judgments (Bard et al. 1996) has led to a surge of interest in formal acceptability-judgment experiments over the past fifteen years. One of the primary reasons for its popularity is that it was developed as a tool to measure actual units of perception, offering the possibility of data that is inherently more informative than previous scaling tasks. However, there are several untested cognitive assumptions that must hold in order for ME to be the perceptual measurement test that it is purported to be. Building on the recent formalization of these assumptions in the psychophysics literature (Narens 1996, Luce 2002), this article presents two experiments designed to test whether these assumptions hold for acceptability-judgment experiments. The results suggest that the cognitive assumptions of magnitude estimation do not hold for participants in acceptability-judgment experiments, eliminating any reason to believe that ME could deliver inherently more meaningful data than other acceptability-judgment tasks.

A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory

Amazon’s Mechanical Turk (AMT) is a Web application that provides instant access to thousands of potential participants for survey-based psychology experiments, such as the acceptability judgment task used extensively in syntactic theory. Because AMT is a Web-based system, syntacticians may worry that the move out of the experimenter-controlled environment of the laboratory and onto the user-controlled environment of AMT could adversely affect the quality of the judgment data collected. This article reports a quantitative comparison of two identical acceptability judgment experiments, each with 176 participants (352 total): one conducted in the laboratory, and one conducted on AMT. Crucial indicators of data quality—such as participant rejection rates, statistical power, and the shape of the distributions of the judgments for each sentence type—are compared between the two samples. The results suggest that aside from slightly higher participant rejection rates, AMT data are almost indistinguishable from laboratory data.

Processing elided verb phrases with flawed antecedents: The recycling hypothesis

Traditional syntactic accounts of verb phrase ellipsis (e.g., “Jason laughed. Sam did [ ] too.”) categorize as ungrammatical many sentences that language users find acceptable (they “undergenerate”); semantic accounts overgenerate. We propose that a processing theory, together with a syntactic account, does a better job of describing and explaining the data on verb phrase–ellipsis. Five acceptability judgment experiments supported a “VP recycling hypothesis,” which claims that when a syntactically matching antecedent is not available, the listener/reader creates one using the materials at hand. Experiments 1 and 2 used verb phrase ellipsis sentences with antecedents ranging from perfect (a verb phrase in matrix verb phrase position) to impossible (a verb phrase containing only a deverbal word). Experiments 3 and 4 contrasted antecedents in verbal vs nominal gerund subjects. Experiment 5 explored the possibility that speakers are particularly likely to go beyond the grammar and produce elided constituents without perfect matching antecedents when the antecedent needed is less marked than the antecedent actually produced. This experiment contrasted active (unmarked) and passive antecedents to show that readers seem to honor such a tendency.

Should given information come before new? Yes and no

It has been noted for some time that given information tends to be placed before new information (e.g., Clark & Clark, 1977). To determine whether this holds generally, double-object and NP-PP sentences like the following were tested in two speeded acceptability judgment experiments. (1A) The senator mailed the woman a report. (1B) The senator mailed a woman the report. (1C) The senator mailed the report to a woman. (1D) The senator mailed a report to the woman. Given-before-new (or definite before indefinite) ordering facilitated processing for double-object structures, with (1A) processed faster and accepted more often than (1B), but did not facilitate processing of (1C) relative to (1D) in NP-PP structures. A third self-paced reading experiment showed that the advantage of the definite-before-indefinite new ordering holds for constructions with a shifted NP (The senator mailed to the woman a report) but also showed that facilitation from having a definite NP immediately after the verb was limited to cases where the two arguments of a verb contrast in definiteness. The results suggest that the presumed given-before-new preference is not general, but is instead limited to certain constructions and is based on the language comprehension system being sensitive to the requirements of language production.

The Breakdown of Functional Categories and the Economy of Derivation

In this paper I argue that in a hierarchical structure of a sentence, the lower the position of the functional head and its projection, the more accessible they are to an agrammatic aphasic. The major empirical basis for this includes spontaneous speech data and an acceptability judgment experiment by Japanese agrammatic patients as well as the available French and Italian data, The previous crosslinguistic agrammatic data from verb second languages are also examined. The unified account for the status of functional categories in agrammatism is presented based on the principles of the Economy of the Minimalist framework (Chomsky 1994). Finally, the empirical consequences of agrammatic data for language acquisition are discussed.