Combinatory Categorial Grammar Research Articles

Combinatory categorial grammars as generators of weighted forests

Combinatory Categorial Grammar (CCG) is an extension of categorial grammar that is well-established in computational linguistics. It is mildly context-sensitive, so it is efficiently parsable and reaches an expressiveness that is suitable for describing natural languages. Weighted CCG (wCCG) are introduced as a natural extension of CCG with weights taken from an arbitrary commutative semiring. Their expressive power is compared to other weighted formalisms with special emphasis on the weighted forests generated by wCCG since the ability to express the underlying syntactic structure of an input sentence is a vital feature of CCG in the area of natural language processing. Building on recent results for the expressivity in the unweighted setting, the corresponding results are derived for the weighted setting for any commutative semiring. More precisely, the weighted forests generatable by wCCG are also generatable by weighted simple monadic context-free tree grammar (wsCFTG). If the rule system is restricted to application rules and composition rules of first degree, then the generatable weighted forests are exactly the regular weighted forests. Finally, when only application rules are allowed, then a proper subset of the regular weighted forests is generatable.

From Combinatory Categorial Grammar to the Calculus of Constructions: A Review of ccg2lambda.

The following is an overview of the concepts and systems necessary to build an understanding of the framework from ccg2lambda: A Compositional Semantics System, which produces logical semantic representations of natural language sentences. These representations are used in tandem with a theorem prover to conduct inferences. But, to the reader unfamiliar with the various tools within computer science, linguistics, and mathematics that were used to construct ccg2lambda and its dependencies, it can be difficult to understand the inner workings of the framework. Thus, an explication of these concepts could be useful for fostering further development. Concepts such as mathematical logic, combinatory categorial grammar, the calculus of constructions, and interactive theorem provers are covered, as well as the two systems C&C and ccg2lambda. A functional version of the ccg2lambda framework can be found at https://github.com/ajung202/ccg2lambda.

Implementing Natural Language Inference for comparatives

This paper presents a computational framework for Natural Language Inference (NLI) using logic-based semantic representations and theorem-proving. We focus on logical inferences with comparatives and other related constructions in English, which are known for their structural complexity and difficulty in performing efficient reasoning. Using the so-called A-not-A analysis of comparatives, we implement a fully automated system to map various comparative constructions to semantic representations in typed first-order logic via Combinatory Categorial Grammar parsers and to prove entailment relations via a theorem prover. We evaluate the system on a variety of NLI benchmarks that contain challenging inferences, in comparison with other recent logic-based systems and neural NLI models.

Tractable Parsing for CCGs of Bounded Degree

Abstract Unlike other mildly context-sensitive formalisms, Combinatory Categorial Grammar (CCG) cannot be parsed in polynomial time when the size of the grammar is taken into account. Refining this result, we show that the parsing complexity of CCG is exponential only in the maximum degree of composition. When that degree is fixed, parsing can be carried out in polynomial time. Our finding is interesting from a linguistic perspective because a bounded degree of composition has been suggested as a universal constraint on natural language grammar. Moreover, ours is the first complexity result for a version of CCG that includes substitution rules, which are used in practical grammars but have been ignored in theoretical work.

Simple Robust Grammar Induction with Combinatory Categorial Grammars

We present a simple EM-based grammar induction algorithm for Combinatory Categorial Grammar (CCG) that achieves state-of-the-art performance by relying on a minimal number of very general linguistic principles. Unlike previous work on unsupervised parsing with CCGs, our approach has no prior language-specific knowledge, and discovers all categories automatically. Additionally, unlike other approaches, our grammar remains robust when parsing longer sentences, performing as well as or better than other systems. We believe this is a natural result of using an expressive grammar formalism with an extended domain of locality.

Strong Equivalence of TAG and CCG

Tree-adjoining grammar (TAG) and combinatory categorial grammar (CCG) are two well-established mildly context-sensitive grammar formalisms that are known to have the same expressive power on strings (i.e., generate the same class of string languages). It is demonstrated that their expressive power on trees also essentially coincides. In fact, CCG without lexicon entries for the empty string and only first-order rules of degree at most 2 are sufficient for its full expressive power.

From Generative Linguistics to Categorial Grammars: Overt Subjects in Control Infinitives

AbstractThe present thesis lies at the interface of logic and linguistics; its object of study are control sentences with overt pronouns in Romance languages (European and Brazilian Portuguese, Italian and Spanish). This is a topic that has received considerably more attention on the part of linguists, especially in recent years, than from logicians. Perhaps for this reason, much remains to be understood about these linguistic structures and their underlying logical properties. This thesis seeks to fill the lacunas in the literature or at least take steps in this direction by way of addressing a number of issues that have so far been under-explored. To this end, we put forward two key questions, one linguistic and the other logical. These are, respectively, (1) What is the syntactic status of the surface pronoun? and (2) What are the available mechanisms to reuse semantic resources in a contraction-free logical grammar? Accordingly, the thesis is divided into two parts: generative linguistics and categorial grammar. Part I starts by reviewing the recent discussion within the generative literature on infinitive clauses with overt subjects, paying detailed attention to the main accounts in the field. Part II does the same on the logical grammar front, addressing in particular the issues of control and of anaphoric pronouns. Ultimately, the leading accounts from both camps will be found wanting. The closing chapter of each of Part I and Part II will thus put forward alternative candidates, that we contend are more successful than their predecessors. More specifically, in Part I, we offer a linguistic account along the lines of Landau’s T/Agr theory of control. In Part II, we present two alternative categorial accounts: one based on Combinatory Categorial Grammar, the other on Type-Logical Grammar. Each of these accounts offers an improved, more fine-grained perspective on control infinitives featuring overt pronominal subjects. Finally, we include an Appendix in which our type-logical proposal is implemented in a categorial parser/theorem-prover.Abstract prepared by María Inés Corbalán.E-mail: inescorbalan@yahoo.com.arURL: http://repositorio.unicamp.br/jspui/handle/REPOSIP/331697

The role of working memory in syntactic sentence realization: A modeling & simulation approach

This paper examines the effects of working memory size in incremental grammatical encoding during language production. Our experiment tests different variants of a computational-cognitive model that combines an empirically validated framework of general cognition, ACT-R, with a linguistic theory, Combinatory Categorial Grammar. The model is induced from a corpus of spoken dialogue. This methodology facilitates comparison of different strategies and working memory capacities according to the similarity of the model’s produced sentences to the corpus sentences. The experiment presented shows that while having more working memory available improves performance, using less working memory during realization does as well, even after controlling sentence length. Sentences realized with a more incremental strategy also appear to more closely track the naturalistic data. As high incrementality is correlated with low working memory usage, this study offers a possible mechanism by which syntactic incrementality can be explained. Finally, this paper proposes a multi-disciplinary modeling and simulation-based approach to empirical psycholinguistic inquiry.

The scope ambiguity in the Korean Left-Node Raising construction

This paper examines the ambiguity that arises from the Left-Node Raising (LNR) construction in Korean. I argue that the ambiguity is induced by the scopal relation between a coordinate phrase and a quantifier-type pivot: the quantificational meaning of a pivot can scope over the coordinate phrase, or it can take a narrow scope within each conjunct. I show that previous proposals such as the ATB-scrambling analysis (Nakao 2010) and the multi-dominance analysis (Chung 2010) have not addressed the ambiguity or have some empirical problems. I develop a compositional analysis of the scopal ambiguity within the framework of Combinatory Categorial Grammar (e.g. Steedman 1996; Steedman 2000; Baldridge 2002; Steedman and Baldridge 2011; Steedman 2012), and propose that it can be extended to the English and Korean RNR sentences with a quantificational pivot. (Sungkyunkwan University)

On the Complexity of CCG Parsing

We study the parsing complexity of Combinatory Categorial Grammar (CCG) in the formalism of Vijay-Shanker and Weir ( 1994 ). As our main result, we prove that any parsing algorithm for this formalism will take in the worst case exponential time when the size of the grammar, and not only the length of the input sentence, is included in the analysis. This sets the formalism of Vijay-Shanker and Weir ( 1994 ) apart from weakly equivalent formalisms such as Tree Adjoining Grammar, for which parsing can be performed in time polynomial in the combined size of grammar and input sentence. Our results contribute to a refined understanding of the class of mildly context-sensitive grammars, and inform the search for new, mildly context-sensitive versions of CCG.

Categorial Analysis of Agreement Asymmetries in Arabic

Agreement asymmetries are the most debated issue in Arabic linguistics. Even though the facts suggest a unified treatment based on the properties of agreement, most of the researchers in this field don’t take into account the essential difference between grammatical agreement and anaphoric agreement. We do propose such a distinction to explain these asymmetries and we propose an analysis that we implement in the Applicative Combinatory Categorial Grammar (ACCG) framework.

CCG supertagging with bidirectional long short-term memory networks

AbstractNeural Network-based approaches have recently produced good performances in Natural language tasks, such as Supertagging. In the supertagging task, a Supertag (Lexical category) is assigned to each word in an input sequence. Combinatory Categorial Grammar Supertagging is a more challenging problem than various sequence-tagging problems, such as part-of-speech (POS) tagging and named entity recognition due to the large number of the lexical categories. Specifically, simple Recurrent Neural Network (RNN) has shown to significantly outperform the previous state-of-the-art feed-forward neural networks. On the other hand, it is well known that Recurrent Networks fail to learn long dependencies. In this paper, we introduce a new neural network architecture based on backward and Bidirectional Long Short-Term Memory (BLSTM) Networks that has the ability to memorize information for long dependencies and benefit from both past and future information. State-of-the-art methods focus on previous information, whereas BLSTM has access to information in both previous and future directions. Our main findings are that bidirectional networks outperform unidirectional ones, and Long Short-Term Memory (LSTM) networks are more precise and successful than both unidirectional and bidirectional standard RNNs. Experiment results reveal the effectiveness of our proposed method on both in-domain and out-of-domain datasets. Experiments show improvements about (1.2 per cent) over standard RNN.

Hindi CCGbank: A CCG treebank from the Hindi dependency treebank

In this paper, we present an approach for automatically creating a combinatory categorial grammar (CCG) treebank from a dependency treebank for the subject–object–verb language Hindi. Rather than a direct conversion from dependency trees to CCG trees, we propose a two stage approach: a language independent generic algorithm first extracts a CCG lexicon from the dependency treebank. An exhaustive CCG parser then creates a treebank of CCG derivations. We also discuss special cases of this generic algorithm to handle linguistic phenomena specific to Hindi. In doing so we extract different constructions with long-range dependencies like coordinate constructions and non-projective dependencies resulting from constructions like relative clauses, noun elaboration and verbal modifiers.

The Applicative Combinatory Categorial Analysis of Arabic

The Applicative Combinatory Categorial Grammar (ACCG), like the whole of the other categorial models conceptualizes the languages as organized systems of linguistic units (words, morphemes, lexemes, etc) of which some function as operators whereas others function as operands. The ACCG is a sufficiently flexible model to give an account of several languages (SVO, VSO, SOV, etc.) and several forms of organisation of the linguistic units of these languages (coordination, subordination, with backward modifier, etc). However, the research tasks were limited almost exclusively to European languages, in particular French, English, German, Dutch. A language like Arabic, in spite of the richness of the tradition of its schools of thought, which go back for several centuries, remains unexplored by this current, except some research works. In our article, we show how the model of Applicative Combinatory Categorial Grammar can give an account of certain forms of Arabic.

A Novel Approach by Injecting CCG Supertags into an Arabic–English Factored Translation Machine

This study addresses the integration and incorporation of rich additional information into the phrase-based approach, aptly called factored translation, which is an extension of phrase-based statistic machine translation (PBSMT). This approach was proven successful when translating English into a morphologically rich language. PBSMT represents the baseline of this work. We extend the phrase-based translation approach by integrating additional linguistic knowledge, namely part-of-speech (POS) tags, to create a factored model. The main contribution of this study is the creation of a new approach for Arabic–English translation via the injection of the factored model into Combinatory Categorial Grammar (CCG) supertags to form an integrated model (POS + CCG). The system was trained on a freely available multi-UN corpus on Arabic–English language pairs. Moses decoder, which is an open-source factored SMT system, was used to integrate these data into the target language model and the target side of the translation model. Results showed improvements to the BLEU automatic score via various high n-gram language models (LMs). The integration of the featured factors (POS + CCG) of the translation has been successfully tested. Overall, the 3-, 5-, 7-, and 9-g LM evaluation with BLEU scores proved that our integrated model performed better than PBSMT. Compared with three other models (PBSMT, POS, and CCG models), the integrated model improved the translation quality by 1.54, 1.29, and 0.21 %, respectively, over the 3-g LM.

Discriminative Syntax-Based Word Ordering for Text Generation

Word ordering is a fundamental problem in text generation. In this article, we study word ordering using a syntax-based approach and a discriminative model. Two grammar formalisms are considered: Combinatory Categorial Grammar (CCG) and dependency grammar. Given the search for a likely string and syntactic analysis, the search space is massive, making discriminative training challenging. We develop a learning-guided search framework, based on best-first search, and investigate several alternative training algorithms.The framework we present is flexible in that it allows constraints to be imposed on output word orders. To demonstrate this flexibility, a variety of input conditions are considered. First, we investigate a “pure” word-ordering task in which the input is a multi-set of words, and the task is to order them into a grammatical and fluent sentence. This task has been tackled previously, and we report improved performance over existing systems on a standard Wall Street Journal test set. Second, we tackle the same reordering problem, but with a variety of input conditions, from the bare case with no dependencies or POS tags specified, to the extreme case where all POS tags and unordered, unlabeled dependencies are provided as input (and various conditions in between). When applied to the NLG 2011 shared task, our system gives competitive results compared with the best-performing systems, which provide a further demonstration of the practical utility of our system.

Weakly-Supervised Grammar-Informed Bayesian CCG Parser Learning

Combinatory Categorial Grammar (CCG) is a lexicalized grammar formalism in which words are associated with categories that, in combination with a small universal set of rules, specify the syntactic configurations in which they may occur. Categories are selected from a large, recursively-defined set; this leads to high word-to-category ambiguity, which is one of the primary factors that make learning CCG parsers difficult, especially in the face of little data. Previous work has shown that learning sequence models for CCG tagging can be improved by using linguistically-motivated prior probability distributions over potential categories. We extend this approach to the task of learning a CCG parser from weak supervision. We present a Bayesian formulation for CCG parser induction that assumes only supervision in the form of an incomplete tag dictionary mapping some word types to sets of potential categories. Our approach outperforms a baseline model trained with uniform priors by exploiting universal, intrinsic properties of the CCG formalism to bias the model toward simpler, more cross-linguistically common categories.

Improving Telugu Dependency Parsing using Combinatory Categorial Grammar Supertags

We show that Combinatory Categorial Grammar (CCG) supertags can improve Telugu dependency parsing. In this process, we first extract a CCG lexicon from the dependency treebank. Using both the CCG lexicon and the dependency treebank, we create a CCG treebank using a chart parser. Exploring different morphological features of Telugu, we develop a supertagger using maximum entropy models. We provide CCG supertags as features to the Telugu dependency parser (MST parser). We get an improvement of 1.8% in the unlabelled attachment score and 2.2% in the labelled attachment score. Our results show that CCG supertags improve the MST parser, especially on verbal arguments for which it has weak rates of recovery.

Integrating Multiple Dependency Corpora for Inducing Wide-Coverage Japanese CCG Resources

A novel method to induce wide-coverage Combinatory Categorial Grammar (CCG) resources for Japanese is proposed in this article. For some languages including English, the availability of large annotated corpora and the development of data-based induction of lexicalized grammar have enabled deep parsing, i.e., parsing based on lexicalized grammars. However, deep parsing for Japanese has not been widely studied. This is mainly because most Japanese syntactic resources are represented in chunk-based dependency structures, while previous methods for inducing grammars are dependent on tree corpora. To translate syntactic information presented in chunk-based dependencies to phrase structures as accurately as possible, integration of annotation from multiple dependency-based corpora is proposed. Our method first integrates dependency structures and predicate-argument information and converts them into phrase structure trees. The trees are then transformed into CCG derivations in a similar way to previously proposed methods. The quality of the conversion is empirically evaluated in terms of the coverage of the obtained CCG lexicon and the accuracy of the parsing with the grammar. While the transforming process used in this study is specialized for Japanese, the framework of our method would be applicable to other languages for which dependency-based analysis has been regarded as more appropriate than phrase structure-based analysis due to morphosyntactic features.

Lexicalization and Generative Power in CCG

The weak equivalence of Combinatory Categorial Grammar (CCG) and Tree-Adjoining Grammar (TAG) is a central result of the literature on mildly context-sensitive grammar formalisms. However, the categorial formalism for which this equivalence has been established differs significantly from the versions of CCG that are in use today. In particular, it allows restriction of combinatory rules on a per grammar basis, whereas modern CCG assumes a universal set of rules, isolating all cross-linguistic variation in the lexicon. In this article we investigate the formal significance of this difference. Our main result is that lexicalized versions of the classical CCG formalism are strictly less powerful than TAG.