Natural Language Syntax Research Articles

Natural language syntax complies with the free-energy principle

Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design—such as “minimal search” criteria from theoretical syntax—adhere to the FEP. This affords a greater degree of explanatory power to the FEP—with respect to higher language functions—and offers linguistics a grounding in first principles with respect to computability. While we mostly focus on building new principled conceptual relations between syntax and the FEP, we also show through a sample of preliminary examples how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel–Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing–Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference.

Semantic requirements construction using ontologies and boilerplates

This paper presents a combination of an ontology and boilerplates, which are requirements templates for the syntactic structure of individual requirements that try to alleviate the problem of ambiguity caused using natural language and make it easier for inexperienced engineers to create requirements. However, still the use of boilerplates restricts the use of natural language only syntactically and not semantically. Boilerplates consists of fixed and attributes elements. Using ontologies, restricts the vocabulary of the words used in the requirements boilerplates to entities, their properties and entity relationships that are semantically meaningful to the application domain, leading thus to fewer errors. In this work we combine the advantages of boilerplates and ontologies. Usually, the attributes of boilerplates are completed with the help of the ontology. The contribution of this paper is that the whole boilerplates are stored in the ontology, based on the fact that RDF triples have similar syntax to the boilerplate syntax, so that attributes and fixed elements are part of the ontology. This combination helps to construct semantically and syntactically correct requirements. The contribution and novelty of our method is that we exploit the natural language syntax of boilerplates mapping them to Resource Description Framework triples which have also a linguistic nature. In this paper we created and present the development of a domain-specific ontology as well as a minimal set of boilerplates for a specific application domain, namely that of engineering software for an ATM, while maintaining flexibility on the one hand and generality on the other.

Syncopation as structure bootstrapping: the role of asymmetry in rhythm and language.

Syncopation - the occurrence of a musical event on a metrically weak position preceding a rest on a metrically strong position - represents an important challenge in the study of the mapping between rhythm and meter. In this contribution, we present the hypothesis that syncopation is an effective strategy to elicit the bootstrapping of a multi-layered, hierarchically organized metric structure from a linear rhythmic surface. The hypothesis is inspired by a parallel with the problem of linearization in natural language syntax, which is the problem of how hierarchically organized phrase-structure markers are mapped onto linear sequences of words. The hypothesis has important consequences for the role of meter in music perception and cognition and, more particularly, for its role in the relationship between rhythm and bodily entrainment.

Merge vs. "Lerge:" Problems of association

It is shown that the proposal of identifying Merge and the Leibnizian addition operator runs up against the obstacle that the latter is associative while the former is not. The confound is attributed to insufficient appreciation of the difference between a calculus for natural language syntax and a calculus of concepts.

Evaluating syntactic proposals using minimalist grammars and minimum description length

Many patterns found in natural language syntax have multiple pos-sible explanations or structural descriptions. Even within the cur-rently dominant Minimalist framework (Chomsky 1995, 2000), it is not uncommon to encounter multiple types of analyses for the same phenomenon proposed in the literature. A natural question, then, is whether one could evaluate and compare syntactic proposals from a quantitative point of view. In this paper, we show how an evaluation measure inspired by the minimum description length principle (Rissa-nen 1978) can be used to compare accounts of syntactic phenomena implemented as minimalist grammars (Stabler 1997), and how argu-ments for and against this kind of analysis translate into quantitative differences.

Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality

We develop a categorical compositional distributional semantics for Lambek Calculus with a Relevant Modality !L∗, a modality that allows for the use of limited editions of contraction and permutation in the logic. Lambek Calculus has been introduced to analyse syntax of natural language and the linguistic motivation behind this modality is to extend the domain of the applicability of the calculus to fragments which witness the discontinuity phenomena. The categorical part of the semantics is a monoidal biclosed category with a !-functor, very similar to the structure of a Differential Category. We instantiate this category to finite dimensional vector spaces and linear maps via "quantisation" functors and work with three concrete interpretations of the !-functor. We apply the model to construct categorical and concrete semantic interpretations for the motivating example of !L∗: the derivation of a phrase with a parasitic gap. The efficacy of the concrete interpretations are evaluated via a disambiguation task, on an extension of a sentence disambiguation dataset to parasitic gap phrase one, using BERT, Word2Vec, and FastText vectors and relational tensors.

A Declarative Metamorphic Testing Framework for Autonomous Driving

Autonomous driving has gained much attention from both industry and academia. Currently, Deep Neural Networks (DNNs) are widely used for perception and control in autonomous driving. However, several fatal accidents caused by autonomous vehicles have raised serious safety concerns about autonomous driving models. Some recent studies have successfully used the metamorphic testing technique to detect thousands of potential issues in some popularly used autonomous driving models. However, prior study is limited to a small set of metamorphic relations, which do not reflect rich, real-world traffic scenarios and are also not customizable. This paper presents a novel declarative rule-based metamorphic testing framework called RMT. RMT provides a rule template with natural language syntax, allowing users to flexibly specify an enriched set of testing scenarios based on real-world traffic rules and domain knowledge. RMT automatically parses human-written rules to metamorphic relations using an NLP-based rule parser referring to an ontology list and generates test cases with a variety of image transformation engines. We evaluated RMT on three autonomous driving models. With an enriched set of metamorphic relations, RMT detected a significant number of abnormal model predictions that were not detected by prior work. Through a large-scale human study on Amazon Mechanical Turk, we further confirmed the authenticity of test cases generated by RMT and the validity of detected abnormal model predictions.

An Improvised Approach of Deep Learning Neural Networks in NLP Applications

Abstract: In recent years, natural language processing (NLP) has drawn a lot of interest for its ability to computationally represent and analyze human language. Its uses have expanded to include machine translation, email spam detection, information extraction, summarization, medical diagnosis, and question answering, among other areas. The purpose of this research is to investigate how deep learning and neural networks are used to analyze the syntax of natural language. This research first investigates a feed-forward neural network-based classifier for a transfer-based dependent syntax analyzer. This study presents a long-term memory neural network-based dependent syntactic analysis paradigm. This model, which will serve as a feature extractor, is based on the feed-forward neural network model mentioned before. After the feature extractor is learned, we train a recursive neural network classifier that is optimized by sentences using a long short-term memory neural network as a classifier of the transfer action and the characteristics retrieved by the syntactic analyzer as its input. Syntactic analysis replaces the method of modeling independent analysis with one that models the analysis of the entire sentence as a whole. The experimental findings demonstrate that the model has improved its performance more than the benchmark techniques.

An Occurrence Description Logic


 
 
 Description Logics (DLs) are a family of well-known terminological knowledge representation formalisms in modern semantics-based systems. This research focuses on analysing how our developed Occurrence Logic (OccL) can conceptually and logically support the development of a description logic. OccL is integrated into the alternative theory of natural language syntax in Deviational Syntactic Structures under the label ‘EFA(X)3’ (or the third version of Epi-Formal Analysis in Syntax, EFA(X), which is a radical linguistic theory). From the logical point of view, OccL is a formal logic that mainly deals with the occurrences of symbols as well as with their priorities within linguistic descriptions, i.e. natural language syntax, semantics and phonology. In this article—based on our OccL-based definitions of the concepts of strong implication and occurrence value as well as of the logical concept Identical Occurrence Constructor (IDOC) that is the most fundamental logical concept in our formalism—we will model Occurrence Description Logic (ODL). Accordingly, we will formally-logically analyse ‘occurrence(s) of symbol(s)’ within descriptions of the world in ODL. In addition, we will analyse and assess the logical concepts of occurrence and occurrence priority in ODL. This research can make a strong logical background for our future research in the development of a Modal Occurrence Description Logic.
 
 

Natural Language Processing with Improved Deep Learning Neural Networks

As one of the core tasks in the field of natural language processing, syntactic analysis has always been a hot topic for researchers, including tasks such as Questions and Answer (Q&A), Search String Comprehension, Semantic Analysis, and Knowledge Base Construction. This paper aims to study the application of deep learning and neural network in natural language syntax analysis, which has significant research and application value. This paper first studies a transfer-based dependent syntax analyzer using a feed-forward neural network as a classifier. By analyzing the model, we have made meticulous parameters of the model to improve its performance. This paper proposes a dependent syntactic analysis model based on a long-term memory neural network. This model is based on the feed-forward neural network model described above and will be used as a feature extractor. After the feature extractor is pretrained, we use a long short-term memory neural network as a classifier of the transfer action, and the characteristics extracted by the syntactic analyzer as its input to train a recursive neural network classifier optimized by sentences. The classifier can not only classify the current pattern feature but also multirich information such as analysis of state history. Therefore, the model is modeled in the analysis process of the entire sentence in syntactic analysis, replacing the method of modeling independent analysis. The experimental results show that the model has achieved greater performance improvement than baseline methods.

Variation in mild context-sensitivity

Abstract Aravind Joshi famously hypothesized that natural language syntax was characterized (in part) by mildly context-sensitive generative power. Subsequent work in mathematical linguistics over the past three decades has revealed surprising convergences among a wide variety of grammatical formalisms, all of which can be said to be mildly context-sensitive. But this convergence is not absolute. Not all mildly context-sensitive formalisms can generate exactly the same stringsets (i.e. they are not all weakly equivalent), and even when two formalisms can both generate a certain stringset, there might be differences in the structural descriptions they use to do so. It has generally been difficult to find cases where such differences in structural descriptions can be pinpointed in a way that allows linguistic considerations to be brought to bear on choices between formalisms, but in this paper we present one such case. The empirical pattern of interest involves wh-movement dependencies in languages that do not enforce the wh-island constraint. This pattern draws attention to two related dimensions of variation among formalisms: whether structures grow monotonically from one end to another, and whether structure-building operations are conditioned by only a finite amount of derivational state. From this perspective, we show that one class of formalisms generates the crucial empirical pattern using structures that align with mainstream syntactic analysis, and another class can only generate that same string pattern in a linguistically unnatural way. This is particularly interesting given that (i) the structurally-inadequate formalisms are strictly more powerful than the structurally-adequate ones from the perspective of weak generative capacity, and (ii) the formalism based on derivational operations that appear on the surface to align most closely with the mechanisms adopted in contemporary work in syntactic theory (merge and move) are the formalisms that fail to align with the analyses proposed in that work when the phenomenon is considered in full generality.

A non-factoid question answering system for prior art search

A patent gives the owner of an invention the exclusive rights to make, use and sell their invention. Before a new patent application is filed, patent lawyers are required to engage in Prior Art Search to determine the likelihood that an invention is novel, valid or to make sense of the domain. To perform this search, existing platforms utilize keywords and Boolean Logic, which disregards the syntax and semantics of natural language and thus, making the search extremely difficult. Consequently, studies regarding semantics using neural embeddings exist, but these only consider a narrow number of unidirectional words. In this study, we propose an end-to-end framework to consider bidirectional semantics, syntax and the thematic nature of natural language for prior art search. The proposed framework goes beyond keywords as input queries and takes a patent as the input. The contributions of this paper is twofold; adapting pre-trained embedding models (e.g., BERT) to address the semantics and syntax of language, followed by the second component, which exploits topic modeling to build a diversified answer that covers all themes across domains of the input patent. We evaluate the performance of the proposed framework on the CLEF-IP 2011 benchmark dataset and a real-world dataset obtained from Google patent repository and show that the proposed framework outperforms existing methods and returns meaningful results for a given patent.

Morphology and word order in Slavic languages: Insights from annotated corpora

Slavic languages are generally assumed to possess rich morphological features with free syntactic word order. Exploring this complexity trade-off can help us better understand the relationship between morphology and syntax within natural languages. However, few quantitative investigations have been carried out into this relationship within Slavic languages. Based on 34 annotated corpora from Universal Dependencies, this paper paid special attention to the correlations between morphology and syntax within Slavic languages by applying two metrics of morphological richness and two of word order freedom, respectively. Our findings are as follows. First, the quantitative metrics adopted can well capture the distributions of morphological richness and word order freedom of languages. Second, the metrics can corroborate the correlation between morphological richness and word order freedom. Within Slavic languages, this correlation is moderate and statistically significant. Precisely, the richer the morphology, the less strict the word order. Third, Slavic languages can be clustered into three subgroups based on classification models. Most importantly, ancient Slavic languages are characterized by richer morphology and more flexible word order than modern ones. Fourth, as two possible disturbing factors, corpus size does not greatly affect the results of the metrics, whereas corpus genre does play an important part in the measurements of word order freedom. Specifically, the word order of formal written genres tends to be more rigid than that of informal written and spoken ones. Overall, based on annotated corpora, the results verify the negative correlation between morphological richness and word order rigidity within Slavic languages, which might shed light on the dynamic relations between morphology and syntax of natural languages and provide quantitative instantiations of how languages encode lexical and syntactic information for the purpose of efficient communication.

Syntactic Structure Distillation Pretraining for Bidirectional Encoders

Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well on challenging tests of syntactic competence. Hence, it remains an open question whether scalable learners like BERT can become fully proficient in the syntax of natural language by virtue of data scale alone, or whether they still benefit from more explicit syntactic biases. To answer this question, we introduce a knowledge distillation strategy for injecting syntactic biases into BERT pretraining, by distilling the syntactically informative predictions of a hierarchical—albeit harder to scale—syntactic language model. Since BERT models masked words in bidirectional context, we propose to distill the approximate marginal distribution over words in context from the syntactic LM. Our approach reduces relative error by 2–21% on a diverse set of structured prediction tasks, although we obtain mixed results on the GLUE benchmark. Our findings demonstrate the benefits of syntactic biases, even for representation learners that exploit large amounts of data, and contribute to a better understanding of where syntactic biases are helpful in benchmarks of natural language understanding.

Logical Instrumentalism and Concatenation

Logical pluralism is the theory that there is more than one right logic. Logical instrumentalism is the view that a logic is a correct logic if it can be used to fruitfully pursue some deductive inquiry. Logical instrumentalism is a version of logical pluralism, since more than one logic can be used fruitfully. In this paper, I will show that a logical instrumentalist must accept linear logic as a correct logic, since linear logic is useful for studying natural language syntax. I further show that this means that the logical instrumentalist must accept a wide range of connectives, in particular concatenation. I end by explaining why this is a feature rather than a bug.

Definition of Natural Languages Using Context-free Grammer: In Context of Dari Language

Formal definition of a natural language lets computers understand it; the languages have challenging complexity to come under representation of any formal language. Modeling assumptions, using formal languages, we can approximate the languages; and better approximation we do, more complexities of formal languages are resolved. Objective is to study current researches and possibilities to model Dari as a context-free language. Researchers have worked for more than 6 decades on definition of syntax of natural languages using context-free grammer. It is crucial that computer scientists to be able to thoroughly understand about which category of formal languages the natural language exist in in. In overall, our skill to decide the grammar type of natural languages performs a significant role in our capability to parse it. This review article concludes that discussed issue of the context-freeness of a certain language is regularly reliant on upon the level of complexity of the language. The common of languages investigated in this regard are languages of European family and Indo-Aryan languages liker Dari, Arabic and Pashto have not been adequately fortunate to be devoted thorough researches in this regard.

The neural basis of combinatory syntax and semantics.

Human language allows us to create an infinitude of ideas from a finite set of basic building blocks. What is the neurobiology of this combinatory system? Research has begun to dissect the neural basis of natural language syntax and semantics by analyzing the basics of meaning composition, such as two-word phrases. This work has revealed a system of composition that involves rapidly peaking activity in the left anterior temporal lobe and later engagement of the medial prefrontal cortex. Both brain regions show evidence of shared processing between comprehension and production, as well as between spoken and signed language. Both appear to compute meaning, not syntactic structure. This Review discusses how language builds meaning and lays out directions for future neurobiological research on the combinatory system.

Finite-state methods in natural language processing and mathematics of language. Introduction to the special issue

For more than half a century, finite-state methods and mathematical linguistics have benefitted from a close relation and fruitful interaction. Both research fields aim to achieve a deeper understanding of human language by means of mathematical techniques. For the automated processing of language by computers such a mathematical basis is an indispensable prerequisite. Historically, the goal pursued by mathematical linguists to formalize natural language syntax in a computer-accessible way was one of the strongest driving forces behind the development of finite-state methods. Thus, it is no exaggeration to say that the field of finite-state methods owes its existence to a large extent to mathematical linguistics. In turn, continued research on finite-state methods has resulted in a categorization of various kinds of language classes and language aspects, together with efficient and provably correct algorithms, thus expanding our understanding of the mathematical properties of language.

Preface to the volume Languages: Bionspired Approaches

Languages, whether they be natural or artificial, are particular cases of a symbol system. And the manipulation of symbols is the stem of formal language theory. The theory of formal languages mainly originated from mathematics and generative linguistics. It was born in the middle of the 20th century as a tool for modelling and investigating the syntax of natural languages. After 1964, it developed as a separate branch with specific problems, techniques and results and since then it has had an important role in the field of computer science. Formal language theory, due to its abstract and formal properties, has been applied to a wide range of fields (besides initial linguistic motivation): economic modelling, developmental biology, cryptography, sociology... Therefore, natural languages, computer science and formal languages had a mutual influence over the years.

The corpus-driven revolution in Polish Sign Language: the interview with Dr. Paweł Rutkowski

Dr. Paweł Rutkowski is head of the Section for Sign Linguistics at the University of Warsaw. He is a general linguist and a specialist in the field of syntax of natural languages, carrying out research on Polish Sign Language (polski język migowy — PJM). He has been awarded a number of prizes, grants and scholarships by such institutions as the Foundation for Polish Science, Polish Ministry of Science and Higher Education, National Science Centre, Poland, Polish–U.S. Fulbright Commission, Kosciuszko Foundation and DAAD.Dr. Rutkowski leads the team developing the Corpus of Polish Sign Language and the Corpus-based Dictionary of Polish Sign Language, the first dictionary of this language prepared in compliance with modern lexicographical standards. The dictionary is an open-access publication, available freely at the following address: www.slownikpjm.uw.edu.pl/en/.This interview took place at eLex 2017, a biennial conference on electronic lexicography, where Dr. Rutkowski was awarded the Adam Kilgarriff Prize and gave a keynote address entitled Sign language as a challenge to electronic lexicography: The Corpus-based Dictionary of Polish Sign Language and beyond. The interview was conducted by Dr. Victoria Nyst from Leiden University, Faculty of Humanities, and Dr. Iztok Kosem from the University of Ljubljana, Faculty of Arts.