Syntactic Distance Research Articles

The Optimal Placement of the Head in the Noun Phrase. The Case of Demonstrative, Numeral, Adjective and Noun

ABSTRACT The word order of a sentence is shaped by multiple principles. The principle of syntactic dependency distance minimization is in conflict with the principle of surprisal minimization (or predictability maximization) in single head syntactic dependency structures: while the former predicts that the head should be placed at the centre of the linear arrangement, the latter predicts that the head should be placed at one of the ends (either first or last). A critical question is when surprisal minimization (or predictability maximization) should surpass syntactic dependency distance minimization. In the context of single head structures, it has been predicted that this is more likely to happen when two conditions are met, i.e. (a) fewer words are involved and (b) words are shorter. Here, we test the prediction on the noun phrase when it is composed of demonstrative, numeral, adjective, and noun. We find that, across preferred orders in languages, the noun tends to be placed at one of the ends, confirming the theoretical prediction. We also show evidence of anti-locality effects: syntactic dependency distances in preferred orders are longer than expected by chance.

Exploring language relations through syntactic distances and geographic proximity

Languages are grouped into families that share common linguistic traits. While this approach has been successful in understanding genetic relations between diverse languages, more analyses are needed to accurately quantify their relatedness, especially in less studied linguistic levels such as syntax. Here, we explore linguistic distances using series of parts of speech (POS) extracted from the Universal Dependencies dataset. Within an information-theoretic framework, we show that employing POS trigrams maximizes the possibility of capturing syntactic variations while being at the same time compatible with the amount of available data. Linguistic connections are then established by assessing pairwise distances based on the POS distributions. Intriguingly, our analysis reveals definite clusters that correspond to well known language families and groups, with exceptions explained by distinct morphological typologies. Furthermore, we obtain a significant correlation between language similarity and geographic distance, which underscores the influence of spatial proximity on language kinships.

Working memory and L2 gender agreement processing in structural distance conditions

Previous studies examining the role of executive function in L2 processing show that working memory (WM) facilitates the processing of agreement in local domains in adult second language (L2) learners. Furthermore, other studies explored whether L2 learners can establish the agreement operation across phrases (i.e., structural distance) and whether WM intervenes in the said linguistic computation. However, these studies have often included both syntactic and linear distance in their stimuli, making it impossible to discern whether WM effects emerge from physical or syntactic reasons. The present study assesses how verbal WM updating and L2 proficiency modulate syntactic processing. Beginner and advanced adult English L2 learners of Spanish and Spanish monolinguals completed a verbal WM updating task, and a self-paced reading task containing Spanish sentences with gender agreement and disagreement within and across phrases. Results show that Spanish monolinguals exhibited sensitivity to gender agreement violations in local domains and in structural distance conditions, while beginner L2 learners were not sensitive to violations in either condition. Advanced learners, on the other hand, detected violations in local domains, and their verbal WM updating spans were associated with sensitivity to violations across phrases. Taken together, the findings suggest that (a) morphosyntactically complex structures consume cognitive resources in great number, and (b) L2 processing is qualitatively similar albeit quantitatively different from native processing, thus providing evidence that late bilinguals may process the L2 in a native-like manner.

Neural Networks as Cognitive Models of the Processing of Syntactic Constraints.

Languages are governed by syntactic constraints-structural rules that determine which sentences are grammatical in the language. In English, one such constraint is subject-verb agreement, which dictates that the number of a verb must match the number of its corresponding subject: "the dogs run", but "the dog runs". While this constraint appears to be simple, in practice speakers make agreement errors, particularly when a noun phrase near the verb differs in number from the subject (for example, a speaker might produce the ungrammatical sentence "the key to the cabinets are rusty"). This phenomenon, referred to as agreement attraction, is sensitive to a wide range of properties of the sentence; no single existing model is able to generate predictions for the wide variety of materials studied in the human experimental literature. We explore the viability of neural network language models-broad-coverage systems trained to predict the next word in a corpus-as a framework for addressing this limitation. We analyze the agreement errors made by Long Short-Term Memory (LSTM) networks and compare them to those of humans. The models successfully simulate certain results, such as the so-called number asymmetry and the difference between attraction strength in grammatical and ungrammatical sentences, but failed to simulate others, such as the effect of syntactic distance or notional (conceptual) number. We further evaluate networks trained with explicit syntactic supervision, and find that this form of supervision does not always lead to more human-like syntactic behavior. Finally, we show that the corpus used to train a network significantly affects the pattern of agreement errors produced by the network, and discuss the strengths and limitations of neural networks as a tool for understanding human syntactic processing.

Semantic Similarity Analysis via Syntax Dependency Structure and Gate Recurrent Unit

Sentences are composed of words, phrases, and clauses. The relationship between them is usually tree-like. In the hierarchical structure of the sentence, the dependency relationships between different components affect the syntactic structure. Syntactic structure is very important for understanding the meaning of the whole sentence. However, the gated recursive unit (GRU) models cannot fully encode hierarchical syntactic dependencies, which leads to its poor performance in various natural language tasks. In this paper, a model called relative syntactic distance bidirectional gated recursive unit (RSD-BiGRU) is constructed to capture syntactic structure dependencies. The model modifies the gating mechanism in GRU through relative syntactic distance. It also offers a transformation gate to model the syntactic structure more directly. Embedding sentence meanings with sentence structure dependency into dense vectors. This model is used to conduct semantic similarity experiments on the QQP and SICK datasets. The results show that the sentence representation obtained by RSD-BiGRU model contains more semantic information. This is helpful for semantic similarity analysis tasks.

Multi-Dimensional Validation of the Integration of Syntactic and Semantic Distance Measures for Clustering Fibromyalgia Patients in the Rheumatic Monitor Big Data Study.

This study primarily aimed at developing a novel multi-dimensional methodology to discover and validate the optimal number of clusters. The secondary objective was to deploy it for the task of clustering fibromyalgia patients. We present a comprehensive methodology that includes the use of several different clustering algorithms, quality assessment using several syntactic distance measures (the Silhouette Index (SI), Calinski-Harabasz index (CHI), and Davies-Bouldin index (DBI)), stability assessment using the adjusted Rand index (ARI), and the validation of the internal semantic consistency of each clustering option via the performance of multiple clustering iterations after the repeated bagging of the data to select multiple partial data sets. Then, we perform a statistical analysis of the (clinical) semantics of the most stable clustering options using the full data set. Finally, the results are validated through a supervised machine learning (ML) model that classifies the patients back into the discovered clusters and is interpreted by calculating the Shapley additive explanations (SHAP) values of the model. Thus, we refer to our methodology as the clustering, distance measures and iterative statistical and semantic validation (CDI-SSV) methodology. We applied our method to the analysis of a comprehensive data set acquired from 1370 fibromyalgia patients. The results demonstrate that the K-means was highly robust in the syntactic and the internal consistent semantics analysis phases and was therefore followed by a semantic assessment to determine the optimal number of clusters (k), which suggested k = 3 as a more clinically meaningful solution, representing three distinct severity levels. the random forest model validated the results by classification into the discovered clusters with high accuracy (AUC: 0.994; accuracy: 0.946). SHAP analysis emphasized the clinical relevance of "functional problems" in distinguishing the most severe condition. In conclusion, the CDI-SSV methodology offers significant potential for improving the classification of complex patients. Our findings suggest a classification system for different profiles of fibromyalgia patients, which has the potential to improve clinical care, by providing clinical markers for the evidence-based personalized diagnosis, management, and prognosis of fibromyalgia patients.

Modeling Structured Dependency Tree with Graph Convolutional Networks for Aspect-Level Sentiment Classification.

Aspect-based sentiment analysis is a fine-grained task where the key goal is to predict sentiment polarities of one or more aspects in a given sentence. Currently, graph neural network models built upon dependency trees are widely employed for aspect-based sentiment analysis tasks. However, most existing models still contain a large amount of noisy nodes that cannot precisely capture the contextual relationships between specific aspects. Meanwhile, most studies do not consider the connections between nodes without direct dependency edges but play critical roles in determining the sentiment polarity of an aspect. To address the aforementioned limitations, we propose a Structured Dependency Tree-based Graph Convolutional Network (SDTGCN) model. Specifically, we explore construction of a structured syntactic dependency graph by incorporating positional information, sentiment commonsense knowledge, part-of-speech tags, syntactic dependency distances, etc., to assign arbitrary edge weights between nodes. This enhances the connections between aspect nodes and pivotal words while weakening irrelevant node links, enabling the model to sufficiently express sentiment dependencies between specific aspects and contextual information. We utilize part-of-speech tags and dependency distances to discover relationships between pivotal nodes without direct dependencies. Finally, we aggregate node information by fully considering their importance to obtain precise aspect representations. Experimental results on five publicly available datasets demonstrate the superiority of our proposed model over state-of-the-art approaches; furthermore, the accuracy and F1-score show a significant improvement on the majority of datasets, with increases of 0.74, 0.37, 0.65, and 0.79, 0.75, 1.17, respectively. This series of enhancements highlights the effective progress made by the STDGCN model in enhancing sentiment classification performance.

Label-Free Data Mining of Scientific Literature by Unsupervised Syntactic Distance Analysis.

Label-free data mining can efficiently feed large amounts of data from the vast scientific literature into artificial intelligence (AI) processing systems. Here, we demonstrate an unsupervised syntactic distance analysis (SDA) approach that is capable of mining chemical substances, functions, properties, and operations without annotation. This SDA approach was evaluated in several areas of research from the physical sciences and achieved performance in information mining comparable to that of supervised learning, as shown by its satisfactory scores of 0.62-0.72, 0.60-0.82, and 0.86-0.95 in precision, recall, and accuracy, respectively. We also showcase how our approach can assist robotic chemists programmed to perform research focused on double-perovskite colloidal nanocrystals, gold colloidal nanocrystals, oxygen evolution reaction catalysts, and enzyme-like catalysts by designing materials, formulations, and synthesis parameters based on data mined from 1.1 million literature references.

MIFINN: A novel multi-information fusion and interaction neural network for aspect-based sentiment analysis

Aspect-based Sentiment Analysis (ABSA) is used to detect corresponding sentiment polarities toward different aspect terms. In recent years, graph convolutional network-based methods achieve great success for ABSA. However, previous studies ignore syntactic dependency types and aspect-related syntactic distances. Besides, effectively fusing semantic and syntactic information remains a challenging problem. Previous studies also neglect interactions between aspect terms and context. To alleviate the above problems, we develop a Multi-Information Fusion and Interaction Neural Network (MIFINN), which consists of an Aspect and Syntax-Aware Multi-Information Fusion Graph Convolutional Network (ASAMIFGCN) and a Multi-Information Interaction and Gating Network (MIIGN). The ASAMIFGCN model can integrate and fuse contextual semantics, syntactic dependency types, and aspect-related syntactic distances. Specifically, two different attentions are designed to learn global and aspect-related semantic and syntactic information, respectively. A syntactic distance gating module is proposed to acquire more precise score matrices. Besides, we propose a semantic and syntactic association module to fuse multi-information simultaneously. The MIIGN model can effectively interact aspect terms and context. Specifically, two interactive attentions are developed to learn different weights toward aspect terms and context via interactions between each other, respectively. Moreover, we propose a gating information module for controlling the information flow of aspect terms and context. We conduct experiments on multiple datasets and MIFINN achieves state-of-the-art performance. The results also demonstrate that effectively fusing and interacting multi-information can improve model performance for ABSA significantly.

Morphologic, Syntactic, and Phonologic Distance Between Japanese and Altaic, Dravidian, Austronesian, and Korean Languages

The present study measures the resemblances of Japanese with Altaic languages (Turkic; Tungstic; Mongolic; Nivkh); the Dravidian language Tamil; Austronesian languages (Western Malayo-Polynesian; Malayo-Sumbawan; Central Luzon; Central Malayo-Polynesian), and Korean, in an effort to pin down the genealogy of Japanese. Morphologic, syntactic, and phonologic distance are calculated using data from corpora. The chi-square homogeneity test and Euclidean distances are used for statistical analysis. The finding brings to light, morphologically, in the light of preferences of causative/inchoative verb alternation patterning and morphemes that convey the alternation, that Japanese and Korean are close for the most part. Syntactically, Altaics and Tamil convey case via suffixes; case in Austronesian languages is marked by prefixes. Japanese and Korean share a similarity in rendering case with particles. Phonologically, the Tamil and Austronesian languages share a resemblance in the harmony of vowel height. The Korean, Altaic languages, and Austronesian languages show similarities in the harmony of vowel backness. Japanese, the Altaic languages, and the Austronesian language Madurese display vowel-consonant harmony. Pulling these strands together, a conclusion is thus drawn that Japanese is most closely related to Korean.

Comparing Germanic, Romance and Slavic: Relationships among linguistic distances

Languages differ along multiple dimensions (lexis, phonology, morphology, syntax). Related languages descend from a common ancestor language but have diverged over time. This paper asks whether languages diverge equally along all dimensions, and, to the extent that they do not, which dimension reflects the traditional language family tree best. We computed measures of (i) lexical distance (ii) phonetic distance, and (iii) syntactic distance. The measures were computed on all words and sentences extracted from a corpus of translations of four relatively short English texts into another four Germanic languages (Danish, Dutch, German, Swedish), five Romance languages (French, Italian, Portuguese, Romanian, Spanish) and six Slavic languages (Bulgarian, Croatian, Czech, Polish, Slovakian, Slovenian). We examined the correlation structure of the distances for all pairs of Germanic (10), Romance (10) and Slavic (15) languages (i.e., within-family comparisons only). The results indicate that the linguistic dimensions are generally correlated (weakly but significantly), and that the correlations are stronger for pairs within families than when all 35 pairs are examined together. Cladistic family trees correlate best with the lexical distance (0.851 < r < 0.887). This confirms that the genealogical language trees are predominantly based on lexical rather than phonetic or syntactic considerations.

Following Hillary Clinton and Donald Trump: A dissection of their tweets in the 2016 U.S. presidential election

In this work, the tweets of Donald Trump and Hillary Clinton in the 2016 U.S. presidential campaign are studied and analyzed from a language-based perspective. The tweets are divided into two batches. The first is from the earliest announcement of candidacy until the last announcement of nomination of candidates. The second is between the end of the first interval and the inauguration of Trump. Readability statistics of tweets are computed and the Scholastic Aptitude Test (SAT) words — subject of much cramming by test takers — in the tweets are analyzed, as well as Ogden’s Simple English words. Some of the readability indexes exhibit minor differences, implying that Clinton’s tweets are more readable whereas the other readability indexes are proximate for the candidates. Clinton’s use of unique SAT words is found to be denser than Trump’s, indicating that employing such words less might be wiser for political campaigns. Simple English analysis does not tell of a noticeable difference. Syntactic Dependency Distance of tweets and Integrative Complexity of tweets were also analyzed but no significant difference for the two candidates was discerned.

Deep adversarial neural network model based on information fusion for music sentiment analysis

Natural language processing (NLP) is a computer-based technology used to process natural language information in written and spoken form that is unique to human society. In the process of mining massive text information, a variety of technologies and research directions in the field of NLP have gradually emerged. And sentiment analysis is an important research direction, which has important research value and practical application value for enterprises and social life. Sentiment analysis is basically a single mining of semantic or grammatical information without establishing the correlation between semantic information and grammatical information. In addition, previous models simply embed the relative distance or grammatical distance of words into the model, ignoring the joint influence of relative distance and grammatical distance on the aspect words. In this paper, we propose a new model that combines deep adversarial neural network model based on information fusion for music sentiment analysis. Firstly, the information of music text sequence is captured by the bidirectional short and long time memory network. Then the sequence information is updated according to the tree structure of dependency syntactic tree. Then, the relative distance and syntactic distance position information are embedded into the music text sequence. Thirdly, the adversarial training is used to expand the alignment boundary of the field distribution and effectively alleviate the problem of fuzzy features leading to misclassification. Semantic information and syntactic information are optimized by attention mechanism. Finally, the fused information is input into the Softmax classifier for music sentiment classification. Experimental results on open data sets show that compared with other advanced methods, the recognition accuracy of the proposed method is more than 90%.

Extract Similarities from Syntactic Contexts: a Distributional Semantic Model Based on Syntactic Distance

Distributional Semantics (DS) models are based on the idea that two words which appear in similar contexts, i.e. similar neighborhoods, have similar meanings. This concept was originally presented by Harris in his Distributional Hypothesis (DH) (Harris 1954). Even though DH forms the basis of the majority of DS models, Harris states in later works that only syntactic analysis can allow for a more precise formulation of the neighborhoods involved: the arguments and the operators.In this work, we present a DS model based on the concept of Syntactic Distance inspired by a study of Harris’s theories concerning the syntactic-semantic interface. In our model, the context of each word is derived from its dependency network generated by a parser. With this strategy, the co-occurring terms of a target word are calculated on the basis of their syntactic relations, which are also preserved in the event of syntactical transformations. The model, named Syntactic Distance as Word Window (SD-W2), has been tested on three state-of-the-art tasks: Semantic Distance, Synonymy and Single Word Priming, and compared with other classical DS models. In addition, the model has been subjected to a new test based on Operator-Argument selection. Although the results obtained by SD-W2 do not always reach those of modern contextualized models, they are often above average and, in many cases, they are comparable with the result of GLOVE or BERT.

Contextual Graph Attention Network for Aspect-Level Sentiment Classification

Aspect-level sentiment classification aims to predict the sentiment polarities towards the target aspects given in sentences. To address the issues of insufficient semantic information extraction and high computational complexity of attention mechanisms in existing aspect-level sentiment classification models based on deep learning, a contextual graph attention network (CGAT) is proposed. The proposed model adopts two graph attention networks to aggregate syntactic structure information into target aspects and employs a contextual attention network to extract semantic information in sentence-aspect sequences, aiming to generate aspect-sensitive text features. In addition, a syntactic attention mechanism based on syntactic relative distance is proposed, and the Gaussian function is cleverly introduced as a syntactic weight function, which can reduce computational complexities and effectively highlight the words related to aspects in syntax. Experiments on three public sentiment datasets show that the proposed model can make better use of semantic information and syntactic structure information to improve the accuracy of sentiment classification.

A syntactic distance sensitive neural network for event argument extraction

A syntactic distance sensitive neural network for event argument extraction

A Local context focus learning model for joint multi-task using syntactic dependency relative distance

A Local context focus learning model for joint multi-task using syntactic dependency relative distance

Effects of Syntactic Distance and Word Order on Language Processing: An Investigation Based on a Psycholinguistic Treebank of English

We conducted a broad-coverage investigation of the effects of syntactic distance and word order on language processing against a dependency-annotated reading time corpus of English. A combined method of quantitative syntax and psycholinguistic analyses was adopted to yield converging evidence. It was found that (i) head-initial structures allow greater structural complexity, i.e., larger head-dependent distance, than head-final structures in both language comprehension and production; (ii) within the capacity limit of working memory, syntactic distance is a positive predictor of reading time for a word with a preceding head, whereas a negative predictor of reading timefor a word with a following head; and (iii) at the sentence level, syntactic distance is a significant predictor of sentencereading time. These results suggest that (i) different word orders may enjoy different processing mechanisms in terms of cognitive difficulty and processes, which can be explained by an incremental language parser; and (ii) in addition to distance, word order should also be considered as a factor affecting language processing, which is an important extension to distance-based language processing models. Taken as a whole, our study paves the way for corpus-based integration of quantitative linguistic and psycholinguistic methods into understanding language processing and its underlying cognitive mechanisms.

LPInsider: a webserver for lncRNA\u2013protein interaction extraction from the literature

BackgroundLong non-coding RNA (LncRNA) plays important roles in physiological and pathological processes. Identifying LncRNA–protein interactions (LPIs) is essential to understand the molecular mechanism and infer the functions of lncRNAs. With the overwhelming size of the biomedical literature, extracting LPIs directly from the biomedical literature is essential, promising and challenging. However, there is no webserver of LPIs relationship extraction from literature.ResultsLPInsider is developed as the first webserver for extracting LPIs from biomedical literature texts based on multiple text features (semantic word vectors, syntactic structure vectors, distance vectors, and part of speech vectors) and logistic regression. LPInsider allows researchers to extract LPIs by uploading PMID, PMCID, PMID List, or biomedical text. A manually filtered and highly reliable LPI corpus is integrated in LPInsider. The performance of LPInsider is optimal by comprehensive experiment on different combinations of different feature and machine learning models.ConclusionsLPInsider is an efficient analytical tool for LPIs that helps researchers to enhance their comprehension of lncRNAs from text mining, and also saving their time. In addition, LPInsider is freely accessible from http://www.csbg-jlu.info/LPInsider/ with no login requirement. The source code and LPIs corpus can be downloaded from https://github.com/qiufengdiewu/LPInsider.

Unsupervised and few-shot parsing from pretrained language models

Pretrained language models are generally acknowledged to be able to encode syntax [46,16,13]. In this article, we propose UPOA, an Unsupervised constituent Parsing model that calculates an Out Association score solely based on the self-attention weight matrix learned in a pretrained language model as the syntactic distance for span segmentation. We further propose an enhanced version, UPIO, which exploits both inside association and outside association scores for estimating the likelihood of a span. Experiments with UPOA and UPIO disclose that the linear projection matrices for the query and key in the self-attention mechanism play an important role in parsing. We therefore extend the unsupervised models to few-shot parsing models (FPOA, FPIO) that use a few annotated trees to learn better linear projection matrices for parsing. Experiments on the Penn Treebank demonstrate that our unsupervised parsing model UPIO achieves results comparable to the state of the art on short sentences (length <= 10). Our few-shot parsing model FPIO trained with only 20 annotated trees outperforms a previous few-shot parsing method trained with 50 annotated trees. Experiments on cross-lingual parsing show that both unsupervised and few-shot parsing methods are better than previous methods on most languages of SPMRL [39].