Natural Language Patterns Research Articles

A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View

Machine learning is one of the most prevailing techniques in computer science, and it has been widely applied in image processing, natural language processing, pattern recognition, cybersecurity, and other fields. Regardless of successful applications of machine learning algorithms in many scenarios, e.g., facial recognition, malware detection, automatic driving, and intrusion detection, these algorithms and corresponding training data are vulnerable to a variety of security threats, inducing a significant performance decrease. Hence, it is vital to call for further attention regarding security threats and corresponding defensive techniques of machine learning, which motivates a comprehensive survey in this paper. Until now, researchers from academia and industry have found out many security threats against a variety of learning algorithms, including naive Bayes, logistic regression, decision tree, support vector machine (SVM), principle component analysis, clustering, and prevailing deep neural networks. Thus, we revisit existing security threats and give a systematic survey on them from two aspects, the training phase and the testing/inferring phase. After that, we categorize current defensive techniques of machine learning into four groups: security assessment mechanisms, countermeasures in the training phase, those in the testing or inferring phase, data security, and privacy. Finally, we provide five notable trends in the research on security threats and defensive techniques of machine learning, which are worth doing in-depth studies in future.

Constraint conjunction in weighted probabilistic grammar

This paper examines a key difference between constraint conjunction and constraint weight additivity, arguing that the two do not have the same empirical coverage. In particular, constraint conjunction in weighted probabilistic grammar allows for superadditive constraint interaction, where the effect of violating two constraints goes beyond the additive combination of the two constraints’ weights alone. A case study from parasitic tone harmony in Dioula d'Odienné demonstrates superadditive local and long-distance segmental feature similarities that increase the likelihood of tone harmony. Superadditivity in Dioula d'Odienné is formally captured in Maximum Entropy Harmonic Grammar by weighted constraint conjunction. Counter to previous approaches that supplant constraint conjunction with weight additivity in Harmonic Grammar, information-theoretic model comparison reveals that weighted constraint conjunction improves the grammar's explanatory power when modelling quantitative natural language patterns.

DDM at Work: Reply to comments on “Dependency distance: A new perspective on syntactic patterns in natural languages”

We provide responses to the commentaries in this volume to evaluate, clarify and extend some of the arguments in Dependency distance: A new perspective on syntactic patterns in natural languages. Evidences show that DDM (dependency distance minimization) is an important linguistic universal, biologically or cognitively motivated, in shaping the language system. As a general tendency, DDM works quite well in theoretical argumentations as well as practical applications. However, this does not mean that DDM is the only linguistic universal that works: it is highly possible that other factors, which might be biologically, physically, socially or culturally motivated, work as well to jointly mold languages.

Dependency distances in natural mixed languages: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distances in natural mixed languages: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Generalizing dependency distance: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

With the support of the comprehensive review in Liu et al. [14], we consider dependency distance minimization to be firmly established as a quantitative property of syntactic trees. In this comment, we consider future empirical and theoretical directions for this concept, including a recent information-theoretic reinterpretation of dependency locality effects as proposed by Futrell and Levy [4].

Towards a theory of word order: Comment on “Dependency distance: a new perspective on syntactic patterns in natural language” by Haitao Liu et al.

Comment on "Dependency distance: a new perspective on syntactic patterns in natural language" by Haitao Liu et al

Planned experiments and corpus based research play a complementary role: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Planned experiments and corpus based research play a complementary role: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Projectivity is the mathematical code of syntax: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Projectivity is the mathematical code of syntax: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance: A new perspective on the syntactic development in second language acquisition: Comment on “Dependency distance: A new perspective on syntactic patterns in natural language” by Haitao Liu et al.

Dependency distance: A new perspective on the syntactic development in second language acquisition: Comment on “Dependency distance: A new perspective on syntactic patterns in natural language” by Haitao Liu et al.

Cross-language dependency treebank—An effective approach for contrastive study of languages: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Cross-language dependency treebank—An effective approach for contrastive study of languages: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance distribution – from the perspective of genre variation: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance distribution – from the perspective of genre variation: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Revisiting syntactic development in deaf and hearing children from a dependency approach: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Revisiting syntactic development in deaf and hearing children from a dependency approach: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance in language evolution: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance in language evolution: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Coevolution of dependency distance, hierarchical structure and word order: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Coevolution of dependency distance, hierarchical structure and word order: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance minimization in understanding of ambiguous structure: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Dependency distance minimization in understanding of ambiguous structure: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Computer simulation as an important approach to explore language universal: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Computer simulation as an important approach to explore language universal: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Cross-language diversity, head-direction and grammars: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Cross-language diversity, head-direction and grammars: Comment on “Dependency distance: A new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

On the relation between dependency distance, crossing dependencies, and parsing: Comment on “Dependency distance: a new perspective on syntactic patterns in natural languages” by Haitao Liu et al.

Liu et al. (2017) provide a comprehensive account of research on dependency distance in human languages. While the article is a very rich and useful report on this complex subject, here I will expand on a few specific issues where research in computational linguistics (specifically natural language processing) can inform DDM research, and vice versa. These aspects have not been explored much in the article by Liu et al. or elsewhere, probably due to the little overlap between both research communities, but they may provide interesting insights for improving our understanding of the evolution of human languages, the mechanisms by which the brain processes and understands language, and the construction of effective computer systems to achieve this goal.

Geographical localization of web domains and organization addresses recognition by employing natural language processing, Pattern Matching and clustering

Nowadays, the World Wide Web is growing at increasing rate and speed, and consequently the online available resources populating Internet represent a large source of knowledge for various business and research interests. For instance, over the past years, increasing attention has been focused on retrieving information related to geographical location of places and entities, which is largely contained in web pages and documents. However, such resources are represented in a wide variety of generally unstructured formats, and this actually does not help final users to find desired information items. The automatic annotation and comprehension of toponyms, location names and addresses (at different resolution and granularity levels) can deliver significant benefits for the whole web community by improving search engines filtering capabilities and intelligent data mining systems. The present paper addresses the problem of gathering geographical information from unstructured text in web pages and documents. In the specific, the proposed method aims at extracting geographical location (at street number resolution) of commercial companies and services, by annotating geo-related information from their web domains. The annotation process is based on Natural Language Processing (NLP) techniques for text comprehension, and relies on Pattern Matching and Hierarchical Cluster Analysis for recognizing and disambiguating geographical entities. Geotagging performances have been assessed by evaluating Precision, Recall and F-Measure of the proposed system output (represented in form of semantic RDF triples) against both a geo-annotated reference database and a semantic Smart City repository.

An automated framework for detection and resolution of cross references in legal texts

When identifying and elaborating compliance requirements, analysts need to follow the cross references in legal texts and consider the additional information in the cited provisions. Enabling easier navigation and handling of cross references requires automated support for the detection of the natural language expressions used in cross references, the interpretation of cross references in their context, and the linkage of cross references to the targeted provisions. In this article, we propose an approach and tool support for automated detection and resolution of cross references. The approach leverages the structure of legal texts, formalized into a schema, and a set of natural language patterns for legal cross reference expressions. These patterns were developed based on an investigation of Luxembourg's legislation, written in French. To build confidence about their applicability beyond the context where they were observed, these patterns were validated against the Personal Health Information Protection Act (PHIPA) by the Government of Ontario, Canada, written in both French and English. We report on an empirical evaluation where we assess the accuracy and scalability of our framework over several Luxembourgish legislative texts as well as PHIPA.