Natural Language Expressions Research Articles

GeoEntity-type constrained knowledge graph embedding for predicting natural-language spatial relations

Natural-language spatial relations between geographic entities (geoentities) reflect diverse perceptions influenced by factors like location, culture, and linguistic conventions. These relations play a crucial role in supporting geospatial tasks, such as question answering and cognitive reasoning. While prior studies focused on a limited set of human-selected spatial terms and geometric attributes, they often overlooked essential semantic attributes. To overcome this limitation, we developed a Spatial Relation-based Knowledge Graph Embedding framework, SR-KGE, with new KG fusion functions to predict spatial relation terms among distinct geoentities. This method not only considers graph structures and the diversity of natural language expressions in the embedding and learning process, but also incorporates geoentity types as a constraint to capture spatial and semantic relations more accurately. Our experiments on two knowledge graph datasets, one small-scale and one large-scale, have both shown its superior performance in spatial relation inference compared to popular KGE models, including TransE, RotatE, and HAKE. We hope our research will advance the classic study of natural language described spatial relations in a more automated and intelligent way.

An effective framework for measuring the novelty of scientific articles through integrated topic modeling and cloud model

Novelty is a critical characteristic of innovative scientific articles, and accurately identifying novelty can facilitate the early detection of scientific breakthroughs. However, existing methods for measuring novelty have two main limitations: (1) Metadata-based approaches, such as citation analysis, are retrospective and do not alleviate the pressures of the peer review process or enable timely tracking of scientific progress; (2) Content-based methods have not adequately addressed the inherent uncertainty between the qualitative concept of novelty and the textual representation of papers. To address these issues, we propose a practical and effective framework for measuring the novelty of scientific articles through integrated topic modeling and cloud model, referred to as MNSA-ITMCM. In this framework, papers are represented as topic combinations, and novelty is reflected in the organic reorganization of these topics. We use the BERTopic model to generate semantically informed topics, and then apply a topic selection algorithm based on maximum marginal relevance to obtain a topic combination that balances similarity and diversity. Furthermore, we leverage the cloud model from fuzzy mathematics to quantify novelty, overcoming the uncertainty inherent in natural language expression and topic modeling to improve the accuracy of novelty measurement. To validate the effectiveness of our framework, we conducted empirical evaluations on papers from the Cell 2021 journal (biomedical domain) and the ICLR 2023 conference (computer science domain). Through correlation analysis and prediction error analysis, our framework demonstrated the ability to identify different types of novel papers and accurately predict their novelty levels. The proposed framework is applicable across diverse scientific disciplines and publication venues, benefiting researchers, librarians, science evaluation agencies, policymakers, and funding organizations by improving the efficiency and comprehensiveness of identifying novelty research.

ChatMol: interactive molecular discovery with natural language.

Natural language is poised to become a key medium for human-machine interactions in the era of large language models. In the field of biochemistry, tasks such as property prediction and molecule mining are critically important yet technically challenging. Bridging molecular expressions in natural language and chemical language can significantly enhance the interpretability and ease of these tasks. Moreover, it can integrate chemical knowledge from various sources, leading to a deeper understanding of molecules. Recognizing these advantages, we introduce the concept of conversational molecular design, a novel task that utilizes natural language to describe and edit target molecules. To better accomplish this task, we develop ChatMol, a knowledgeable and versatile generative pretrained model. This model is enhanced by incorporating experimental property information, molecular spatial knowledge, and the associations between natural and chemical languages. Several typical solutions including large language models (e.g. ChatGPT) are evaluated, proving the challenge of conversational molecular design and the effectiveness of our knowledge enhancement approach. Case observations and analysis offer insights and directions for further exploration of natural-language interaction in molecular discovery. Codes and data are provided in https://github.com/Ellenzzn/ChatMol/tree/main.

Language conditioned multi-scale visual attention networks for visual grounding

Visual grounding (VG) is a task that requires to locate a specific region in an image according to a natural language expression. Existing efforts on the VG task are divided into two-stage, one-stage and Transformer-based methods, which have achieved high performance. However, most of the previous methods extract visual information at a single spatial scale and ignore visual information at other spatial scales, which makes these models unable to fully utilize the visual information. Moreover, the insufficient utilization of linguistic information, especially failure to capture global linguistic information, may lead to failure to fully understand language expressions, thus limiting the performance of these models. To better address the task, we propose a language conditioned multi-scale visual attention network (LMSVA) for visual grounding, which can sufficiently utilize visual and linguistic information to perform multimodal reasoning, thus improving performance of model. Specifically, we design a visual feature extractor containing a multi-scale layer to get the required multi-scale visual features by expanding the original backbone. Moreover, we exploit pooling the output of the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model to extract sentence-level linguistic features, which can enable the model to capture global linguistic information. Inspired by the Transformer architecture, we present the Visual Attention Layer guided by Language and Multi-Scale Visual Features (VALMS), which is able to better learn the visual context guided by multi-scale visual and linguistic features, and facilitates further multimodal reasoning. Extensive experiments on four large benchmark datasets, including ReferItGame, RefCOCO, RefCOCO+ and RefCOCOg, demonstrate that our proposed model achieves the state-of-the-art performance.

SURE: Structure for Unambiguous Requirement Expression in Natural Language

This study presents three structures for clearly expressing functional requirements (FRs) and quantitative non-functional requirements (qt-NFRs). Expressing requirements with these structures will allow the understanding of requirements by stakeholders and software developers. The first structure is the SURE format, which is composed of three main sections: a title, a short definition, and a detailed description. The second proposed structure is a template to facilitate the definition of the title and description of unambiguous FRs. It is based on the application of CRUD operations on a certain entity, calling it the “CRUDE” structure. Finally, the third structure serves as a template to make it easier to clearly define the description and title of qt-NFRs. It is based on the application of system properties to computer events or actions, calling it the “PROSE” structure. In this, it is very important to specify those metric values that are desired or expected by the stakeholder. To know how much the definition of FRs and qt-NFRs improved when the proposed structures were used, 46 requirement specification documents elaborated as homework by students of the “Requirement Engineering” course offered at the University of Guayaquil between 2020 and 2022 were evaluated by five experts with more than 10 years of experience in software development for Ecuadorian companies. The findings showed that students reduced the percentage of unambiguous FRs and qt-NFRs from over 80% to about 10%. In conclusion, the findings demonstrate how crucial the three structures proposed in this paper are to helping students develop the ability to clearly express requirements.

A survey of methods for addressing the challenges of referring image segmentation

Referring image segmentation is guided by natural language descriptions to separate the target objects in an image. This task is different from semantic segmentation and instance segmentation in that it involves unique challenges such as multimodal information fusion, variability of natural language expressions, and model robustness. In recent years, the emergence of deep learning techniques has led to innovative ideas and methods for solving these problems. We systematically analyze the main challenges of referring image segmentation and summarize the existing solutions. These include strategies such as multimodal fusion, expression query, multimodal pre-training, and robustness. In addition, we provide an overview of several datasets commonly used in referring image segmentation and analyze the performance of various representative approaches in comparison to different datasets, visual backbone models and threshold settings. Our focus also extends to the challenges and future developments in the field of referring image segmentation. Our survey paper will provide a comprehensive technical reference for future researchers.

Construction grammar and procedural semantics for human-interpretable grounded language processing

Abstract Grounded language processing is a crucial component in many artificial intelligence systems, as it allows agents to communicate about their physical surroundings. State-of-the-art approaches typically employ deep learning techniques that perform end-to-end mappings between natural language expressions and representations grounded in the environment. Although these techniques achieve high levels of accuracy, they are often criticized for their lack of interpretability and their reliance on large amounts of training data. As an alternative, we propose a fully interpretable, data-efficient architecture for grounded language processing. The architecture is based on two main components. The first component comprises an inventory of human-interpretable concepts learned through task-based communicative interactions. These concepts connect the sensorimotor experiences of an agent to meaningful symbols that can be used for reasoning operations. The second component is a computational construction grammar that maps between natural language expressions and procedural semantic representations. These representations are grounded through their integration with the learned concepts. We validate the architecture using a variation on the CLEVR benchmark, achieving an accuracy of 96 %. Our experiments demonstrate that the integration of a computational construction grammar with an inventory of interpretable grounded concepts can effectively achieve human-interpretable grounded language processing in the CLEVR environment.

End-to-end Visual Grounding Based on Query Text Guidance and Multi-stage Reasoning

<p>Visual grounding locates target objects or areas in the image based on natural language expression. Most current methods extract visual features and text embeddings independently, and then carry out complex fusion reasoning to locate target objects mentioned in the query text. However, such independently extracted visual features often contain many features that are irrelevant to the query text or misleading, thus affecting the subsequent multimodal fusion module, and deteriorating target localization. This study introduces a combined network model based on the transformer architecture, which realizes more accurate visual grounding by using query text to guide visual feature generation and multi-stage fusion reasoning. Specifically, the visual feature generation module reduces the interferences of irrelevant features and generates visual features related to query text through the guidance of query text features. The multi-stage fused reasoning module uses the relevant visual features obtained by the visual feature generation module and the query text embeddings for multi-stage interactive reasoning, further infers the correlation between the target image and the query text, so as to achieve the accurate localization of the object described by the query text. The effectiveness of the proposed model is experimentally verified on five public datasets and the model outperforms state-of-the-art methods. It achieves an improvement of 1.04%, 2.23%, 1.00% and +2.51% over the previous state-of-the-art methods in terms of the top-1 accuracy on TestA and TestB of the RefCOCO and RefCOCO+ datasets, respectively.</p> <p> </p>

Cross‐modal fusion encoder via graph neural network for referring image segmentation

AbstractReferring image segmentation identifies the object masks from images with the guidance of input natural language expressions. Nowadays, many remarkable cross‐modal decoder are devoted to this task. But there are mainly two key challenges in these models. One is that these models usually lack to extract fine‐grained boundary information and gradient information of images. The other is that these models usually lack to explore language associations among image pixels. In this work, a Multi‐scale Gradient balanced Central Difference Convolution (MG‐CDC) and a Graph convolutional network‐based Language and Image Fusion (GLIF) for cross‐modal encoder, called Graph‐RefSeg, are designed. Specifically, in the shallow layer of the encoder, the MG‐CDC captures comprehensive fine‐grained image features. It could enhance the perception of target boundaries and provide effective guidance for deeper encoding layers. In each encoder layer, the GLIF is used for cross‐modal fusion. It could explore the correlation of every pixel and its corresponding language vectors by a graph neural network. Since the encoder achieves robust cross‐modal alignment and context mining, a light‐weight decoder could be used for segmentation prediction. Extensive experiments show that the proposed Graph‐RefSeg outperforms the state‐of‐the‐art methods on three public datasets. Code and models will be made publicly available at https://github.com/ZYQ111/Graph_refseg.

Intermediate quantifiers and valid syllogisms on EQ-algebras

Intermediate quantifiers are expressions of natural language, for example “most, almost all, many, a few” using which we quantify a number of some objects in a given universe. We have shown in [23] that all valid syllogisms with intermediate quantifiers are a consequence of only two algebraic inequalities and one equality. The result was obtained in the formalism of Lukasiewicz fuzzy type theory whose truth values form a linearly ordered complete MV-algebra. In this paper we will prove that the same holds if we replace MV-algebra by a much more general IEQ-algebra (involutive EQ-algebra).

Extraction of UML class diagrams using deep learning: Comparative study and critical analysis

Several approaches and tools have been proposed to facilitate the automatic generation of Unified Modeling Language (UML) class diagrams from natural language specifications, based on advances in Natural Language Processing (NLP). However, these tools suffer from difficulties due to the inherent imprecision and ambiguity commonly found in natural language expressions. In this article, we present an overview and a study of approaches and tools designed to extract UML diagrams from textual requirements using NLP and computational linguistics techniques. Furthermore, we introduce approaches employing deep learning techniques. Next, we provide a descriptive study and comparative analysis of the limitations and contributions of these automatic and semiautomatic tools, as well as solutions for improving the existing state of the art.

ВИНИКНЕННЯ ТА РОЗВИТОК ОКСФОРДСЬКОЇ ФІЛОСОФІЇ ЯК ПЕРЕДУМОВА ФОРМУВАННЯ ФІЛОСОФІЇ ПОВСЯКДЕННОЇ МОВИ

B a c k g r o u n d . According to a common prejudice, especially prevalent among philosophers in the continental tradition, the philosophy of everyday language (sometimes referred to as Oxford philosophy), with its focus on what we usually say and mean, fundamentally expresses a positivist attitude. Herbert Marcuse (1898-1979), who significantly contributed to the spread of this view, interprets the appeal to the concept analysis in these philosophers' writings as purely ideological: it fails to recognize the constructed nature of the social and views the social as a domain of brute facts where critical thinking should cease. Therefore, to grasp the essence of the ordinary language philosophy, it is crucial to identify the prerequisites for the emergence of this philosophical style and delve into its historical context. M e t h o d s . In this article, the author employs the method of historical-philosophical reconstruction to clarify the conditions of the formation and genesis of Oxford philosophy. General scientific methods of cognition – such as analysis, synthesis, and generalization – are used to elucidate key ideas and concepts that influenced the development of Oxford philosophy's method. Additionally, methods of systematization and comparison are applied. R e s u l t s . The article identifies the historical prerequisites for the formation of Oxford philosophy, specifying the key ideas and concepts that impacted the development of the ordinary language philosophy. It clarifies the distinctions and interconnections between the concepts of the philosophy of everyday language and Oxford philosophy. C o n c l u s i o n s . For ten years following the end of the Second World War, the University of Oxford became a hub of exceptionally productive philosophical activity. This period laid the groundwork for the emergence of a new and original philosophical movement known as the "Ordinary language philosophy" or "Oxford philosophy". However, equating the ordinary language philosophy entirely with Oxford philosophy is not accurate. The former term pertains specifically to the methodology of analyzing natural language expressions in the post-World War II era. In contrast, Oxford philosophy encompasses a broader historical context, including the pre-war period of philosophical activity at the University of Oxford. Thus, Oxford philosophy should be understood as a historical-philosophical stage that ultimately led to the formation of the philosophy of everyday language.

SATR: Semantics-Aware Triadic Refinement network for referring image segmentation

Referring image segmentation (RIS) is a fundamental cross-modal task that aims to predict the pixel-level segmentation mask of the target referred by a natural language expression. Existing methods usually focus on leveraging word-to-pixel interaction mechanisms to directly generate final masks, while they ignore semantic alignment between the query and visual context, as well as rich fine-grained spatial details of the referred object, resulting in inaccurate identification, blurry boundary, and missing small objects.To address these issues, we propose a Semantics-Aware Triadic Refinement (SATR) network for referring image segmentation. Specifically, to bridge the gap between visual and linguistic modalities, we propose a Language-Guided Pixel Modulation (LGPM), which utilizes word- and sentence-level features to facilitate word-to-pixel interaction and sentence-to-object alignment, respectively. Meanwhile, the LGPM is plugged into an off-the-shelf pre-trained visual backbone to jointly learn and extract multi-modal features, which avoids the tedious learning phase of low-level features from scratch. In addition, without any post-processing for refining the final mask, we design a triadic refinement decoder to selectively extract and aggregate salient object features, pixel-level details, and boundary information, which preserves the rich spatial features to generate a high-quality mask. Further, we use a multi-task strategy to capture the target-specific context during training. Experimental results demonstrate the proposed method performs favorably against the previous approaches on the challenging RefCOCO, RefCOCO+, and RefCOCOg datasets.

Fuzzy‐logical self‐organisation system modelling—Illustrating case of self‐organisation Czech restaurants during the Covid‐19 pandemic

AbstractThe aim of this paper is to show how Niklas Luhmann's concept of self‐organization can be formalized using fuzzy logic. This approach is based on the shared assumptions of systems theory and fuzzy logic and focuses on natural language expression and the complexity of social processes. It involves modelling the operations of the system in relation to the fuzziness of the environment, that is, the transformation of environmental stimuli in gradual uncertainty to binary codes of communication. Based on these operations, it is possible to formalize their unified pattern, which shows the observation of past operations and the simultaneous observation of the environment. An example is given of the formalization of restaurant self‐organization in the Czech Republic during the Covid‐19 pandemic. The model illustrates and explains how the restaurant system reduces lockdown as an environment, in other words, how the system stabilizes its behaviour in a complex pandemic period.

Three-way decisions with evaluative linguistic expressions

The theory of three-way decisions (3WD) requires dividing a finite, non-empty universe into three disjoint sets called positive, negative, and boundary regions. Three types of decisions are then made on the objects in each region: acceptance, rejection, and abstention (or non-commitment), respectively. Until today, a large number of 3WD extensions and applications have been proposed; some of the most recent ones also include aspects of linguistics. In this article, we first propose an innovative linguistic interpretation of three-way decisions, where the positive, negative, and boundary regions are constructed by means of the so-called evaluative linguistic expressions. These are expressions of natural language, such as small, medium, very short, quite roughly strong, extremely good, etc., and they are described within a logical theory based on the formal system of higher-order fuzzy logic. Furthermore, in line with our linguistic 3WD approach, we introduce the novel notion of linguistic rough sets, thus contributing to the development of Rough Set Theory. Finally, we connect the theory of linguistic three-way decisions with the standard 3WD model based on probabilistic rough sets, establishing conditions under which the two approaches coincide. Our results highlight connections between two different research areas: three-way decisions and the theory of evaluative linguistic expressions.

Dual-graph hierarchical interaction network for referring image segmentation

Referring Image Segmentation (RIS) aims to extract the object or stuff from an image according to the given natural language expression. As a representative multi-modal reasoning task, the main challenge of RIS lies in accurately understanding and aligning two types of heterogeneous data (i.e. image and text). Existing methods typically conduct this task via inexplicit cross-modal fusion toward the visual and linguistic features, which are separately extracted from different encoders and hard to capture accurate image–text alignment information due to their distinct latent representation structures. In this paper, we propose a Dual-Graph Hierarchical Interaction Network (DGHIN) to facilitate the explicit and comprehensive alignment between the image and text data. Firstly, two graphs are separately built for the initial visual and linguistic features extracted with different pre-trained encoders. By means of graph reasoning, we obtain a unified representation structure for different modalities to capture the intra-modal entities and their contexts, where each projected node incorporates the long-range dependencies into the latent representation. Then, the Hierarchical Interaction Module (HIM) is applied to the visual and linguistic graphs to extract comprehensive inter-modal correlations from the entity level and graph level, which not only capture the corresponding keywords and visual patches but also draws the whole sentence closer to the image region with the consistent context in the latent space. Extensive experiments on RefCOCO, RefCOCO+, G-Ref, and ReferIt demonstrate that the proposed DGHIN outperforms many state-of-the-art methods. Code is available at https://github.com/ZhaofengSHI/referring-DGHIN.

Non-Axiomatic Logic Modeling of English Texts for Knowledge Discovery and Commonsense Reasoning

Non-axiomatic logic (NAL) is a term-based, non-monotonic, multi-valued logic with evidence-based formal semantics. All those characteristics position NAL as an excellent candidate for modeling natural language expressions and supporting artificial agents while performing knowledge discovery and commonsense reasoning tasks. In this article, we propose a set of rules for the automatic translation of natural language (NL) text into the formal language of non-axiomatic logic (NAL). Several free available tools are used to support a previous linguistic analysis, and a common sense ontology is used to populate a background knowledge base that helps to delimit the scope and the semantics of logical formulas translated. Experimentation shows our set to be the most comprehensive NL-to-NAL translation rule set known so far. Furthermore, we included an extensive set of examples to show how our proposed set of rules can be used for translating a wide range of English statements with varying grammatical structures.

A contrastive learning framework for Event Detection via semantic type prototype representation modelling

The diversity of natural language expressions for describing events poses a challenge for the task of Event Detection (ED) with machine learning methods. To detect and classify event mentions, ED models essentially need to construct a semantic linkage between representations of the mentions and a set of target types. Unfortunately, most existing models use meaningless homogeneous one-hot vectors to represent the event type classes in ED, ignoring the fact that the event type labels also consist of meaningful words and can provide important clues for type representation learning. In this paper, we propose a Contrastive Semantic Prototype Representation Learning Framework for Event Detection (SemPRE), which exploits the pre-defined event type label words to inject the semantic information of the types and guide event detection. Specifically, we utilize pre-trained BERT to fuse text and event type into a joint representation space, and employ a contrastive-regularized module to enhance cross-type interaction. We conduct extensive experiments on the ACE 2005 and MAVEN benchmark datasets. The performance results show that our proposed SemPRE model achieves state-of-the-art performance on the datasets and outperforms existing baselines on limited annotated data and without using any external resources. Further analysis shows that our model is also effective in detecting multiple events and ambiguous trigger words.

INTERLEAVING LOGIC AND COUNTING

AbstractReasoning with quantifier expressions in natural language combines logical and arithmetical features, transcending strict divides between qualitative and quantitative. Our topic is this cooperation of styles as it occurs in common linguistic usage and its extension into the broader practice of natural language plus ‘grassroots mathematics’.We begin with a brief review of $\mathsf {FO}(\#)$ , first-order logic with counting operators and cardinality comparisons. This system is known to be of very high complexity, and drowns out finer aspects of the combination of logic and counting. We therefore move to a small fragment that can represent numerical syllogisms and basic reasoning about comparative size: monadic first-order logic with counting, $\mathsf {MFO}(\#)$ . We provide normal forms that allow for axiomatization, determine which arithmetical notions can be defined on finite and on infinite models, and conversely, we discuss which logical notions can be defined out of purely arithmetical ones, and what sort of (non-)classical logics can be induced.Next, we investigate a series of strengthenings of $\mathsf {MFO}(\#)$ , again using normal form methods. The monadic second-order version is close, in a precise sense, to additive Presburger Arithmetic, while versions with the natural device of tuple counting take us to Diophantine equations, making the logic undecidable. We also define a system $\mathsf {ML}(\#)$ that combines basic modal logic over binary accessibility relations with counting, needed to formulate ubiquitous reasoning patterns such as the Pigeonhole Principle. We prove decidability of $\mathsf {ML}(\#)$ , and provide a new kind of bisimulation matching the expressive power of the language.As a complement to the fragment approach pursued here, we also discuss two other ways of lowering the complexity of $\mathsf {FO}(\#)$ by changing the semantics of counting in natural ways. A first approach replaces cardinalities by abstract but well-motivated values of ‘mass’ or other mereological aggregating notions. A second approach keeps the cardinalities but generalizes the meaning of counting to work in models that allow dependencies between variables.Finally, we return to our starting point in natural language, confronting the architecture of our formal systems with linguistic quantifier vocabulary and syntax, as well as with natural reasoning modules such as the monotonicity calculus. In addition to these encounters with formal semantics, we discuss the role of counting in semantic evaluation procedures for quantifier expressions and determine, for instance, which binary quantifiers are computable by finite ‘semantic automata’. We conclude with some general thoughts on yet further entanglements of logic and counting in formal systems, on rethinking the qualitative/quantitative divide, and on connecting our analysis to empirical findings in cognitive science.

Reasoning with Concepts: A Unifying Framework

Over the past few decades, cognitive science has identified several forms of reasoning that make essential use of conceptual knowledge. Despite significant theoretical and empirical progress, there is still no unified framework for understanding how concepts are used in reasoning. This paper argues that the theory of conceptual spaces is capable of filling this gap. Our strategy is to demonstrate how various inference mechanisms which clearly rely on conceptual information—including similarity, typicality, and diagnosticity-based reasoning—can be modeled using principles derived from conceptual spaces. Our first topic analyzes the role of expectations in inductive reasoning and their relation to the structure of our concepts. We examine the relationship between using generic expressions in natural language and common-sense reasoning as a second topic. We propose that the strength of a generic can be described by distances between properties and prototypes in conceptual spaces. Our third topic is category-based induction. We demonstrate that the theory of conceptual spaces can serve as a comprehensive model for this type of reasoning. The final topic is analogy. We review some proposals in this area, present a taxonomy of analogical relations, and show how to model them in terms of distances in conceptual spaces. We also briefly discuss the implications of the model for reasoning with concepts in artificial systems.