Natural Language Sentences Research Articles

The Use of Large Language Models in Combination with the Ontological Approach for the Synthesis of Natural Language Text

This research presented an approach based on the application of structured prompt instructions to large language models (LLM). Methodological foundations for using an ontology-driven system of structured prompts in interaction with ChatGPT were developed. The ChatGPT system allows expanding the knowledge base by obtaining new information from existing knowledge units based on a set of contexts. Thus, methods for generating meaningful responses in natural language were considered using a large language model using ontological approaches and natural language contexts. Using the proposed methodology, the OntoChatGPT system was developed, which effectively extracts entities from contexts, classifies them and generates appropriate responses. An experiment on the reverse synthesis of natural language sentences based on their ontological representation using large language models allows to clearly demonstrate the effectiveness of using the concept of large language models in dialogue systems. The study highlights the versatility of the methodology, emphasizing its applicability not only to ChatGPT but also to other chatbot systems based on LLMs, such as Google’s Bard utilizing the PaLM 2 LLM. The implementation of this technology is demonstrated using the Ukrainian language

How is Logical Analysis Viable?

The idea and the actual exercises of logical analysis as a philosophical method are at the heart of the emergence of analytic philosophy in the beginning of the 20th century. Although analytic philosophy is most commonly said to emerge with the critique devised by G. E. Moore and Bertrand Russell of the then-dominant idealist currents of philosophy in Britain, the central conception of analysis at work in this process is not the dominant traditional conception of analysis as decomposition of concepts, but a fairly new, distinctive conception which we find to be exemplified exclusively in Frege’s and Russell’s works (specifically Frege 1879, 1893 and Russell 1905). This distincive conception has been specified under a few various rubrics, such as paraphrastic or transformative-interpretive analysis, to separate it not only from the decompositional but the regressive conception as well, the latter characterizing the analysis-synthesis method of ancient Greek geometry. The present paper first locates logical analysis (as a philosophical method) in this picture, by proposing to define it as a definite kind of paraphrastic (or transformative/interpretive) analysis, the kind where the language in the analysans position is a logical language. The paper highlights the fact that the whole point of analyzing natural language forms by means of translating them into a logical language is essentially solving problems of inference – i.e. determining whether certain natural language sentence types follow from/contradict with/entail certain others, and shows that this particular conception of analysis is not as self-consistent a notion as it may first seem to be. The key to the argument is the undeniable connection between the meaning (at large) of a sentence form and the principal inferential relations that the sentence form is supposed (pre-analytically) to enter into with other related forms. The argument employs as a clear example the common modern logical analysis of the so-called problem of existential import of traditional categorical forms in order to problematize the viability of logical analysis, and concludes that one of the necessary conditions of the self-consistency (viability, possibility) of logical analysis is the acceptance of a certain negative thesis about meaning.

Evaluating What Others Say: The Effect of Accuracy Assessment in Shaping Mental Models of AI Systems

Forming accurate mental models that align with the actual behavior of an AI system is critical for successful user experience and interactions. One way to develop mental models is through information shared by other users. However, this social information can be inaccurate and there is a lack of research examining whether inaccurate social information influences the development of accurate mental models. To address this gap, our study investigates the impact of social information accuracy on mental models, as well as whether prompting users to validate the social information can mitigate the impact. We conducted a between-subject experiment with 39 crowdworkers where each participant interacted with our AI system that automates a workflow given a natural language sentence. We compared participants' mental models between those exposed to social information of how the AI system worked, both correct and incorrect, versus those who formed mental models through their own usage of the system. Specifically, we designed three experimental conditions: 1) validation condition that presented the social information followed by an opportunity to validate its accuracy through testing example utterances, 2) social information condition that presented the social information only, without the validation opportunity, and 3) control condition that allowed users to interact with the system without any social information. Our results revealed that the inclusion of the validation process had a positive impact on the development of accurate mental models, especially around the knowledge distribution aspect of mental models. Furthermore, participants were more willing to share comments with others when they had the chance to validate the social information. The impact of inaccurate social information on altering user mental models was found to be non-significant, while 69.23% of participants incorrectly judged the social information accuracy at least once. We discuss the implications of these findings for designing tools that support the validation of social information and thereby improve human-AI interactions.

Fully exploring object relation interaction and hidden state attention for video captioning

Video Captioning (VC) is a challenging task of automatically generating natural language sentences for describing video contents. As a video often contains multiple objects, it is comprehensively crucial to identify multiple objects and model relationships between them. Previous models usually adopt Graph Convolutional Networks (GCN) to infer relational information via object nodes, but there exist uncertainty and over-smoothing issues of relational reasoning. To tackle these issues, we propose a Knowledge Graph based Video Captioning Network (KG-VCN) by fully exploring object relation interaction, hidden state and attention enhancement. In encoding stages, we present a Graph and Convolution Hybrid Encoder (GCHE), which uses an object detector to find visual objects with bounding boxes for Knowledge Graph (KG) and Convolutional Neural Network (CNN). To model intrinsic relations between detected objects, we propose a knowledge graph based Object Relation Graph Interaction (ORGI) module. In ORGI, we design triplets (head, relation, tail) to efficiently mine object relations, and create a global node to enable adequate information flow among all graph nodes for avoiding possibly missed relations. To produce accurate and rich captions, we propose a hidden State and Attention Enhanced Decoder (SAED) by integrating hidden states and dynamically updated attention features. Our SAED accepts both relational and visual features, adopts Long Short-Term Memory (LSTM) to produce hidden states, and dynamically update attention features. Unlike existing methods, we concatenate state and attention features to predict next word sequentially. To demonstrate the effectiveness of our model, we conduct experiments on three well-known datasets (MSVD, MSR-VTT, VaTeX), and our model achieves impressive results significantly outperforming existing state-of-the-art models.

ImageSchemaNet: A framester graph for embodied commonsense knowledge

Commonsense knowledge is a broad and challenging area of research which investigates our understanding of the world as well as human assumptions about reality. Deriving directly from the subjective perception of the external world, it is intrinsically intertwined with embodied cognition. Commonsense reasoning is linked to human sense-making, pattern recognition and knowledge framing abilities. This work presents a new resource that formalizes the cognitive theory of image schemas. Image schemas are dynamic conceptual building blocks originating from our sensorimotor interactions with the physical world, and enable our sense-making cognitive activity to assign coherence and structure to entities, events and situations we experience everyday. ImageSchemaNet is an ontology that aligns pre-existing resources, such as FrameNet, VerbNet, WordNet and MetaNet from the Framester hub, to image schema theory. This article describes an empirical application of ImageSchemaNet, combined with semantic parsers, on the task of annotating natural language sentences with image schemas.

Enhancing E-Commerce Recommendation Systems with Multiple Item Purchase Data: A Bidirectional Encoder Representations from Transformers-Based Approach

This study proposes how to incorporate concurrent purchase data into e-commerce recommendation systems to improve their predictive accuracy. We identified that concurrent purchases account for about 23% of total orders on Katcher’s, a Korean e-commerce platform. Despite the prevalence of concurrent purchases, existing algorithms often overlook this aspect. We introduce a novel transformer-based recommendation algorithm to process a user’s order history, including concurrent purchases. Each order is represented as a natural language sentence, capturing the order timestamp, product names and their attribute values, their corresponding categories, and whether multiple products were purchased together in a single order (i.e., a concurrent purchase). These sentences form a sequence, which serves as a training dataset for fine-tuning Bidirectional Encoder Representations from Transformers (BERT) with the Next Sentence Prediction objective. We validate our ideas by conducting experiments on Katcher’s platform, demonstrating the proposed method’s improved prediction performance compared to existing recommendation systems, with enhanced accuracy and F1 score. Notably, the normalized discounted cumulative gain (NDCG) showed a significant improvement with a large margin. Furthermore, we demonstrate the beneficial impact of integrating concurrent purchase information on prediction performance.

ImageSchemaNet: A knowledge graph for embodied commonsense knowledge

Commonsense knowledge is a broad and challenging area of research which investigates our understanding of the world as well as human assumptions about reality. Deriving directly from the subjective perception of the external world, it is intrinsically intertwined with embodied cognition. Commonsense reasoning is linked to human sense-making, pattern recognition and knowledge framing abilities. This work presents a new resource that formalizes the cognitive theory of image schemas. Image schemas are dynamic conceptual building blocks originating from our sensorimotor interactions with the physical world, and enable our sense-making cognitive activity to assign coherence and structure to entities, events and situations we experience everyday. ImageSchemaNet is an ontology that aligns pre-existing resources, such as FrameNet, VerbNet, WordNet and MetaNet from the Framester hub, to image schema theory. This article describes an empirical application of ImageSchemaNet, combined with semantic parsers, on the task of annotating natural language sentences with image schemas.

SDN: Semantic Decoupling Network for Temporal Language Grounding.

Temporal language grounding (TLG) is one of the most challenging cross-modal video understanding tasks, which aims at retrieving the most relevant video segment from an untrimmed video according to a natural language sentence. The existing methods can be separated into two dominant types: 1) proposal-based and 2) proposal-free methods, where the former conduct contextual interactions and the latter localizes timestamps flexibly. However, the constant-scale candidates in proposal-based methods limit the localization precision and bring extra computational costs. In contrast, the proposal-free methods perform well on high-precision metrics-based on the fine-grained features but suffer from a lack of coarse-grained interactions, which cause degeneration when the video becomes complex. In this article, we propose a novel framework termed semantic decoupling network (SDN) that combines the advantages of proposal-based and proposal-free methods and overcomes their defects. It contains three key components: 1) semantic decoupling module (SDM); 2) context modeling block (CMB); and 3) semantic cross-level aggregation module (SCAM). By capturing the video-text contexts in multilevel semantics, the SDM and CMB effectively utilize the benefits of proposal-based methods. Meanwhile, the SCAM maintains the merit of proposal-free methods in that it localizes timestamps precisely. The experiments on three challenge datasets, i.e., Charades-STA, TACoS, and ActivityNet-Caption, show that our proposed SDN method significantly outperforms recent state-of-the-art methods, especially the proposal-free methods. Extensive analyses, as well as the implementation code of the proposed SDN method, are provided at https://github.com/CFM-MSG/Code_SDN.

A Benchmark for UAV-View Natural Language-Guided Tracking

We propose a new benchmark, UAVNLT (Unmanned Aerial Vehicle Natural Language Tracking), for the UAV-view natural language-guided tracking task. UAVNLT consists of videos taken from UAV cameras from four cities for vehicles on city roads. For each video, vehicles’ bounding boxes, trajectories, and natural language are carefully annotated. Compared to the existing data sets, which are only annotated with bounding boxes, the natural language sentences in our data set can be more suitable for many application fields where humans take part in the system for that language, being not only more friendly for human–computer interaction but also capable of overcoming the appearance features’ low uniqueness for tracking. We tested several existing methods on our new benchmarks and found that the performance of the existing methods was not satisfactory. To pave the way for future work, we propose a baseline method suitable for this task, achieving state-of-the-art performance. We believe our new data set and proposed baseline method will be helpful in many fields, such as smart city, smart transportation, vehicle management, etc.

Advanced Query Functionalities in Natural Logic Knowledge Bases

Natural logics are formal logics characterized by expressions that bear a resemblance to simplified natural language sentences. The inherent readability of natural logics renders them suitable also for large knowledge bases. Within the realm of natural logics, the rules of logical proof are applicable directly to the sentences constructed using this framework. This direct application ensures explainability of query inferences. In the paper we describe a specific natural logic named NaturaLog, suited for ontology-structured knowledge bases. We outline how NaturaLog can be effectively implemented into a database system to facilitate deductive querying. The primary focus of this paper lies in exploring various query functionalities and elucidating methods to achieve these capabilities, also when dealing with large-scale knowledge bases.

Explicitly Representing Syntax Improves Sentence-to-Layout Prediction of Unexpected Situations

Abstract Recognizing visual entities in a natural language sentence and arranging them in a 2D spatial layout require a compositional understanding of language and space. This task of layout prediction is valuable in text-to-image synthesis as it allows localized and controlled in-painting of the image. In this comparative study it is shown that we can predict layouts from language representations that implicitly or explicitly encode sentence syntax, if the sentences mention similar entity-relationships to the ones seen during training. To test compositional understanding, we collect a test set of grammatically correct sentences and layouts describing compositions of entities and relations that unlikely have been seen during training. Performance on this test set substantially drops, showing that current models rely on correlations in the training data and have difficulties in understanding the structure of the input sentences. We propose a novel structural loss function that better enforces the syntactic structure of the input sentence and show large performance gains in the task of 2D spatial layout prediction conditioned on text. The loss has the potential to be used in other generation tasks where a tree-like structure underlies the conditioning modality. Code, trained models, and the USCOCO evaluation set are available via Github.1

A Survey of Source Code Search: A 3-Dimensional Perspective

(Source) code search is widely concerned by software engineering researchers because it can improve the productivity and quality of software development. Given a functionality requirement usually described in a natural language sentence, a code search system can retrieve code snippets that satisfy the requirement from a large-scale code corpus, e.g., GitHub. To realize effective and efficient code search, many techniques have been proposed successively. These techniques improve code search performance mainly by optimizing three core components, including query understanding component, code understanding component, and query-code matching component. In this paper, we provide a 3-dimensional perspective survey for code search. Specifically, we categorize existing code search studies into query-end optimization techniques, code-end optimization techniques, and match-end optimization techniques according to the specific components they optimize. These optimization techniques are proposed to enhance the performance of specific components, and thus the overall performance of code search. Considering that each end can be optimized independently and contributes to the code search performance, we treat each end as a dimension. Therefore, this survey is 3-dimensional in nature, and it provides a comprehensive summary of each dimension in detail. To understand the research trends of the three dimensions in existing code search studies, we systematically review 68 relevant literatures. Different from existing code search surveys that only focus on the query end or code end or introduce various aspects shallowly (including codebase, evaluation metrics, modeling technique, etc.), our survey provides a more nuanced analysis and review of the evolution and development of the underlying techniques used in the three ends. Based on a systematic review and summary of existing work, we outline several open challenges and opportunities at the three ends that remain to be addressed in future work.

A data model for enabling deep learning practices on discovery services of cyber‐physical systems

AbstractThe W3C Web of Things (WoT) is a leading technology that facilitates dynamic information management in the Internet of Things (IoT). In most IoT scenarios, devices and their associated information change continuously, generating a large amount of data. Hence, to correctly use the information and the data generated by different devices, a new perspective of managing and ensuring data quality is recommended. Applying Data Science techniques to create the data model can help to manage and ensure data quality by creating a common schema that can be reused in future projects, as well as producing recommendations to facilitate Service Discovery. In addition, due to the dynamic devices that change over time or under specific circumstances, the data model created must be sufficiently abstract to add new instances and to support new requirements that devices should incorporate. The use of models helps to raise the abstraction level, adapting it to the continuous changes of devices by defining instances associated with the data model. This paper proposes two data models: one for Cyber‐Physical Systems (CPS) to define device information fetched by a Discovery Service, and another for applying Deep Learning in natural language problems through a Transformer approach. The latter matches user queries in natural language sentences with WoT devices or services. These data models expand the Thing Description model to help find similar CPSs by giving a confidence level to each CPS based on features such as security and the number of times the device was accessed. The results show how the proposed models support the search process of CPSs in syntactic and natural language searches. Furthermore, the four levels of the FAIR principles are validated for the proposed data models, thus ensuring the data's transparency, reproducibility, and reusability.

Automating Knowledge Acquisition for Content-Centric Cognitive Agents Using LLMs

The paper describes a system that uses large language model (LLM) technology to support automatic learning of new entries in an intelligent agent’s semantic lexicon. The process is bootstrapped by an existing non-toy lexicon and a natural language generator that converts formal, ontologically-grounded representations of meaning into natural language sentences. The learning method involves a sequence of LLM requests and includes an automatic quality control step. To date, this learning method has been applied to learning multiword expressions whose meanings are equivalent to those of transitive verbs in the agent’s lexicon. The experiment demonstrates the benefits of a hybrid learning architecture that integrates knowledge-based methods and resources with both traditional data analytics and LLMs.

Agile Logical Semantics for Natural Languages

This paper presents an agile method of logical semantics based on high-order Predicate Logic. An operator of predicate abstraction is introduced that provides a simple mechanism for logical aggregation of predicates and for logical typing. Monadic high-order logic is the natural environment in which predicate abstraction expresses the semantics of typical linguistic structures. Many examples of logical representations of natural language sentences are provided. Future extensions and possible applications in the interaction with chatbots are briefly discussed as well.

Text2EL + : Expert Guided Event Log Enrichment using Unstructured Text

Through the application of process mining, business processes can be improved on the basis of process execution data captured in event logs. Naturally, the quality of this data determines the quality of the improvement recommendations. Improving data quality is non-trivial and there is great potential to exploit unstructured text, e.g. from notes, reviews, and comments, for this purpose and to enrich event logs. To this end, this paper introduces Text2EL + a three-phase approach to enrich event logs using unstructured text. In its first phase, events and (case and event) attributes are derived from unstructured text linked to organisational processes. In its second phase, these events and attributes undergo a semantic and contextual validation before their incorporation in the event log. In its third and final phase, recognising the importance of human domain expertise, expert guidance is used to further improve data quality by removing redundant and irrelevant events. Expert input is used to train a Named Entity Recognition (NER) model with customised tags to detect event log elements. The approach applies natural language processing techniques, sentence embeddings, training pipelines and models, as well as contextual and expression validation. Various unstructured clinical notes associated with a healthcare case study were analysed and completeness, concordance, and correctness of the derived event log elements were evaluated through experiments. The results show that the proposed method is feasible and applicable.

N-gram Language Model for Chinese Function-word-centered Patterns

N-gram language modelling, a proven and effective method in NLP, is widely used to calculate the probability of a sentence in natural language. Language pattern is a linguistic level between word/character and sentence, which exists in pattern grammar. In this research, the approach of language model and language pattern are combined for the first time, and language patterns are studied by use of the N-gram model. Chinese function-word-centered patterns are extracted from the LCMC corpus, and aligned into pattern chains. The language model is trained from these chains to investigate the properties and distribution of Chinese function words, the interaction of content words and function words, and the interaction between patterns. The results indicate that there are approximately 10,000 function-word-centered patterns in the texts, which are distributed exponentially. This research summarizes the most common function-word-centered patterns and content-word-centered patterns, and discusses the interactions of patterns based on corpus data. The bigram language model of these patterns reflects the restrictions of function words. In addition, the research adopts an innovative method to visualize the interactions between patterns. This research fills the research gap between word/character and sentence and reveals basic Chinese pattern categories and the interactions between patterns, which makes a significant contribution to Chinese linguistic research, and improves the efficiency of NLP.

Comprehensive Attribute Prediction Learning for Person Search by Language.

Person search by language refers to searching for the interested pedestrian images given natural language sentences, which requires capturing fine-grained differences to accurately distinguish different pedestrians, while still far from being well addressed by most of the current solutions. In this paper, we propose the Comprehensive Attribute Prediction Learning (CAPL) method, which explicitly carries out attribute prediction learning, for improving the modeling capabilities of fine-grained semantic attributes and obtaining more discriminative visual and textual representations. First, we construct the semantic ATTribute Vocabulary (ATT-Vocab) based on sentence analysis. Second, the complementary context-wise and attribute-wise attribute predictions are simultaneously conducted to better model the high-frequency in-vocab attributes in our In-vocab Attribute Prediction (IAP) module. Third, to additionally consider the out-of-vocab semantics, we present the Attribute Completeness Learning (ACL) module for better capturing the low-frequency attributes outside the ATT-Vocab, obtaining more comprehensive representations. Combining the IAP and ACL modules together, our CAPL method has obtained the currently state-of-the-art retrieval performance on two widely-used benchmarks, i.e., CUHK-PEDES and ICFG-PEDES datasets. Extensive experiments and analyses have been carried out to validate the effectiveness and generalization capacities of our CAPL method.

DreamAnime: Learning Style-Identity Textual Disentanglement for Anime and Beyond.

Text-to-image generation models have significantly broadened the horizons of creative expression through the power of natural language. However, navigating these models to generate unique concepts, alter their appearance, or reimagine them in unfamiliar roles presents an intricate challenge. For instance, how can we exploit language-guided models to transpose an anime character into a different art style, or envision a beloved character in a radically different setting or role? This paper unveils a novel approach named DreamAnime, designed to provide this level of creative freedom. Using a minimal set of 2-3 images of a user-specified concept such as an anime character or an art style, we teach our model to encapsulate its essence through novel "words" in the embedding space of a pre-existing text-to-image model. Crucially, we disentangle the concepts of style and identity into two separate "words", thus providing the ability to manipulate them independently. These distinct "words" can then be pieced together into natural language sentences, promoting an intuitive and personalized creative process. Empirical results suggest that this disentanglement into separate word embeddings successfully captures a broad range of unique and complex concepts, with each word focusing on style or identity as appropriate. Comparisons with existing methods illustrate DreamAnime's superior capacity to accurately interpret and recreate the desired concepts across various applications and tasks. Code is available at https://github.com/chnshx/DreamAnime.

HRCL: Hierarchical Relation Contrastive Learning for Low-Resource Relation Extraction.

Low-resource relation extraction (LRE) aims to extract the relationships between given entities from natural language sentences in low-resource application scenarios, which has been an incredibly challenging task due to the limited annotated corpora. Existing studies either leverage self-training schemes to expand the scale of labeled data, while the error accumulation of pseudo-labels' selection bias provoke the gradual drift problem in subsequent relation prediction, or utilize the instance-wise contrastive learning that fails to distinguish those sentence pairs with similar semantics. To alleviate these defects, this article introduces a novel contrastive learning framework called hierarchical relation contrastive learning (HRCL) for LRE. HRCL leverages task-related instruction description and schema-constrained as prompts to generate high-level relation representations. To enhance the efficacy of contrastive learning, we further employ hierarchical affinity propagation clustering (HiPC) to derive hierarchical signals from relational feature space with a hierarchy cross-attention (HCA) mechanism and effectively optimize pair-level relation features through relation-wise contrastive learning. Exhaustive experiments have been conducted on five public relation extraction (RE) datasets in low-resource settings. The results demonstrate the effectiveness and robustness of HRCL and outperform the current state-of-the-art (SOTA) model by 6.56% on average in terms of B3F1 . Our source code is publicly available at https://github.com/Phevos75/HRCLRE.