Semantic Description Research Articles

A comprehensive review on zero-shot-learning techniques

Advancements in computational capabilities have enabled the implementation of advanced deep learning models across various domains of knowledge, yet the increasing complexity and scarcity of data in specialized areas pose significant challenges. Zero-shot learning (ZSL), a subset of transfer learning, has emerged as an innovative solution to these challenges, focusing on classifying unseen categories present in the test set but absent during training. Unlike traditional methods, ZSL utilizes semantic descriptions, like attribute lists or natural language phrases, to map intermediate features from the training data to unseen categories effectively, enhancing the model’s applicability across diverse and complex domains. This review provides a concise synthesis of the advancements, methodologies, and applications in the field of zero-shot learning, highlighting the milestones achieved and possible future directions. We aim to offer insights into the contemporary developments in ZSL, serving as a comprehensive reference for researchers exploring the potentials and challenges of implementing ZSL-based methodologies in real-world scenarios.

Contrastive Prototype-Guided Generation for Generalized Zero-Shot Learning

Generalized zero-shot learning (GZSL) aims to recognize both seen and unseen classes, while only samples from seen classes are available for training. The mainstream methods mitigate the lack of unseen training data by simulating the visual unseen samples. However, the sample generator is actually learned with just seen-class samples, and semantic descriptions of unseen classes are just provided to the pre-trained sample generator for unseen data generation, therefore, the generator would have bias towards seen categories, and the unseen generation quality, including both precision and diversity, is still the main learning challenge. To this end, we propose a Prototype-Guided Generation for Generalized Zero-Shot Learning (PGZSL), in order to guide the sample generation with unseen knowledge. First, unseen data generation is guided and rectified in PGZSL by contrastive prototypical anchors with both class semantic consistency and feature discriminability. Second, PGZSL introduces Certainty-Driven Mixup for generator to enrich the diversity of generated unseen samples, while suppress the generation of uncertain boundary samples as well. Empirical results over five benchmark datasets show that PGZSL significantly outperforms the SOTA methods in both ZSL and GZSL tasks.

Consistency-guided pseudo labeling for transductive zero-shot learning

Zero-shot learning (ZSL) aims to recognize unseen classes during training. Transductive methods have advanced in ZSL, however, often rely on pseudo labels based on confidence scores, leading to semantic misalignment between unseen-class image features and corresponding class semantic descriptions due to noisy pseudo labels. In this paper, we introduce a novel Consistency-Guided Pseudo-Labeling (CGPL) to generate high-quality pseudo labels, achieving robust mapping from visual to semantic space for unseen classes. CGPL incorporates a large-scale vision-language model as a collaborator with the ZSL model to generate high-quality pseudo-labels. Then, pseudo-labeled samples with consistent prediction of two models are added to the training set, to learn the visual-to-semantic mapping for unseen classes. Furthermore, we design a quasi-classification loss based on reconstructed unseen prototypes to learn accurate visual-semantic mapping. Consequently, CGPL is further encouraged to obtain higher-quality pseudo labels, and progressively learn the precise visual-semantic mapping for unseen classes throughout the iterative process. Our extensive experimental results across four benchmark datasets highlight the superior performance of CGPL in both CZSL and GZSL settings.

Research on Early Warning Method of Plough Layer Thickness of Cultivated Land in Black Soil Driven by Rainfall Events

The thinning of the arable layer causes soil quality degradation, seriously affecting crop growth and even threatening national food security. This article focuses on the problems and needs of poor timeliness, high cost, and inability to dynamically warn about the thinning of the tillage layer in black soil farmland, and conducts research on corresponding warning methods. Based on daily rainfall data, the percentile method is used to calculate the threshold for erosive rainfall event driven warning. Based on erosive rainfall event driven coupling of external soil erosion stress meteorological elements and influencing processes, as well as the connotation of geographic events, the concept definition of geographic events for thinning warning of arable layer thickness is given. The semantic description and discovery method of geographic events are provided, and geographic event driven rules and the technical process of thinning warning of arable layer thickness are formulated. There is a significant correlation between rainfall and topsoil thickness, with a correlation coefficient of 0.83. A soil thickness thinning warning simulation was conducted based on the daily rainfall data of Hailun City, Heilongjiang Province in 2020. On June 14, 2020, when the daily rainfall exceeded the event threshold, an automatic warning for thinning of the plow layer was triggered, achieving a dynamic warning process for thinning of the plow layer. The research results indicate that this method is helpful in solving the problem of thinning warning of the cultivated layer thickness in black soil farmland, and can provide technical support for the quality construction, protection, and management of black soil farmland.

The Concept of “Dreams (While Sleeping)” in the Proverbial Context of the Russian and Kyrgyz Languages

This article conducts a linguistic and cultural analysis of proverbial expressions, we tried to identify the specifics of linguistic verbalization of the linguistic and cultural concept of “dreams (while sleeping) / сон (Russian) / түш (Kyrgyz)” in the Russian and Kyrgyz linguistic proverbial space. The combination of sustainable forms produces a proverbial space of language, which reflects the entire history of life, way of life and experience, folk observations, conclusions, traditions and customs of each nation. By conducting linguistic studies of each language, linguists identify the ethnocultural characteristics of each language, depending on such extralinguistic factors as: geographical location, mental processes in the consciousness of a certain ethnic group and worldview. The relevance of this survey is the importance of the linguistic and cultural concepts, semantic descriptions of the features of Russian and Kyrgyz paroemia, which are an integral part of the national linguistic picture of the world. The concepts of “dreams (while sleeping) / сон (Russian) / туш (Kyrgyz)” considered by are an important state of human existence, the basic concept of any linguistic culture.

Promoting Unified Generative Framework with Descriptive Prompts for Joint Multi-Intent Detection and Slot Filling

Natural language understanding is a crucial aspect of task-oriented dialogue systems, encompassing intent detection (ID) and slot filling (SF). Conventional approaches for ID and SF solve the problems in a separate manners, while recent studies are now leaning toward joint modeling to tackle multi-intent detection and SF. Although the advancements in prompt learning offer a unified framework for ID and SF, current prompt-based methods fail to fully exploit the semantics of intent and slot labels. Additionally, the potential of using prompt learning to model the correlation between ID and SF in multi-intent scenarios remains unexplored. To address the issue, we propose a text-generative framework that unifies ID and SF. The prompt templates are constructed with label semantical descriptions. Moreover, we introduce an auxiliary task to explicitly capture the correlation between ID and SF. The experimental results on two benchmark datasets show that our method achieves an overall accuracy improvement of 0.4–1.5% in a full-data scenario and 1.4–2.7% in a few-shot setting compared with a prior method, establishing it as a new state-of-the-art approach.

Semantic-aware automatic extraction method for bottom sediment annotations in raster nautical charts

ABSTRACT The automatic extraction of bottom sediment annotations in large-scale raster nautical charts has limitations, including an imprecise semantic information description and low efficiency. To overcome them, we propose a convolutional neural network (CNN)-based method for the automatic extraction of bottom sediment annotations in raster nautical charts, using image processing techniques to improve it. First, an adaptive chart partitioning model that considers element completeness is constructed. Second, a principle for the unique identification of elements based on spatial conflicts is designed. Finally, a model for accurately extracting semantic information for bottom sediment annotations is established. To evaluate the effectiveness of the proposed method, we implemented a model based on the PyTorch framework and used the PIL library to analyze the results. We also conducted comparative experiments on multiple CNN models to recommend the selection of such models in the proposed method by comparing their classification and recognition performance. The experimental results indicate that (1) the proposed model can achieve high-precision extraction of bottom sediment annotations in raster nautical charts. (2) Furthermore, the proposed model generally has high recognition accuracy and semantic completeness, with better recognition precision than traditional pattern recognition methods.

An ontology‐based semantic description model of ubiquitous map images

AbstractMap images with various themes and cartographic representations have become ubiquitous on the Internet. Such ubiquitously and openly accessible data, named ubiquitous map images in this study, are a potential resource for many geographic information applications such as cartographic design. However, there is a semantic gap between the simple physical form and the complex connotation of ubiquitous map images, which hinders their further applications. To mitigate such barrier, this article develops an ontology‐based semantic description model for ubiquitous map images. First, we discuss the design concerns and principles of the semantic description model of ubiquitous map images. Second, three semantic layers of the semantic description model are proposed, that is, image semantic description layer, cognitive tool layer, and information source layer, and detailed semantic description items are defined for each layer. Furthermore, a formalized semantic description model for ubiquitous map images is developed using ontology construction tools, which lays the foundation for automated and fine‐grained reasoning with the information embedded in map images. We construct a small test dataset consisting of weather maps, and use three types of constraints, namely “time‐topic,” “region‐topic,” and “map auxiliary elements” for the semantic retrieval experiments. The experiments show that the proposed semantic ontology model can enable complex semantic retrieval of ubiquitous map images. Finally, the scalability of the model is discussed from three perspectives: the depth of description, the combination with intelligent methods, and the integration with other open knowledge bases. The proposed model provides a semantic label system for applying data‐driven approaches to decode ubiquitous map images, which also paves the path to the development of cartographic theory in the era of information and communications technologies.

The Empire of Affectivity: Qualitative Evidence of the Subjective Orgasm Experience.

The subjective orgasm experience (SOE) refers to its perception and/or assessment from a psychological viewpoint. Few works have approached this construct from a qualitative perspective and have never taken a consolidated theoretical model as a reference. This study aims to provide qualitative validity evidence to the Multidimensional Model of Subjective Orgasmic Experience, derived from the Orgasm Rating Scale (ORS), to qualitatively address SOE in the contexts of sexual relationships and solitary masturbation, analyzing the terms self-generated by individuals and examining the coincidence with the semantic descriptions of orgasm proposed by the ORS. Four hundred Spanish adults aged 18 to 64 years participated. The Technique of Free Association of Words was applied, and prototypical, frequency, and similitude analyses were performed. A similar description was observed concerning the terms generated in both contexts, with a higher frequency and intensity in the context of sexual relationships. In the context of solitary masturbation, negative orgasmic descriptions were evoked. Participants were able to elicit the vast majority of ORS adjectives, with Affective being the most notable dimension, followed closely by Rewards, especially in masturbation. Most of the adjectives were evoked simultaneously with those of the Affective, with "pleasurable" standing out as the most predominant one. This work provides qualitative evidence to the SOE study, ratifying the semantic composition of the ORS and thus endorsing the Multidimensional Model of Subjective Orgasmic Experience as a good theoretical model from which to continue studying the subjective orgasmic experience.

An Image Retrieval Pipeline in a Medical Data Integration Center.

Medical images need annotations with high-level semantic descriptors, so that domain experts can search for the desired dataset among an enormous volume of visual media within a Medical Data Integration Center. This article introduces a processing pipeline for storing and annotating DICOM and PNG imaging data by applying Elasticsearch, S3 and Deep Learning technologies. The proposed method processes both DICOM and PNG images to generate annotations. These image annotations are indexed in Elasticsearch with the corresponding raw data paths, where they can be retrieved and analyzed.

A corpus-based analysis of light verb constructions with MAKE and DO in British English

The comparison of light verb constructions with the verbs make and do has not received much attention. The present paper is an attempt to contribute to the study of these constructions. It aims to analyze the light verbs make and do in combination with the same deverbal nouns to identify similarities and differences between the two light verbs and examine the contribution of the light verbs and deverbal nouns to the light verb construction in terms of semantic and syntactic features. The research is corpus-based, and the data for analysis are collected from the British National Corpus (BNC). The semantic description deals with characteristics such as generality, polysemy and aspectual meaning, and the syntactic description focuses on complementation. The analysis of findings demonstrates that, despite some similarities, almost all constructions with the two light verbs show either meaning or/and complementation differences. The study also reveals that both light verbs and deverbal nouns have an impact on the semantics and syntax of the construction. Light verbs may affect light verb constructions semantically in terms of generality and polysemy, and the aspectual meaning of constructions depends on both light verbs and deverbal nouns. Likewise, both light verbs and deverbal nouns have an influence on complementation, but in their own way: light verbs on the selection of complements and deverbal nouns on complementation patterns.

"A Rare Vocabulary Dictionary based on school curriculum literature": theoretical issues of creation and methodological prospects for use

The article describes the principles of compiling the Rare Vocabulary Dictionary based on school curriculum literature (hereinafter referred to as the Dictionary) and its methodological potential for being used as an electronic educational resource in teaching schoolchildren. Moreover, the study examines the specific features of the Dictionary in comparison with similar existing lexicographic publications. The Dictionary creation is relevant as there is a need to make Russian classical literature texts more understandable for the modern reader. As a result, the resource can promote the culture of reading literary texts and the formation of students’ linguistic personality, as well as enhance learning efficiency. The Dictionary materials were comprised of Russian authors’ works included in the school literature curriculum for years 5 to 11. The paper pays particular attention to the innovative solutions concerning the principles of glossary compilation and dictionary entry creation. The research relied on corpus linguistics methodology and its tools for working with text, which enabled us to select vocabulary based on objective criteria (low frequency in modern Russian) and describe the meaning of a particular word relying on the entire set of contexts of its use in the literature under study. The focus of lexicographic interpretation was an extensive layer of rare vocabulary of various semantic groups (including proper names and occasionalisms) and different parts of speech. Semantic description demonstrates the history of word meanings in the language in conjunction with contextual interpretation of a lexical unit. The entries contain encyclopedic information and a detailed cultural and/or philological commentary. Maximal attention is given to authorial neologisms. Some entries are accompanied by images of the item being described. The Dictionary is implemented as a module of the new Russian Classics cross platform mobile application. The article also describes the operating principles and capabilities of the application.

Indoor functional subspace division from point clouds based on graph neural network

Indoor scenes are closely related to human life and contain rich geometric and semantic information. Dividing indoor spaces from data is crucial for multiple applications, such as navigation and digital twins. However, achieving this task is challenging. Traditional indoor space division methods only represent buildings, floors, and rooms to some extent, lacking semantic descriptions of indoor elements and their spatial arrangements. To divide an indoor space more finely, a novel indoor space subdivision method is proposed in this paper. Our method leverages the flexible space subdivision framework (FSS) to categorize indoor space into free navigation subspace, object subspace, and functional subspace. To define functional subspaces, we present a taxonomy for the spatial layout patterns of common indoor elements (like tables, chairs, ceilings, walls, floors, etc.). Then, we introduce scene graphs to represent indoor 3D scenes, where each node represents an indoor element and each edge encodes the spatial relationship between the elements. Finally, a node classification network is proposed to segment indoor scene into subspaces and predicts their (semantic) functions. We select 9 buildings of Matterport3D and 6 areas in S3DIS and merge them to form our dataset for training and testing our method. Experiments yield good results with up to 90.42% accuracy and 85.28% F1-scores in overall space subdivision. Moreover, compared with the various graph node classification networks, our method has achieved the best performance in indoor space subdivisions.

A Novel Semantic IoT Middleware for Secure Data Management: Blockchain and AI-Driven Context Awareness

In the modern digital landscape of the Internet of Things (IoT), data interoperability and heterogeneity present critical challenges, particularly with the increasing complexity of IoT systems and networks. Addressing these challenges, while ensuring data security and user trust, is pivotal. This paper proposes a novel Semantic IoT Middleware (SIM) for healthcare. The architecture of this middleware comprises the following main processes: data generation, semantic annotation, security encryption, and semantic operations. The data generation module facilitates seamless data and event sourcing, while the Semantic Annotation Component assigns structured vocabulary for uniformity. SIM adopts blockchain technology to provide enhanced data security, and its layered approach ensures robust interoperability and intuitive user-centric operations for IoT systems. The security encryption module offers data protection, and the semantic operations module underpins data processing and integration. A distinctive feature of this middleware is its proficiency in service integration, leveraging semantic descriptions augmented by user feedback. Additionally, SIM integrates artificial intelligence (AI) feedback mechanisms to continuously refine and optimise the middleware’s operational efficiency.

Marqueurs de discours et rasoir d’Ockham

Most studies of discourse markers aim to establish a typology of uses for the unit under consideration. Authors seek to identify distinctive properties that characterize each use, and propose a more or less detailed semantic description for each one. In this article, we propose a critical analysis of the assumptions underlying this “taxonomic” approach to semantic description, and propose instead an “explanatory” approach based on the notion of “gloss”.

Generative Mixup Networks for Zero-Shot Learning.

Zero-shot learning casts light on lacking unseen class data by transferring knowledge from seen classes via a joint semantic space. However, the distributions of samples from seen and unseen classes are usually imbalanced. Many zero-shot learning methods fail to obtain satisfactory results in the generalized zero-shot learning task, where seen and unseen classes are all used for the test. Also, irregular structures of some classes may result in inappropriate mapping from visual features space to semantic attribute space. A novel generative mixup networks with semantic graph alignment is proposed in this article to mitigate such problems. To be specific, our model first attempts to synthesize samples conditioned with class-level semantic information as the prototype to recover the class-based feature distribution from the given semantic description. Second, the proposed model explores a mixup mechanism to augment training samples and improve the generalization ability of the model. Third, triplet gradient matching loss is developed to guarantee the class invariance to be more continuous in the latent space, and it can help the discriminator distinguish the real and fake samples. Finally, a similarity graph is constructed from semantic attributes to capture the intrinsic correlations and guides the feature generation process. Extensive experiments conducted on several zero-shot learning benchmarks from different tasks prove that the proposed model can achieve superior performance over the state-of-the-art generalized zero-shot learning.

Polarity Loss: Improving Visual-Semantic Alignment for Zero-Shot Detection.

Conventional object detection models require large amounts of training data. In comparison, humans can recognize previously unseen objects by merely knowing their semantic description. To mimic similar behavior, zero-shot object detection (ZSD) aims to recognize and localize "unseen" object instances by using only their semantic information. The model is first trained to learn the relationships between visual and semantic domains for seen objects, later transferring the acquired knowledge to totally unseen objects. This setting gives rise to the need for correct alignment between visual and semantic concepts so that the unseen objects can be identified using only their semantic attributes. In this article, we propose a novel loss function called "polarity loss" that promotes correct visual-semantic alignment for an improved ZSD. On the one hand, it refines the noisy semantic embeddings via metric learning on a "semantic vocabulary" of related concepts to establish a better synergy between visual and semantic domains. On the other hand, it explicitly maximizes the gap between positive and negative predictions to achieve better discrimination between seen, unseen, and background objects. Our approach is inspired by embodiment theories in cognitive science that claim human semantic understanding to be grounded in past experiences (seen objects), related linguistic concepts (word vocabulary), and visual perception (seen/unseen object images). We conduct extensive evaluations on the Microsoft Common Objects in Context (MS-COCO) and Pascal Visual Object Classes (VOC) datasets, showing significant improvements over state of the art. Our code and evaluation protocols available at: https://github.com/salman-h-khan/PL-ZSD_Release.

ROBUST DETECTION OF LINK COMMUNITIES WITH SUMMARY DESCRIPTION IN SOCIAL NETWORKS

Community detection has been extensively studied for various applications. Recent research has started to explore node contents to identify semantically meaningful communities. However, links in real networks typically have semantic descriptions and communities of links can better characterize community behaviors than communities of nodes. The second issue in community finding is that the most existing methods assume network topologies and descriptive contents carry the same or compatible information of node group membership, restricting them to one topic per community, which is generally violated in real networks. The third issue is that the existing methods use top ranked words or phrases to label topics when interpreting communities, which is often inadequate for comprehension. To address these issues altogether, we propose a new Bayesian probabilistic approach for modeling real networks and developing an efficient variational algorithm for model inference. Our new method explores the intrinsic correlation between communities and topics to discover link communities and extract semantically meaningful community summaries at the same time. If desired, it is able to derive more than one topical summary per community to provide rich explanations. We present experimental results to show the effectiveness of our new approach and evaluate the method by a case study.

Querying Semantic OpenAPI Descriptions with OASL

Querying Semantic OpenAPI Descriptions with OASL

Querying semantic OpenAPI descriptions with OASL

Querying semantic OpenAPI descriptions with OASL