Semantic Mapping Research Articles

Unsupervised model adaptation for source-free segmentation of medical images

The recent prevalence of deep neural networks has led semantic segmentation networks to achieve human-level performance in the medical field, provided they are given sufficient training data. However, these networks often fail to generalize when tasked with creating semantic maps for out-of-distribution images, necessitating re-training on new distributions. This labor-intensive process requires expert knowledge for generating training labels. In the medical field, distribution shifts can naturally occur due to the choice of imaging devices, such as MRI or CT scanners. To mitigate the need for labeling images in a target domain after successful model training in a fully annotated source domain with a different data distribution, unsupervised domain adaptation (UDA) can be employed. Most UDA approaches ensure target generalization by generating a shared source/target latent feature space, allowing a source-trained classifier to maintain performance in the target domain. However, such approaches necessitate joint source and target data access, potentially leading to privacy leaks with respect to patient information. We propose a UDA algorithm for medical image segmentation that does not require access to source data during adaptation, thereby preserving patient data privacy. Our method relies on approximating the source latent features at the time of adaptation and creates a joint source/target embedding space by minimizing a distributional distance metric based on optimal transport. We demonstrate that our approach is competitive with recent UDA medical segmentation works, even with the added requirement of privacy. 11Early partial results of this work has been presented in 2022 British Machine Vision Conference (Stan and Rostami, 2022a).

Семантико-синтаксические свойства отглагольного релятива «исходя из»

The aim of the study is to determine the degree of grammaticalization of the deverbal relative ‘исходя из’ (‘based on’). The paper examines the lexical meaning of the relative and the presence of the functional meaning in dictionaries and works dedicated to prepositional units. The paper explores the meaning of the construction with ‘исходя из’, the semantics of the right (dependent) and left (main) components of the construction and outlines features indicative of the transition of this unit into a prepositional one. The study is novel in that it is the first to describe the semantic and syntactic functioning and to determine the degree of grammaticalization of the deverbal relative ‘исходя из’, which is a language unit that does not belong to the core of the preposition category. As a result, the study finds that the semantics of the deverbal relative, the meaning of the construction that expresses cause-effect relationships of phenomena reflecting various aspects of intellectual activities (denoted by verbs) and phenomena subject to interpretation (denoted by nouns), as well as the broadening of usage contexts are indicative of the grammaticalization of the examined unit.

Escalating Students' Vocabulary Through Semantic Mapping Learning Model

The biggest obstacle for teachers and tutors is the lack of students' vocabulary. It causes students to feel difficulty understanding the meaning of English, particularly their performance in applying and developing four skills of English. Therefore, this research aims to enhance students' English vocabulary mastery through a semantic mapping learning model. This model was applied to seventh-grade students in the 7.2 class of MTs Al-Yusra Gorontalo, consisting of ten students. Classroom Action Research (CAR) was used as the method of this study. For collecting data, there are four phases of CAR used; they were planning, action, observation, and reflection. Two cycles were applied in this study because the first cycle did not improve students' vocabulary. Based on the data finding on cycle one, the scoring average of students' task results was 56,2. It is because 80% of students still got a low score, and 20% got a perfect score. The content of the task was about the questions of material "Noun". Then, in cycle two, it was applied by involving students in the classroom action; they were asked to present the result of their group discussion of semantic mapping of Noun material. After that, they did the second task, and the average of their score was 79,5. It could be concluded that the semantic mapping model greatly affected the students' mastery of the vocabulary taught by the teacher.

Iconicity mediates semantic networks of sound symbolisma).

One speech sound can be associated with multiple meanings through iconicity, indexicality, and/or systematicity. It was not until recently that this "pluripotentiality" of sound symbolism attracted serious attention, and it remains uninvestigated how pluripotentiality may arise. In the current study, Japanese, Korean, Mandarin, and English speakers rated unfamiliar jewel names on three semantic scales: size, brightness, and hardness. The results showed language-specific and cross-linguistically shared pluripotential sound symbolism. Japanese speakers associated voiced stops with large and dark jewels, whereas Mandarin speakers associated [i] with small and bright jewels. Japanese, Mandarin, and English speakers also associated lip rounding with darkness and softness. These sound-symbolic meanings are unlikely to be obtained through metaphorical or metonymical extension, nor are they reported to colexify. Notably, in a purely semantic network without the mediation of lip rounding, softness can instead be associated with brightness, as illustrated by synesthetic metaphors such as yawaraka-na hizashi /jawaɾakanaçizaɕi/ "a gentle (lit. soft) sunshine" in Japanese. These findings suggest that the semantic networks of sound symbolism may not coincide with those of metaphor or metonymy. The current study summarizes the findings in the form of (phono)semantic maps to facilitate cross-linguistic comparisons of pluripotential sound symbolism.

The Arabic Translation of Cohesive Devices used in “The Midnight Library” Novel

This study deals with English cohesive devices as discourse markers, which are used for various purposes, such as connecting ideas and expressing attitudes. In fact, such devices may cause problems in case of translation, especially when dealing with them in literary works. Therefore, this study aims to identify the English cohesive devices and clarify how the translators translate these devices into Arabic. The study data is collected from the English novel “The Midnight Library” written by Matt Haig. The study sample includes (5) texts extracted from this novel to be translated by (3) MA students in the Translation Department, College of Arts, Tikrit University. Concerning the analysis of data, the study adopts two models: Halliday and Hasan's (1976) model for identifying the cohesive devices in the novel under study; and Newmark’s (1988) model for determining the method of translation. The study reveals that cohesive devices are used to demonstrate a semantic function rather than a communicative one, causing a challenge for translators to translate them into Arabic.

Лингвофилософская концепция В.И. Абаева и осетино-кабардино-балкарские идеосеманти-ческие параллели

The article is devoted to the linguophilosophical concept of the outstanding Russian theorist, his-torian and philosopher of language, the largest Iranian scholar Vasily Ivanovich Abaev. The as-pects of the author’s linguophilosophical concept are presented based on two works by V.I. Abaev: “Reflection of the work of consciousness in the lexico-semantic system of language” (1970) and “Language as ideology and language as technique” (1934). In these works, he pays special atten-tion to the problem of the origin of human language. The theory of the origin of language itself, in his opinion, should be based on an optimal scientific theory that proceeds from the correct theo-retical premises and is consistent with the available empirical data. To explain the origin of a lan-guage, it is not the grammar of the language that is important, but the subject–significant vocabu-lary, as he believes that the origin of vocabulary is a single and integral problem, and the origin of grammar is inevitably divided into several particular issues. Language, according to V.I.Abaev, arose not from biological, but from the social needs of a person, from the need to relate things to their collective, to impose their own “brand” on them. Special attention is paid to the concept of ideosemantics, firstly introduced by V.I. Abaev in-to scientific circulation. He distinguishes two types of semantics: small semantics (or signal, technical), which includes a mandatory minimum of semantic functions that determine the mod-ern communicative use of the word, and large semantics, which is the sum of those concomitant cognitive and emotional representations that reflect the complex inner life of the word in its past and present. He defined this broader understanding of semantics by the term “ideosemantics”. Id-eosemantics, according to V. Abaev, is the most important criterion for the historical characteriza-tion of linguistic phenomena. V. Abaev sees the value of language at an early stage of its for-mation in its “ethnodemarkation function”. In conclusion, the article presents ideosemantic parallels in three genetically and structurally diverse languages: Ossetian, Kabardino-Circassian and Karachay-Balkar.

ODN-Pro: An Improved Model Based on YOLOv8 for Enhanced Instance Detection in Orchard Point Clouds

In an unmanned orchard, various tasks such as seeding, irrigation, health monitoring, and harvesting of crops are carried out by unmanned vehicles. These vehicles need to be able to distinguish which objects are fruit trees and which are not, rather than relying on human guidance. To address this need, this study proposes an efficient and robust method for fruit tree detection in orchard point cloud maps. Feature extraction is performed on the 3D point cloud to form a two-dimensional feature vector containing three-dimensional information of the point cloud and the tree target is detected through the customized deep learning network. The impact of various feature extraction methods such as average height, density, PCA, VFH, and CVFH on the detection accuracy of the network is compared in this study. The most effective feature extraction method for the detection of tree point cloud objects is determined. The ECA attention module and the EVC feature pyramid structure are introduced into the YOLOv8 network. The experimental results show that the deep learning network improves the precision, recall, and mean average precision by 1.5%, 0.9%, and 1.2%, respectively. The proposed framework is deployed in unmanned orchards for field testing. The experimental results demonstrate that the framework can accurately identify tree targets in orchard point cloud maps, meeting the requirements for constructing semantic orchard maps.

Real-time semantic segmentation based on BiSeNetV2 for wild road

Abstract State-of-the-art segmentation models have shown great performance in structured road segmentation. However, these models are not suitable for the wild roads, which are highly unstructured. To tackle the problem of real-time semantic segmentation of wild roads, we propose a Multi-Information Concatenate Network based on BiSeNetV2 and construct a segmentation dataset Dalle Molle institute for artificial intelligence feature segmentation (IDSIAFS) based on Dalle Molle institute for artificial intelligence. The proposed model removes structural redundancy and optimizes the semantic branch based on BiSeNetV2. Moreover, the Dual-Path Semantic Inference Layer (TPSIL) reduces computation by designing the channel dimension of the semantic branch feature map and aggregates feature maps of different depths. Finally, the segmentation results are achieved by fusing both shallow detail information and deep semantic information. Experiments on the IDSIAFS dataset demonstrate that our proposed model achieves an 89.5% Intersection over Union. The comparative experiments on Cityscapes and India driving dataset benchmarks show that proposed model achieves good inference accuracy and faster inference speed.

Regulating Intermediate 3D Features for Vision-Centric Autonomous Driving

Multi-camera perception tasks have gained significant attention in the field of autonomous driving. However, existing frameworks based on Lift-Splat-Shoot (LSS) in the multi-camera setting cannot produce suitable dense 3D features due to the projection nature and uncontrollable densification process. To resolve this problem, we propose to regulate intermediate dense 3D features with the help of volume rendering. Specifically, we employ volume rendering to process the dense 3D features to obtain corresponding 2D features (e.g., depth maps, semantic maps), which are supervised by associated labels in the training. This manner regulates the generation of dense 3D features on the feature level, providing appropriate dense and unified features for multiple perception tasks. Therefore, our approach is termed Vampire, stands for ``Volume rendering As Multi-camera Perception Intermediate feature REgulator''. Experimental results on the Occ3D and nuScenes datasets demonstrate that Vampire facilitates fine-grained and appropriate extraction of dense 3D features, and is competitive with existing SOTA methods across diverse downstream perception tasks like 3D occupancy prediction, LiDAR segmentation and 3D objection detection, while utilizing moderate GPU resources. We provide a video demonstration in the supplementary materials and Codes are available at github.com/cskkxjk/Vampire.

RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation

3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. However, image-based scene perception encounters significant challenges in achieving accurate prediction due to the absence of geometric priors. In this paper, we address this issue by exploring cross-modal knowledge distillation in this task, i.e., we leverage a stronger multi-modal model to guide the visual model during training. In practice, we observe that directly applying features or logits alignment, proposed and widely used in bird's-eye-view (BEV) perception, does not yield satisfactory results. To overcome this problem, we introduce RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction. By employing differentiable volume rendering, we generate depth and semantic maps in perspective views and propose two novel consistency criteria between the rendered outputs of teacher and student models. Specifically, the depth consistency loss aligns the termination distributions of the rendered rays, while the semantic consistency loss mimics the intra-segment similarity guided by vision foundation models (VLMs). Experimental results on the nuScenes dataset demonstrate the effectiveness of our proposed method in improving various 3D occupancy prediction approaches, e.g., our proposed methodology enhances our baseline by 2.2% in the metric of mIoU and achieves 50% in Occ3D benchmark.

Local-Global Multi-Modal Distillation for Weakly-Supervised Temporal Video Grounding

This paper for the first time leverages multi-modal videos for weakly-supervised temporal video grounding. As labeling the video moment is labor-intensive and subjective, the weakly-supervised approaches have gained increasing attention in recent years. However, these approaches could inherently compromise performance due to inadequate supervision. Therefore, to tackle this challenge, we for the first time pay attention to exploiting complementary information extracted from multi-modal videos (e.g., RGB frames, optical flows), where richer supervision is naturally introduced in the weaklysupervised context. Our motivation is that by integrating different modalities of the videos, the model is learned from synergic supervision and thereby can attain superior generalization capability. However, addressing multiple modalities† would also inevitably introduce additional computational overhead, and might become inapplicable if a particular modality is inaccessible. To solve this issue, we adopt a novel route: building a multi-modal distillation algorithm to capitalize on the multi-modal knowledge as supervision for model training, while still being able to work with only the single modal input during inference. As such, we can utilize the benefits brought by the supplementary nature of multiple modalities, without compromising the applicability in practical scenarios. Specifically, we first propose a cross-modal mutual learning framework and train a sophisticated teacher model to learn collaboratively from the multi-modal videos. Then we identify two sorts of knowledge from the teacher model, i.e., temporal boundaries and semantic activation map. And we devise a local-global distillation algorithm to transfer this knowledge to a student model of single-modal input at both local and global levels. Extensive experiments on large-scale datasets demonstrate that our method achieves state-of-the-art performance with/without multi-modal inputs.

Knowledge Distillation from Single-Task Teachers to Multi-Task Student for End-to-End Autonomous Driving

In the domain of end-to-end autonomous driving, conventional sensor fusion techniques exhibit inadequacies, particularly when facing challenging scenarios with numerous dynamic agents. Imitation learning hampers the performance by the expert and encounters issues with out-of-distribution challenges. To overcome these limitations, we propose a transformer-based algorithm designed to fuse diverse representations from RGB-D cameras through knowledge distillation. This approach leverages insights from multi-task teachers to enhance the learning capabilities of single-task students, particularly in a Reinforcement Learning (RL) setting. Our model consists of two primary modules: the perception module, responsible for encoding observation data acquired from RGB-D cameras and performing tasks such as semantic segmentation, semantic depth cloud mapping (SDC), ego vehicle speed estimation, and traffic light state recognition. Subsequently, the control module decodes these features, incorporating additional data, including a rough simulator for static and dynamic environments, to anticipate waypoints within a latent feature space. Vehicular controls (e.g., steering, throttle, and brake) are obtained directly from measurement features and environmental states using the RL agent and are further refined by a PID algorithm that dynamically follows waypoints. The model undergoes rigorous evaluation and comparative analysis on the CARLA simulator across various scenarios, encompassing normal to adversarial conditions. Our code is available at https://github.com/pagand/e2etransfuser/ to facilitate future studies.

Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference

Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making minor modifications to their input textual descriptions for several rounds of diffusion inference. However, such an image editing process suffers from the low inference efficiency of many existing diffusion models even using GPU accelerators. To solve this problem, we introduce Fast Image Semantically Edit (FISEdit), a cached-enabled sparse diffusion model inference engine for efficient text-to-image editing. The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image. For each text editing step, FISEdit can 1) automatically identify the affected image regions and 2) utilize the cached unchanged regions' feature map to accelerate the inference process. For the former, we measure the differences between cached and ad hoc feature maps given the modified textual description, extract the region with significant differences, and capture the affected region by masks. For the latter, we develop an efficient sparse diffusion inference engine that only computes the feature maps for the affected region while reusing the cached statistics for the rest of the image. Finally, extensive empirical results show that FISEdit can be 3.4 times and 4.4 times faster than existing methods on NVIDIA TITAN RTX and A100 GPUs respectively, and even generates more satisfactory images.

Modeling of cross-scale human activity for digital twin workshop

Digital Twin Workshop(DTW), as an important approach to digitalization and intelligentization of workshop, has gained significant attention in manufacturing industry. Currently, digital twin models for manufacturing resources have progressed from theoretical research to practical implementation. However, as a crucial component of workshop, modeling of human activity in workshop still faces challenges due to the autonomy and uncertainty of human beings. Therefore, we propose a comprehensive approach to the modeling cross-scale human activity in digital twin workshop, which comprises macro activity and micro activity. Macro activity contains human’s occupation and spatial positions in workshop, while micro activity refers to real-time posture and production actions at work. In this paper, we build and integrate macro activity digital twin model and micro activity digital twin model. With the combination of closed-loop interaction between virtual models and physical entities, we achieve semantic mapping and control of production activities, thereby facilitating practical management of human activity in workshop. Finally, we take certain factory’s manufacturing workshop as an example to introduce the application of the proposed approach.

К вопросу о синтаксическом статусе субстантивов в русских предложениях тождества

A multi aspect study of bisubstantive sentences of characterizing semantics is necessary to solve complex problems of syntactic analysis. The grammatical specificity of bisubstantial sentences lies in the morphological identity of the predicative minimum. The study has theoretical and practical tasks: identify the structural, semantic and communicative organization of sentences, propose methodological recommendations for syntactic analysis of sentences. The material of the study is modern literature. Analytical and descriptive methods are used to analyze the linguistic material. The problems of teaching syntactic analysis of sentences are identified through experimental methods. The main conclusions of the study: syntactic analysis of bisubstantial sentences should consist of traditional structural and semantic parameters, roles and functions in the semantic structure and communicative organization of a sentence, take into account context, stylistic and pragmatic factors. Syntactic phenomena should be studied from the functional position at all stages and levels of teaching syntax of the modern Russian language.

Re-investigating the classification of definite CL-NP constructions in Chinese dialect: An empirical study based on semantic maps

This study aims to assess the fitness of the “quasi-article/quasi-demonstrative” framework for classifying definite classifiers in varieties of Chinese. Taking the Suzhou (Wu) variety of Chinese as our target, we utilize an approach of the Probabilistic Semantic Maps within a trilingual parallel corpus (English-Mandarin-Suzhou Wu). Contrary to the earlier understanding that only one type of definite classifiers exists in Suzhou, functioning exclusively either as definite articles or demonstratives in English, our research indicates the presence of two distinct types. Additionally, it appears that these two types are not aptly categorized under the “quasi-article/quasi-demonstrative” framework. Our results indicate no distinct association between the choice of classifiers for expressing identifiability in Suzhou and the one of utilizing definite articles or demonstratives in English. The observed discrepancies suggest a reconsideration of current assumptions due to the limited similarities between English grammatical categories of definiteness and varieties of Chinese definite expressions. Consequently, this study advocates that grammatical categories should be established within a particular language, rather than relying on external theoretical frameworks of other languages.

Scalable semantic 3D mapping of coral reefs with deep learning

Abstract Coral reefs are among the most diverse ecosystems on our planet, and essential to the livelihood of hundreds of millions of people who depend on them for food security, income from tourism and coastal protection. Unfortunately, most coral reefs are existentially threatened by global climate change and local anthropogenic pressures. To better understand the dynamics underlying deterioration of reefs, monitoring at high spatial and temporal resolution is key. However, conventional monitoring methods for quantifying coral cover and species abundance are limited in scale due to the extensive manual labor required. Although computer vision tools have been employed to aid in this process, in particular structure‐from‐motion (SfM) photogrammetry for 3D mapping and deep neural networks for image segmentation, analysis of the data products creates a bottleneck, effectively limiting their scalability. This paper presents a new paradigm for mapping underwater environments from ego‐motion video, unifying 3D mapping systems that use machine learning to adapt to challenging conditions under water, combined with a modern approach for semantic segmentation of images. The method is exemplified on coral reefs in the northern Gulf of Aqaba, Red Sea, demonstrating high‐precision 3D semantic mapping at unprecedented scale with significantly reduced required labor costs: given a trained model, a 100 m video transect acquired within 5 min of diving with a cheap consumer‐grade camera can be fully automatically transformed into a semantic point cloud within 5 min. We demonstrate the spatial accuracy of our method and the semantic segmentation performance (of at least 80% total accuracy), and publish a large dataset of ego‐motion videos from the northern Gulf of Aqaba, along with a dataset of video frames annotated for dense semantic segmentation of benthic classes. Our approach significantly scales up coral reef monitoring by taking a leap towards fully automatic analysis of video transects. The method advances coral reef transects by reducing the labor, equipment, logistics, and computing cost. This can help to inform conservation policies more efficiently. The underlying computational method of learning‐based Structure‐from‐Motion has broad implications for fast low‐cost mapping of underwater environments other than coral reefs.

Graded and sharp transitions in semantic function in left temporal lobe

Recent work has focussed on how patterns of functional change within the temporal lobe relate to whole-brain dimensions of intrinsic connectivity variation (Margulies et al., 2016). We examined two such ‘connectivity gradients’ reflecting the separation of (i) unimodal versus heteromodal and (ii) visual versus auditory-motor cortex, examining visually presented verbal associative and feature judgments, plus picture-based context and emotion generation. Functional responses along the first dimension sometimes showed graded change between modality-tuned and heteromodal cortex (in the verbal matching task), and other times showed sharp functional transitions, with deactivation at the extremes and activation in the middle of this gradient (internal generation). The second gradient revealed more visual than auditory-motor activation, regardless of content (associative, feature, context, emotion) or task process (matching/generation). We also uncovered subtle differences across each gradient for content type, which predominantly manifested as differences in relative magnitude of activation or deactivation.

Shared structure of fundamental human experience revealed by polysemy network of basic vocabularies across languages

How are concepts related to fundamental human experiences organized within the human mind? Our insights are drawn from a semantic network created using the Cross-Linguistic Database of Polysemous Basic Vocabulary, which focuses on a broad range of senses extracted from dictionary entries. The database covers 60 basic vocabularies in 61 languages, providing 11,841 senses from 3736 entries, revealing cross-linguistic semantic connections through automatically generated weighted semantic maps. The network comprises 2941 nodes connected by 3573 edges. The nodes representing body parts, motions, and features closely related to human experience occupy wide fields or serve as crucial bridges across semantic domains in the network. The polysemous network of basic vocabularies across languages represents a shared cognitive network of fundamental human experiences, as these semantic connections should be conceived as generally independent of any specific language and are driven by universal characteristics of the real world as perceived by the human mind. The database holds the potential to contribute to research aimed at unraveling the nature of cognitive proximity.

An ontology‐based rule mapping approach for integrating IFC and CityGML

AbstractBuilding information modeling (BIM) and geographic information system (GIS) are often integrated to achieve more refined urban management. Industry Foundation Classes (IFC) and City Geography Markup Language (CityGML) are widely used standards in BIM and GIS. The main problem of the transformation of these two standards lies in the gap between them in geometry and semantics. This article introduces an ontology‐based rule mapping approach, achieving the transformation from IFC to CityGML through coordinate system conversion, geometric expression conversion, and semantic mapping. For semantic mapping, we have designed and established rules, enabling the effective mapping of the IFC model to the CityGML level of detail 0 (LOD0) to LOD4 model based on entities, appearance properties and building properties. To fully demonstrate the entire process, we give a detailed elaboration of the conversion of several IFC models.