Semantic Model Research Articles

Applications for Semantic 3D Streetspace Models and Their Requirements—A Review and Look at the Road Ahead

In addition to geometric accuracy, topological information, appearance and georeferenced data, semantic capabilities are key strengths of digital 3D city models. This provides the foundation for a growing number of use cases, far beyond visualization. While these use cases mostly focused on models of buildings or the terrain so far, the increasing availability of data on roads and other transportation infrastructure opened up a range of emerging use cases in the field of semantic 3D streetspace models. While there are already a number of implemented examples, there is also a potential for new use cases not yet established in the field of 3D city modeling, which benefit from detailed representations of roads and their environment. To ensure clarity in our discussions, we introduce an unambiguous distinction between the terms `application domain’, `use case’, `functionality’ and `software application’. Based on these definitions, use cases are categorized according to their primary application domain and discussed with respect to relevant literature and required functionalities. Furthermore, requirements of functionalities towards semantic 3D streetspace models are determined and evaluated in detail with regard to geometric, semantic, topological, temporal and visual aspects. This article aims to give an overview on use cases in the context of semantic 3D streetspace models and to present requirements of respective functionalities, in order to provide insight for researchers, municipalities, companies, data providers, mapping agencies and other stakeholders interested in creating and using a digital twin of the streetspace.

GASCOM: Graph-based Attentive Semantic Context Modeling for Online Conversation Understanding

Online conversation understanding is an important yet challenging NLP problem which has many useful applications (e.g., hate speech detection). However, online conversations typically unfold over a series of posts and replies to those posts, forming a tree structure within which individual posts may refer to semantic context from elsewhere in the tree. Such semantic cross-referencing makes it difficult to understand a single post by itself; yet considering the entire conversation tree is not only difficult to scale but can also be misleading as a single conversation may have several distinct threads or points, not all of which are relevant to the post being considered. In this paper, we propose a Graph-based Attentive Semantic COntext Modeling (GASCOM) framework for online conversation understanding. Specifically, we design two novel algorithms that utilize both the graph structure of the online conversation as well as the semantic information from individual posts for retrieving relevant context nodes from the whole conversation. We further design a token-level multi-head graph attention mechanism to pay different attentions to different tokens from different selected context utterances for fine-grained conversation context modelling. Using this semantic conversational context, we re-examine two well-studied problems: polarity prediction and hate speech detection. Our proposed framework significantly outperforms state-of-the-art methods on both tasks, improving macro-F1 scores by 4.5% for polarity prediction and by 5% for hate speech detection. The GASCOM context weights also enhance interpretability.

U-Net Semantic Segmentation-Based Calorific Value Estimation of Straw Multifuels for Combined Heat and Power Generation Processes

This paper proposes a system for real-time estimation of the calorific value of mixed straw fuels based on an improved U-Net semantic segmentation model. This system aims to address the uncertainty in heat and power generation per unit time in combined heat and power generation (CHPG) systems caused by fluctuations in the calorific value of straw fuels. The system integrates an industrial camera, moisture detector, and quality sensors to capture images of the multi-fuel straw. It applies the improved U-Net segmentation network for semantic segmentation of the images, accurately calculating the proportion of each type of straw. The improved U-Net network introduces a self-attention mechanism in the skip connections of the final layer of the encoder, replacing traditional convolutions by depthwise separable convolutions, as well as replacing the traditional convolutional bottleneck layers with Transformer encoder. These changes ensure that the model achieves high segmentation accuracy and strong generalization capability while maintaining good real-time performance. The semantic segmentation results of the straw images are used to calculate the proportions of different types of straw and, combined with moisture content and quality data, the calorific value of the mixed fuel is estimated in real time based on the elemental composition of each straw type. Validation using images captured from an actual thermal power plant shows that, under the same conditions, the proposed model has only a 0.2% decrease in accuracy compared to the traditional U-Net segmentation network, while the number of parameters is significantly reduced by 74%, and inference speed is improved 23%.

Enhancing bridge damage detection with Mamba-Enhanced HRNet for semantic segmentation.

With the acceleration of urbanization, bridges, as crucial infrastructure, their structural health and stability are paramount to public safety. This paper proposes Mamba-Enhanced HRNet for bridge damage detection. Mamba-Enhanced HRNet integrates the advantages of HRNet's multi-resolution parallel design and VMamba's visual state space model. By replacing the residual convolutional blocks in HRNet with a combination of VSS blocks and convolution, this model enhances the network's capability to capture global contextual information while maintaining computational efficiency. This work builds an extensive dataset with multiple damage kinds and uses Mean Intersection over Union (Mean IoU) as the assessment metric to assess the performance of Mamba-Enhanced HRNet. Experimental results demonstrate that Mamba-Enhanced HRNet achieves significant performance improvements in bridge damage semantic segmentation tasks, with Mean IoU scores of 0.963, outperforming several other semantic segmentation models.

UNeXt: An Efficient Network for the Semantic Segmentation of High-Resolution Remote Sensing Images

The application of deep neural networks for the semantic segmentation of remote sensing images is a significant research area within the field of the intelligent interpretation of remote sensing data. The semantic segmentation of remote sensing images holds great practical value in urban planning, disaster assessment, the estimation of carbon sinks, and other related fields. With the continuous advancement of remote sensing technology, the spatial resolution of remote sensing images is gradually increasing. This increase in resolution brings about challenges such as significant changes in the scale of ground objects, redundant information, and irregular shapes within remote sensing images. Current methods leverage Transformers to capture global long-range dependencies. However, the use of Transformers introduces higher computational complexity and is prone to losing local details. In this paper, we propose UNeXt (UNet+ConvNeXt+Transformer), a real-time semantic segmentation model tailored for high-resolution remote sensing images. To achieve efficient segmentation, UNeXt uses the lightweight ConvNeXt-T as the encoder and a lightweight decoder, Transnext, which combines a Transformer and CNN (Convolutional Neural Networks) to capture global information while avoiding the loss of local details. Furthermore, in order to more effectively utilize spatial and channel information, we propose a SCFB (SC Feature Fuse Block) to reduce computational complexity while enhancing the model’s recognition of complex scenes. A series of ablation experiments and comprehensive comparative experiments demonstrate that our method not only runs faster than state-of-the-art (SOTA) lightweight models but also achieves higher accuracy. Specifically, our proposed UNeXt achieves 85.2% and 82.9% mIoUs on the Vaihingen and Gaofen5 (GID5) datasets, respectively, while maintaining 97 fps for 512 × 512 inputs on a single NVIDIA GTX 4090 GPU, outperforming other SOTA methods.

A Study on the Identification of the Water Army to Improve the Helpfulness of Online Product Reviews

Based on the perspective of identifying the water army, this paper uses the methods of machine learning and data visualization to analyze the helpfulness of online produce reviews, portray product portraits, and provide real and helpful product reviews. In order to identify and eliminate the water army, the Term Frequency-Inverse Document Frequency Model (TF-IDF) and Latent Semantic Index Model (LSI) are used. After eliminating the water army, three classification methods were selected to perform sentimental analysis, including logistics, SnowNLP, and Convolutional Neural Network for text(TextCNN). The TextCNN has the highest F1 score among the three classification methods. At the same time, the Latent Dirichlet Allocation mode (LDA) is used to extract the topics of various reviews. Finally, targeted countermeasures are proposed to manufacturers, consumers, and regulators.

A generic framework for the semantic contextualization of indicators

Indicators are quantitative or qualitative measures used to gauge various aspects of society and assess change over time (such as monitoring the progress or effectiveness of a public policy). Ideally, indicators should be precisely defined and measured according to harmonized procedures that may not be feasible in practice, especially in domains such as health, where indicators are often derived from preexisting, heterogeneous datasets. Integrating such data has posed a persistent challenge, but semantic technologies offer advantages by enriching data in a relatively simple, linkable, and non-disruptive way. However, without harmonized frameworks, the difficulties associated with data integration are unlikely to be resolved. In this article, we propose a generic, domain-neutral indicator contextualization framework for structuring and linking distributed datasets with contextual metadata according to a standard model. The framework integrates the concepts of the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 11179 metadata registry standard with the common core ontologies (CCO) mid-level ontology suite, and incorporates other semantic technologies to make it adaptable and interoperable within and across domains. Application of the framework to an example indicator illustrates the versatility and adaptability of the approach in a federated data architecture. The contextual information can be dereferenced using standard query tools to provide data users a comprehensive understanding and overview of the indicator. The framework is amenable to deep learning applications via the principles of semantic data models, linked open data, and knowledge organization systems. The ideas are presented to stimulate further reflection and consolidation of standard data contextualization frameworks.

Advancing radiation therapy safety in cancer-care: Leveraging AI for adverse reaction prediction

The development of advanced technologies for radiation cancer treatment plays a pivotal role in anticipating potential complications, safeguarding patient well-being, and elevating treatment satisfaction. Access to curated medical data coupled with specialized knowledge in radiation oncology is indispensable for equipping healthcare professionals with the insights needed to make informed treatment decisions. This study endeavors to provide systematic support to practitioners in predicting personalized side effects of radiation oncology treatments before their administration to patients. For this, a pioneering ontology-based methodology for evidence-based medicine is introduced, drawing upon information and evidence from various scientific outlets. This approach utilizes semantic modeling and rule-based reasoning to assess radiation therapy pathways, ensuring adjustments are made when necessary to mitigate health risks and enhance safety. Focused on radiation oncology, the intricacies of knowledge are simplified and validated through real reported cases obtained from scientific literature. The resulted predictive model demonstrated a notable compatibility rate of 76.7% with real reported risks, surpassing conventional methods by 4% during testing achieves 89% success with single therapies but faces challenges with complex regimens. Data enrichment may increase accuracy by 20% over five years. Current method accuracy is 92%, with potential for 97% through quantum-inspired methods. Testing reveals 80.1% compatibility, exceeding conventional approaches by 3%. These findings hold significant potential of ontology-based models for revolutionizing radiation oncology treatment strategies in the realm of radiation protection.

Semantic Segmentation of Satellite Images for Landslide Detection Using Foreground-Aware and Multi-Scale Convolutional Attention Mechanism.

Advancements in satellite and aerial imagery technology have made it easier to obtain high-resolution remote sensing images, leading to widespread research and applications in various fields. Remote sensing image semantic segmentation is a crucial task that provides semantic and localization information for target objects. In addition to the large-scale variation issues common in most semantic segmentation datasets, aerial images present unique challenges, including high background complexity and imbalanced foreground-background ratios. However, general semantic segmentation methods primarily address scale variations in natural scenes and often neglect the specific challenges in remote sensing images, such as inadequate foreground modeling. In this paper, we present a foreground-aware remote sensing semantic segmentation model. The model introduces a multi-scale convolutional attention mechanism and utilizes a feature pyramid network architecture to extract multi-scale features, addressing the multi-scale problem. Additionally, we introduce a Foreground-Scene Relation Module to mitigate false alarms. The model enhances the foreground features by modeling the relationship between the foreground and the scene. In the loss function, a Soft Focal Loss is employed to focus on foreground samples during training, alleviating the foreground-background imbalance issue. Experimental results indicate that our proposed method outperforms current state-of-the-art general semantic segmentation methods and transformer-based methods on the LS dataset benchmark.

A New Winter Wheat Crop Segmentation Method Based on a New Fast-UNet Model and Multi-Temporal Sentinel-2 Images

Mapping and monitoring crops are the most complex and difficult tasks for experts processing and analyzing remote sensing (RS) images. Classifying crops using RS images is the most expensive task, and it requires intensive labor, especially in the sample collection phase. Fieldwork requires periodic visits to collect data about the crop’s physiochemical characteristics and separating them using the known conventional machine learning algorithms and remote sensing images. As the problem becomes more complex because of the diversity of crop types and the increase in area size, sample collection becomes more complex and unreliable. To avoid these problems, a new segmentation model was created that does not require sample collection or high-resolution images and can successfully distinguish wheat from other crops. Moreover, UNet is a well-known Convolutional Neural Network (CNN), and the semantic method was adjusted to become more powerful, faster, and use fewer resources. The new model was named Fast-UNet and was used to improve the segmentation of wheat crops. Fast-UNet was compared to UNet and Google’s newly developed semantic segmentation model, DeepLabV3+. The new model was faster than the compared models, and it had the highest average accuracy compared to UNet and DeepLabV3+, with values of 93.45, 93.05, and 92.56 respectively. Finally, new datasets of time series NDVI images and ground truth data were created. These datasets, and the newly developed model, were made available publicly on the Web.

Optimisation of U-Net Semantic Segmentation Model Based on Residual Connectivity and Attention Mechanisms

Optimisation of U-Net Semantic Segmentation Model Based on Residual Connectivity and Attention Mechanisms

Hierarchical Event-RGB Interaction Network for single-eye expression recognition

The Single-eye Expression Recognition task stands as a crucial vision task, aimed at decoding human emotional states through careful examination of the eye region. Nevertheless, traditional cameras face challenges in detecting and capturing relevant biological information, especially under demanding lighting conditions such as dim environments, high exposure scenarios, or when other radiation sources are present. In this regard, we use a new type of sensor data that can resist extreme lighting conditions, namely event camera data, to improve the performance of single-eye expression recognition. To this end, we propose a novel Hierarchical Event-RGB Interaction Network (HI-Net), to fully integrate RGB and event data to overcome the extreme lighting challenges faced by the single-eye expression recognition task. The HI-Net contains two novel designs: Event-RGB Semantic Interaction Mechanism (ER-SIM) and Hierarchical Semantics Modeling (HSM) Scheme. The former aims to achieve interaction between Event and RGB modality semantics, while the latter aims to obtain high-quality modality semantic representations. In the ER-SIM, we employ an effective cross-attention mechanism to facilitate information fusion, to adaptively integrate and complement multi-scale Event and RGB semantics to cope with extreme lighting conditions. In HSM Scheme, we first explore multi-scale contextual semantics for the event modality and the RGB modality respectively. Then, we perform a semantics interaction strategy for these multi-scale contextual semantics, to enhance each modality's semantic representation. Extensive experiments demonstrate that our HI-Net significantly outperforms many state-of-the-art methods on the single-eye expression recognition task, especially under degraded lighting conditions.

Asset management knowledge graph for production plants in process industry

Abstract The introduction and adoption of various semantic models for describing assets within a plant have paved the way for new opportunities. This contribution centers on the establishment and continuous maintenance of a plant asset management system based on knowledge graphs, utilizing the information contained in various semantic models. Additionally, it highlights how the graph can be leveraged in the creation and configuration of AI-based anomaly detection models, aimed at the monitoring of measurement values in a process plant. The concept paves the way for the wide deployment of AI models in monitoring applications in process industry.

A Modified Deep Semantic Segmentation Model for Analysis of Whole Slide Skin Images

Automated segmentation of biomedical image has been recognized as an important step in computer-aided diagnosis systems for detection of abnormalities. Despite its importance, the segmentation process remains an open challenge due to variations in color, texture, shape diversity and boundaries. Semantic segmentation often requires deeper neural networks to achieve higher accuracy, making the segmentation model more complex and slower. Due to the need to process a large number of biomedical images, more efficient and cheaper image processing techniques for accurate segmentation are needed. In this article, we present a modified deep semantic segmentation model that utilizes the backbone of EfficientNet-B3 along with UNet for reliable segmentation. We trained our model on Non-melanoma skin cancer segmentation for histopathology dataset to divide the image in 12 different classes for segmentation. Our method outperforms the existing literature with an increase in average class accuracy from 79 to 83%. Our approach also shows an increase in overall accuracy from 85 to 94%.

A Dependent Nominal Physical Type System for Static Analysis of Memory in Low Level Code

We tackle the problem of checking non-proof-carrying code , i.e. automatically proving type-safety (implying in our type system spatial memory safety) of low-level C code or of machine code resulting from its compilation without modification. This requires a precise static analysis that we obtain by having a type system which (i) is expressive enough to encode common low-level idioms, like pointer arithmetic, discriminating variants by bit-stealing on aligned pointers, storing the size and the base address of a buffer in distinct parts of the memory, or records with flexible array members, among others; and (ii) can be embedded in an abstract interpreter. We propose a new type system that meets these criteria. The distinguishing feature of this type system is a nominal organization of contiguous memory regions, which (i) allows nesting, concatenation, union, and sharing parameters between regions; (ii) induces a lattice over sets of addresses from the type definitions; and (iii) permits updates to memory cells that change their type without requiring one to control aliasing. We provide a semantic model for our type system, which enables us to derive sound type checking rules by abstract interpretation, then to integrate these rules as an abstract domain in a standard flow-sensitive static analysis. Our experiments on various challenging benchmarks show that semantic type-checking using this expressive type system generally succeeds in proving type safety and spatial memory safety of C and machine code programs without modification, using only user-provided function prototypes.

Correction to: Measuring the semantic headedness of English blends with token-based semantic vector space modeling: a corpus-based study

Correction to: Measuring the semantic headedness of English blends with token-based semantic vector space modeling: a corpus-based study

Automated vision-based concrete crack measurement system

Maintaining and ensuring the safety of ageing infrastructures is becoming of uttermost importance. This is particularly critical in Europe, where several infrastructures like tunnels and bridges are approaching their end-of-life. Such a condition justifies the need for a prompt recognition of the deterioration status of these structures as well as the identification of strategies for effective inspection and monitoring operations. This paper focuses on cracks, and proposes a vision-based measurement system aimed at precisely and accurately detecting cracks on concrete surfaces and measuring their geometric features. The system mainly leverages RGB-D cameras to make the pixel to real world measurement units conversion possible (by assessing the camera to target relative distance and pose), and exploits an image processing pipeline consisting of the following steps: (a) an artificial intelligence-based semantic segmentation model for identifying cracks on the target structure; (b) a ridge detection algorithm for identifying the centre line of the cracks; (c) a geometric feature extraction algorithm to extract key metrics of the crack, like its width and length. Variance analysis of the crack mean width measurement shows a repeatability standard deviation of 0.01 mm and a reproducibility standard deviation of 0.02 mm, with a Type A expanded uncertainty of 0.004 mm. The system performance is discussed in the paper referring both to lab testing activities, targeted to the assessment of its metrological characteristics, and to a real field application, i.e. concrete inspection in a tunnel. This latter application is of particular interests as it is targeted to demonstrate the robustness of the approach in harsh environments (i.e., strong presence of dust, total absence of natural light) and on surfaces whose roughness generates important light gradients and shadows (e.g., sprayed concrete).

Mapping and modeling the semantic space of math concepts

Mathematics is an underexplored domain of human cognition. While many studies have focused on subsets of math concepts such as numbers, fractions, or geometric shapes, few have ventured beyond these elementary domains. Here, we attempted to map out the full space of math concepts and to answer two specific questions: can distributed semantic models, such a GloVe, provide a satisfactory fit to human semantic judgements in mathematics? And how does this fit vary with education? We first analyzed all of the French and English Wikipedia pages with math contents, and used a semi-automatic procedure to extract the 1000 most frequent math terms in both languages. In a second step, we collected extensive behavioral judgements of familiarity and semantic similarity between them. About half of the variance in human similarity judgements was explained by vector embeddings that attempt to capture latent semantic structures based on cooccurence statistics. Participants' self-reported level of education modulated familiarity and similarity, allowing us to create a partial hierarchy among high-level math concepts. Our results converge onto the proposal of a map of math space, organized as a database of math terms with information about their frequency, familiarity, grade of acquisition, and entanglement with other concepts.

QSIM: A Quantum-inspired hierarchical semantic interaction model for text classification

Semantic interaction modeling is a fundamental technology in natural language understanding that guides models to extract deep semantic information from text. Currently, the attention mechanism is one of the most effective techniques in semantic interaction modeling, which learns word-level attention representation by measuring the relevance between different words. However, the attention mechanism is limited to word-level semantic interaction, it cannot meet the needs of fine-grained interactive information for some text classification tasks. In recent years, quantum-inspired language modeling methods have successfully constructed quantized representations of language systems in Hilbert spaces, which use density matrices to achieve fine-grained semantic interaction modeling.This paper presents a Quantum-inspired hierarchical Semantic Interaction Model (QSIM), which follows the sememe-word-sentence language construction principle and utilizes quantum entanglement theory to capture hierarchical semantic interaction information in Hilbert space. Our work builds on the idea of the attention mechanism and extends it. Specifically, we explore the original semantic space from a quantum theory perspective and derive the core semantic space using the Schmidt decomposition technique, where: (1) Sememe is represented as the unit vector in the two-dimensional minimum semantic space; (2) Word is represented as reduced density matrices in the core semantic space, where Schmidt coefficients quantify sememe-level semantic interaction. Compared to density matrices, reduced density matrices capture fine-grained semantic interaction information with lower computational cost; (3) Sentence is represented as quantum superposition states of words, and the degree of word-level semantic interaction is measured using entanglement entropy.To evaluate the model’s performance, we conducted experiments on 15 text classification datasets. The experimental results demonstrate that our model is superior to classical neural network models and traditional quantum-inspired language models. Furthermore, the experiment also confirms two distinct advantages of QISM: (1) flexibility, as it can be integrated into various mainstream neural network text classification architectures; and (2) practicability, as it alleviates the problem of parameter growth inherent in density matrix calculation in quantum language model.

An embedded computational framework of memory: Accounting for the influence of semantic information in verbal short-term memory

We introduce the Embedded Computational Framework of Memory (eCFM), a model that integrates structured semantic word representations with an instance-based memory model to account for the influence of semantic information in verbal short-term memory. The eCFM combines principles from the episodic MINERVA 2 model and the semantic Latent Semantic Analysis model. After reviewing how semantic information impacts verbal short-term memory performance, we demonstrate eCFM’s ability to reconcile various phenomena within a common computational framework. Our model captures key findings, such as the influence of semantic information in serial recall, its reduction in serial reconstruction, and the impact of task difficulty on semantic information. In five experiments, we tested predictions derived from the eCFM. Experiments 1 and 2 manipulated list organization, with Experiment 1 using alternating lists of related or unrelated words and Experiment 2 using blocked lists. Experiment 3 varied presentation rates, Experiment 4 revisited the detrimental effect of semantic information on order information, and Experiment 5 explored false recall. We found that semantic information interacts with list composition, presentation rate affects the magnitude of its influence, and semantic information impacts order information contrary to the dominant view. Additionally, increasing the number of related study words to a non-studied semantic lure boosts false recall. The eCFM captured these findings as well as memory at the item level. Our demonstration provides insight into the cognitive mechanisms underlying verbal short-term memory and the interplay of semantic and episodic memory processes in recall.