Semantic Correctness Research Articles

Automatic differentiation for ML-family languages: Correctness via logical relations

Abstract We give a simple, direct, and reusable logical relations technique for languages with term and type recursion and partially defined differentiable functions. We demonstrate it by working out the case of automatic differentiation (AD) correctness: namely, we present a correctness proof of a dual numbers style AD code transformation for realistic functional languages in the ML-family. We also show how this code transformation provides us with correct forward- and reverse-mode AD. The starting point is to interpret a functional programming language as a suitable freely generated categorical structure. In this setting, by the universal property of the syntactic categorical structure, the dual numbers AD code transformation and the basic $\boldsymbol{\omega } \mathbf{Cpo}$ -semantics arise as structure preserving functors. The proof follows, then, by a novel logical relations argument. The key to much of our contribution is a powerful monadic logical relations technique for term recursion and recursive types. It provides us with a semantic correctness proof based on a simple approach for denotational semantics, making use only of the very basic concrete model of $\omega$ -cpos.

Enhancing Text-To-Image Translation with Stack-Generative Adversarial Networks

Text-to-image translation is the process of generating visually coherent images from textual descriptions. Traditional GANs suffer from inadequacy in capturing intricate details and therefore pose an obstacle in achieving this task. Hence, in this paper, we propose an improved approach based on the concept of Stack Generative Adversarial Networks, in which the generative process is applied in two stages. Stage I produces a 64x64 low-resolution image that offers solid shapes and colors. Then, in Stage II, this is further intensified into a resolution of 256x256 such that it closely resembles the text input. The architecture of StackGAN surpasses the earlier variant GANs and other derivatives for it produces high-resolution images with semantic correctness closely following the description in the given text. The staged nature of this approach improves image quality while ensuring that a stronger relationship lies between the produced image and the input text. Future works will attempt to enhance the model's capabilities while applying it in other areas of synthesis in images.

Position-Aware Relational Transformer for Knowledge Graph Embedding.

Although Transformer has achieved success in language and vision tasks, its capacity for knowledge graph (KG) embedding has not been fully exploited. Using the self-attention (SA) mechanism in Transformer to model the subject-relation-object triples in KGs suffers from training inconsistency as SA is invariant to the order of input tokens. As a result, it is unable to distinguish a (real) relation triple from its shuffled (fake) variants (e.g., object-relation-subject) and, thus, fails to capture the correct semantics. To cope with this issue, we propose a novel Transformer architecture, namely, Knowformer, for KG embedding. It incorporates relational compositions in entity representations to explicitly inject semantics and capture the role of an entity based on its position (subject or object) in a relation triple. The relational composition for a subject (or object) entity of a relation triple refers to an operator on the relation and the object (or subject). We borrow ideas from the typical translational and semantic-matching embedding techniques to design relational compositions. We carefully design a residual block to integrate relational compositions into SA and efficiently propagate the composed relational semantics layer by layer. We formally prove that the SA with relational compositions is able to distinguish the entity roles in different positions and correctly capture relational semantics. Extensive experiments and analyses on six benchmark datasets show that Knowformer achieves state-of-the-art performance on both link prediction and entity alignment.

A transformer-based Urdu image caption generation

Image caption generation has emerged as a remarkable development that bridges the gap between Natural Language Processing (NLP) and Computer Vision (CV). It lies at the intersection of these fields and presents unique challenges, particularly when dealing with low-resource languages such as Urdu. Limited research on basic Urdu language understanding necessitates further exploration in this domain. In this study, we propose three Seq2Seq-based architectures specifically tailored for Urdu image caption generation. Our approach involves leveraging transformer models to generate captions in Urdu, a significantly more challenging task than English. To facilitate the training and evaluation of our models, we created an Urdu-translated subset of the flickr8k dataset, which contains images featuring dogs in action accompanied by corresponding Urdu captions. Our designed models encompassed a deep learning-based approach, utilizing three different architectures: Convolutional Neural Network (CNN) + Long Short-term Memory (LSTM) with Soft attention employing word2Vec embeddings, CNN+Transformer, and Vit+Roberta models. Experimental results demonstrate that our proposed model outperforms existing state-of-the-art approaches, achieving 86 BLEU-1 and 90 BERT-F1 scores. The generated Urdu image captions exhibit syntactic, contextual, and semantic correctness. Our study highlights the inherent challenges associated with retraining models on low-resource languages. Our findings highlight the potential of pre-trained models for facilitating the development of NLP and CV applications in low-resource language settings.

GMAlignNet: multi-scale lightweight brain tumor image segmentation with enhanced semantic information consistency

Although the U-shaped architecture, represented by UNet, has become a major network model for brain tumor segmentation, the repeated convolution and sampling operations can easily lead to the loss of crucial information. Additionally, directly fusing features from different levels without distinction can easily result in feature misalignment, affecting segmentation accuracy. On the other hand, traditional convolutional blocks used for feature extraction cannot capture the abundant multi-scale information present in brain tumor images. This paper proposes a multi-scale feature-aligned segmentation model called GMAlignNet that fully utilizes Ghost convolution to solve these problems. Ghost hierarchical decoupled fusion unit and Ghost hierarchical decoupled unit are used instead of standard convolutions in the encoding and decoding paths. This transformation replaces the holistic learning of volume structures by traditional convolutional blocks with multi-level learning on a specific view, facilitating the acquisition of abundant multi-scale contextual information through low-cost operations. Furthermore, a feature alignment unit is proposed that can utilize semantic information flow to guide the recovery of upsampled features. It performs pixel-level semantic information correction on misaligned features due to feature fusion. The proposed method is also employed to optimize three classic networks, namely DMFNet, HDCNet, and 3D UNet, demonstrating its effectiveness in automatic brain tumor segmentation. The proposed network model was applied to the BraTS 2018 dataset, and the results indicate that the proposed GMAlignNet achieved Dice coefficients of 81.65%, 90.07%, and 85.16% for enhancing tumor, whole tumor, and tumor core segmentation, respectively. Moreover, with only 0.29 M parameters and 26.88G FLOPs, it demonstrates better potential in terms of computational efficiency and possesses the advantages of lightweight. Extensive experiments on the BraTS 2018, BraTS 2019, and BraTS 2020 datasets suggest that the proposed model exhibits better potential in handling edge details and contour recognition.

Privacy-Preserving and Cross-Domain Human Sensing by Federated Domain Adaptation with Semantic Knowledge Correction

Privacy-Preserving and Cross-Domain Human Sensing by Federated Domain Adaptation with Semantic Knowledge Correction

Sec-Auditor: A Blockchain-Based Data Auditing Solution for Ensuring Integrity and Semantic Correctness

Sec-Auditor: A Blockchain-Based Data Auditing Solution for Ensuring Integrity and Semantic Correctness

DFBU-Net: Double-branch flat bottom U-Net for efficient medical image segmentation

In the field of medical image processing, segmenting tissues and organs in CT/MRI and other medical sequence images is a vital yet challenging task. Analyzing the MICCAI competition, we have identified two problems in current methods for medical image organ segmentation: (1) There is a bottleneck in organ segmentation, with marginal room for improvement, as algorithmic capabilities have already surpassed the task’s inherent difficulty. (2) Most current research focuses on stacking and enhancing new modules for segmentation while overlooking the inherent characteristics of medical sequence images. To overcome these two problems, firstly, we have encapsulated the three characteristics of CT/MRI medical sequence image segmentation: semantic correctness, edge accuracy, and 3D structure. Secondly, we delved into the most information-rich downsampling stage in terms of detail and semantics. Subsequently, we designed a flat-bottom double-branch network (DFBU-Net) based on the U-Net architecture. The high-resolution flat bottom branch of this network maintained a 1/4 feature map size to ensure the preservation of rich detail information, while the low-resolution branch underwent progressive downsampling to capture more semantic information. To prevent information loss, cross-fusion was performed at each stage of the model’s two branches. Finally, DFBU-Net was evaluated on the MICCAI FLARE2021 dataset (DSC:93.61%, NSD:85.01%). Particularly, in the challenging task of pancreatic segmentation, our model outperformed the first-place model by 0.72% in DSC and 2.92% in NSD. Furthermore, in the MICCAI PARSE2022 competition, DFBU-Net ranked ninth with a DICE score of 79.28%, demonstrating its excellent segmentation performance and generalization ability.

Semantic correction, enrichment and enhancement of social and transport infrastructure BIM models

The use of Building Information Modelling (BIM) models in the design, construction and operation of buildings and infrastructure is leading to a stronger focus on the quality of the models. Models may need correction, enrichment or enhancement to meet the expectations for quality and completeness, especially if models are to be taken as legal documents, for example for regulatory approval. Past work on semantic development has looked at specific scenarios such as scanned geometry or missing classification. This paper describes an innovative unified approach to the documentation of semantic expectations by actors in the AECO (Architectural, Engineering, Construction and Operations) domain and the means to put them into effect. RASE (Requirements, Applications, Selections and Exceptions) semantic mark-up is used to make both the requirements and any supporting resources both human-readable and machine-operable. Two example models from industry, a motorway bridge and a healthcare space, are used to demonstrate applying geometric, schema and classification knowledge. This knowledge is represented in a number of different styles. This extends our understanding of the nature of the knowledge found in dictionaries, classifications and development specifications, demonstrating how this knowledge can be made operable. This bridges the gap between the application of static compliance knowledge and the accurate and efficient application of correction, enrichment and enhancement knowledge.

Knowledge representation of mathematical optimization problems and constructs for modeling

We introduce two new knowledge bases that we have developed to categorize mixed-integer linear programming (MILP) problems and standardize the element definitions of MILP models. MILP is a commonly used mathematical programming technique for modeling and solving real-life scheduling, routing, planning, resource allocation, and timetabling optimization problems providing optimized business solutions for industry sectors such as manufacturing, agriculture, defense, healthcare, medicine, energy, finance, and transportation among others. Despite the numerous real-life combinatorial optimization problems (COPs) found and solved, and many yet to be discovered and formulated, the number of types of constraints (the building blocks of a MILP) is relatively small. In the search for a uniform categorization of MILP problems and a machine-readable knowledge representation structure for MILP models, we have developed an optimization modeling tree (OMT) and a MILP model ontology. The two knowledge structures can serve as a standardized, uniform representation in understanding MILP problems and developing MILP models for combinatorial business optimization problems. While there are several algebraic modeling languages (AMLs) for developing and solving MILP models, the semantic correctness of such models cannot be guaranteed using the syntactic grammar of such AMLs. The MILP model ontology will act as the main resource for semantic validation of MILP models through ontology instantiations and axiom assertions.

Semantic Consistency and Correctness Verification of Digital Traffic Rules

The consensus of the automotive industry and traffic management authorities is that autonomous vehicles must follow the same traffic laws as human drivers. Using formal or digital methods, natural language traffic rules can be translated into machine language and used by autonomous vehicles. In this paper, a translation flow is designed. Beyond the translation, a deeper examination is required, because the semantics of natural languages are rich and complex, and frequently contain hidden assumptions. The issue of how to ensure that digital rules are accurate and consistent with the original intent of the traffic rules they represent is both significant and unresolved. In response, we propose a method of formal verification that combines equivalence verification with model checking. Reasonable and reassuring digital traffic rules can be obtained by utilizing the proposed traffic rule digitization flow and verification method. In addition, we offer a number of simulation applications that employ digital traffic rules to assess vehicle violations. The experimental findings indicate that our digital rules utilizing metric temporal logic (MTL) can be easily incorporated into simulation platforms and autonomous driving systems (ADS).

SCFNet: Semantic correction and focus network for remote sensing image object detection

SCFNet: Semantic correction and focus network for remote sensing image object detection

CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics

Considerable progress has recently been made in leveraging CLIP (Contrastive Language-Image Pre-Training) models for text-guided image manipulation. However, all existing works rely on additional generative models to ensure the quality of results, because CLIP alone cannot provide enough guidance information for fine-scale pixel-level changes. In this paper, we introduce CLIPVG, a text-guided image manipulation framework using differentiable vector graphics, which is also the first CLIP-based general image manipulation framework that does not require any additional generative models. We demonstrate that CLIPVG can not only achieve state-of-art performance in both semantic correctness and synthesis quality, but also is flexible enough to support various applications far beyond the capability of all existing methods.

Target-Free Text-Guided Image Manipulation

We tackle the problem of target-free text-guided image manipulation, which requires one to modify the input reference image based on the given text instruction, while no ground truth target image is observed during training. To address this challenging task, we propose a Cyclic-Manipulation GAN (cManiGAN) in this paper, which is able to realize where and how to edit the image regions of interest. Specifically, the image editor in cManiGAN learns to identify and complete the input image, while cross-modal interpreter and reasoner are deployed to verify the semantic correctness of the output image based on the input instruction. While the former utilizes factual/counterfactual description learning for authenticating the image semantics, the latter predicts the "undo" instruction and provides pixel-level supervision for the training of cManiGAN. With the above operational cycle-consistency, our cManiGAN can be trained in the above weakly supervised setting. We conduct extensive experiments on the datasets of CLEVR and COCO datasets, and the effectiveness and generalizability of our proposed method can be successfully verified. Project page: sites.google.com/view/wancyuanfan/projects/cmanigan.

A Shared Hippocampal Network in Retrieving Science-related Semantic Memories.

In responding to the calls for revisiting the role that hippocampus (HIP) plays in semantic memory retrieval, this study used functional neuroimaging-based connectivity technique to elucidate the functional brain network involved in retrieving the correct and incorrect science-related semantic memories. Unlike episodic memory retrieval, the 40 scientific concepts learned during middle and high school were selected to assess 46 science majors' semantic memory retrieval and correctness monitoring, which requires neither the support of spatial information nor events to retrieve the memory. Our results demonstrated that HIP was significantly and robustly engaged in the semantic memory retrieval of correct scientific concepts than incorrect ones. Importantly, the Granger causality analysis indicated that effective connectivity of [Formula: see text] and [Formula: see text] was shared by the semantic memory retrieval of both correct and incorrect scientific concepts. On the other hand, the strengths of connectivity in the [Formula: see text] and [Formula: see text] brain networks appeared more pronounced during the processing of correct scientific concepts than of incorrect ones. The shared hippocampal networks highlight the role of the HIP as a hub to coordinate the INS, ACC, and MTG, in turn, support the semantic memory retrieval of scientific concepts.

Pedestrian Re-ID based on feature consistency and contrast enhancement

For robust person re-identification(Re-ID), the key is effectively learning the features of body parts and their long-distance dependence. ResNet and Transformer are respectively good at learning local dependence and long-distance dependence between region features due to their respective special structures. In order to fully integrate the advantages of the two models, we propose a novel person Re-ID framework that effectively incorporates pixel-level region features, posture-level relation features and the long-distance dependence of region features. Specifically, we design a Semantic Correction Module (SCM) that corrects pixel-level region features and posture-level relation features in a masked manner to generate discriminative fine-grained features with high pose semantics. Considering the semantic inconsistency between relation features and region features, we propose a Contrastive Association Module (CAM) to interactively enhances the long-distance correlation and local saliency of features in a self-attention way. Finally, to improve the robustness of local and global features, we construct a CAM layer to enhance the representation of features based on their potential relationships. Extensive experiment results on general and occlusion datasets demonstrate that our approach performs favorably against the state-of-the-art methods, e.g. 96% Rank-1 on Market-1501.

Improving Efficiency and Accuracy in English Translation Learning: Investigating a Semantic Analysis Correction Algorithm

ABSTRACT Translation serves as a vital link in connecting individuals from diverse cultural backgrounds, assuming greater significance in the context of globalization. With the continued growth of international communication, the importance of effective translation in fostering mutual understanding, exploring new horizons, and building relationships cannot be overstated. However, the persistent language barrier necessitates improvements in translation accuracy and efficiency. In recent years, information technology has facilitated the development of innovative algorithms for English translation. This study aims to investigate correction algorithms for English translation learning, proposing a method that encompasses semantic word similarity calculations, a log-linear model construction, and the selection of appropriate translations using a dependency tree-to-string approach. Our research examines the performance of this system through comprehensive testing, showcasing noteworthy enhancements in the efficacy and precision of the translations in English. By addressing the users’ needs for English translation correction, this work contributes to bridging language gaps and fostering effective cross-cultural communication.

Improvement and Application of Fusion Scheme in Automatic Medical Image Analysis

The research in this paper provides generalization and new ideas for research topics in computer-assisted medicine. The main improvement efforts in deep learning-based multimodal fusion schemes, which provide alternative directions and robust feature fitting performance for fusion schemes, are building complex structures, migrating knowledge or experience, processing and enhancing data, and targeting features for semantic correction based on contextual features. At the application level, the brain, liver, and lungs are the main targets of scientific research, so this paper surveys related work and analyzes the reasons for performance gains. Taken together, deep learning-based image fusion schemes can assist physicians in understanding information about lesion sites, lesion types, and sizes, providing an important basis for developing personalized treatment plans, which is important for improving diagnosis and specifying precise treatment plans. Therefore, the investigation of medical image fusion schemes is promising and beneficial.

Irregular mask image inpainting based on progressive generative adversarial networks

ABSTRACT Existing image inpainting algorithms have achieved good results in regular mask inpainting tasks, but there are still many limitations for irregular mask, and the inpainting results lack edge consistency and semantic correctness. To address these problems, we designed a progressive generative adversarial network in this paper. The generative network mainly contains a progressive feature generation module and an adaptive consistent attention module, and the discriminator network uses SN-PatchGAN. The network performs recursive inpainting from the edge to the centre, and the repaired features are used as the conditions for the next feature generation, which makes the constraints of the central content gradually strengthened. In order to obtain distant information from the feature maps as well as to consider the possible problem of inconsistency between feature mappings under different recursions when using attention directly in the progressive inpainting network, we designed an adaptive consistent attention module in this paper, which can adaptively combine scores obtained from different recursions to capture more feature information. The discriminator used in this paper is SN-PatchGAN, which directly calculates the hinge loss of each point on the output graph and can focus on different locations as well as different semantics. Comparing this paper's method on CelebA dataset and CMP Facade dataset with the latest work, the experimental results show that this paper's method performs well in the irregular mask image inpainting task, and the inpainting results have better performance in terms of edge consistency, semantic correctness, and overall image structure.

Can Linguistic Correctness Provide Us with Categorical Semantic Norms?

Saul Kripke’s paradoxical argument in Wittgenstein on Rules and Private Language (1982) has generated an extravagant number of responses. A major debate prompted by this book has focused on the plausibility and role of the supposed normative character of meaning; the argument itself is often taken to rely on the assumption that meaning is irreducibly normative. Following Boghossian (1989), the normativity of meaning has been understood as closely tied to the existence of semantic correctness conditions. After a brief introduction to the background of the debate, this work will focus on whether the normativity of meaning may be better understood as stemming from a different type of correctness, namely linguistic correctness. Linguistic correctness differs from semantic correctness insofar as it is related to conventional, and not truth-functional, meaning. I will begin by clarifying some of the features of linguistic correctness. First, I will outline some reasons why the distinction between linguistic and semantic correctness should be maintained. Then, I will anticipate a possible criticism and argue that linguistic correctness does not belong to the domain of pragmatics, as it is relevant to our understanding of conventional meaning. Finally, I will try to show that linguistic “oughts” are constitutive of meaning. Having established these basic features of linguistic correctness, I will investigate whether the fact that it is constitutive of meaning can vindicate the idea that meaning is robustly, irreducibly normative. By applying arguments from the realm of moral philosophy – within which, too, there have been attempts to show that constitutive facts can give rise to categorical moral norms – I will argue that linguistic correctness cannot give rise to categorical semantic norms. Linguistic correctness may be, nevertheless, a useful tool for explaining some of our intuitions about meaning.