Semantic Inconsistencies Research Articles

Towards better semantic consistency of 2D medical image segmentation

The latest deep neural networks for medical segmentation typically utilize transposed convolutional filters and atrous convolutional filters for spatial restoration and larger receptive fields, leading to dilution and inconsistency of visual semantics. To address such issues, we propose a novel attentional up-concatenation structure to build an auxiliary path for direct access to multi-level features. In addition, we employ a new structural loss to bring better morphological awareness and reduce the segmentation flaws caused by the semantic inconsistencies. Thorough experiments on the challenging optic cup/disc segmentation, cellular segmentation and lung segmentation tasks were performed to evaluate the proposed methods. Further ablation analysis demonstrated the effectiveness of the different components of the model and illustrated its efficiency. The proposed methods achieved the best performance and speed compared to the state-of-the-art models in three tasks on seven public datasets, including DRISHTI-GS, RIM-r3, REFUGE, MESSIDOR, TNBC, GlaS and LUNA.

The magician's choice: Providing illusory choice and sense of agency with the equivoque forcing technique.

Forcing techniques allow magicians to subtly influence spectators' choices and the outcome of their actions, and they provide powerful tools to study decision-making and the illusory sense of agency and freedom over choices we make. We investigated the equivoque force, a technique that exploits semantic ambiguities and people's failure to notice inconsistencies, to ensure that a spectator ends up with a predetermined outcome. Similar to choice blindness paradigms, the equivoque forces participants to end up with an item they did not choose in the first place. However, here, the subterfuge is accomplished in full view. In 3 experiments, we showed that the equivoque is highly effective in providing participants an illusory sense of agency over the outcome of their actions, even after 2 repetitions of the trick (Experiment 2) and using items for which preexisting preferences can be present (Experiment 3). Across all experiments, participants were oblivious to inconsistencies in the procedure used to guide their decisions, and they were genuinely surprised by the experimenter's matching prediction. Contrary to our prediction, the equivoque force did not significantly change participants' preference for the chosen item. We discuss the results with regard to other illusions of agency (e.g., forcing, choice blindness), failures in noticing semantic inconsistencies (e.g., Moses illusion), and issues surrounding choice-induced-preference literature. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

CFFNet: Coordinated feature fusion network for crowd counting

Long skip connection or encoder-decoder networks for crowd counting have proven to be effective methods to generate high-resolution density maps. However, the simple and coarse feature fusion ignores the disharmony between features, that is, spatial misalignment and semantic inconsistency, which will weaken feature representation and degrade network performance. In this paper, we propose an end-to-end trainable architecture called Coordinated Feature Fusion Network (CFFNet) to tackle the aforementioned problems. The proposed model contains a powerful baseline network and embeds two primary modules: Spatial Alignment Module (SAM) and Semantic Consistency Module (SCM). Specifically, the SAM can learn the transformation offset of pixels to alleviate the spatial misalignment caused by the feature resolution difference; the SCM based on the multi-scale attention mechanism can capture pixel-wise weight to alleviate the semantic inconsistency due to the feature level gap. Extensive experiments on four benchmark crowd datasets (the ShanghaiTech, the UCF-QNRF, the JHU-CRWORD++ and the NWPU-Crowd), indicate that CFFNet can achieve state-of-the-art counting performance and high robustness.

Disrupted object-scene semantics boost scene recall but diminish object recall in drawings from memory.

Humans are highly sensitive to the statistical relationships between features and objects within visual scenes. Inconsistent objects within scenes (e.g., a mailbox in a bedroom) instantly jump out to us and are known to catch our attention. However, it is debated whether such semantic inconsistencies result in boosted memory for the scene, impaired memory, or have no influence on memory. Here, we examined the relationship of scene-object consistencies on memory representations measured through drawings made during recall. Participants (N = 30) were eye-tracked while studying 12 real-world scene images with an added object that was either semantically consistent or inconsistent. After a 6-minute distractor task, they drew the scenes from memory while pen movements were tracked electronically. Online scorers (N = 1,725) rated each drawing for diagnosticity, object detail, spatial detail, and memory errors. Inconsistent scenes were recalled more frequently, but contained less object detail. Further, inconsistent objects elicited more errors reflecting looser memory binding (e.g., migration across images). These results point to a dual effect in memory of boosted global (scene) but diminished local (object) information. Finally, we observed that participants fixate longest on inconsistent objects, but these fixations during study were not correlated with recall performance, time, or drawing order. In sum, these results show a nuanced effect of scene inconsistencies on memory detail during recall.

Consensus Graph Representation Learning for Better Grounded Image Captioning

The contemporary visual captioning models frequently hallucinate objects that are not actually in a scene, due to the visual misclassification or over-reliance on priors that resulting in the semantic inconsistency between the visual information and the target lexical words. The most common way is to encourage the captioning model to dynamically link generated object words or phrases to appropriate regions of the image, i.e., the grounded image captioning (GIC). However, GIC utilizes an auxiliary task (grounding objects) that has not solved the key issue of object hallucination, i.e., the semantic inconsistency. In this paper, we take a novel perspective on the issue above: exploiting the semantic coherency between the visual and language modalities. Specifically, we propose the Consensus Rraph Representation Learning framework (CGRL) for GIC that incorporates a consensus representation into the grounded captioning pipeline. The consensus is learned by aligning the visual graph (e.g., scene graph) to the language graph that consider both the nodes and edges in a graph. With the aligned consensus, the captioning model can capture both the correct linguistic characteristics and visual relevance, and then grounding appropriate image regions further. We validate the effectiveness of our model, with a significant decline in object hallucination (-9% CHAIRi) on the Flickr30k Entities dataset. Besides, our CGRL also evaluated by several automatic metrics and human evaluation, the results indicate that the proposed approach can simultaneously improve the performance of image captioning (+2.9 Cider) and grounding (+2.3 F1LOC}).

Rich CNN Features for Water-Body Segmentation from Very High Resolution Aerial and Satellite Imagery

Extracting water-bodies accurately is a great challenge from very high resolution (VHR) remote sensing imagery. The boundaries of a water body are commonly hard to identify due to the complex spectral mixtures caused by aquatic vegetation, distinct lake/river colors, silts near the bank, shadows from the surrounding tall plants, and so on. The diversity and semantic information of features need to be increased for a better extraction of water-bodies from VHR remote sensing images. In this paper, we address these problems by designing a novel multi-feature extraction and combination module. This module consists of three feature extraction sub-modules based on spatial and channel correlations in feature maps at each scale, which extract the complete target information from the local space, larger space, and between-channel relationship to achieve a rich feature representation. Simultaneously, to better predict the fine contours of water-bodies, we adopt a multi-scale prediction fusion module. Besides, to solve the semantic inconsistency of feature fusion between the encoding stage and the decoding stage, we apply an encoder-decoder semantic feature fusion module to promote fusion effects. We carry out extensive experiments in VHR aerial and satellite imagery respectively. The result shows that our method achieves state-of-the-art segmentation performance, surpassing the classic and recent methods. Moreover, our proposed method is robust in challenging water-body extraction scenarios.

Формальные и семантические сближения в сфере русско-польской межъязыковой омонимии

This article deals with the issue of structural and semantic relationships in the area of Russian-Polish interlingual homonymy, such as exclusion, inclusion, and hybridization. The analysis of Russian-Polish homonymous doublets made it possible to specify two basic reasons for the differences in meaning of the lexemes in both languages: semantic divergence of the words with a common etymon (disintegration of polysemy) and phonetic convergence. The study revealed that interlanguage Russian-Polish correspondences with a partial coincidence of lexical meanings make up the largest group of interlanguage homo-pairs. The types of semantic relations described at the word level do not exhaust the variety of semantic relations between Russian-Polish interlanguage homonyms. The depth of meaning development is another theoretical problem of modern lexicology and lexicography. In addition to the semantic differences between Russian-Polish homo-pairs, there may be more complex relationships. They are observed in cases where stylistic and functional discrepancies are layered on semantic inconsistencies. The differences in pragmatic significance are the most significant and can form the subject of independent study.

A Semi-Automatic Semantic Consistency-Checking Method for Learning Ontology from Relational Database

To tackle the issues of semantic collision and inconsistencies between ontologies and the original data model while learning ontology from relational database (RDB), a semi-automatic semantic consistency checking method based on graph intermediate representation and model checking is presented. Initially, the W-Graph, as an intermediate model between databases and ontologies, was utilized to formalize the semantic correspondences between databases and ontologies, which were then transformed into the Kripke structure and eventually encoded with the SMV program. Meanwhile, description logics (DLs) were employed to formalize the semantic specifications of the learned ontologies, since the OWL DL showed good semantic compatibility and the DLs presented an excellent expressivity. Thereafter, the specifications were converted into a computer tree logic (CTL) formula to improve machine readability. Furthermore, the task of checking semantic consistency could be converted into a global model checking problem that could be solved automatically by the symbolic model checker. Moreover, an example is given to demonstrate the specific process of formalizing and checking the semantic consistency between learned ontologies and RDB, and a verification experiment was conducted to verify the feasibility of the presented method. The results showed that the presented method could correctly check and identify the different kinds of inconsistencies between learned ontologies and its original data model.

Fighting Fake News: Two Stream Network for Deepfake Detection via Learnable SRM

Benefitting from the development of deep generative networks, modern fake news generation methods called Deepfake rapidly go viral over the Internet, calling for efficient detection methods. Existing Deepfake detection methods basically use binary classification networks trained on frame-level inputs and lack leveraging temporal information in videos. Besides, the accuracy of these methods will rapidly decrease when processing low-quality data. In this work, we propose a two-stream network to detect Deepfake in video level with the capability of handling low-quality data. The proposed architecture firstly divides the input video into segments and then feeds selected frames of each segment into two streams: The first stream takes RGB information as input and tries to learn the semantic inconsistency. The second stream parallelly leverages noise features extracted by spatial rich model (SRM) filters. Additionally, our experiments found that traditional SRM filters with fixed weights contribute insignificant improvement, we thus design novel learnable SRM filters, which can better fit the noise inconsistency in tampered regions. Segmental fusion and stream fusion are conducted at last to combine the information from segments and streams. We evaluate our algorithm on the existing largest Deepfake dataset FaceForensics++ and the experimental results show that we obtain state-of-the-art performance.

Zero-shot Cross-modal Retrieval by Assembling AutoEncoder and Generative Adversarial Network

Conventional cross-modal retrieval models mainly assume the same scope of the classes for both the training set and the testing set. This assumption limits their extensibility on zero-shot cross-modal retrieval (ZS-CMR), where the testing set consists of unseen classes that are disjoint with seen classes in the training set. The ZS-CMR task is more challenging due to the heterogeneous distributions of different modalities and the semantic inconsistency between seen and unseen classes. A few of recently proposed approaches are inspired by zero-shot learning to estimate the distribution underlying multimodal data by generative models and make the knowledge transfer from seen classes to unseen classes by leveraging class embeddings. However, directly borrowing the idea from zero-shot learning (ZSL) is not fully adaptive to the retrieval task, since the core of the retrieval task is learning the common space. To address the above issues, we propose a novel approach named Assembling AutoEncoder and Generative Adversarial Network (AAEGAN), which combines the strength of AutoEncoder (AE) and Generative Adversarial Network (GAN), to jointly incorporate common latent space learning, knowledge transfer, and feature synthesis for ZS-CMR. Besides, instead of utilizing class embeddings as common space, the AAEGAN approach maps all multimodal data into a learned latent space with the distribution alignment via three coupled AEs. We empirically show the remarkable improvement for ZS-CMR task and establish the state-of-the-art or competitive performance on four image-text retrieval datasets.

HTD: Heterogeneous Task Decoupling for Two-Stage Object Detection.

Decoupling the sibling head has recently shown great potential in relieving the inherent task-misalignment problem in two-stage object detectors. However, existing works design similar structures for the classification and regression, ignoring task-specific characteristics and feature demands. Besides, the shared knowledge that may benefit the two branches is neglected, leading to potential excessive decoupling and semantic inconsistency. To address these two issues, we propose Heterogeneous task decoupling (HTD) framework for object detection, which utilizes a Progressive Graph (PGraph) module and a Border-aware Adaptation (BA) module for task-decoupling. Specifically, we first devise a Semantic Feature Aggregation (SFA) module to aggregate global semantics with image-level supervision, serving as the shared knowledge for the task-decoupled framework. Then, the PGraph module performs progressive graph reasoning, including local spatial aggregation and global semantic interaction, to enhance semantic representations of region proposals for classification. The proposed BA module integrates multi-level features adaptively, focusing on the low-level border activation to obtain representations with spatial and border perception for regression. Finally, we utilize the aggregated knowledge from SFA to keep the instance-level semantic consistency (ISC) of decoupled frameworks. Extensive experiments demonstrate that HTD outperforms existing detection works by a large margin, and achieves single-model 50.4%AP and 33.2% APs on COCO test-dev set using ResNet-101-DCN backbone, which is the best entry among state-of-the-arts under the same configuration. Our code is available at https://github.com/CityU-AIM-Group/HTD.

Tradução para o português brasileiro e adaptação cultural da Definição de Pontuações de Componentes (Component Scores Definition) pertencentes ao Modified Barium Swallow Impairment Profile – MBSImP(™)

Translate into Brazilian Portuguese and culturally adapt the component scores definition from the Modified Barium Swallow Impairment Profile - MBSImP videofluoroscopy protocol. This study was conducted based on international guidelines for creation, translation and transcultural adaptation according to domestic publications for the validation of international speech-language pathology tests. A specialist committee was convened with two otorhinolaryngologists and three speech-language pathologists familiar with videofluoroscopy examinations. Translation into Brazilian Portuguese was carried out by two otorhinolaryngologists with subsequent back translation performed independently by two U.S.-born translators. The final Brazilian Portuguese version was written by speech-language pathologists after revision of translations and back translations resolved semantic, idiomatic, conceptual, linguistic and contextual inconsistencies. This version was tested by committee members, who assessed subjects with no disease, with head and neck cancer and with mild cognitive impairment. Translation of the component scores definition from MBSImP was performed independently and translators agreed on a final version. The MBSImP protocol was renamed "Martin-Harris scale for swallowing videofluoroscopy". Some terms were adapted for the final Brazilian Portuguese version. Use of the translated protocol did not reveal any deviations from the original. Translation of the MBSImP fragment into Brazilian Portuguese was consistent with the original version. Approval from specialist committee members refined the protocol adaptation, allowing precise concepts to be accurately translated.

Metaphor identification: A contextual inconsistency based neural sequence labeling approach

Metaphor identification helps improve the performance of various natural language understanding tasks such as word sense disambiguation and sentiment analysis. Though many efforts have been made to deal with metaphor identification, most existing studies largely overlook a fact of contextual inconsistency of the metaphors in natural language. We observe that the greater the semantic inconsistency between current word and contextual words is, the more likely the word belongs to the metaphorical category. In this paper, we formulate the metaphor identification as a sequential tagging problem, and then develop a novel contextual inconsistency based neural sequence labeling approach, which can leverage the semantic contextual inconsistency among words of a sentence to address the problem. We propose to rely on distance metric to measure the contextual inconsistency, and evaluate four widely used distance functions in experiments. Experimental results on publicly available datasets validate the benefit of the proposed model over state-of-the-art baselines for metaphor identification.

The Development of Scene Semantics: First ERP Indications for the Processing of Semantic Object-Scene Inconsistencies in 24-Month-Olds

Finding a bottle of milk in the bathroom would probably be quite surprising to most of us. Such a surprised reaction is driven by our strong expectations, learned through experience, that a bottle of milk belongs in the kitchen. Our environment is not randomly organized but governed by regularities that allow us to predict what objects can be found in which types of scene. These scene semantics are thought to play an important role in the recognition of objects. But when during development are the semantic predictions so far implemented that such scene-object inconsistencies would lead to semantic processing difficulties? Here we investigated how toddlers perceive their environments, and what expectations govern their attention and perception. To this aim, we used a purely visual paradigm in an ERP experiment and presented 24-month-olds with familiar scenes in which either a semantically consistent or an inconsistent object would appear. The scene-inconsistency effect has been previously studied in adults by means of the N400, a neural marker responding to semantic inconsistencies across many types of stimuli. Our results show that semantic object-scene inconsistencies indeed elicited an enhanced N400 over the left anterior brain region between 750 and 1150 ms post stimulus onset. This modulation of the N400 marker provides first indications that by the age of two toddlers have already established their scene semantics allowing them to detect a purely visual, semantic object-scene inconsistency. Our data suggest the presence of specific semantic knowledge regarding what objects occur in a certain scene category.

AUTOMATIC ONTOLOGY GENERATION OF BIM AND GIS DATA

Abstract. Data represented in the form of geospatial context and detailed building information are prominently nurturing infrastructure development and smart city applications. Bringing open-formats from data acquisition level to information engineering accelerates geospatial technologies towards urban sustainability and knowledge-based systems. BIM and GIS technologies are known to excel in this domain. However, fundamental level differences lie among their data-formats, which developed integration methods to bridge the gap between these distinct domains. Several studies have conducted data, process, and application-level integration, considering the significance of collaboration among these information systems. Although integration methods have narrowed the gap of geometric dissimilarity, semantic inconsistency, and information loss yet add constraints towards achieving interoperability. Integration using semantic web technology is more flexible and enables process-level integration without changing data format and structure. However, due to its developing nature and complex BIM-GIS data-formats, most approaches adapted requires human intervention. This paper presents a method, named OGGD (Ontology Generation for Geospatial Data), that implements a formal method for automatic ontology generation from XSD documents using transformation patterns following three extensive processes; first, formalization of XSD elements and transformation patterns; the second process identifies corresponding patterns explicitly, and the last process generates ontology for XSD schema. XSD elements from open-standard data models of BIM and GIS, ifcXML and CityGML, are manipulated and transformed into a semantically rich OWL model. The ontology models created can be applicable for information-based integration systems that will nurture knowledge-discovery and urban applications.

Semantic Loop Closure Detection With Instance-Level Inconsistency Removal in Dynamic Industrial Scenes

A novel semantic loop closure detection (SLCD) method is proposed in this article for visual simultaneous localization and mapping systems. SLCD aims to relieve the instance-level semantic inconsistency issue that arose from dynamic industrial scenes (e.g., autonomous driving in big cities). As the first step in this direction, SLCD fully exploits both low- and high-level video frame information, in a coarse-to-fine way. In SLCD, we adopt a convolutional neural network based object detection to acquire object information from the consecutive frames. Meanwhile, we perform a bag of visual words based similarity calculation to narrow the frames to coarse loop closure candidates. For these candidates, we perform an object matching on them to find their semantic inconsistency cases and remove involved semantic inconsistencies according to their cases. Then, we recalculate the similarity scores for these candidates. Finally, loop closures are determined by the similarity scores and a geometrical verification. Favorable performance of the proposed method is demonstrated by comparing it to other state-of-the-art methods using data from several public datasets and our new Dynamic Scenes dataset.

GapFinder: Finding Inconsistency of Security Information From Unstructured Text

Textual data mining of open source intelligence on the Web has become an increasingly important topic across a wide range of domains such as business, law enforcement, military, and cybersecurity. Text mining efforts utilize natural language processing to transform unstructured web content into structured forms that can drive various machine learning applications and data indexing services. For example, applications for text mining in cybersecurity have produced a range of threat intelligence services that serve the IT industry. However, a less studied problem is that of automating the identification of semantic inconsistencies among various text input sources. In this paper, we introduce GapFinder, a new inconsistency checking system for identifying semantic inconsistencies within the cybersecurity domain. Specifically, we examine the problem of identifying technical inconsistencies that arise in the functional descriptions of open source malware threat reporting information. Our evaluation, using tens of thousands of relations derived from web-based malware threat reports, demonstrates the ability of GapFinder to identify the presence of inconsistencies.

Unifying logic rules and machine learning for entity enhancing

This paper proposes a notion of entity enhancing, which unifies entity resolution and conflict resolution, to identify tuples that refer to the same real-world entity and at the same time, correct semantic inconsistencies. We propose to unify rule-based and machine learning (ML) methods for entity enhancing, by embedding ML classifiers as predicates in logic rules. We model entity enhancing by extending the chase. We show that the chase warrants correctness justification and the Church-Rosser property. Moreover, we settle fundamental problems associated with entity enhancing, including the enhancing, consistency, satisfiability, and implication problems, ranging from NP-complete and coNP-complete to Π 2 -complete. Taken together, these provide a new theoretical framework for unifying entity resolution and conflict resolution.

Zero shot learning based on class visual prototypes and semantic consistency

Zero shot classification is to recognize unseen images which are not present in training set, which is a quite difficult task. Traditional zero shot classification methods ignore the semantic inconsistencies between visual and semantic spaces which make those methods less effective. Projecting semantic representations to visual space can alleviate hubness problem. However, directly utilizing the semantic to visual mapping function learnt by seen classes to unseen classes will lead to domain shift problem. We propose a zero shot learning method which learns visual prototypes and preserves semantic consistency across visual and semantic spaces simultaneously, to handle semantic inconsistency problem and domain shift problem. The semantic consistency is represented by a shared sparse graph of visual space and semantic space. Our key insight is that the visual prototypes learning and the sparse graph learning are unified into a single process. Extensive experiments demonstrate that the results by the proposed method could be boosted significantly.

Reply to the Comment on “Realization of Lewis Basic Sodium Anion in the NaBH3− Cluster”

AbstractWe reply to the comment by S. Pan and G. Frenking who challenged our interpretation of the Na−:→BH3 dative bond in the recently synthesized NaBH3− cluster. Our conclusion remains the same as that in our original paper (https://doi.org/10.1002/anie.201907089 and https://doi.org/10.1002/ange.201907089). This conclusion is additionally supported by the energetic pathways and NBO charges calculated at UCCSD and CASMP2(4,4) levels of theory. We also discussed the suitability of the Laplacian of electron density (QTAIM) and Adaptive Natural Density Partitioning (AdNDP) method for bond type assignment. It seems that AdNDP yields more sensible results. This discussion reveals that the complex realm of bonding is full of semantic inconsistencies, and we invite experimentalists and theoreticians to elaborate this topic and find solutions incorporating different views on the dative bond.