Semantic Correlation Research Articles

Geometric Boundary Guided Feature Fusion and Spatial-semantic Context Aggregation for Semantic Segmentation of Remote Sensing Images.

Semantic segmentation of remote sensing images aims to achieve pixel-level semantic category assignment for input images. This task has achieved significant advances with the rapid development of deep neural network. Most current methods mainly focus on effectively fusing the low-level spatial details and high-level semantic cues. Other methods also propose to incorporate the boundary guidance to obtain boundary preserving segmentation. However, current methods treat the multi-level feature fusion and the boundary guidance as two separate tasks, resulting in sub-optimal solutions. Moreover, due to the large inter-class difference and small intra-class consistency within remote sensing images, current methods often fail to accurately aggregate the long-range contextual cues. These critical issues make current methods fail to achieve satisfactory segmentation predictions, which severely hinder downstream applications. To this end, we first propose a novel boundary guided multi-level feature fusion module to seamlessly incorporate the boundary guidance into the multi-level feature fusion operations. Meanwhile, in order to further enforce the boundary guidance effectively, we employ a geometric-similarity-based boundary loss function. In this way, under the explicit guidance of boundary constraint, the multi-level features are effectively combined. In addition, a channel-wise correlation guided spatial-semantic context aggregation module is presented to effectively aggregate the contextual cues. In this way, subtle but meaningful contextual cues about pixel-wise spatial context and channel-wise semantic correlation are effectively aggregated, leading to spatial-semantic context aggregation. Extensive qualitative and quantitative experimental results on ISPRS Vaihingen and GaoFen-2 datasets demonstrate the effectiveness of the proposed method.

Category-Aware Multimodal Attention Network for Fashion Compatibility Modeling

Fashion compatibility modeling, which is used to estimate the matching degree of a given set of fashion items, has received increasing attention in recent years. However, existing studies often fail to fully leverage multimodal information or ignore the semantic guidance of clothing categories in elevating the reliability of multimodal information. In this paper, we propose a fashion compatibility modeling approach with a category-aware multimodal attention network, termed as FCM-CMAN. In FCM-CMAN, we focus on enriching and aggregating multimodal representations of fashion items by means of the dynamic representations of categories and a contextual attention mechanism simultaneously. Specifically, considering that category correlations are always dynamic and varied for different fashion items, we design a categorical dynamic graph convolutional network to adaptively learn the semantic correlations between categories. When combined with the multi-layered visual outputs of a convolutional neural network and the surrounding contextual information, multiple content-aware category representations and context-aware attention weights are obtained to better characterize fashion items from different aspects. On this basis, two pieces of aware information are integrated by a multimodal factorized bilinear pooling strategy to generate visual-semantic embeddings, which are further improved by a multi-head self-attention mechanism to capture significant elements related to fashion compatibility. Extensive experiments conducted on the FashionVC and ExpFashion datasets demonstrate the superiority of FCM-CMAN over state-of-the-art methods.

Random Online Hashing for Cross-Modal Retrieval.

In the past decades, supervised cross-modal hashing methods have attracted considerable attentions due to their high searching efficiency on large-scale multimedia databases. Many of these methods leverage semantic correlations among heterogeneous modalities by constructing a similarity matrix or building a common semantic space with the collective matrix factorization method. However, the similarity matrix may sacrifice the scalability and cannot preserve more semantic information into hash codes in the existing methods. Meanwhile, the matrix factorization methods cannot embed the main modality-specific information into hash codes. To address these issues, we propose a novel supervised cross-modal hashing method called random online hashing (ROH) in this article. ROH proposes a linear bridging strategy to simplify the pair-wise similarities factorization problem into a linear optimization one. Specifically, a bridging matrix is introduced to establish a bidirectional linear relation between hash codes and labels, which preserves more semantic similarities into hash codes and significantly reduces the semantic distances between hash codes of samples with similar labels. Additionally, a novel maximum eigenvalue direction (MED) embedding method is proposed to identify the direction of maximum eigenvalue for the original features and preserve critical information into modality-specific hash codes. Eventually, to handle real-time data dynamically, an online structure is adopted to solve the problem of dealing with new arrival data chunks without considering pairwise constraints. Extensive experimental results on three benchmark datasets demonstrate that the proposed ROH outperforms several state-of-the-art cross-modal hashing methods.

Семантические и функциональные особенности фотокомментария в англоязычном новостном тексте

The study aims to determine the semantic features and functional load of the photo commentary in English-language news texts. The paper examines the role of the photo commentary in the interaction of visual and verbal components of a polycode news text. The work is novel in that it is the first to analyse the semantic relations between the photo commentary and the image, as well as between the photo commentary and the text of a news article, determine the functional features of the photo commentary in news materials. As a result of the study, it has been found that the photo commentary helps to identify and interpret the image, incorporate it into the text of an article content-wise; the types of semantic correlation of the photo commentary with the image and the text, which are based on the relations of logical subordination, intersection and equivalence, have been identified; the most common functions of the photo commentary in the texts under consideration have been described: the explanatory, coherent, informative, distinguishing, expressive, attractive functions, the function of attributing the source of photographic material. One of the important functional features of the photo commentary in news articles is its multifunctionality.

Enhancing Semantic Correlation between Instances and Relations for Zero-Shot Relation Extraction

Enhancing Semantic Correlation between Instances and Relations for Zero-Shot Relation Extraction

Intelligent mining of safety hazard information from construction documents using semantic similarity and information entropy

Project construction on-site is known to be very dangerous workplace environments due to large numbers of safety hazards. Analysis of construction safety hazards is essential to formulate rational safety management plans and prevent accidents. Construction documents contain large volumes of safety hazard information available for analysis. However, such analyses are challenging because the safety hazard information in the construction documents is presented in an unstructured or semi-structured format. This study proposes a method for intelligent mining of safety hazard information, which comprises safety hazard technical term recognition and safety hazard information analysis. The safety hazard technical term recognition model is developed based on semantic similarity and information correlation to build a safety hazard technical term library. The safety hazard information based on the technical term library is mined and analyzed using the term frequency-inverse document frequency method (TF-IDF). Finally, the proposed method is applied to build the safety hazard technical term library, which contains 2697 technical terms, and develop a hydraulic project construction safety hazard analysis system, which can realize the intelligent recognition and application of technical terms. Meanwhile, this system can automatically extract safety hazard information and provide a visualization interface to intuitively show the safety hazard analysis results, which improves the extraction efficiency of safety hazard information. The study provides a new approach for recognizing technical terms and mining safety hazard information, which can lead to enhancing management efficiency and practical knowledge discovery for safety management.

Unsupervised Word Embedding with Ensemble Deep Learning for Twitter Rumor Identification

In social networks, rumor identification is a major problem. The structural data in a topic is applied to derive useful attributes for rumor identification. Most standard rumor identification methods concentrate on local structural attributes, ignoring the global structural attributes that exist between the source tweet and its responses. To tackle this issue, a Source-Replies relation Graph (SR-graph) has been built to develop an Ensemble Graph Convolutional neural Net (EGCN) with a Nodes Proportion Allocation Mechanism (NPAM) which identifies the rumor. But, the word vectors were trained by the standard word-embedding model which does not increase the accuracy for large Twitter databases. To solve this problem, an unsupervised word-embedding method is needed for large Twitter corpora. As a result, the Twitter word-embedded EGCN (T-EGCN) model is proposed in this article, which uses unsupervised learning-based word embedding to find rumors in huge Twitter databases. Initially, the latent contextual semantic correlation and co-occurrence statistical attributes among words in tweets are extracted. Then, to create a rumor attribute vector of tweets, these word embeddings are concatenated with the GloVe model's word attribute vectors, Twitter-specific attributes, and n-gram attributes. Further, the EGCN is trained by using this attribute vector to identify rumors in a huge Twitter database. Finally, the testing results exhibit that the T-EGCN achieves 87.56% accuracy, whereas the RNN, GCN, PGNN, EGCN, and BiLSTM-CNN attain 65.38%, 68.41%, 75.04%, 81.87%, and 86.12%, respectively for rumor identification.

Multi-label weak-label learning via semantic reconstruction and label correlations

In the multi-label classification task, an instance is simultaneously associated with multiple semantic labels. Due to the high complexity of the semantic space in practical applications, obtaining instances with full labels is difficult, leading to the weak-label problem. Existing methods focus on the low-rank and instance manifold regularization assumptions of the label matrix to recover the ground-truth label matrix but ignore the influence that the above assumptions may not hold due to the semantic noise caused by missing labels. To address the problem, this paper proposes a simple and effective method to recover the label space by reconstructing the label semantic space through joint label correlation to solve the multi-label weak-label classification task. Specifically, we leverage the label information consistency and feature-label dependency assumptions to reconstruct the semantic space and consider label correlations to enhance the information of semantic views. Moreover, l2,1-norm is utilized to mitigate the effect of missing label space noise. Additionally, the linear model of the proposed method is expanded to a nonlinear version of the kernel method to address the problem of the inseparability of linear data. Extensive experiments on several real-world tasks show that the proposed method outperforms some state-of-the-art methods.

Goal model structuring based on semantic correlation of user reviews

User review is a crucial component of open mobile app market, such as the Google Play Store. These markets allow users to submit reviews for downloaded apps. By analyzing these reviews, app developers can find new or missing features. However, owing to the huge number of reviews, this manual process is time-consuming and unscalable. An automatic method for user review clustering and goal-model identification has been proposed. However, this method limits the numbers of sub-goals in the goal model. To address this, we propose a comprehensive user review clustering method. This method comprises two components. One is a latent Dirichlet allocation (LDA) model, which clusters user reviews into several topics. The other is a distance-based clustering algorithm, which is an improved version of the existing clustering method.

Bra Emotional Design Based on Emotion Model and Design Features

Emotional design can produce resonance with consumers, guide user behavior through positive emotions, and improve product awareness. This article aims to explore a method of bra emotional design that can give users a pleasant experience. First, emotional words are collected and then ordered in the coordinate system using the semantic correlation between emotions and the method of statistics for circular series. As a result, the affect model is established with two pairs of opposite emotions, relaxed–excited and disappointed–enjoying, in the vertical and horizontal directions and other four emotions, namely joyful, accepted, boredom, and disgust, in the diagonal direction. Then, 89 bra design features are analyzed through subjective evaluations. Compared with emotions stimulated by the functionality and aesthetics of bras, emotions only visually induced by bra design features range on a smaller scale and are mostly located in the upper right of the emotion model without extremely negative emotions. Finally, we design a loving bra using bra design features which are highly related to the emotional experience of love. The prospect of this study is to explore the feasibility of bra emotional design and pave the way for emotion recognition in fashion consumption.

Exploring the Benefits of Cross-Modal Coding

Multi-modal services, typically integrating such signals as audio, video, and haptic, will become an inevitable application trend of the 5G and beyond. However, due to the essential differences among the haptic and audio/video signals, the existing coding schemes usually fail to satisfy the critical requirements in terms of the rate distortion performance. Inspired by the phenomenon that hearing, sight and touch are highly correlated, we provide an affirmative answer by proposing the framework of cross-modal coding, which compresses multi-modal signals aided by their semantic correlation. In particular, the highlights of this work lie in addressing three fundamental technical problems: i) how to exploit the semantic correlation among different modalities, ii) to what extent of benefit we can get from cross-modal coding, and iii) how to design a general cross-modal codec. On the theoretical end, we determine the minimum number of bits required to compress haptic signals under the rate conditions of video streams through investigating their semantic correlation. On the technical end, we design a general cross-modal codec to approach the optimal compression limit by using the AI-enabled cross-modal prediction and channel coding. Numerical results demonstrate that the proposed cross-modal coding can achieve significant benefits relative to the existing schemes, especially when multi-modal signals have strong semantic correlation.

A Person Re-Identification Method with Multi-Scale and Multi-Feature Fusion

In order to solve the problem of insufficient expression of pedestrian features and ignoring the semantic relevance of adjacent areas in person re-identification, a multi-scale and multi-feature fusion person re-identification model is proposed. Firstly, feature maps containing different information are obtained by multi-branch structure. Secondly, the semantic correlation between local features is emphasized by combining adjacent local regions. Finally, combining the advantages of maximum pooling and average pooling, the model can learn more comprehensive feature information from different directions. Experiments are carried out on Market-1501, DukeMTMC-reID and MSMT17 respectively, it can be evaluated that the model can effectively extract pedestrian feature and has high recognition accuracy under different lighting and camera angles.

Exploiting Japanese–Chinese Cognates with Shared Private Representations for NMT

Neural machine translation has achieved remarkable progress over the past several years; however, little attention has been paid to machine translation (MT) between Japanese and Chinese, which share a large proportion of cognate words that can be utilized as additional linguistic knowledge to enhance translation performance. In this article, we seek to strengthen the semantic correlation between Japanese and Chinese by leveraging cognate words that share common Chinese characters. Specifically, we experiment with three strategies: (1) a shared vocabulary with cognate lexicon induction, which models the commonality between source and target cognates; (2) a shared private representation with a dynamic gating mechanism, which models the language-specific features on the source side; and (3) an embedding shortcut, which enables the decoder to access the shared private representation with shortest distance and aids the training process. The experiments and analysis presented in this article demonstrate that our proposed approaches can significantly improve the performance of both Japanese-to-Chinese and Chinese-to-Japanese translations and verify the effectiveness of exploiting Japanese–Chinese cognates for MT.

Attention-enabled gated spiking neural P model for aspect-level sentiment classification

Gated spiking neural P (GSNP) model is a recently developed recurrent-like network, which is abstracted by nonlinear spiking mechanism of nonlinear spiking neural P systems. In this study, a modification of GSNP is combined with attention mechanism to develop a novel model for sentiment classification, called attention-enabled GSNP model or termed as AGSNP model. The AGSNP model has two channels that process content words and aspect item respectively, where two modified GSNPs are used to obtain dependencies between content words and between aspect words. Moreover, two attention components are used to establish semantic correlation between content words and aspect item. Comparative experiments on three real data sets and several baseline models are conducted to verify the effectiveness of the AGSNP model. The comparison results demonstrate that the AGSNP model is competent for aspect-level sentiment classification tasks.

Unsupervised Cross‐Media Hashing Learning via Knowledge Graph

With the rapid growth of multimedia data, cross-media hashing has become an important technology for fast cross-media retrieval. Because the manual annotations are difficult to obtain in real-world application, unsupervised cross-media hashing is studied to address the hashing learning without manual annotations. Existing unsupervised cross-media hashing methods generally focus on calculating the similarities through the features of multimedia data, while the learned hashing code cannot reflect the semantic relationship among the multimedia data, which hinders the accuracy in the cross-media retrieval. When humans try to understand multimedia data, the knowledge of concept relations in our brain plays an important role in obtaining high-level semantic. Inspired by this, we propose a knowledge guided unsupervised cross-media hashing (KGUCH) approach, which applies the knowledge graph to construct high-level semantic correlations for unsupervised cross-media hash learning. Our contributions in this paper can be summarized as follows: 1) The knowledge graph is introduced as auxiliary knowledge to construct the semantic graph for the concepts in each image and text instance, which can bridge the multimedia data with high-level semantic correlations to improve the accuracy of learned hash codes for cross-media retrieval. 2) The proposed KGUCH approach constructs correlation of the multimedia data from both the semantic and the feature aspects, which can exploit complementary information to promote the unsupervised cross-media hash learning. The experiments are conducted on three widely-used datasets, which verify the effectiveness of our proposed KGUCH approach.

TEXT MINING DAN KLASIFIKASI MULTI LABEL MENGGUNAKAN XGBOOST

The conventional classification process is applied to find a single criterion or label. The multi-label classification process is more complex because a large number of labels results in more classes. Another aspect that must be considered in multi-label classification is the existence of mutual dependencies between data labels. In traditional binary classification, classification analysis only aims to determine the label in the text, whether positive or negative. This method is sub-optimal because the relationship between labels cannot be determined. To overcome the weaknesses of these traditional methods, multi-label classification is one of the solutions in data labeling. With multi-label text classification, it allows the existence of many labels in a document and there is a semantic correlation between these labels. This research performs multi-label classification on research article texts using the ensemble classifier approach, namely XGBoost. Classification performance evaluation is based on several metrics criteria of confusion matrix, accuracy, and f1 score. Model evaluation is also carried out by comparing the performance of XGBoost with Logistic Regression. The results of the study using the train test split and cross-validation obtained an average accuracy of training and testing for Regression Logistics of 0.81, and an average f1 score of 0.47. The average accuracy for XGBoost is 0.88, and the average f1 score is 0.78. The results show that the XGBoost classifier model can be applied to produce a good classification performance.

TFUN: Trilinear Fusion Network for Ternary Image-Text Retrieval

Recently, a particular type of image-text retrieval task named Ternary Image-Text Retrieval (TITR) has drawn increasing attention. In this task, the total inputs of query and target consist of three components, rather than two inputs in the widely-studied retrieval case. The typical TITR scenarios include recipe retrieval (e.g., ingredients text, instructions text and food images) and fashion search (e.g., original images, text and modified images). A few recently proposed TITR methods mainly focus on learning the semantic correlations of two modality data by projecting them to the same embedding space to capture the alignment between image and text modalities. Nevertheless, two limitations still exist in these methods: 1) the underlying difference between data in the same modality (e.g., ingredients and instructions) is neglected; and 2) the trilinear interaction among the three inputs is implicitly captured. To this end, we propose a novel fusion framework named Trilinear FUsion Network (TFUN) to utilize high-level associations between these three inputs simultaneously and learn an accurate cross-modal similarity function via bi-directional triplet loss explicitly, which is generic for the TITR task. To reduce the model complexity, we introduce the advanced method of tensor decomposition to ensure computational efficiency and accessibility. We also develop a three-stage hard triplet sampling scheme to ensure fast convergence. Extensive experiments on three large-scale TITR datasets Recipe1M, Fashion200k and FashionIQ demonstrate the superiority of our proposed TFUN model compared to the state-of-the-art approaches. The implementation code and additional instructions are provided at https://github.com/CFM-MSG/Code_TFUN.

What Happened in 2020: a Topic Modeling Approach based on a Topic Similarity Metric

2020 was atypical mainly due to the Covid-19 pandemic's beginning which has become a vastly discussed subject worldwide. Unsurprisingly, online news websites have followed this trend, besides publishing traditional subjects (e.g., sports, business, and politics). Understanding how the subjects interact with each other over the year is a challenge. In this paper, we intend to build a 2020 timeline based on the subjects and their similarity using a topic modeling approach (LDA) and a novel topic similarity metric. To accomplish that, we scrap news articles websites to build a collection of 2020 news. After that, the collection is pre-processed and sliced monthly. We use an LDA approach to discover the latent topics from all temporal collections. Next, we calculate the similarity between the topics across 2020 using five semantic correlations: born, death, keep, merge, and split. The discovered topics and the drift semantic between them show that building a meaningful 2020 time line is possible.

An Information Retrieval-Based Joint System for Complex Chinese Knowledge Graph Question Answering

Knowledge graph-based question answering is an intelligent approach to deducing the answer to a natural language question from structured knowledge graph information. As one of the mainstream knowledge graph-based question answering approaches, information retrieval-based methods infer the correct answer by constructing and ranking candidate paths, which achieve excellent performance in simple questions but struggle to handle complex questions due to rich entity information and diverse relations. In this paper, we construct a joint system with three subsystems based on the information retrieval methods, where candidate paths can be efficiently generated and ranked, and a new text-matching method is introduced to capture the semantic correlation between questions and candidate paths. Results of the experiment conducted on the China Conference on Knowledge Graph and Semantic Computing 2019 Chinese Knowledge Base Question Answering dataset verify the superiority and efficiency of our approach.

A Novel Hybrid Attention-Driven Multistream Hierarchical Graph Embedding Network for Remote Sensing Object Detection

Multiclass geospatial object detection in high-spatial-resolution remote-sensing images (HSRIs) has recently attracted considerable attention in many remote-sensing applications as a fundamental task. However, the complexity and uncertainty of spatial distribution among multiclass geospatial objects are still huge challenges for object detection in HSRIs. Most current remote-sensing object-detection approaches fall back on deep convolutional neural networks (CNNs). Nevertheless, most existing methods only focus on mining visual characteristics and lose sight of spatial or semantic relation discriminations, eventually degrading object-detection performance in HSRIs. To tackle these challenges, we propose a novel hybrid attention-driven multistream hierarchical graph embedding network (HA-MHGEN) to explore complementary spatial and semantic patterns for improving remote-sensing object-detection performance. Specifically, we first constructed hierarchical spatial graphs for multiscale spatial relation representation. Then, semantic graphs were also constructed by integrating them with the word embedding of object category labels on graph nodes. Afterwards, we developed a self-attention-aware multiscale graph convolutional network (GCN) to derive stronger for intra- and interobject hierarchical spatial relations and contextual semantic relations, respectively. These two relation networks were followed by a novel cross-attention-driven spatial- and semantic-feature fusion module that utilizes a multihead attention mechanism to learn associations between diverse spatial and semantic correlations, and guide them to endowing a more powerful discrimination ability. With the collaborative learning of the three relation networks, the proposed HA-MHGEN enables grasping explicit and implicit relations from spatial and semantic patterns, and boosts multiclass object-detection performance in HRSIs. Comprehensive and extensive experimental evaluation results on three benchmarks, namely, DOTA, DIOR, and NWPU VHR-10, demonstrate the effectiveness and superiority of our proposed method compared with that of other advanced remote-sensing object-detection methods.