Rich Semantic Information Research Articles

Charged With a Crime: The Neuronal Signature of Processing Negatively Evaluated Faces Under Different Attentional Conditions

Our brains rapidly respond to human faces and can differentiate between many identities, retrieving rich semantic emotional-knowledge information. Studies provide a mixed picture of how such information affects event-related potentials (ERPs). We systematically examined the effect of feature-based attention on ERP modulations to briefly presented faces of individuals associated with a crime. The tasks required participants (N = 40 adults) to discriminate the orientation of lines overlaid onto the face, the age of the face, or emotional information associated with the face. Negative faces amplified the N170 ERP component during all tasks, whereas the early posterior negativity (EPN) and late positive potential (LPP) components were increased only when the emotional information was attended to. These findings suggest that during early configural analyses (N170), evaluative information potentiates face processing regardless of feature-based attention. During intermediate, only partially resource-dependent, processing stages (EPN) and late stages of elaborate stimulus processing (LPP), attention to the acquired emotional information is necessary for amplified processing of negatively evaluated faces.

Semantic Relation Model and Dataset for Remote Sensing Scene Understanding

A deep understanding of our visual world is more than an isolated perception on a series of objects, and the relationships between them also contain rich semantic information. Especially for those satellite remote sensing images, the span is so large that the various objects are always of different sizes and complex spatial compositions. Therefore, the recognition of semantic relations is conducive to strengthen the understanding of remote sensing scenes. In this paper, we propose a novel multi-scale semantic fusion network (MSFN). In this framework, dilated convolution is introduced into a graph convolutional network (GCN) based on an attentional mechanism to fuse and refine multi-scale semantic context, which is crucial to strengthen the cognitive ability of our model Besides, based on the mapping between visual features and semantic embeddings, we design a sparse relationship extraction module to remove meaningless connections among entities and improve the efficiency of scene graph generation. Meanwhile, to further promote the research of scene understanding in remote sensing field, this paper also proposes a remote sensing scene graph dataset (RSSGD). We carry out extensive experiments and the results show that our model significantly outperforms previous methods on scene graph generation. In addition, RSSGD effectively bridges the huge semantic gap between low-level perception and high-level cognition of remote sensing images.

Robust Vehicle Speed Measurement Based on Feature Information Fusion for Vehicle Multi-Characteristic Detection.

A robust vehicle speed measurement system based on feature information fusion for vehicle multi-characteristic detection is proposed in this paper. A vehicle multi-characteristic dataset is constructed. With this dataset, seven CNN-based modern object detection algorithms are trained for vehicle multi-characteristic detection. The FPN-based YOLOv4 is selected as the best vehicle multi-characteristic detection algorithm, which applies feature information fusion of different scales with both rich high-level semantic information and detailed low-level location information. The YOLOv4 algorithm is improved by combing with the attention mechanism, in which the residual module in YOLOv4 is replaced by the ECA channel attention module with cross channel interaction. An improved ECA-YOLOv4 object detection algorithm based on both feature information fusion and cross channel interaction is proposed, which improves the performance of YOLOv4 for vehicle multi-characteristic detection and reduces the model parameter size and FLOPs as well. A multi-characteristic fused speed measurement system based on license plate, logo, and light is designed accordingly. The system performance is verified by experiments. The experimental results show that the speed measurement error rate of the proposed system meets the requirement of the China national standard GB/T 21555-2007 in which the speed measurement error rate should be less than 6%. The proposed system can efficiently enhance the vehicle speed measurement accuracy and effectively improve the vehicle speed measurement robustness.

PaI‐Net: A modified U‐Net of reducing semantic gap for surgical instrument segmentation

Tracking the instruments in a surgical scene is an essential task in minimally invasive surgery. However, due to the unpredictability of scenes, automatically segmenting the instruments is very challenging. In this paper, a novel method named parallel inception network (PaI-Net) is proposed, in which an attention parallel module (APM) and an output fusion module (OFM) are integrated with U-Net to improve the segmentation ability. Specially, APM utilizes multi-scale convolution kernels and global average pooling operations to extract semantic information and global context information of different scales, while OFM combines the feature maps of the decoder part to aggregate the abundant boundary information of shallow layers and the rich semantic information of deep layers together, which achieve a significant improvement in generating segmentation masks. Finally, the evaluation of proposed method on robotic instruments segmentation task from Medical Image Computing and Computer Assisted Intervention Society (MICCAI) and retinal image segmentation task from International Symposium on Biomedical Imaging (ISBI) show that our model has achieved advanced performance on multi-scale semantic segmentation and is superior to the current state-of-the-art models.

Hybrid Chinese text classification model based on pretraining model

In order to solve the problems of fuzzy boundary of Chinese words, polysemy of one word and the inability of traditional models to reflect the importance of each word in the text, a hybrid text classification model based on Ernie CNN and bilstm attention (mecba) is proposed. Firstly, Ernie model is used to generate the corresponding word vector, which can retain rich semantic information and enhance the semantic representation of words. Then, the word vector is input into CNN to extract local information, and the context feature is extracted by bilstm attention, and the output of the two is spliced; Finally, softmax classifier is used for classification. The experimental results show that the model is better than CNN, bilstm and other classification models, and can effectively improve the performance of Chinese text classification.

Monitoring the summer flooding in the Poyang Lake area of China in 2020 based on Sentinel-1 data and multiple convolutional neural networks

Precise monitoring of floods is significant in disaster management and loss reduction; however, remote sensing data resource and methods can largely affect the monitoring accuracy of flooded areas. In this study, we use cloud-free Sentinel-1 Synthetic Aperture Radar (SAR) imagery, preferable to the optical imagery. We have used 5 convolutional neural networks (CNNs), including HRNet, DenseNet, SegNet, ResNet and DeepLab v3 + for flood monitoring in the Poyang Lake area, and compared their performances with the traditional methods — the bimodal threshold segmentation (BTS) and the OSTU method. The HRNet has superior performance in water body identification with the highest precision and efficiency, based on a parallel structure to not only extract rich semantic information but also maintain high-resolution features in the whole process. Besides, speckle noise reduction by deep convolutional neural networks in SAR imagery is better compared with the Refined Lee filter. The CNNs are then used to monitor the temporal evolution of summer flooding (May-Nov.) in 2020. Results show the smallest water coverage of Poyang Lake in late May; it gradually increases to the maximum in mid-July, and then shows a downward trend until November.

OWL2Vec*: embedding of OWL ontologies

Semantic embedding of knowledge graphs has been widely studied and used for prediction and statistical analysis tasks across various domains such as Natural Language Processing and the Semantic Web. However, less attention has been paid to developing robust methods for embedding OWL (Web Ontology Language) ontologies, which contain richer semantic information than plain knowledge graphs, and have been widely adopted in domains such as bioinformatics. In this paper, we propose a random walk and word embedding based ontology embedding method named OWL2Vec*, which encodes the semantics of an OWL ontology by taking into account its graph structure, lexical information and logical constructors. Our empirical evaluation with three real world datasets suggests that OWL2Vec* benefits from these three different aspects of an ontology in class membership prediction and class subsumption prediction tasks. Furthermore, OWL2Vec* often significantly outperforms the state-of-the-art methods in our experiments.

Clustering Heterogeneous Information Network by Joint Graph Embedding and Nonnegative Matrix Factorization

Many complex systems derived from nature and society consist of multiple types of entities and heterogeneous interactions, which can be effectively modeled as heterogeneous information network (HIN). Structural analysis of heterogeneous networks is of great significance by leveraging the rich semantic information of objects and links in the heterogeneous networks. And, clustering heterogeneous networks aims to group vertices into classes, which sheds light on revealing the structure–function relations of the underlying systems. The current algorithms independently perform the feature extraction and clustering, which are criticized for not fully characterizing the structure of clusters. In this study, we propose a learning model by joint <underline>G</underline>raph <underline>E</underline>mbedding and <underline>N</underline>onnegative <underline>M</underline>atrix <underline>F</underline>actorization (aka GEjNMF ), where feature extraction and clustering are simultaneously learned by exploiting the graph embedding and latent structure of networks. We formulate the objective function of GEjNMF and transform the heterogeneous network clustering problem into a constrained optimization problem, which is effectively solved by l 0 -norm optimization. The advantage of GEjNMF is that features are selected under the guidance of clustering, which improves the performance and saves the running time of algorithms at the same time. The experimental results on three benchmark heterogeneous networks demonstrate that GEjNMF achieves the best performance with the least running time compared with the best state-of-the-art methods. Furthermore, the proposed algorithm is robust across heterogeneous networks from various fields. The proposed model and method provide an effective alternative for heterogeneous network clustering.

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both deep and shallow features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) at the inference speeds of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

Deep Graph Matching and Searching for Semantic Code Retrieval

Code retrieval is to find the code snippet from a large corpus of source code repositories that highly matches the query of natural language description. Recent work mainly uses natural language processing techniques to process both query texts (i.e., human natural language) and code snippets (i.e., machine programming language), however, neglecting the deep structured features of query texts and source codes, both of which contain rich semantic information. In this article, we propose an end-to-end deep graph matching and searching (DGMS) model based on graph neural networks for the task of semantic code retrieval. To this end, we first represent both natural language query texts and programming language code snippets with the unified graph-structured data, and then use the proposed graph matching and searching model to retrieve the best matching code snippet. In particular, DGMS not only captures more structural information for individual query texts or code snippets, but also learns the fine-grained similarity between them by cross-attention based semantic matching operations. We evaluate the proposed DGMS model on two public code retrieval datasets with two representative programming languages (i.e., Java and Python). Experiment results demonstrate that DGMS significantly outperforms state-of-the-art baseline models by a large margin on both datasets. Moreover, our extensive ablation studies systematically investigate and illustrate the impact of each part of DGMS.

Gesture Detection and Recognition Based on Pyramid Frequency Feature Fusion Module and Multiscale Attention in Human-Computer Interaction

Aiming at the problem of the absence of detail texture and other high-frequency features in the feature extraction process of the deep network employing the upsampling operation, the accuracy of gesture recognition is seriously affected in complex scenes. This study integrates object detection and gesture recognition into one model and proposes a gesture detection and recognition based on the pyramid frequency feature fusion module and multiscale attention in human-computer interaction. Pyramid fusion module is used to perform efficient feature fusion and is proposed to obtain feature layers with rich details and semantic information, which is helpful to improve the efficiency and accuracy of gesture recognition. In addition, the multiscale attention module is further adopted to adaptively mine important and effective feature information from both temporal and spatial channels and embedded into the detection layer. Finally, our proposed network realizes the enhancement of the effective information and the suppression of the invalid information of the detection layer. Experimental results show that our proposed model makes full use of the high-low frequency feature fusion module without replacing the basic backbone network, which can greatly reduce the computational overhead while improving the detection accuracy.

Automatic detection of Acacia longifolia invasive species based on UAV-acquired aerial imagery

The Acacia longifolia species is known for its rapid growth and dissemination, causing loss of biodiversity in the affected areas. In order to avoid the uncontrolled spread of this species, it is important to effectively monitor its distribution on the agroforestry regions. For this purpose, this paper proposes the use of Convolutional Neural Networks (CNN) for the detection of Acacia longifolia, from images acquired by an unmanned aerial vehicle. Two models based on the same CNN architecture were elaborated. One classifies image patches into one of nine possible classes, which are later converted into a binary model; this model presented an accuracy of 98.6% and 98.5% in the validation and training sets, respectively. The second model was trained directly for binary classification and showed an accuracy of 98.8% and 98.7% for the validation and test sets, respectively. The results show that the use of multiple classes, useful to provide the aerial vehicle with richer semantic information regarding the environment, does not hamper the accuracy of Acacia longifolia detection in the classifier’s primary task. The presented system also includes a method for increasing classification’s accuracy by consulting an expert to review the model’s predictions on an automatically selected sub-set of the samples.

Self-attention feature fusion network for semantic segmentation

Capturing rich contextual information is very helpful for semantic segmentation. In addition, the effective use of low-level details and high-level semantics is crucial for semantic segmentation. In this paper, we start from these two aspects, and we propose a self-attention feature fusion network for semantic segmentation (SA-FFNet) to improve semantic segmentation performance. Specifically, we introduced the vertical and horizontal compression attention module (VH-CAM) and the unequal channel pyramid pooling module (UC-PPM). The previous position attention module, such as the position attention module (PAM) in DANet, calculates the similarity between each pixel and all other pixels. However, the amount of information contained in a single isolated pixel is too small, so the resulting position attention weight is not perfect. Our approach is to compress a feature map from both vertical and horizontal directions so that the information contained in each pixel will be richer, and the spatial feature map obtained from this will be better. Experiments show that the position attention weight generated by our vertical and horizontal compression attention module is better than PAM in DANet. Additionally, our UC-PPM on each decoder can provide high-level rich semantic information to guide the selection of low-level feature maps. The proposed model achieves a 76.42% mean IoU on PASCAL VOC2012 and a 73.13% mean IoU on Cityscapes.

Image resizing by reconstruction from deep features

Traditional image resizing methods usually work in pixel space and use various saliency measures. The challenge is to adjust the image shape while trying to preserve important content. In this paper we perform image resizing in feature space using the deep layers of a neural network containing rich important semantic information. We directly adjust the image feature maps, extracted from a pre-trained classification network, and reconstruct the resized image using neural-network based optimization. This novel approach leverages the hierarchical encoding of the network, and in particular, the high-level discriminative power of its deeper layers, that can recognize semantic regions and objects, thereby allowing maintenance of their aspect ratios. Our use of reconstruction from deep features results in less noticeable artifacts than use of imagespace resizing operators. We evaluate our method on benchmarks, compare it to alternative approaches, and demonstrate its strengths on challenging images.

PGRA: Projected graph relation-feature attention network for heterogeneous information network embedding

• We propose PGRA for embedding heterogeneous information networks (HIN). • PGRA is a graph neural network with three mechanisms for handling the heterogeneity. • PGRA seamlessly fuses ideas of projection, attention, and neighborhood to capture HIN. • PGRA is flexible containing several function modules that can be freely specified. • PGRA outperforms the state-of-the-art methods by a large margin. Graph neural networks (GNNs) have achieved superior performance and gained significant interest in various domains. However, most of the existing GNNs are considered for homogeneous graphs, whereas real-world systems are usually modeled as heterogeneous graphs or heterogeneous information networks (HINs). Designing a GNN to fully capture the rich semantic information of HINs is significantly challenging owing to the heterogeneity and incompatibility of relations in HINs. To address these issues while utilizing the power of GNNs, we propose a novel unsupervised embedding approach, named Projected Graph Relation-Feature Attention Network (PGRA). PGRA is based on three mechanisms: 1) specific-relation projection that projects the representation vector of each node to a relation-specific space, 2) aggregation with a relation-feature attention network that learns salient neighbors in the aggregation by considering the features of the nodes and compatibility between the connected and target relations, 3) an elegantly designed loss function that captures both the first- and second-order proximities between nodes. The results of extensive experiments on seven real-world datasets illustrate that PGRA outperforms the state-of-the-art methods by a large margin.

Computer vision detection of foreign objects in coal processing using attention CNN

Foreign objects in coal seriously affect the efficiency and safety of clean coal production. Currently, the removal of foreign objects in coal preparation plant mainly depends on manual picking, which has disadvantages of high labor intensity and low efficiency. Therefore, there is an urgent need for rapid detection and removal of foreign objects. However, due to the inference of the background and surround objects, it is a challenge for the accurate detection of foreign objects. In this study, a convolutional neural network (CNN) with attention modules was designed to accurately segment foreign objects from a complex background in real-time. The proposed network consists of an encoder and a decoder, and the attention mechanism was introduced into the decoder to capture rich semantic information. The visualization results proved that the attention modules could focus on the features of the salient region and inhibit the irrelevant background, which significantly improved the accuracy of the detection The results showed that the proposed model correctly recognized 97% of the foreign objects in the 1871 sets of test images. The mean intersection over union (MIOU) of the optimal model was 91.24%, and the inference speed was greater than 15 fps/s, which satisfied the real-time requirement.

Multi-Scale Safety Helmet Detection Based on SAS-YOLOv3-Tiny

In the practical application scenarios of safety helmet detection, the lightweight algorithm You Only Look Once (YOLO) v3-tiny is easy to be deployed in embedded devices because its number of parameters is small. However, its detection accuracy is relatively low, which is why it is not suitable for detecting multi-scale safety helmets. The safety helmet detection algorithm (named SAS-YOLOv3-tiny) is proposed in this paper to balance detection accuracy and model complexity. A light Sandglass-Residual (SR) module based on depthwise separable convolution and channel attention mechanism is constructed to replace the original convolution layer, and the convolution layer of stride two is used to replace the max-pooling layer for obtaining more informative features and promoting detection performance while reducing the number of parameters and computation. Instead of two-scale feature prediction, three-scale feature prediction is used here to improve the detection effect about small objects further. In addition, an improved spatial pyramid pooling (SPP) module is added to the feature extraction network to extract local and global features with rich semantic information. Complete-Intersection over Union (CIoU) loss is also introduced in this paper to improve the loss function for promoting positioning accuracy. The results on the self-built helmet dataset show that the improved algorithm is superior to the original algorithm. Compared with the original YOLOv3-tiny, the SAS-YOLOv3-tiny has significantly improved all metrics (including Precision (P), Recall (R), Mean Average Precision (mAP), F1) at the expense of only a minor speed while keeping fewer parameters and amounts of calculation. Meanwhile, the SAS-YOLOv3-tiny algorithm shows advantages in accuracy compared with lightweight object detection algorithms, and its speed is faster than the heavyweight model.

Efficient estimation of Hindi WSD with distributed word representation in vector space

Word Sense Disambiguation (WSD) is significant for improving the accuracy of the interpretation of a Natural language text. Various supervised learning-based models and knowledge-based models have been developed in the literature for WSD of the language text. However, these models do not provide good results for low-resource languages, due to the lack of labelled and tagged data. Therefore, in this work, we have examined different word embedding techniques for word sense disambiguation of the Hindi language texts. Several studies in the literature show that these embeddings have been utilized for different foreign languages in the field of word sense disambiguation. However, to the best of our knowledge, no such work exists for the Hindi language. Therefore, in this paper, we utilize various existing word embeddings for WSD of Hindi text. Moreover, we have created Hindi word embeddings on articles taken from Wikipedia and test the quality of the created word embeddings using Pearson correlation. In this direction, we perform different experiments and observe that Word2Vec model gives best performance among all the considered embeddings on the used Hindi dataset. In our method, the proposed model directly takes input that is trained with word embedding methods and helps to develop a sense inventory using clustering that has been employed for performing disambiguation. Experimental observations indicate that the performance of the proposed approach is moderate and competent in terms of accuracy. The paper, thus, presents how WSD can leverage these representations to encode rich semantic information.

Dual attention enhancement feature fusion network for segmentation and quantitative analysis of paediatric echocardiography.

Paediatric echocardiography is a standard method for screening congenital heart disease (CHD). The segmentation of paediatric echocardiography is essential for subsequent extraction of clinical parameters and interventional planning. However, it remains a challenging task due to (1) the considerable variation of key anatomic structures, (2) the poor lateral resolution affecting accurate boundary definition, (3) the existence of speckle noise and artefacts in echocardiographic images. In this paper, we propose a novel deep network to address these challenges comprehensively. We first present a dual-path feature extraction module (DP-FEM) to extract rich features via a channel attention mechanism. A high- and low-level feature fusion module (HL-FFM) is devised based on spatial attention, which selectively fuses rich semantic information from high-level features with spatial cues from low-level features. In addition, a hybrid loss is designed to deal with pixel-level misalignment and boundary ambiguities. Based on the segmentation results, we derive key clinical parameters for diagnosis and treatment planning. We extensively evaluate the proposed method on 4,485 two-dimensional (2D) paediatric echocardiograms from 127 echocardiographic videos. The proposed method consistently achieves better segmentation performance than other state-of-the-art methods, whichdemonstratesfeasibility for automatic segmentation and quantitative analysis of paediatric echocardiography. Our code is publicly available at https://github.com/end-of-the-century/Cardiac.

A survey on heterogeneous network representation learning

Heterogeneous information networks usually contain different kinds of nodes and distinguishing types of relations, which can preserve more information than homogeneous information networks. Heterogeneous network representation learning attempts to learn a low-dimensional representation for each node and capture rich semantic information of the given network. Most of existing surveys focus on heterogeneous information network analysis and homogeneous information network representation learning. Although considerable research efforts concentrate on heterogeneous network representation learning, there are few surveys that systematically review the state-of-the-art heterogeneous network representation learning techniques. Motivated by this, we propose a taxonomy of heterogeneous network representation learning algorithms according to different approaches of capturing semantic information in heterogeneous networks, including path based algorithms and semantic unit based algorithms. Moreover, we introduce the typical heterogeneous network representation learning techniques in detail and make a comparative analysis of these techniques. In addition, the research challenges in terms of semantics preserving, data sparsity and scalability are discussed. To tackle these challenges, several potential future research directions for heterogeneous network representation learning are pointed out, including semantic relations extraction, dynamic heterogeneous networks, very large heterogeneous networks and heterogeneous networks construction.