Semantic Dependency Research Articles

Ontology of knowledge representation about the prescribed personalized treatment

The paper describes a generalized ontology that makes it possible to form knowledge about various types of treatment (medicamental, restorative, surgical), regardless of the branch of medicine. Ontology is a necessary component for creating an information system focused on solving a class of treatment planning problems. Planning allows you to combine treatments for various diseases and branches of medicine. A method for adapting a generalized ontology to various types of treatment and terminology, which is familiar to the developer of the knowledge base, an expert in the subject area, is described. All specialized ontologies for specific types of treatment are inherited from the generalized ontology, preserving its structure and semantic dependencies. This approach ensures the creation of knowledge bases in decision support systems for the complex prescription of patient treatment, opening up the possibility of creating a single solver, which provides a significant reduction in system development costs. The generalized ontology was developed on the IACPaaS cloud platform and is currently used to create knowledge bases in various fields of medicine.

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

Self-attention-based networks have obtained impressive performance in parallel training and global context modeling. However, it is weak in local dependency capturing, especially for data with strong local correlations such as utterances. Therefore, we will mine linguistic information of the original text based on a semantic dependency and the semantic relationship between nodes is regarded as prior knowledge to revise the distribution of self-attention. On the other hand, given the strong correlation between input characters, we introduce a one-dimensional (1-D) convolution neural network (CNN) producing query(Q) and value(V) in the self-attention mechanism for a better fusion of local contextual information. Then, we migrate this variant of the self-attention networks to speech synthesis tasks and propose a non-autoregressive (NAR) neural Text-to-Speech (TTS): SeDepTTS. Experimental results show that our model yields good performance in speech synthesis. Specifically, the proposed method yields significant improvement for the processing of pause, stress, and intonation in speech.

Sentiment Analysis of Social Network Comment Text Based on LSTM and Bert

This paper has the aim of solving problems in research studies on the analysis tasks of text emotion; the problems are the low utilization of text, the difficulty of effective information extraction, the failure of recognizing word polysemy with effectiveness. Thus, based on LSTM and Bert, the method of sentiment analysis on text is adopted. To be precise, word embedding of dataset in view of the skip-gram model is used for training course. In each sample, the word embeddings combine matric with the two-dimensional feature to be neural network input. Next, construction of analysis model for text sentiment combines Bert pre-training language model and long short-term memory (LSTM) network, using the word vector pre-trained by Bert instead of that trained in the traditional way to dynamically generate the semantic vector according to the word context. Finally, the semantic representation of words from text is improved by effectively identifying the polysemy of words, and the semantic vector is input into the LSTM to capture the semantic dependencies, thereby enhancing the ability to extract valid information. The Accuracy, Precision, Recall and F-Measure for the method of Bert–LSTM based on analysis of text sentiment are 0.89, 0.9, 0.84 and 0.87, indicating high value than the compared ones. Thus, the proposed method significantly outperforms the comparison methods in text sentiment analysis.

Multilevel Federated Learning-Based Intelligent Traffic Flow Forecasting for Transportation Network Management

Accurate traffic flow forecasting is crucial to improving traffic safety and alleviating road congestion for intelligent transportation network management. Recently, spatial-temporal graph-based deep learning methods have achieving significant performance improvements in traffic flow forecasting. However, they only consider spatial-temporal correlation of traffic network but ignore a mass of semantic correlation. In addition, they need to centralize data for training models, leading to privacy leakage concern. To tackle these problems, we introduce a federated learning-based intelligent traffic flow forecasting model that integrates our proposed spatial-temporal graph-based deep learning model into the devised Multilevel Federated Learning framework(MFL), named MFVSTGNN. This MFL is used to allow data collaboration among different data owners to train an efficient model without sharing their private data, while achieving the trade-off between communication overhead and computation performance. The proposed spatial-temporal graph-based deep learning model is composed of two phases. The first phase utilizes Variational Graph Autoencoder (VGAE) to dynamically generate adjacency matrix that contains both the spatial and semantic dependencies, contributing to preserving valuable information for improving prediction accuracy, and the second phase employs general spatial-temporal graph neural network to conduct prediction. We evaluate the performance of MFVSTGNN with two large-scale traffic datasets from California and Los Angeles County. The experimental results demonstrate the superior performance of MFVSTGNN in reducing communication overhead, and improving prediction accuracy, validating the effectiveness of our proposed model.

Denoising Graph Inference Network for Document-Level Relation Extraction

Relation Extraction (RE) is to obtain a predefined relation type of two entities mentioned in a piece of text, e.g., a sentence-level or a document-level text. Most existing studies suffer from the noise in the text, and necessary pruning is of great importance. The conventional sentence-level RE task addresses this issue by a denoising method using the shortest dependency path to build a long-range semantic dependency between entity pairs. However, this kind of denoising method is scarce in document-level RE. In this work, we explicitly model a denoised document-level graph based on linguistic knowledge to capture various long-range semantic dependencies among entities. We first formalize a Syntactic Dependency Tree forest (SDT-forest) by introducing the syntax and discourse dependency relation. Then, the Steiner tree algorithm extracts a mention-level denoised graph, Steiner Graph (SG), removing linguistically irrelevant words from the SDT-forest. We then devise a slide residual attention to highlight word-level evidence on text and SG. Finally, the classification is established on the SG to infer the relations of entity pairs. We conduct extensive experiments on three public datasets. The results evidence that our method is beneficial to establish long-range semantic dependency and can improve the classification performance with longer texts.

Cross-image Dependency Modelling for Breast Ultrasound Segmentation.

We present a novel deep network (namely BUSSeg) equipped with both within- and cross-image long-range dependency modeling for automated lesions segmentation from breast ultrasound images, which is a quite daunting task due to (1) the large variation of breast lesions, (2) the ambiguous lesion boundaries, and (3) the existence of speckle noise and artifacts in ultrasound images. Our work is motivated by the fact that most existing methods only focus on modeling the within-image dependencies while neglecting the cross-image dependencies, which are essential for this task under limited training data and noise. We first propose a novel cross-image dependency module (CDM) with a cross-image contextual modeling scheme and a cross-image dependency loss (CDL) to capture more consistent feature expression and alleviate noise interference. Compared with existing cross-image methods, the proposed CDM has two merits. First, we utilize more complete spatial features instead of commonly used discrete pixel vectors to capture the semantic dependencies between images, mitigating the negative effects of speckle noise and making the acquired features more representative. Second, the proposed CDM includes both intra- and inter-class contextual modeling rather than just extracting homogeneous contextual dependencies. Furthermore, we develop a parallel bi-encoder architecture (PBA) to tame a Transformer and a convolutional neural network to enhance BUSSeg's capability in capturing within-image long-range dependencies and hence offer richer features for CDM. We conducted extensive experiments on two representative public breast ultrasound datasets, and the results demonstrate that the proposed BUSSeg consistently outperforms state-of-the-art approaches in most metrics.

THE TERM COMBINATION AND THE METAPHOR IN THE OFFICIAL BUSINESS DOCUMENT: COGNITIVE ASPECT

THE TERM COMBINATION AND THE METAPHOR IN THE OFFICIAL BUSINESS DOCUMENT: COGNITIVE ASPECT

A Joint-Learning-Based Dynamic Graph Learning Framework for Structured Prediction

Graph neural networks (GNNs) have achieved remarkable success in structured prediction, owing to the GNNs’ powerful ability in learning expressive graph representations. However, most of these works learn graph representations based on a static graph constructed by an existing parser, suffering from two drawbacks: (1) the static graph might be error-prone, and the errors introduced in the static graph cannot be corrected and might accumulate in later stages, and (2) the graph construction stage and graph representation learning stage are disjoined, which negatively affects the model’s running speed. In this paper, we propose a joint-learning-based dynamic graph learning framework and apply it to two typical structured prediction tasks: syntactic dependency parsing, which aims to predict a labeled tree, and semantic dependency parsing, which aims to predict a labeled graph, for jointly learning the graph structure and graph representations. Experiments are conducted on four datasets: the Universal Dependencies 2.2, the Chinese Treebank 5.1, the English Penn Treebank 3.0 in 13 languages for syntactic dependency parsing, and the SemEval-2015 Task 18 dataset in three languages for semantic dependency parsing. The experimental results show that our best-performing model achieves a new state-of-the-art performance on most language sets of syntactic dependency and semantic dependency parsing. In addition, our model also has an advantage in running speed over the static graph-based learning model. The outstanding performance demonstrates the effectiveness of the proposed framework in structured prediction.

A Fluid Self-Concept: How the Brain Maintains Coherence and Positivity across an Interconnected Self-Concept While Incorporating Social Feedback.

People experience instances of social feedback as interdependent with potential implications for their entire self-concept. How do people maintain positivity and coherence across the self-concept while updating self-views from feedback? We present a network model describing how the brain represents the semantic dependency relations among traits and uses this information to avoid an overall loss of positivity and coherence. Both male and female human participants received social feedback during a self-evaluation task while undergoing functional magnetic resonance imaging. We modeled self-belief updating by incorporating a reinforcement learning model within the network structure. Participants learned more rapidly from positive than negative feedback and were less likely to change self-views for traits with more dependencies in the network. Further, participants back propagated feedback across network relations while retrieving prior feedback on the basis of network similarity to inform ongoing self-views. Activation in ventromedial prefrontal cortex (vmPFC) reflected the constrained updating process such that positive feedback led to higher activation and negative feedback to less activation for traits with more dependencies. Additionally, vmPFC was associated with the novelty of a trait relative to previously self-evaluated traits in the network, and angular gyrus was associated with greater certainty for self-beliefs given the relevance of prior feedback. We propose that neural computations that selectively enhance or attenuate social feedback and retrieve past relevant experiences to guide ongoing self-evaluations may support an overall positive and coherent self-concept.SIGNIFICANCE STATEMENT We humans experience social feedback throughout our lives, but we do not dispassionately incorporate feedback into our self-concept. The implications of feedback for our entire self-concept plays a role in how we either change or retain our prior self-beliefs. In a neuroimaging study, we find that people are less likely to change their beliefs from feedback when the feedback has broader implications for the self-concept. This resistance to change is reflected in processing in the ventromedial prefrontal cortex, a region that is central to self-referential and social cognition. These results are broadly applicable given the role that maintaining a positive and coherent self-concept plays in promoting mental health and development throughout the lifespan.

Real-time load forecasting model for the smart grid using bayesian optimized CNN-BiLSTM

A smart grid is a new type of power system based on modern information technology, which utilises advanced communication, computing and control technologies and employs advanced sensors, measurement, communication and control devices that can monitor the status and operation of various devices in the power system in real-time and optimise the dispatch of the power system through intelligent algorithms to achieve efficient operation of the power system. However, due to its complexity and uncertainty, how to effectively perform real-time prediction is an important challenge. This paper proposes a smart grid real-time prediction model based on the attention mechanism of convolutional neural network (CNN) combined with bi-directional long and short-term memory BiLSTM.The model has stronger spatiotemporal feature extraction capability, more accurate prediction capability and better adaptability than ARMA and decision trees. The traditional prediction models ARMA and decision tree can often only use simple statistical methods for prediction, which cannot meet the requirements of high accuracy and efficiency of real-time load prediction, so the CNN-BiLSTM model based on Bayesian optimisation has the following advantages and is more suitable for smart grid real-time load prediction compared with ARMA and decision tree. CNN is a hierarchical neural network structure containing several layers such as a convolutional layer, pooling layer and fully connected layer. The convolutional layer is mainly used for extracting features from data such as images, the pooling layer is used for the dimensionality reduction of features, and the fully connected layer is used for classification and recognition. The core of CNN is the convolutional operation, a locally weighted summation operation on the input data that can effectively extract features from the data. In the convolution operation, different features can be extracted by setting different convolution kernels to achieve feature extraction and classification of data. BiLSTM can capture semantic dependencies in both directions. The BiLSTM structure consists of two LSTM layers that process the input sequence in the forward and backward directions to combine the information in both directions to obtain more comprehensive contextual information. BiLSTM can access both the front and back inputs at each time step to obtain more accurate prediction results. It effectively prevents gradient explosion and gradient disappearance while better capturing longer-distance dependencies. The CNN-BiLSTM extracts features of the data and then optimises them by Bayes. By collecting real-time data from the power system, including power, load, weather and other factors, our model uses the features of CNN-BiLSTM to deeply learn real-time load data from smart grids and extract key features to achieve future load prediction. Meanwhile, the Bayesian optimisation algorithm based on the model can optimise the model’s hyperparameters, thus improving the model’s prediction performance. The model can achieve accurate prediction of a real-time power system load, provide an important reference for the dispatch and operation of the power system, and help optimise the operation efficiency and energy utilisation efficiency of the power system.

Novel Approach for Thorax Disease Classification using Deep Learning

The task of thorax disease diagnosis on chest X- ray (CXR) pictures is discussed in this work. The majority of available approaches learn a network using global images as input. Thorax disease, on the other hand, frequently occurs in disease-specific (small) localized areas. As a result, the (excessive) irrelevant noisy areas may impact CNN training utilizing a global image. Furthermore, the presence of uneven borders hampered network performance due to poor alignment of some CXR pictures. In this research, we suggest using two-branch architecture called ConsultNet to train discriminative features and satisfy both of these goals at the same time. ConsultNet is made up of two parts. 1) a feature selector bound by an information bottleneck retrieves key disease-specific features based on their relevance. 2) a feature integrator based on spatial and channel encoding improves the latent semantic dependencies in the feature space. ConsultNet combines these distinguishing characteristics to improve thoracic illness categorization in CXRs. Experiments on the ChestX-ray14 and CheXpert datasets have shown that the proposed strategy is effective.

THE PARABOLIC STRUCTURE OF GOSPEL PARABLES

The article is devoted to the analysis of the special parabolic structure of the evangelical parables. It is not only about the actual literary understanding of the parabola as a stylistic device, with the help of which foreign language expression is carried out, but also about a mathematical graph – a curved curve, divided at the point of intersection into two symmetrical branches. In the structure of the evangelical parable, these branches can be represented as real-factographic and symbolic interpretive plans, between which there is a direct semantic dependence. All narrative situations that make up the real plan of the parable and are represented by the right branch of the parabola must be interpreted in the same sequence to build a symmetrical left branch. In the article, the parabolic structure of two symmetrical plans is demonstrated on the example of two gospel parables well-known in culture: The Parable of the Prodigal Son [Luk. 15, 11 32] and Proverbs about ten virgins [Matt. 15, 1–11]. Obviously, a similar way of interpretive reading of the Gospel parables can be qualified as a fullfledged hermeneutic technique of analysis of other biblical works.

Improving Chinese Named Entity Recognition by Interactive Fusion of Contextual Representation and Glyph Representation

Named entity recognition (NER) is a fundamental task in natural language processing. In Chinese NER, additional resources such as lexicons, syntactic features and knowledge graphs are usually introduced to improve the recognition performance of the model. However, Chinese characters evolved from pictographs, and their glyphs contain rich semantic information, which is often ignored. Therefore, in order to make full use of the semantic information contained in Chinese character glyphs, we propose a Chinese NER model that combines character contextual representation and glyph representation, named CGR-NER (Character–Glyph Representation for NER). First, CGR-NER uses the large-scale pre-trained language model to dynamically generate contextual semantic representations of characters. Secondly, a hybrid neural network combining a three-dimensional convolutional neural network (3DCNN) and bi-directional long short-term memory network (BiLSTM) is designed to extract the semantic information contained in a Chinese character glyph, the potential word formation knowledge between adjacent glyphs and the contextual semantic and global dependency features of the glyph sequence. Thirdly, an interactive fusion method with a crossmodal attention and gate mechanism is proposed to fuse the contextual representation and glyph representation from different models dynamically. The experimental results show that our proposed model achieves 82.97% and 70.70% F1 scores on the OntoNotes 4 and Weibo datasets. Multiple ablation studies also verify the advantages and effectiveness of our proposed model.

Planarized sentence representation for nested named entity recognition

One strategy to recognize nested entities is to enumerate overlapped entity spans for classification. However, current models independently verify every entity span, which ignores the semantic dependency between spans. In this paper, we first propose a planarized sentence representation to represent nested named entities. Then, a bi-directional two-dimensional recurrent operation is implemented to learn semantic dependencies between spans. Our method is evaluated on seven public datasets for named entity recognition. It achieves competitive performance in named entity recognition. The experimental results show that our method is effective to resolve nested named entities and learn semantic dependencies between them.

A trajectory prediction method based on bayonet importance encoding and bidirectional LSTM

Predicting vehicle trajectories can not only provide users with efficient travel suggestions, but also help traffic managers make better decisions. This paper aims to address the next position prediction problem to enable the prediction of the movement trajectory of an individual vehicle by considering where it will go. With this aim, it investigates using deep learning methods for predicting a vehicle’s next position based on its journey history. A deep learning model based on bidirectional long short-term memory (Bi-LSTM) with self-attention mechanism that fuses bayonet importance encoding (Bi-LSTMAI) is proposed, which can well capture both local and global semantic dependency. In specific, word embedding in natural language processing (NLP) is introduced to measure the relevance of bayonets and the PageRank algorithm is improved to obtain the bayonet importance encoding. By using real bayonet data, this research compares the prediction performance of the proposed model with several baselines, including LSTM, GRU, Bi-LSTM and Bi-GRU. The result shows that the proposed model yields better performance in all experimental metrics for the task of trajectory prediction.

Event Detection Using a Self-Constructed Dependency and Graph Convolution Network

The extant event detection models, which rely on dependency parsing, have exhibited commendable efficacy. However, for some long sentences with more words, the results of dependency parsing are more complex, because each word corresponds to a directed edge with a dependency parsing label. These edges do not all provide guidance for the event detection model, and the accuracy of dependency parsing tools decreases with the increase in sentence length, resulting in error propagation. To solve these problems, we developed an event detection model that uses a self-constructed dependency and graph convolution network. First, we statistically analyzed the ACE2005 corpus to prune the dependency parsing tree, and combined the named entity features in the sentence to generate an undirected graph. Second, we implemented an enhanced graph convolution network using the multi-head attention mechanism to understand the representation of nodes in the graph. Finally, a gating mechanism combined the semantic and structural dependency information of the sentence, enabling us to accomplish the event detection task. A series of experiments conducted on the ACE2005 corpus demonstrates that the proposed method enhances the performance of the event detection model.

AMDNet: A Modern UAV RGB Remote-Sensing Tree Species Image Segmentation Model Based on Dual-Attention Residual and Structure Re-Parameterization

Tree species classification is an important and challenging task in image recognition and the management of forest resources. Moreover, the task of tree species classification based on remote sensing images can significantly improve the efficiency of the tree species survey and save costs. In recent years, many large models have achieved high accuracy in the task of tree species classification in an airborne remote-sensing manner, but due to their fixed geometric structure, traditional convolutional neural networks are inherently limited to the local receptive field and can only provide segmental context information. The limitation of insufficient context information greatly affects the segmentation accuracy. In this paper, a dual-attention residual network (AMDNet) and a re-parameterized model approach are proposed to capture the global context information, fuse the weight, reduce the model volume, and maintain the computational efficiency. Firstly, we propose MobileNetV2 as the backbone network for feature extraction to further improve the feature identification by modeling semantic dependencies in the spatial dimension and channel dimension and adding the output of the two attention modules. Then, the attention perception features are generated by stacking the attention modules, and the in-depth residual attention network is trained using attention residual learning, through which more accurate segmentation results can be obtained. Secondly, we adopt the approach of structure re-parameterization, use a multi-branch topology for training, carry out weighted averaging on multiple trained models, and fuse multiple branch modules into a completely equivalent module in inference. The proposed approach results in a reduction in the number of parameters and an accelerated inference speed while also achieving improved classification accuracy. In addition, the model training strategy is optimized based on Transformer to enhance the accuracy of segmentation. The model was used to conduct classification experiments on aerial orthophotos of Hongya Forest Farm in Sichuan, China, and the MIOU of tree species recognition using the test equipment reached 93.8%. Compared with current models such as UNet, our model exhibits a better performance in terms of both speed and accuracy, in addition to its enhanced deployment capacity, and its speed advantage is more conducive to real-time segmentation, thereby representing a novel approach for the classification of tree species in remote sensing imagery with significant potential for practical applications.

Progressive Perception Learning for Main Coronary Segmentation in X-ray Angiography.

Main coronary segmentation from the X-ray angiography images is important for the computer-aided diagnosis and treatment of coronary disease. However, it confronts the challenge at three different image granularities (the semantic, surrounding, and local levels). The challenge includes the semantic confusion between the main and collateral vessels, low contrast between the foreground vessel and background surroundings, and local ambiguity near the vessel boundaries. The traditional hand-crafted feature-based methods may be insufficient because they may lack the semantic relationship information and may not distinguish the main and collateral vessels. The existing deep learning-based methods seem to have issues due to the deficiency in the long-distance semantic relationship capture, the foreground and background interference adaptability, and the boundary detail information preservation. To solve the main coronary segmentation challenge, we propose the progressive perception learning (PPL) framework to inspect these three different image granularities. Specifically, the PPL contains the context, interference, and boundary perception modules. The context perception is designed to focus on the main coronary vessel based on the semantic dependence capture among different coronary segments. The interference perception is designed to purify the feature maps based on the foreground vessel enhancement and background artifact suppression. The boundary perception is designed to highlight the boundary details based on boundary feature extraction through the intersection between the foreground and background predictions. Extensive experiments on 1085 subjects show that the PPL is effective (e.g., the overall Dice is greater than 95%), and superior to thirteen state-of-the-art coronary segmentation methods.

A deep penetration network for sentence classification

Sentence classification is an important task in natural language processing. The task makes use of deep networks to enclose a mass of features with different granularities in a sentence. However, the classification usually suffer from severe performance degradation when stacking a large number of networks. The main reason is that, in a deep architecture, the silent feature representations are easily weakened and mixed with noisy information, which is not effective in learning contextual features and constructing semantic dependencies in a sentence. In this paper, a deep penetration network (DPN) is designed to improve deep architectures’ ability to preserve the favourable semantic features. The DPN enables salient features to penetrate through a deeper architecture and to construct long semantic dependencies between them. This approach is evaluated on seven public datasets. Our experiments show that the DPN exhibits a stable performance with deeper architectures. It improves the performance on three types of sentence classification tasks, outperforming the existing state-of-the-art models.

Urban hotspot forecasting via automated spatio-temporal information fusion

Urban hotspot forecasting is one of the most important tasks for resource scheduling and security in future smart cities. Most previous works employed fixed neural architectures based on many complicated spatial and temporal learning modules. However, designing appropriate neural architectures is challenging for urban hotspot forecasting. One reason is that there is currently no adequate support system for how to fuse multi-scale spatio-temporal information rationally by integrating different spatial and temporal learning modules. Another one is that the empirical fixed neural architecture is difficult to adapt to different data scenarios from different domains or cities. To address the above problems, we propose a novel framework based on neural architecture search for urban hotspot forecasting, namely Automated Spatio-Temporal Information Fusion Neural Network (ASTIF-Net). In the search space of our ASTIF-Net, normal convolution and graph convolution operations are adopted to capture spatial geographic neighborhood dependencies and spatial semantic neighborhood dependencies, and different types of temporal convolution operations are adopted to capture short-term and long-term temporal dependencies. In addition to combining spatio-temporal learning operations from different scales, ASTIF-Net can also search appropriate fusion methods for aggregating multi-scale spatio-temporal hidden information. We conduct extensive experiments to evaluate ASTIF-Net on three real-world urban hotspot datasets from different domains to demonstrate that our proposed model can obtain effective neural architectures and achieve superior performance (about 5%∼10% improvements) compared with the existing state-of-art baselines.