Features Of Sentences Research Articles

Piecewise convolutional neural network relation extraction with self-attention mechanism

The task of relation extraction in natural language processing is to identify the relation between two specified entities in a sentence. However, the existing model methods do not fully utilize the word feature information and pay little attention to the influence degree of the relative relation extraction results of each word. In order to address the aforementioned issues, we propose a relation extraction method based on self-attention mechanism (SPCNN-VAE) to solve the above problems. First, we use a multi-head self-attention mechanism to process word vectors and generate sentence feature vector representations, which can be used to extract semantic dependencies between words in sentences. Then, we introduce the word position to combine the sentence feature representation with the position feature representation of words to form the input representation of piecewise convolutional neural network (PCNN). Furthermore, to identify the word feature information that is most useful for relation extraction, an attention-based pooling operation is employed to capture key convolutional features and classify the feature vectors. Finally, regularization is performed by a variational autoencoder (VAE) to enhance the encoding ability of model word information features. The performance analysis is performed on SemEval 2010 task 8, and the experimental results show that the proposed relation extraction model is effective and outperforms some competitive baselines.

An attention-based hybrid model for spatial and temporal sentiment analysis of COVID-19 related tweets in the contiguous United States

ABSTRACT Understanding the sentiments of social media posts can help health authorities respond to disease outbreaks, through a proxy measure of fear, confidence, and community compliance. Sentiment analysis identifies a pattern of emotion through the written word and assigns a positive, neutral, or negative value to it. As of February 2023, there were 677.7 million confirmed cases of coronavirus disease (COVID-19) and more than 6.7 million confirmed deaths. In this paper, around 170,000 COVID-19-related tweets were collected between September 2020 and January 2021 in the contiguous United States. Data preprocessing and exploratory investigation were completed for analysis of the collected dataset. Further, a novel and unified architecture called attention-based one-dimensional convolution with bidirectional long short-term memory layers (CNN-BiLSTM-ATT) is proposed to classify people’s sentiments as positive, neutral, and negative based on COVID-19-related tweets. In the CNN-BiLSTM-ATT model, the CNN layer can extract the low-level semantic features from textual data, and the BiLSTM layer can extract both the previous and future contextual representations. The attention module can improve the information focus from the outputted layer of the BiLSTM. The proposed method can extract both the local phrase representations and the global feature of sentences. Numerical experiments were conducted on COVID-19-related tweets using the proposed method and other baseline models to compare their performances. Our experimental results demonstrate that the CNN-BiLSTM-ATT model achieves an average accuracy of 95.16% and a macro-average F1-score of 95.12%, which outperforms the baseline models.

FRGEM: Feature integration pre-training based Gaussian embedding model for Chinese word representation

Most Chinese word representation approaches mine the lexeme of Chinese words by predicting the target word or the contextual words. However, Chinese words may have multiple meanings, above ways may only capture one aspect implication. In this paper, for better modeling multiple connotations of Chinese words, a Feature integRation pre-training based Gaussian Embedding Model (FRGEM) is proposed with three stages, feature integration pre-training, relevancy Gaussian representation and similarity-based training. In the first stage, the internal sequences with three inner-character stroke, structure and pinyin features of Chinese words are generated to learn their semantic relevancy, then we combine BERT with internal sequences to extract sentence relevance and integrate the internal and sentence features to learn pre-training representations. In the second stage, relevancy Gaussian representation with pre-training embeddings is proposed to fuse inner-character features with Gaussian through estimating its mean to analyze multiple implications of Chinese words. In the third stage, a similarity-based objective is proposed to distinguish true contextual words and learn Chinese word representations. Extensive experiments on word similarity, word analogy, named entity recognition and text classification show that FRGEM is superior to majority state-of-the-art algorithms.

Metaphor in grammatical works of linguists

The object of the research is metaphorical nominations in scientific linguistic discourse. The purpose of the study is to systematically describe the structural and semantic features of metaphors in grammatical works. The identification of expressive linguistic units in scientific texts is important for a comprehensive study of the relationship between rational and emotional in strict speech styles. The appeal to this topic is dictated by the need to find ways to present complex theoretical material in university syntax teaching. The article considers metaphors-words, metaphors-phrases and metaphors-predicative units characterizing transitional phenomena in the grammar of the Russian language. Metaphorical designations in the description of the desemanticized verbal components of analytical constructions and verbal connectives are highlighted. Metaphors characterizing other grammatical phenomena are analyzed. The role of verbal metaphors in the description of constitutive features of sentences of different types is established. The features of substantive and adjectival metaphors in grammatical works are noted. It is shown that the expanded metaphors demonstrate the multidimensional nature of syntactic categories. It is concluded that verbal and substantive metaphors convey the desire of linguists to present the grammatical system as dynamic, actively developing. The use of metaphorical names makes it possible to present grammatical theory in a more accessible way, to draw attention to interesting linguistic facts.

Chinese text classification method based on sentence information enhancement and feature fusion

Text classification involves annotating text data with specific labels and is a crucial research task in the field of natural language processing. Chinese text classification presents significant challenges due to the complex semantics of the language, difficulties in semantic feature extraction, and the interleaving and irregularity of lexical features. Traditional methods often struggle to manage the relationships between words and sentences in Chinese, hindering the model's ability to capture deep semantic information and resulting in poor classification performance. To address these issues, a Chinese text classification method based on utterance information enhancement and feature fusion is proposed. This method first embeds the text into a unified space and obtains feature representations of word vectors and sentence vectors using the BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model. Subsequently, an utterance information enhancement module is constructed to perform syntactic enhancement and feature extraction on the sentence information within the text. Additionally, a feature fusion strategy is introduced to combine the enhanced sentence-level information features with the word-level features extracted by the Bi-GRU (Bidirectional Gated Recurrent Unit network), culminating in the classification output. This approach effectively enhances the feature representation of Chinese text and significantly filters out irrelevant and noisy information. Evaluations on several Chinese datasets demonstrate that the proposed method surpasses existing mainstream classification models in terms of classification accuracy and F1 value, validating its effectiveness and feasibility.

A text guided cross modal joint inpainting algorithm for ancient murals

Existing deep learning algorithms for ancient mural inpainting only consider prior information of mural images to guide the inpainting of damaged areas, lacking guidance from textual semantic information, which often leads to problems such as inconsistent semantics and disordered textures in the inpainting results. In this work, we propose a text guided cross modal joint inpainting algorithm for ancient murals. Firstly, a cross modal joint inpainting network for ancient murals was constructed, which introduced text semantic information as the control guidance for mural inpainting, providing contextual information for mural image inpainting and guiding the inpainting of damaged areas. Then, design a text encoder and an image encoder to extract text and image features respectively, including word and sentence features. Input the sentence features into the conditional enhancement module, resample the text sentences, and solve the problem of text sparsity. Secondly, design a text decoupling module that utilizes a multimodal attention mechanism to extract the semantic information of the missing parts of damaged murals, and design a text semantic enhancement module to improve the semantic representation ability of the text, achieve the repair of local details of murals, and solve the problem of texture disorder. Finally, the reconstruction and repair of murals are completed through a residual discriminator game and a spectral normalized Markov dual discriminator against the mural generation network. Through inpainting experiments on real Dunhuang murals, it has been shown that the proposed method is more effective than comparative methods in repairing damaged murals, achieving better visual perception and coordination in reconstruction, and outperforming comparative algorithms in both subjective and objective evaluations.

Predicting the Sentiment of Review Aspects in the Peer Review Text using Machine Learning

This paper develops a Machine Learning (ML) model to classify the sentiment of review aspects in the peer review text. Reviewers use the review aspect as paper quality indicators such as motivation, originality, clarity, soundness, substance, replicability, meaningful comparison, and summary during the review process. The proposed model addresses the critique of the existing peer review process, including a high volume of submitted papers, limited reviewers, and reviewer bias. This paper uses citation functions, representing the author motivation to cites previous researches, as the main predictor. Specifically, the predictor comprises citing sentence features representing the scheme of citation functions, regular sentence features representing the scheme of citation functions for non-citation sentence, and reference-based representing the source of citation. This paper utilizes the paper dataset from the International Conference on Learning Representations (ICLR) 2017-2020, which includes sentiment values (positive or negative) for all review aspects. Our experiment on combining XGBoost, oversampling, and hyper-parameter optimization revealed that not all review aspects can be effectively estimated by the ML model. The highest results were achieved when predicting Replicability sentiment with 97.74% accuracy. It also demonstrated accuracies of 94.03% for Motivation and 93.93% for Meaningful Comparison. However, the model exhibited lower effectiveness on Originality and Substance (85.21% and 79.94%) and performed less effectively on Clarity and Soundness with accuracies of 61.22% and 61.11%, respectively. The combination predictor was the best for the 5 review aspects, while the other 2 aspects were effectively estimated by regular sentence and reference-based predictors.

THE EMERGENCE OF THE STYLISTICS OF THE ARABIC LANGUAGE

This article provides detailed information about the stylistic aspect of formal communication in Arabic, the emergence of stylistics in Arabic and its features in sentences.

TCKGCN: Graph convolutional network for aspect-based sentiment analysis with three-channel knowledge fusion

At present, most aspect-based sentiment analysis studies addresses the syntactic dependency problem of review statements by building syntactic dependency graphs. However, this approach is insensitive to syntactic structural information and susceptible to noise interference. Existing fusion methods are unable to effectively capture the complementary relationships between information, leading to large and less efficient model structures. To tackle these issues, we introduce a unique and original scheme that fuses semantic information, ConceptNet conceptual knowledge, and SenticNet affective knowledge, intending to improve the capability of the model to capture sentence features. Specifically, the process involves four steps: first, a semantic attention mechanism is constructed as the first channel to parse the semantic relationship within the context; second, the expressive meaning of the aspects is enhanced with concept mapping as the second channel, with concepts injected into aspectual words; third, the representation ability of sentence dependency graphs is strengthened through adding the affective knowledge of SenticNet as a third channel. Subsequently, the sensitivity of the model to affective information is increased, and finally, the optimization of aspect and context coordination is strengthened by integrating an interactive attention mechanism. The overall findings underscore the efficacy of the model, showing accuracy improvements of 1.05 % and 2.80 % on the Twitter and Lap14 datasets, respectively, compared to state-of-the-art models. Moreover the macro-F1 scores showed significant improvements of 1.99 % and 1.61 %, respectively, enabling more effective capturing of aspect-specific sentiment expressions.

Interaction-and-Response Network for Distantly Supervised Relation Extraction.

Distantly supervised relation extraction (DSRE) aims to identify semantic relations from massive plain texts. A broad range of the prior research has leveraged a series of selective attention mechanisms over sentences in a bag to extract relation features without considering dependencies among the relation features. As a result, potential discriminative information existed in the dependencies is ignored, causing a decline in the performance of extracting entity relations. In this article, we focus on going beyond the selective attention mechanisms and propose a new framework termed interaction-and-response network (IR-Net) that adaptively recalibrates the features of sentence, bag, and group levels by explicitly modeling interdependencies among the features on each level. The IR-Net consists of a series of interactive and responsive modules throughout feature hierarchy, seeking to strengthen its power of learning salient discriminative features for distinguishing entity relations. We conduct extensive experiments on three benchmark DSRE datasets, including NYT-10, NYT-16, and Wiki-20m. The experimental results demonstrate that the IR-Net brings obvious improvements in performance when comparing ten state-of-the-art DSRE methods for entity relation extraction.

Label-semantics enhanced multi-layer heterogeneous graph convolutional network for Aspect Sentiment Quadruplet Extraction

Aspect-based sentiment analysis (ABSA) is a pivotal area within natural language processing (NLP) that focuses on extracting fine-grained sentiment information from text. A particularly demanding task within ABSA is Aspect Sentiment Quadruplet Extraction (ASQE), which aims to identify quadruplets comprising aspect terms, their corresponding opinion terms, sentiment polarity, and categories. This level of analysis is crucial for downstream applications such as sentiment monitoring and user experience research, offering nuanced insights into textual data. However, current methodologies fall short of fully leveraging the rich linguistic features of sentences and the semantic information of labels embedded within sentences. To address this gap, this paper introduces a novel framework, the Label-Semantics Enhanced Multi-layer Heterogeneous Graph Convolutional Network (LSEMH-GCN), specifically designed for ASQE. Our model integrates sentence linguistic features and label semantics to construct a graph neural network tailored for this task. It employs a multi-layer graph convolutional network that synergizes various linguistic features, and utilizes Biaffine attention to enrich the label probability distribution for token pairs with semantic label information. Furthermore, our approach introduces a token pair vector concatenation strategy alongside an advanced asymmetric label tagging scheme to enhance quadruplet extraction. Comprehensive evaluations on benchmark datasets reveal that LSEMH-GCN significantly surpasses existing state-of-the-art models, establishing a new benchmark for ASQE. Our model achieves an average F1 score improvement of 15.52% and 12.30% on the Restaurant-ACOS and Laptop-ACOS datasets, respectively.

Text Classification Model Based on Graph Attention Networks and Adversarial Training

Text information on the internet often has a strong sense of immediacy, constantly reflecting societal dynamics and evolving events. This is especially crucial in the field of news text, where the classification and analysis of these immediate and varied text data become essential. Existing text classification models frequently struggle to effectively represent the semantic and local feature information of texts, limiting their effectiveness. The primary challenge lies in improving the representation of both semantic and local feature information in text classification models, which is critical for capturing the nuanced meanings in rapidly evolving news texts. This paper proposes a deep learning-driven framework designed to enhance the effectiveness of text classification models. The method incorporates noise perturbation during training for adversarial training, thereby enhancing the model’s generalization ability on original samples and increasing its robustness. A graph attention network is employed to extract the contextual semantic information of vocabulary from sequential texts. This information is then combined with extracted sentence feature information to enrich the feature representation of the sequence. An attention mechanism is also introduced to extract more critical feature information from the text, thereby deepening the understanding of textual semantic information. Experimental results demonstrate that this method successfully integrates the boundary and semantic information of vocabulary into the classification task. The approach comprehensively and deeply mines the semantic features of the text, leading to improved classification performance.

Aspect category sentiment analysis based on multiple attention mechanisms and pre-trained models

The rapid rise of e-commerce platforms has changed people's shopping habits, driving the popularity of online shopping. Users express their opinions on products and services by purchasing products on platforms and posting comments. These comment data contain rich user experiences, which are crucial for enterprises to understand user needs and improve product quality. Sentiment analysis of comment text is an important research direction in text mining, focusing on how to extract user evaluations of products from comment data to provide comprehensive, authentic, and accurate product feedback. This paper mainly investigates aspect-based fine-grained sentiment analysis. Because users express multiple sentiments towards different aspects in comments, coarse-grained sentiment analysis cannot accurately capture users' sentiment tendencies. This study utilizes artificial intelligence technology based on deep learning, firstly, it constructs auxiliary training samples to transform aspect sentiment tasks into machine reading comprehension or language inference tasks, using BERT model to extract text features and sentence features from comment data. To guide the model to focus on the features most relevant to the given aspect, a cross-attention mechanism is utilized to cross-focus the features of text with aspect category features. Finally, the sentiment polarity of given aspect category text can be predicted through a forward network. Experimental results on multiple datasets demonstrate that this method outperforms other deep learning models.

교실 담화에서 교사의 청유문 분석

This study explores the functions of grammatical forms in specific discourse contexts by analyzing propositional sentences based on sentence-ending types. Propositional sentences, similar to imperative sentences, are identified as having a notably directive attribute. Particularly, classroom discourse, which is grounded in a clear hierarchical structure between teachers and students, may highlight the use of propositional sentences more than in other discourse situations. Therefore, this research utilized a spoken corpus to investigate propositional sentences through an inductive method. Initially, propositional sentences and their functions were defined within the spoken corpus from a grammatical perspective. The sentences were then analyzed and categorized according to their functions. The findings suggest that propositional sentences in classroom discourse primarily exhibit directive functions. Moreover, new functions of propositional sentences, previously undiscussed, were discovered. These newly identified functions are closely associated with the type of discourse. The results of this study allow for the differentiation of unique features of propositional sentences in classroom discourse, potentially aiding learners in enhancing their intuitive understanding of the Korean language.

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model’s performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

ESIE-BERT: Enriching sub-words information explicitly with BERT for intent classification and slot filling

Natural language understanding (NLU) has two core tasks: intent classification and slot filling. The success of pre-training language models resulted in a significant breakthrough in the two tasks. The architecture based on autoencoding (BERT-based model) can optimize the two tasks jointly. However, we note that BERT-based models convert each complex token into multiple sub-tokens by the Wordpiece algorithm, which generates an out-of-alignment between the length of the tokens and the labels. This leads to BERT-based models not performing well in label prediction, limiting model performance improvement. Many existing models can address this issue, but some hidden semantic information is discarded during fine-tuning. We addressed the problem by introducing a novel joint method on top of BERT. This method explicitly models multiple sub-token features after the Wordpiece tokenization, contributing to both tasks. Our proposed method effectively extracts contextual features from complex tokens using the Sub-words Attention Adapter (SAA), preserving overall utterance information. Additionally, we propose an Intent Attention Adapter (IAA) to acquire comprehensive sentence features, assisting users in predicting intent. Experimental results confirm that our proposed model exhibits significant improvements on two public benchmark datasets. Specifically, the slot-filling F1 score improves from 96.5 to 98.2 (an absolute increase of 1.7%) on the Airline Travel Information Systems (ATIS) dataset.

A Multi-sentence Music Humming Retrieval Algorithm Based on Relative Features and Deep Learning

This project will study a fast retrieval method for music humming speech recognition based on sentence features and deep learning. The method proposed in this paper can realize the fast extraction of songs. According to the characteristics of the natural pause mode of the song, the song database and the song fragments provided by the user are divided into different sentences. The deep learning algorithm of BDTW is used to calculate the similarity of the song's pitch, and users can set matching conditions according to their preferences. It can identify the most significant differences between music fragments and the order of queries in the database. Then, a retrieval method of a music database based on DIS is proposed. It can shorten the acquisition time. Experiments show that the algorithm can recognize humming songs quickly and efficiently.

<span>Building compound-sentence exercises for 5<sup>th</sup>-graders in competency-based teaching</span>

Surveying compound-sentence exercises types in the Vietnamese-Language subject of the 2006 Primary School Program and those currently in some primary schools, Dong Thap province, the article clarifies a number of issues on building compound sentence exercises teaching 5th-graders for competency development. In particular, the article focuses on those exercises aligned with 3 lesson types in the cognitive-level order: awareness, understanding and application. These exercises provide teachers with more resources to apply to teaching; thereby, helping students identify the structural features of compound sentences and use them effectively in communication.

Intelligent extraction of medical entity relationship based on graph neural network and optimization strategy

Machine Learning technologies have obtained breakthrough in various fields, and medical information extraction has also successfully made great progress through deep learning methods. However, entity relationship overlap is a key issue and challenge in the field of medical entity relationship extraction. Hence, we propose an intelligent extraction method for Chinese medicine entity relationship through improving graph neural network and structure optimization strategy. We optimize model structure and introduce global pointer network. The sentence feature information is captured by attention mechanism and Bi-Long Short Term Memory (LSTM). Grammatical relations between entities are parsed using Graph Convolution Network (GCN) layer. Finally, multiple decoders are used for joint extraction of entity relations through the global pointer network. The model learns the graph structure data by GCN, while the challenge of entity relationship overlap is solved through using the pointer network. The experiments demonstrate that the model achieves 83.77% in terms of F1 value, which is better than other baseline models.

CourseKG: An Educational Knowledge Graph Based on Course Information for Precision Teaching

With the rapid development of advanced technologies, such as artificial intelligence and deep learning, educational informatization has entered a new era. However, the explosion of information has brought numerous challenges. Knowledge graphs, as a crucial component of artificial intelligence, can contribute to the quality of teaching. This study proposes an educational knowledge graph based on course information named CourseKG for precision teaching. Precision teaching seeks to individualize the curriculum for each learner and optimize learning efficiency. CourseKG aims to establish a correct and comprehensive curriculum knowledge system and promote personalized learning paths. CourseKG can address the issue that current general-purpose knowledge graphs are not suitable for the education field. Particularly, this study proposes a framework for educational entity recognition based on the pre-trained BERT model. This framework captures relevant information in the educational domain using the BERT model and combines it with the BiGRU and multi-head self-attention mechanism to extract multi-scale and multi-level global dependency relationships. In addition, the CRF is used for character-label decoding. Further, a relationship extraction method based on the BERT model, which integrates sentence features and educational entities and estimates the similarity between knowledge pairs using cosine similarity, is proposed. The proposed CourseKG is verified by experiments using real-world C programming course data. The experimental results demonstrate the effectiveness of CourseKG. Finally, the results show that the proposed CourseKG can significantly enhance the precision teaching quality and realize multi-directional adaptation among teachers, courses, and students.