Sentence Information Research Articles

Enhancing emerging technology discovery in nanomedicine by integrating innovative sentences using BERT and NLDA

Abstract Purpose Nanomedicine has significant potential to revolutionize biomedicine and healthcare through innovations in diagnostics, therapeutics, and regenerative medicine. This study aims to develop a novel framework that integrates advanced natural language processing, noise-free topic modeling, and multidimensional bibliometrics to systematically identify emerging nanomedicine technology topics from scientific literature. Design/methodology/approach The framework involves collecting full-text articles from PubMed Central and nanomedicine-related metrics from the Web of Science for the period 2013–2023. A fine-tuned BERT model is employed to extract key informative sentences. Noiseless Latent Dirichlet Allocation (NLDA) is applied to model interpretable topics from the cleaned corpus. Additionally, we develop and apply metrics for novelty, innovation, growth, impact, and intensity to quantify the emergence of novel technological topics. Findings By applying this methodology to nanomedical publications, we identify an increasing emphasis on research aligned with global health priorities, particularly inflammation and biomaterial interactions in disease research. This methodology provides deeper insights through full-text analysis and leading to a more robust discovery of emerging technologies. Research limitations One limitation of this study is its reliance on the existing scientific literature, which may introduce publication biases and language constraints. Additionally, manual annotation of the dataset, while thorough, is subject to subjectivity and can be time-consuming. Future research could address these limitations by incorporating more diverse data sources, and automating the annotation process. Practical implications The methodology presented can be adapted to explore emerging technologies in other scientific domains. It allows for tailored assessment criteria based on specific contexts and objectives, enabling more precise analysis and decision-making in various fields. Originality/value This study offers a comprehensive framework for identifying emerging technologies in nanomedicine, combining theoretical insights and practical applications. Its potential for adaptation across scientific disciplines enhances its value for future research and decision-making in technology discovery.

Contextualised retrieval practice: the effect of context type and working memory capacity on word retention

ABSTRACT The present study on intentional retrieval practice compared the benefits of presenting words in either informative or uninformative sentence contexts. Participants first studied a list of English words with their translations. Then, they were all exposed to half of the words with informative sentences containing meaning clues in the Context Inference (CI) condition and half with uninformative sentences devoid of such clues in the Memory Retrieval (MR) condition as part of retrieval-based practising. Participants were required to type the L1 translation for each word presented using a mobile application. Data were collected by both form-recall and meaning-recall tests immediately afterwards and then a week later. In addition, this study focused on the relationship between working memory capacity (WMC) and word retention in these two conditions to explore the suggestion that individuals may benefit differently from retrieval practices. Although the results showed that both conditions contributed to word retention, the MR condition was significantly more effective than the CI condition for the participants’ long-term retention. Further, the results revealed an overall positive effect of WMC on word retention in both conditions, with high-WMC individuals achieving higher retention scores than low-WMC individuals. However, this effect was not modulated by the type of context condition.

Similar Data Points Identification with LLM: A Human-in-the-Loop Strategy Using Summarization and Hidden State Insights

This study introduces a simple yet effective method for identifying similar data points across non-free text domains, such as tabular and image data, using Large Language Models (LLMs). Our two-step approach involves data point summarization and hidden state extraction. Initially, data is condensed via summarization using an LLM, reducing complexity and highlighting essential information in sentences. Subsequently, the summarization sentences are fed through another LLM to extract hidden states, serving as compact, feature-rich representations. This approach leverages the advanced comprehension and generative capabilities of LLMs, offering a scalable and efficient strategy for similarity identification across diverse datasets. We demonstrate the effectiveness of our method in identifying similar data points on multiple datasets. Additionally, our approach enables non-technical domain experts, such as fraud investigators or marketing operators, to quickly identify similar data points tailored to specific scenarios, demonstrating its utility in practical applications. In general, our results open new avenues for leveraging LLMs in data analysis across various domains.

Effective sentence-level relation extraction model using entity-centric dependency tree.

The syntactic information of a dependency tree is an essential feature in relation extraction studies. Traditional dependency-based relation extraction methods can be categorized into hard pruning methods, which aim to remove unnecessary information, and soft pruning methods, which aim to utilize all lexical information. However, hard pruning has the potential to overlook important lexical information, while soft pruning can weaken the syntactic information between entities. As a result, recent studies in relation extraction have been shifting from dependency-based methods to pre-trained language model (LM) based methods. Nonetheless, LM-based methods increasingly demand larger language models and additional data. This trend leads to higher resource consumption, longer training times, and increased computational costs, yet often results in only marginal performance improvements. To address this problem, we propose a relation extraction model based on an entity-centric dependency tree: a dependency tree that is reconstructed by considering entities as root nodes. Using the entity-centric dependency tree, the proposed method can capture the syntactic information of an input sentence without losing lexical information. Additionally, we propose a novel model that utilizes entity-centric dependency trees in conjunction with language models, enabling efficient relation extraction without the need for additional data or larger models. In experiments with representative sentence-level relation extraction datasets such as TACRED, Re-TACRED, and SemEval 2010 Task 8, the proposed method achieves F1-scores of 74.9%, 91.2%, and 90.5%, respectively, which are state-of-the-art performances.

B - 51 Are my Constructs Bass-Ackwards? A Hierarchical Approach to Neuropsychological Factors in Traumatic Brain Injury

Abstract Objective This study examined neuropsychological measures within a hierarchical framework in a traumatic brain injury (TBI) sample. Methods 270 individuals (M = 32 years old, SD = 14.8; 70% male, 94% white) with mixed TBI severity (73.7% mild TBI) completed the Meyers Neuropsychological Battery (MNB). Principal Components Analyses (PCAs) were conducted using Goldberg’s (2006) “bass-ackwards” hierarchical approach commonly used to evaluate “stratum” in personality and psychopathology. This involves an iterative top-down approach extracting a single general factor, then using the same data to extract two factors, then three factors, and continues as long as theoretically feasible. Extracted factors are saved as factor scores and correlated to reveal pathways between stratum. Results Iterative PCAs revealed a 4-stratum model. The first unrotated principal component (FUPC) was defined by loadings for Arithmetic, AVLT, RCFT, and Block Design. The second stratum factors were defined by AVLT and BNT (factor 1) and RCFT and Block Design (factor 2). The third stratum was defined by RCFT, Block Design, and Finger Localization nondominant (factor 1); Information and BNT (factor 2); and AVLT, Coding, and Finger Tapping dominant (factor 3). Stratum 4 was RCFT and Block Design (factor 1); AVLT (factor 2); Sentence Repetition, Similarities, and Information (factor 3); and Finger Tapping bilateral, Coding, and Trails Making Test (factor 4). Correlational pathways between factors ranged from 0.50 to 0.98. Conclusion Factors at lower stratum generally aligned with functions for cortical lobes. For example, Stratum 4 can be conceptualized as right temporoparietal (factor 1), left temporal (factor 2), left frontotemporal (factor 3), and bilateral frontal (factor 4).

Fusion Network for Aspect-Level Sentiment Classification Based on Graph Neural Networks—Enhanced Syntactics and Semantics

Aspect-level sentiment classification (ALSC) struggles with correctly trapping the aspects and corresponding sentiment polarity of a statement. Recently, several works have combined the syntactic structure and semantic information of sentences for more efficient analysis. The combination of sentence knowledge with graph neural networks has also proven effective at ALSC. However, there are still limitations on how to effectively fuse syntactic structure and semantic information when dealing with complex sentence structures and informal expressions. To deal with these problems, we propose an ALSC fusion network that combines graph neural networks with a simultaneous consideration of syntactic structure and semantic information. Specifically, our model is composed of a syntactic attention module and a semantic enhancement module. First, the syntactic attention module builds a dependency parse tree with the aspect term being the root, so that the model focuses better on the words closely related to the aspect terms, and captures the syntactic structure through a graph attention network. In addition, the semantic enhancement module generates the adjacency matrix through self-attention, which is processed by the graph convolutional network to obtain the semantic details. Lastly, the extracted features are merged to achieve sentiment classification. As verified by experiments, the model we propose can effectively enhance ALSC’s behavior.

Chinese text classification method based on sentence information enhancement and feature fusion

Text classification involves annotating text data with specific labels and is a crucial research task in the field of natural language processing. Chinese text classification presents significant challenges due to the complex semantics of the language, difficulties in semantic feature extraction, and the interleaving and irregularity of lexical features. Traditional methods often struggle to manage the relationships between words and sentences in Chinese, hindering the model's ability to capture deep semantic information and resulting in poor classification performance. To address these issues, a Chinese text classification method based on utterance information enhancement and feature fusion is proposed. This method first embeds the text into a unified space and obtains feature representations of word vectors and sentence vectors using the BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model. Subsequently, an utterance information enhancement module is constructed to perform syntactic enhancement and feature extraction on the sentence information within the text. Additionally, a feature fusion strategy is introduced to combine the enhanced sentence-level information features with the word-level features extracted by the Bi-GRU (Bidirectional Gated Recurrent Unit network), culminating in the classification output. This approach effectively enhances the feature representation of Chinese text and significantly filters out irrelevant and noisy information. Evaluations on several Chinese datasets demonstrate that the proposed method surpasses existing mainstream classification models in terms of classification accuracy and F1 value, validating its effectiveness and feasibility.

Distance-reconstructed dependency enhanced aspect-based sentiment analysis with sentiment strength

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that requires the detection of sentiment polarity towards specific aspects. Dependency tree-based graph convolutional network models (GCN) have been widely applied in ABSA. These studies utilize syntactic analysis tools to derive syntactic dependency graphs, which are then inputted into GCN. It enables iterative learning of sentiment dependency information from contextual words to aspect terms. However, directly using syntactic dependency graphs can lead to two shortcomings. First, the long-distance sentiment information in long sentences will be weakened during GCN iterative computations. Second, the syntactic dependency graphs of short sentences lack sentiment strength information. To address these two shortcomings, we propose a Distance-Reconstructed Dependency Enhanced Sentiment analysis approach with Sentiment Strength based on GCN (DRD-SE-GCN) approach, in which the syntactic dependency graph of long sentences is reconstructed based on distance, thus narrowing the gap between sentiment words and target aspect words. It ensures that the sentiment dependency information carried by the long-distance sentiment words will not be weakened or lost during the GCN iteration. Additionally, the weights of the syntactic dependency graph for short sentences are reassigned so as to carry the strength information of sentiment words. Experimental results on four standard datasets show that our proposed model can effectively improve the accuracy of aspect-level sentiment analysis.

Fine-grained sentiment analysis using multidimensional feature fusion and GCN

ABSTRACT The demand for analysing sentiment information in social media data is increasing. However, current fine-grained sentiment analysis methods fail to consider both global and local semantic features simultaneously, leading to the oversight of grammatical information in sentences and an inability to address the issue of polysemy. To address these challenges, we propose a microblog fine-grained sentiment analysis model based on multidimensional feature fusion and graph convolutional neural networks (GCN). Built upon the ALBERT model, we utilize BiLSTM and capsule network models to extract global and local semantic features, thereby capturing bidirectional semantic dependencies and textual positional semantic information. Finally, we employ multi-head self-attention and GCN to select key features and sentence information, ensuring the integrity of fine-grained features. The experimental results indicate that the model outperforms several other models on fine-grained sentiment analysis datasets SMP2020-EWECT, NLPCC2013, NLPCC2014, and the binary classification dataset weibo_senti_100k, achieving accuracies of 80.64%, 67.19%, 71.37%, and 98.43%, respectively.

Document-Level Relation Extraction Based on Machine Reading Comprehension and Hybrid Pointer-sequence Labeling

Document-level relational extraction requires reading, memorization, and reasoning to discover relevant factual information in multiple sentences. It is difficult for the current hierarchical network and graph network methods to fully capture the structural information behind the document and make natural reasoning from the context. Different from the previous methods, this article reconstructs the relation extraction task into a machine reading comprehension task. Each pair of entities and relationships is characterized by a question template, and the extraction of entities and relationships is translated into identifying answers from the context. To enhance the context comprehension ability of the extraction model and achieve more precise extraction, we introduce large language models (LLMs) during question construction, enabling the generation of exemplary answers. Besides, to solve the multi-label and multi-entity problems in documents, we propose a new answer extraction model based on hybrid pointer-sequence labeling, which improves the reasoning ability of the model and realizes the extraction of zero or multiple answers in documents. Extensive experiments on three public datasets show that the proposed method is effective.

Rhetorical Renewal in the Kinds of Informative Sentences by Mohammed Mohammed Abu Musa, Survey and Analysis

Rhetorical Renewal in the Kinds of Informative Sentences by Mohammed Mohammed Abu Musa, Survey and Analysis

KGRED: Knowledge-graph-based rule discovery for weakly supervised data labeling

In weakly supervised learning, labeling rules can automatically label data to train models. However, due to insufficient prior knowledge, rule discovery often suffers from semantic drift. Since misclassified rules are generated from wrongly matched sentences, the sentences matched by rules shift from the target labels to other labels. It is worth noting that rules do not exist in isolation. The multi-dimensional semantic associations among rules can impose semantic constraints for rule generation, as well as enrich the semantic information of rules for rule matching. Therefore, we propose a Knowledge-Graph-based RulE Discovery method (KGRED), which can leverage the multi-dimensional semantic associations among rules to alleviate semantic drift in rule discovery. Specifically, to decrease misclassified rules, we design a label-aware rule generation approach to attentively propagate prior knowledge from seed rules to candidate rules based on rule KG. To reduce wrongly-matched sentences, we present a cross-attention-based semantic matching mechanism to refine the semantic information of sentences while enriching that of rules. Moreover, we propose an inconsistency-directed active learning strategy to verify rules that perform inconsistently in rule generation and matching. Experiments on two public datasets prove that KGRED can achieve at least 5.1 % gain in F1 score compared to state-of-the-art methods.

A Sentence-Matching Model Based on Multi-Granularity Contextual Key Semantic Interaction

In the task of matching Chinese sentences, the key semantics within sentences and the deep interaction between them significantly affect the matching performance. However, previous studies mainly relied on shallow interactions based on a single semantic granularity, which left them vulnerable to interference from overlapping terms. It is particularly challenging to distinguish between positive and negative examples within datasets from the same thematic domain. This paper proposes a sentence-matching model that incorporates multi-granularity contextual key semantic interaction. The model combines multi-scale convolution and multi-level convolution to extract different levels of contextual semantic information at word, phrase, and sentence granularities. It employs multi-head self-attention and cross-attention mechanisms to align the key semantics between sentences. Furthermore, the model integrates the original, similarity, and dissimilarity information of sentences to establish deep semantic interaction. Experimental results on both open- and closed-domain datasets demonstrate that the proposed model outperforms existing baseline models in terms of matching performance. Additionally, the model achieves matching effectiveness comparable to large-scale pre-trained language models while utilizing a lightweight encoder.

Fusing semantic information for syntax-guided paraphrase generation

Syntax-guided paraphrase generation (SGPG) refers to generating a paraphrase sentence that satisfies the given syntactic structure without changing the source sentences’ semantics. The commonly utilized syntactic structures are part-of-speech (POS) sequence, constituency parse tree, and masked template, with constituency parse tree achieving State-of-The-Art (SOTA) performance because of its rich syntactic information. As a result, further mining of syntactic information in parse trees has grown popular, yet fewer works pay attention to investigating semantic information in source sentences. A sentence is made up of two parts: syntax and semantics. Multiple studies have shown that improving the model’s ability to learn semantic information is critical for paraphrase construction as well as syntax learning. In this paper, we propose Fusing Semantic Information for Syntax-guided Paraphrase Generation (FS-SPG). Specifically, we propose a transformer-based semantic encoder to obtain detailed semantics from source sentences. This encoder contains a Semantics-Aware Attention mechanism for mining semantic information. In addition, we apply contrastive learning to improve the accuracy of parse tree nodes’ guidance to semantic sentences. Experiments on ParaNMT and QQP-Pos show that our model beats the SOTA model SI-SCP by 4.92% in syntactic metrics and 1.35% in semantic metrics.

Learning hierarchical embedding space for image-text matching

There are two mainstream strategies for image-text matching at present. The one, termed as joint embedding learning, aims to model the semantic information of both image and sentence in a shared feature subspace, which facilitates the measurement of semantic similarity but only focuses on global alignment relationship. To explore the local semantic relationship more fully, the other one, termed as metric learning, aims to learn a complex similarity function to directly output score of each image-text pair. However, it significantly suffers from more computation burden at retrieval stage. In this paper, we propose a hierarchically joint embedding model to incorporate the local semantic relationship into a joint embedding learning framework. The proposed method learns the shared local and global embedding spaces simultaneously, and models the joint local embedding space with respect to specific local similarity labels which are easy to access from the lexical information of corpus. Unlike the methods based on metric learning, we can prepare the fixed representations of both images and sentences by concatenating the normalized local and global representations, which makes it feasible to perform the efficient retrieval. And experiments show that the proposed model can achieve competitive performance when compared to the existing joint embedding learning models on two publicly available datasets Flickr30k and MS-COCO.

RB-GAT: A Text Classification Model Based on RoBERTa-BiGRU with Graph ATtention Network.

With the development of deep learning, several graph neural network (GNN)-based approaches have been utilized for text classification. However, GNNs encounter challenges when capturing contextual text information within a document sequence. To address this, a novel text classification model, RB-GAT, is proposed by combining RoBERTa-BiGRU embedding and a multi-head Graph ATtention Network (GAT). First, the pre-trained RoBERTa model is exploited to learn word and text embeddings in different contexts. Second, the Bidirectional Gated Recurrent Unit (BiGRU) is employed to capture long-term dependencies and bidirectional sentence information from the text context. Next, the multi-head graph attention network is applied to analyze this information, which serves as a node feature for the document. Finally, the classification results are generated through a Softmax layer. Experimental results on five benchmark datasets demonstrate that our method can achieve an accuracy of 71.48%, 98.45%, 80.32%, 90.84%, and 95.67% on Ohsumed, R8, MR, 20NG and R52, respectively, which is superior to the existing nine text classification approaches.

The Use of Particles in Ngapak Bumiayu Language: A Study of Systemic Functional Linguistics

This study explores the use of particles in Ngapak Bumiayu, a Javanese dialect spoken in Bumiayu, Brebes, Central Java, using Michael A.K. Halliday's Systemic Functional Linguistics theory. Data was collected from everyday conversations of native speakers using a descriptive analysis method. The identified particles—kok, lho, kok ya, toh, wis, and bar—each have unique roles in sentence structure and meaning. The ideational metafunction shows that these particles enrich the information in sentences. Without particles, sentences present straightforward facts, but with particles, they carry additional nuances that enhance the message. For the interpersonal metafunction, particle placement affects meaning. Kok can be in any sentence position, conveying surprise or emphasis. Lho emphasizes attention. Kok ya, typically at the beginning or middle, shows wonder and affirmation. Toh, usually at the end, indicates finality. Wis, in the middle or end, signifies completion or agreement. Bar indicates ongoing or recent actions. The textual metafunction highlights particles' role in maintaining text coherence and cohesion. They ensure consistency and continuity in discourse through repetition, serving as connectors (referential cohesion), and maintaining consistent meanings across contexts (lexical cohesion). These particles are essential for structuring and organizing information in sentences and conversations. These findings illustrate the significant role of particles in enhancing communication, structuring information, and providing nuanced meanings in the Ngapak Bumiayu language.

SAW-GAN: Multi-granularity Text Fusion Generative Adversarial Networks for text-to-image generation

Text-to-image generation is a challenging task that aims to generate visually realistic images semantically consistent for a given text. Existing methods mainly exploit the global semantic information of a single sentence while ignoring fine-grained semantic information such as aspects and words, which are critical factors in bridging the semantic gap in text-to-image generation. We propose a Multi-granularity Text (Sentence-level, Aspect-level, and Word-level) Fusion Generative Adversarial Network (SAW-GAN), which comprehensively represents textual information from multiple granularities. To effectively fuse multi-granularity information, we design a Double-granularity-text Fusion Module (DFM) fusing sentence and aspect information through parallel affine transformation and a Triple granularity-text Fusion Module (TFM) fusing sentence, aspect and word information by designing a novel Coordinate Attention Module (CAM), which can precisely locate the visual areas associated with each aspect and word. Furthermore, we use CLIP (Contrastive Language-Image Pre-training) to provide visual information to bridge the semantic gap and improve the model’s generalization ability. Our results show significant performance improvements over state-of-the-art methods using Conditional Generation Adversarial Network (CGAN) on CUB (FID from 13.91 to 10.45) and COCO (FID from 14.60 to 11.17) datasets with photorealistic images of richer details and text–image consistency.

The influence of sentence focus on mental simulation: A possible cause of ACE instability.

Recent studies have revealed the instability of the action-sentence compatibility effect (ACE). The current study was designed to demonstrate the hypothesis that the instability of the ACE may be attributed to the instability of focused information in a sentence. A pilot study indicated that the focused information of sentences was relatively stable in the sentence-picture verification task but exhibited significant interindividual variability in the action-sentence compatibility paradigm in previous studies. Experiments 1 and 2 examined the effect of sentence focus on the shape match effect and the ACE by manipulating the focused information of sentences using the focus marker word "" (is). Experiment 1 found that the shape match effect occurred in the original sentence, while it disappeared when the word "" (is) was used to make an object noun no longer the focus of a sentence. Experiment 2 failed to observe the ACE regardless of whether the sentence focus was on the action information. Experiment 3 modified the focus manipulation to observe its impact on the ACE using different fonts and underlines to highlight the focused information. The results indicated that the ACE only occurred when the action information was the sentence focus. These findings suggest that sentence focus influences mental simulation, and the instability of the ACE is likely to be associated with the instability of sentence focus in previous studies. This outcome highlights the crucial role of identifying specific information as the critical element expressed in the current linguistic context for successful simulation.

Fake News Classification Using Tensor Decomposition and Graph Convolutional Network

With the widespread of fake news on social media, its impact has become a major concern of the public, so accurate detection methods are urgently needed. As content-based features are the most natural clues for fake news detection, many methods that rely solely on text content have been proposed. However, these methods rarely investigate the sentence interaction patterns of different news articles, and most of them do not consider fine-grained fake news classification. To overcome these issues, this article constructs a graph representation for news articles and employs a graph neural network (GNN) to classify fake news. The proposed method uses the local word co-occurrence information of sentences to obtain the interaction relationship between sentences, which is abstracted by the weight matrix of the graph representation. Accordingly, a third-order co-occurrence tensor is built, and the weight matrix is calculated based on the canonical polyadic (CP) decomposition of this tensor. Since our method considers the sentence interaction patterns, the computed representations can capture more accurate contextual information of news articles. The results on two real-world datasets demonstrate that our method outperforms the competing methods in both binary and multiclass classification tasks. In particular, for multiclass classification on the selected dataset with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$70$</tex-math> </inline-formula> % of the training set for training, the improvements of accuracy and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$</tex-math> </inline-formula> -score are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.82$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.98$</tex-math> </inline-formula> %p and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.85$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.65$</tex-math> </inline-formula> %p, respectively.