Sentence Extraction Research Articles

FRGEM: Feature integration pre-training based Gaussian embedding model for Chinese word representation

Most Chinese word representation approaches mine the lexeme of Chinese words by predicting the target word or the contextual words. However, Chinese words may have multiple meanings, above ways may only capture one aspect implication. In this paper, for better modeling multiple connotations of Chinese words, a Feature integRation pre-training based Gaussian Embedding Model (FRGEM) is proposed with three stages, feature integration pre-training, relevancy Gaussian representation and similarity-based training. In the first stage, the internal sequences with three inner-character stroke, structure and pinyin features of Chinese words are generated to learn their semantic relevancy, then we combine BERT with internal sequences to extract sentence relevance and integrate the internal and sentence features to learn pre-training representations. In the second stage, relevancy Gaussian representation with pre-training embeddings is proposed to fuse inner-character features with Gaussian through estimating its mean to analyze multiple implications of Chinese words. In the third stage, a similarity-based objective is proposed to distinguish true contextual words and learn Chinese word representations. Extensive experiments on word similarity, word analogy, named entity recognition and text classification show that FRGEM is superior to majority state-of-the-art algorithms.

Comparative Analysis of Semantic and Syntactic Approaches in Automatic Text Summarization: A Comprehensive Review and Evaluation

It is very important to use automatic text summarization to turn long chunks of text into short, useful snippets. In the area of automatic text summary, this study looks at both semantic and syntactic methods in great detail. Using methods like natural language understanding and semantic processing, semantic approaches try to get meaning and context from writing. These methods try to get at the main ideas and connections in the material so that recaps can be more complex and full of information. Syntactic methods, on the other hand, focus on the structure of language, such as grammar and syntax. To find important sentences and phrases for summarization, methods like sentence extraction based on grammar patterns and grammatical analysis are used. Syntactic methods often make sure that the outlines they create are grammatically right and make sense. This review compares both methods in a number of areas, such as how well they capture the spirit of the original text, how well the generated snippets make sense, how quickly they can be produced, and how well they can be used with different types of material. The pros and cons of each method are shown in comparative studies, which help us understand how they can be used in various areas and situations. Recent improvements that combine semantic and syntactic methods have also shown promise in making automatic recaps better overall and more helpful. The goal of these mixed methods is to make reports that are both verbally and semantically correct by using the benefits of both semantic understanding and syntactic structure. This review gives us a more complete picture of how automatic text summarization methods are changing. It stresses how important it is to look at both semantic and syntactic approaches in order to make the field move forward and create better and more advanced summary systems.

Aspect category sentiment analysis based on multiple attention mechanisms and pre-trained models

The rapid rise of e-commerce platforms has changed people's shopping habits, driving the popularity of online shopping. Users express their opinions on products and services by purchasing products on platforms and posting comments. These comment data contain rich user experiences, which are crucial for enterprises to understand user needs and improve product quality. Sentiment analysis of comment text is an important research direction in text mining, focusing on how to extract user evaluations of products from comment data to provide comprehensive, authentic, and accurate product feedback. This paper mainly investigates aspect-based fine-grained sentiment analysis. Because users express multiple sentiments towards different aspects in comments, coarse-grained sentiment analysis cannot accurately capture users' sentiment tendencies. This study utilizes artificial intelligence technology based on deep learning, firstly, it constructs auxiliary training samples to transform aspect sentiment tasks into machine reading comprehension or language inference tasks, using BERT model to extract text features and sentence features from comment data. To guide the model to focus on the features most relevant to the given aspect, a cross-attention mechanism is utilized to cross-focus the features of text with aspect category features. Finally, the sentiment polarity of given aspect category text can be predicted through a forward network. Experimental results on multiple datasets demonstrate that this method outperforms other deep learning models.

SIGNSense: Auditory -Optic Impairment Communication Bridge

Language experts have recognized sign languages as natural languages with the ability to convey human emotions and ideas. Translation from written language into sign videos or extraction of spoken language sentences from sign videos is the aim of sign language translation. Sign language is the principal means of communication for the deaf and hard of hearing community, which comprises 32 million children and 328 million adults worldwide who suffer from hearing impairment. However, the inability of current systems to accurately translate and transmit sign language motions in real-time prevents effective and spontaneous communication. This research provides a revolutionary technique that enables real-time recognition of ISL gestures by integrating natural language processing with cross-modal integration. The methodology uses cutting-edge methods like the Single Shot Multibox Detector (SSD) with MobileNetV2 architecture for data collection, preprocessing, model selection, and training. In real-time inference, the trained model attains an impressive 94% accuracy rate, showcasing strong performance and encouraging outcomes for enhancing communication accessibility for people with hearing impairments

Automated Extraction and Analysis of Sentences under Production: A Theoretical Framework and Its Evaluation

Sentences are generally understood to be essential communicative units in writing that are built to express thoughts and meanings. Studying sentence production provides a valuable opportunity to shed new light on the writing process itself and on the underlying cognitive processes. Nevertheless, research on the production of sentences in writing remains scarce. We propose a theoretical framework and an open-source implementation that aim to facilitate the study of sentence production based on keystroke logs. We centre our approach around the notion of sentence history: all the versions of a given sentence during the production of a text. The implementation takes keystroke logs as input and extracts sentence versions, aggregates them into sentence histories and evaluates the sentencehood of each sentence version. We provide detailed evaluation of the implementation based on a manually annotated corpus of texts in French, German and English. The implementation yields strong results on the three processing aspects.

LERCause: Deep learning approaches for causal sentence identification from nuclear safety reports.

Identifying causal sentences from nuclear incident reports is essential for advancing nuclear safety research and applications. Nonetheless, accurately locating and labeling causal sentences in text data is challenging, and might benefit from the usage of automated techniques. In this paper, we introduce LERCause, a labeled dataset combined with labeling methods meant to serve as a foundation for the classification of causal sentences in the domain of nuclear safety. We used three BERT models (BERT, BioBERT, and SciBERT) to 10,608 annotated sentences from the Licensee Event Report (LER) corpus for predicting sentence labels (Causal vs. non-Causal). We also used a keyword-based heuristic strategy, three standard machine learning methods (Logistic Regression, Gradient Boosting, and Support Vector Machine), and a deep learning approach (Convolutional Neural Network; CNN) for comparison. We found that the BERT-centric models outperformed all other tested models in terms of all evaluation metrics (accuracy, precision, recall, and F1 score). BioBERT resulted in the highest overall F1 score of 94.49% from the ten-fold cross-validation. Our dataset and coding framework can provide a robust baseline for assessing and comparing new causal sentences extraction techniques. As far as we know, our research breaks new ground by leveraging BERT-centric models for causal sentence classification in the nuclear safety domain and by openly distributing labeled data and code to enable reproducibility in subsequent research.

Leveraging Transformer Summarizer to Extract Sentences for Arabic Text Summarization

Automatic Text Summarization (ATS) is one of the fastest-growing areas of Artificial Intelligence (AI), Machine Learning (ML), and Natural Language Processing (NLP). Automatic text summarizing aims to create summaries by extracting relevant information from input texts. Being able to study text non-sequentially allowed for large model training, making the Transformer the most widely used NLP model. This study focuses on extractive Arabic text summarization using various transformer-based summarization models. Two distinct scenarios are explored: summarization extracted from scored sentences and summarization generated from ranked (reversed) sentences. Experimental results demonstrate the effectiveness and performance of the transformer-based summarization models in the context of Arabic text summarization. The EASC corpus is used to evaluate our proposed summarization approach and ROUGE evaluation method is applied to determine the accuracy of the proposed approach. The findings of the study demonstrate notable performance when compared to other relevant works. The comparison between the two scenarios provides insights into the strengths and weaknesses of each approach. The scored sentence extraction scenario offers a more direct and content-focused summary generation, while the ranked sentence scenario provides flexibility in summarizing diverse text sources.

Intent recognition model based on sequential information and sentence features

In human–computer dialogue systems, intent recognition is crucial for determining the intentions or purposes of users during interactions with the system, enabling the system to provide appropriate responses or actions. This paper proposes an intent recognition model that integrates sequential information and sentence structural features. Specifically, the approach utilizes a CNN to capture local salient features in the text, followed by a BILSTM to extract sequential information within the local context. The sequential information is then fed into a multi-head attention mechanism to focus on more relevant sequential details. Additionally, the original data is processed by BERT to extract sentence structural features. Finally, the sequential information features and sentence structural features are concatenated and fused to achieve enhanced intent recognition performance. This approach effectively leverages contextual and semantic information within the text, leading to improved accuracy in intent recognition. Experimental results demonstrate the effectiveness of the proposed method in intent recognition and its high relevance for practical applications.

Discovering new applications: Cross-domain exploration of patent documents using causal extraction and similarity analysis

Determining a technology’s potential applications is crucial in assessing its level of innovation and evaluating its commercial viability. However, patent documents offer limited insights into a technology’s full potential. As a solution, this study suggests an approach to explore a technology’s applicability beyond what is explicitly stated in patents. The approach employs causal extraction to extract sentences expressing technologies and their applications from patents, followed by deep learning-based similarity analysis to compare the similarity of these sentences. Experimental results show its effectiveness in extracting sentences about technologies and applications and its superiority in terms of F1 score compared to benchmark models. This study enables cross-domain comparisons of technologies and applications, identifies multiple prospective applications for a given technology, and offers new opportunities for patent value analysis and intellectual property management in the industry. A cross-domain application network of the proposed method demonstrates how to find all potential cross-domain connections of a given data and we provide open access to the code.

The Impact of Collaborative Documentation on Person-Centered Care: Textual Analysis of Clinical Notes.

Collaborative documentation (CD) is a behavioral health practice involving shared writing of clinic visit notes by providers and consumers. Despite widespread dissemination of CD, research on its effectiveness or impact on person-centered care (PCC) has been limited. Principles of PCC planning, a recovery-based approach to service planning that operationalizes PCC, can inform the measurement of person-centeredness within clinical documentation. This study aims to use the clinical informatics approach of natural language processing (NLP) to examine the impact of CD on person-centeredness in clinic visit notes. Using a dictionary-based approach, this study conducts a textual analysis of clinic notes from a community mental health center before and after staff were trained in CD. This study used visit notes (n=1981) from 10 providers in a community mental health center 6 months before and after training in CD. LIWC-22 was used to assess all notes using the Linguistic Inquiry and Word Count (LIWC) dictionary, which categorizes over 5000 linguistic and psychological words. Twelve LIWC categories were selected and mapped onto PCC planning principles through the consensus of 3 domain experts. The LIWC-22 contextualizer was used to extract sentence fragments from notes corresponding to LIWC categories. Then, fixed-effects modeling was used to identify differences in notes before and after CD training while accounting for nesting within the provider. Sentence fragments identified by the contextualizing process illustrated how visit notes demonstrated PCC. The fixed effects analysis found a significant positive shift toward person-centeredness; this was observed in 6 of the selected LIWC categories post CD. Specifically, there was a notable increase in words associated with achievement (β=.774, P<.001), power (β=.831, P<.001), money (β=.204, P<.001), physical health (β=.427, P=.03), while leisure words decreased (β=-.166, P=.002). By using a dictionary-based approach, the study identified how CD might influence the integration of PCC principles within clinical notes. Although the results were mixed, the findings highlight the potential effectiveness of CD in enhancing person-centeredness in clinic notes. By leveraging NLP techniques, this research illuminated the value of narrative clinical notes in assessing the quality of care in behavioral health contexts. These findings underscore the promise of NLP for quality assurance in health care settings and emphasize the need for refining algorithms to more accurately measure PCC.

The role of natural language processing techniques versus conventional methods to gain ICI safety insights from unstructured EHR data.

136 Background: Studies using International Classification of Diseases (ICD) codes have aimed to characterize the safety profile of cancer therapies in the Real-World Data setting, but this approach has limitations including an inability to illustrate patient journey and offer insights into drug-Adverse Event (AE) causality. Using data from a cohort of ~9000 patients treated with immune checkpoint inhibitors (ICI) at Mayo Clinic, we recently demonstrated the feasibility of applying Augmented Curation (AC), a natural language processing (NLP)-based innovation, on unstructured electronic health record (EHR) data to detect and characterize immune related AEs (irAEs). While the ability to identify and extract clinical details from EHRs has been previously demonstrated, our approach focuses on applying these techniques to identify implied textual causality of drug-event pairs. This study aims to build upon our prior findings and compare the accuracy, speed, and resource savings enabled by AC analysis versus manual review of patient notes. Methods: We compared the prevalence of adrenal insufficiency, colitis, and hypophysitis (A.C.H) among ICI patients at Mayo Clinic using ICD codes and AC which leverages SciBERT to perform scientific NLP tasks. AC models trained to identify drug-adverse event relationships were used to assess irAEs and subsequent treatments and demonstrated strong performance with F1 scores of 0.85. Manual curation of relevant clinical data, i.e. medications and adverse events, was employed to create a “gold standard” A.C.H. cohort. Time required for manual review compared to AC was documented for overall comparison. Corticosteroid/immunosuppressant use and ICI discontinuations were used as proxies of severity. Results: In our cohort, 540 unique patients experienced at least one of the A.C.H. irAEs. A.C.H patients were found to receive corticosteroids for their respective irAE 79% of the time and discontinued the ICI due to the irAE 7.7% of the time. 6.1% of Colitis patients were treated with a 2L immunosuppressant. Similar to the previous study, AC showed higher sensitivity, PPV and NPV compared to ICD codes alone within the A.C.H. cohort. Sentence extraction and entity classification using AC models on all 9,000-patients took ~10 minutes, whereas manual review of this cohort would take roughly 9 weeks (~10 minutes per patient). Conclusions: The results from this study reinforce that AC models are a valuable tool to comprehensively extract information and to impute relationships between entities, such as therapies and AEs. The AC models demonstrated better accuracy at a much greater speed compared to manual review of patient EHRs. Leveraging AC models provides the ability to manipulate unstructured clinical text married to structured EHR data at scale while minimizing need for manual curation.

Maximum Marginal Relevance and Vector Space Model for Summarizing Students' Final Project Abstracts

Automatic summarization is reducing a text document with a computer program to create a summary that retains the essential parts of the original document. Automatic summarization is necessary to deal with information overload, and the amount of data is increasing. A summary is needed to get the contents of the article briefly. A summary is an effective way to present extended information in a concise form of the main contents of an article, and the aim is to tell the reader the essence of a central idea. The simple concept of a summary is to take an essential part of the entire contents of the article. Which then presents it back in summary form. The steps in this research will start with the user selecting or searching for text documents that will be summarized with keywords in the abstract as a query. The proposed approach performs text preprocessing for documents: sentence breaking, case folding, word tokenizing, filtering, and stemming. The results of the preprocessed text are weighted by term frequency-inverse document frequency (tf-idf), then weighted for query relevance using the vector space model and sentence similarity using cosine similarity. The next stage is maximum marginal relevance for sentence extraction. The proposed approach provides comprehensive summarization compared with another approach. The test results are compared with manual summaries, which produce an average precision of 88%, recall of 61%, and f-measure of 70%.

An Entity based Approach for Extracting Relationship between Chinese Characters

Knowledge extraction includes the extraction of entity relationships, which is crucial. Deep learning algorithms are increasingly prevalent in relation extraction tasks when compared to conventional pattern recognition techniques. The majority of current research on remotely supervised methods and kernel functions in relationship extraction strategies for Chinese language cannot disregard the detrimental effects of noisy data in the dataset on the experimental findings. A bidirectional GRU neural network, a two-layer attention mechanism, and a Chinese relationship extraction model are proposed. For the forgetfulness problem, a bidirectional GRU neural network is utilised to fuse the input vectors. Integrating the structural properties of the Mandarin language, word vectors are employed as the input. Sentence-level attention mechanisms are utilised to extract sentence features, and word-level feature information is retrieved from a sentence. About 1300 bits of data are extracted from news websites using a remotely supervised methodology for validation. According to the experimental findings, the neural network model with a two-layer attention mechanism is able to fully utilise all of the feature information in a sentence, and it performs significantly better than the neural network model without an attention mechanism in terms of accuracy and recall rates. Compared to the model without the attention mechanism, the accuracy and recall rates are noticeably greater.

Mining heuristic evidence sentences for more interpretable document-level relation extraction

Current research on evidence sentences is aimed at developing document-level relational extraction models with improved interpretability. Evidence sentences extracted using existing methods are often incomplete, leading to poor relationship prediction accuracy. To address this problem, we developed a novel efficient heuristic rule and entity representation method. First, a heuristic rule is constructed according to the interactions between different mentions of the head and tail entities of the target entity pair, and evidence sentences are subsequently extracted. Second, pseudo documents, constructed according to the original document order, are used as input text to remove noisy statements. Finally, different representations of the same entity in different entity pairs are learned to represent it more accurately through the interactive mention of head and tail entities. Experiments on the document-level general domain dataset DocRED indicated that our heuristic rules improved sentence extraction by 6.01% compared to that achieved by the baseline model Paths-BiLSTM. In terms of relation prediction, the accuracy of the proposed method was comparable to those of existing models that use the entire document as input text; however, the input text used by the proposed method was shorter and more interpretable.

Attention-based Sentence Extraction for Aspect-based Sentiment Analysis with Implicit Aspect Cases in Hotel Review Using Machine Learning Algorithm, Semantic Similarity, and BERT

Attention-based Sentence Extraction for Aspect-based Sentiment Analysis with Implicit Aspect Cases in Hotel Review Using Machine Learning Algorithm, Semantic Similarity, and BERT

Automatic Extraction of Comprehensive Drug Safety Information from Adverse Drug Event Narratives in the Korea Adverse Event Reporting System Using Natural Language Processing Techniques.

Concerns have been raised over the quality of drug safety information, particularly data completeness, collected through spontaneous reporting systems (SRS), although regulatory agencies routinely use SRS data to guide their pharmacovigilance programs. We expected that collecting additional drug safety information from adverse event (ADE) narratives and incorporating it into the SRS database would improve data completeness. The aims of this study were to define the extraction of comprehensive drug safety information from ADE narratives reported through the Korea Adverse Event Reporting System (KAERS) as natural language processing (NLP) tasks and to provide baseline models for the defined tasks. This study used ADE narratives and structured drug safety information from individual case safety reports (ICSRs) reported through KAERS between 1 January 2015 and 31 December 2019. We developed the annotation guideline for the extraction of comprehensive drug safety information from ADE narratives based on the International Conference on Harmonisation (ICH) E2B(R3) guideline and manually annotated 3723 ADE narratives. Then, we developed a domain-specific Korean Bidirectional Encoder Representations from Transformers (KAERS-BERT) model using 1.2 million ADE narratives in KAERS and provided baseline models for the task we defined. In addition, we performed an ablation experiment to investigate whether named entity recognition (NER) models were improved when a training dataset contained more diverse ADE narratives. We defined 21 types of word entities, six types of entity labels, and 49 types of relations to formulate the extraction of comprehensive drug safety information as NLP tasks. We obtained a total of 86,750 entities, 81,828 entity labels, and 45,107 relations from manually annotated ADE narratives. The KAERS-BERT model achieved F1-scores of 83.81 and 76.62% on the NER and sentence extraction tasks, respectively, while outperforming other baseline models on all the NLP tasks we definedexcept the sentence extraction task. Finally, utilizing the NER model for extracting drug safety information from ADE narratives resulted in an average increase of 3.24% in data completeness for KAERS structured data fields. We formulated the extraction of comprehensive drug safety information from ADE narratives as NLP tasks and developed the annotated corpus and strong baseline models for the tasks. The annotated corpus and models for extracting comprehensive drug safety information can improve the data quality of an SRS database.

Optimizing Multidocument Summarization by Blending Reinforcement Learning Policies

We consider extractive summarization within a cluster of related texts (multi-document summarization). Unlike single-document summarization, redundancy is particularly important because sentences across related documents might convey overlapping information. Thus, sentence extraction in such a setting is difficult because one will need to determine which pieces of information are relevant while avoiding unnecessary repetitiveness. To solve this difficult problem, we propose a novel reinforcement learning based method <inline-formula><tex-math notation="LaTeX">$\mathbf{PoBRL}$</tex-math></inline-formula> (<b>Po</b>licy <b>B</b>lending with maximal marginal relevance and <b>R</b>einforcement <b>L</b>earning) for solving multi-document summarization. PoBRL jointly optimizes over the following objectives necessary for a high-quality summary: importance, relevance, and length. Our strategy decouples this multi-objective optimization into different sub-problems that can be solved individually by reinforcement learning. Utilizing PoBRL, we then blend each learned policies to produce a summary that is a concise and a complete representation of the original input. Our empirical analysis shows high performance on several multi-document datasets. Human evaluation also shows that our method produces high-quality output.

Chinese–Vietnamese Pseudo-Parallel Sentences Extraction Based on Image Information Fusion

Parallel sentences play a crucial role in various NLP tasks, particularly for cross-lingual tasks such as machine translation. However, due to the time-consuming and laborious nature of manual construction, many low-resource languages still suffer from a lack of large-scale parallel data. The objective of pseudo-parallel sentence extraction is to automatically identify sentence pairs in different languages that convey similar meanings. Earlier methods heavily relied on parallel data, which is unsuitable for low-resource scenarios. The current mainstream research direction is to use transfer learning or unsupervised learning based on cross-lingual word embeddings and multilingual pre-trained models; however, these methods are ineffective for languages with substantial differences. To address this issue, we propose a sentence extraction method that leverages image information fusion to extract Chinese–Vietnamese pseudo-parallel sentences from collections of bilingual texts. Our method first employs an adaptive image and text feature fusion strategy to efficiently extract the bilingual parallel sentence pair, and then, a multimodal fusion method is presented to balance the information between the image and text modalities. The experiments on multiple benchmarks show that our method achieves promising results compared to a competitive baseline by infusing additional external image information.

Automatic generation of short-answer questions in reading comprehension using NLP and KNN.

In general, making evaluations requires a lot of time, especially in thinking about the questions and answers. Therefore, research on automatic question generation is carried out in the hope that it can be used as a tool to generate question and answer sentences, so as to save time in thinking about questions and answers. This research focuses on automatically generating short answer questions in the reading comprehension section using Natural Language Processing (NLP) and K-Nearest Neighborhood (KNN). The questions generated use article sources from news with reliable grammar. To maintain the quality of the questions produced, machine learning methods are also used, namely by conducting training on existing questions. The stages of this research in outline are simple sentence extraction, problem classification, generating question sentences, and finally comparing candidate questions with training data to determine eligibility. The results of the experiment carried out were for the Grammatical Correctness parameter to produce a percentage of 59.52%, for the Answer Existence parameter it yielded 95.24%, while for the Difficulty Index parameter it produced a percentage of 34.92%. So that the resulting average is 63.23%. So, this software deserves to be used as an alternative to automatically create reading comprehension questions.

Improving unsupervised keyphrase extraction by modeling hierarchical multi-granularity features

Existing unsupervised keyphrase extraction methods typically emphasize the importance of the candidate keyphrase itself, ignoring other important factors such as the influence of uninformative sentences. We hypothesize that the salient sentences of a document are particularly important as they are most likely to contain keyphrases, especially for long documents. To our knowledge, our work is the first attempt to exploit sentence salience for unsupervised keyphrase extraction by modeling hierarchical multi-granularity features. Specifically, we propose a novel position-aware graph-based unsupervised keyphrase extraction model, which includes two model variants. The pipeline model first extracts salient sentences from the document, followed by keyphrase extraction from the extracted salient sentences. In contrast to the pipeline model which models multi-granularity features in a two-stage paradigm, the joint model accounts for both sentence and phrase representations of the source document simultaneously via hierarchical graphs. Concretely, the sentence nodes are introduced as an inductive bias, injecting sentence-level information for determining the importance of candidate keyphrases. We compare our model against strong baselines on three benchmark datasets including Inspec, DUC 2001, and SemEval 2010. Experimental results show that the simple pipeline-based approach achieves promising results, indicating that keyphrase extraction task benefits from the salient sentence extraction task. The joint model, which mitigates the potential accumulated error of the pipeline model, gives the best performance and achieves new state-of-the-art results while generalizing better on data from different domains and with different lengths. In particular, for the SemEval 2010 dataset consisting of long documents, our joint model outperforms the strongest baseline UKERank by 3.48%, 3.69% and 4.84% in terms of F1@5, F1@10 and F1@15, respectively. We also conduct qualitative experiments to validate the effectiveness of our model components.