Sentence Extraction Research Articles

Kazakh Text Summarization using Fuzzy logic

In this paper we present an extractive summarization method for the Kazakh language based on fuzzy logic. We aimed to extract and concatenate important sentences from the primary text to obtain its shorter form. With the rapid growth of information on the Internet thereis a demand on its efficient and cost-effective summarization. Thereforethe creation of automatic summarization methods is considered as a very important task of natural language processing. Our approach is based on the preprocessing of the sentences by applying morphological analysis and pronoun resolution techniques in order to avoid their early rejections. Afterwards, we determine the features of the processed sentences need for exploiting fuzzy logic methods. Additionally, since there is no available data for the given task, we collected and manually annotated our own dataset from the different Internet resources in the Kazakh language for the experimentation. We also applied our method on CNN/Daily Maildataset. The ROUGE-N indicators were calculated to assess the quality of the proposed method. The ROUGE-L(f-score) score by the proposed method with pronoun resolution for the former dataset is 0.40, whereas for the latter one it is 0.38.

Question-Led object attention for visual question answering

Question plays a leading role for Visual Question Answering (VQA) because it specifies the particular visual objects or conjures vivid visual that the machine should attend. However, existing approaches predominantly predict the answer by utilizing the question and the whole image without considering the leading role of the question. Also, recent object spatial inference is usually conducted on pixel level instead of object level. Therefore, we propose a novel but simple framework, namely Question-Led Object Attention (QLOB), to improve the VQA performance by exploring question semantics, fine-grained object information, and the relationship between those two modalities. First, we extract sentence semantics by a question model, and utilize the efficient object detection network to obtain a global visual feature and local features from top r object region proposals. Second, our QLOB attention mechanism selects those question-related object regions. Third, we optimize question model and QLOB attention by a softmax classifier to predict the final answer. Extensive experimental results on three public VQA datasets demonstrate that our QLOB outperforms the state-of-the-arts.

EcForest: Extractive document summarization through enhanced sentence embedding and cascade forest

SummaryWe present EcForest, an extractive summarization model through Enhanced Sentence Embedding and Cascade Forest. Sentence representation is of great significance for many summarization methods. Bag‐of‐words mostly fails to grasp the semantics, and typical embedding models cannot capture more complex semantic features, such as polysemy and the meaning of a phrase, which is usually ignored by simply averaging the word embeddings included in a sentence. To this end, we propose Enhanced Sentence Embedding (ESE) model to solve such drawbacks via mapping several valid features to dense vectors. Essentially, the enhanced sentence embedding is a novel model for improving the distributed representation of sentence. Our sentence embedding model is universally applicable and it can be adapted to other NLP tasks. Moreover, deep forest is used as a sentence extraction algorithm for its robustness to the hyper‐parameters and its efficient training algorithm compared to deep neural network. The evaluation of variant models proposed in this work proves the validation of the enhanced sentence embedding. The comparison results between EcForest and several baselines on two different datasets demonstrate that the proposed summarization model performs better than or with high competitiveness to the state‐of‐the‐art.

Movie Question Answering via Textual Memory and Plot Graph

Movies provide us with a mass of visual content as well as attracting stories. Existing methods have illustrated that understanding movie stories through only visual content is still a hard problem. In this paper, for answering questions about movies, we introduce a new dataset called PlotGraphs, as external knowledge. The dataset contains massive graph-based information of movies. In addition, we put forward a model that can utilize movie clip, subtitle, and graph-based external knowledge. The model contains two main parts: a layered memory network (LMN) and a plot graph representation network (PGRN). In particular, the LMN can represent frame-level and clip-level movie content by the fixed word memory module and the adaptive subtitle memory module, respectively. And the plot graph representation network can represent the entire graph. We first extract words and sentences from the training movie subtitles and then the hierarchically formed movie representations, which are learned from LMN. At the same time, the PGRN can represent the semantic information and the relationships in the graph. We conduct extensive experiments on the MovieQA dataset and the PlotGraphs dataset. With only visual content as inputs, the LMN with frame-level representation obtains a large performance improvement. When incorporating subtitles into LMN to form the clip-level representation, we achieve the state-of-the-art performance on the online evaluation task of “Video+Subtitles.” After the integration of external knowledge, the performance of the model consisting of the LMN and the PGRN is further improved. The good performance successfully demonstrates that the external knowledge and the proposed model are effective for movie understanding.

Recognizing sentences concerning future research from the full text of JASIST

ABSTRACTIn the conclusion part of an academic paper, sentences concerning future research provide an outlook about future focus of research and reflect the latest development trend in a specific topic or field. These sentences are of high value for further investigation. However, very little has been done on sentences talking about future research in academic full texts. Here, papers in JASIST were studied with respect to these sentences. Deep learning model was trained on articles containing sentences annotated as relating to “future research” by BERT model. The model capable of automatic extraction of such sentences from academic full texts was obtained. Later this model was verified on the unannotated corpus, which demonstrated good performance. Finally, clustering was performed for the extracted sentences. It was found by cluster analysis that the sentences concerning future research were divided into four types. Our findings provide support for classification of sentences concerning future research.

A Pipeline Approach to Free‐Description Question Answering in Chinese Gaokao Reading Comprehension

This study attempted to answer complicated free-description questions in Chinese Gaokao Reading comprehension (RC) tasks. We found that quite a few questions can be answered by extracting sentences from the document and combining them, so we used a pipeline approach with two components: Answer sentence extraction (ASE) and Answer sentence fusion (ASF). Semantic vector similarity and topical distribution similarity were explored for ASE. Integer linear programming strategy was used for ASF, which combined dependencies with the language model, based on word importance. As a first step towards the new challenge, we obtained some encouraging results on actual exam questions in Chinese subject's RC tasks of Beijing Gaokao, which helped us obtain insights into techniques needed to solve real-word complex questions.

Multi-Channel CNN Based Inner-Attention for Compound Sentence Relation Classification

Relation classification is a vital task in natural language processing, and it is screening for semantic relation between clauses in texts. This paper describes a study of relation classification on Chinese compound sentences without connectives. There exists an implicit relation in a compound sentence without connectives, which makes it difficult to realize the recognition of relation. The major challenges that relation classification modeling faces are how to obtain the contextual representation of sentence and relation dependence features between clauses. To solve this problem, we propose a novel Inatt-MCNN model to extract sentence features and classify relations by combining multi-channel CNN and Inner-attention mechanism. This network structure utilizes CNN to extract local features of sentences and Inner-attention to capture sentence-level feature representations for this relation classification task. Besides, since the Inner-attention is based on Bi-LSTM, the global and long-term dependence semantic information can be well obtained in Inatt-MCNN to promote the model performance. We conduct experiments on two public Chinese discourse datasets: the Chinese compound sentence corpus (CCCS) dataset and the Tsinghua Chinese Treebank(TCT) dataset. Compared with the previous public methods, Inatt-MCNN model has superior performance and achieves the highest accuracy, especially on the CCCS dataset.

Predication Transformation: A parallel corpus-based study of Russian verbless sentences and their English translations

<p class="Default">Artykuł przedstawia opartą na korpusie, kontrastywną analizę predykacji związanej z tłuma­czeniem zdań bezczasownikowych z języka rosyjskiego na język angielski w oparciu o pilotażowy paralelny korpus składający się z opartego na dialogach języka rosyjskiego <em>Braci Karamazow </em>Fiodora Dostojewskiego (1880) i angielskiego tłumaczenia Peveara i Volokhonsky <em>The Brothers Karamazov </em>(1990). W przeciwieństwie do angielskiego, znanego z zależności od frazy werbalnej z osobową formą czasownika, rosyjski pozwala na użycie zdań bezczasownikowych bardziej pro­duktywnie niż jakikolwiek inny język indoeuropejski (McShane, 2000; Kopotev, 2007). Łącząc założenia tekstu paralelnego w odniesieniu do językoznawstwa kontrastywnego opracowane przez Guillemin-Flescher (2003) z nową metodą automatycznej ekstrakcji zdań bezczasownikowych, niniejsze studium bada powtarzające się wzorce dotyczące sposobu, w jaki struktury językowe zyskują lub tracą wartość predykacji w tłumaczeniu. Po automatycznej segmentacji, morfosyn­taktycznej adnotacji i ekstrakcji zdania bezczasownikowych i ich tłumaczeniowe odpowiedniki są ręcznie przypisywane do typu predykacji werbalnej i niewerbalnej zgodnie z definicjami Hen­gevelda (1992). Wyniki przedstawiają typologię zjawiska zwanego “transformacją predykacyjną”, w której odpowiedniki translacyjne są efektem transformacji ze względu na typ predykacyjny. Wyniki ilościowe ukazują, w jakim stopniu czasowniki są wykorzystywane w tłumaczeniu zdań bezczasownikowych z języka rosyjskiego na język angielski, a także są wskaźnikiem werbalizacji predykatów. Twierdzimy, że powiązane z czasownikiem pojęcie semantycznej predykacji nie jest stabilne międzyjęzykowo.</p>

Improving Extractive Multi-Document Text Summarization Through Multi-Objective Optimization

Multi-document summarization is an optimization problem demanding optimization of more than one objective function simultaneously. The proposed work regards balancing of the two significant objectives: content coverage and diversity when generating summaries from a collection of text documents. Any automatic text summarization system has the challenge of producing high quality summary. Despite the existing efforts on designing and evaluating the performance of many text summarization techniques, their formulations lack the introduction of any model that can give an explicit representation of – coverage and diversity – the two contradictory semantics of any summary . In this work, the design of generic text summarization model based on sentence extraction is redirected into more semantic measure reflecting individually both content coverage and content diversity as two explicit optimization models. The problem is defined by projecting the first criterion, i.e. content coverage in the light of text similarity. The proposed model hypothesizes a possible decomposition of text similarity into three different levels of optimization formula. First, aspire to global optimization, the candidate summary should cover the summary of the document collection. Then, to attain, less global optimization, the sentences of the candidate summary should cover the summary of the document collection. The third level of optimization is content with local optimization, where the difference between the magnitude of terms covered by the candidate summary and those of the document collection should be small. This coverage model is coupled with a proposed diversity model and defined as a Multi-Objective Optimization ( MOO ) problem. Moreover, heuristic perturbation and heuristic local repair operators have been proposed and injected into the adopted evolutionary algorithm to harness its strength. Assessment of the proposed model has been performed using document sets supplied by Document Understanding Conference 2002 ( ) and a comparison has been made with other state-of-the-art methods. Metric used to measure performance of the proposed work is Recall-Oriented Understudy for Gisting Evaluation ( ) toolkit. Results obtained support strong proof for the effectiveness and the significant performance awarded to the proposed MOO model over other state-of-the-art models.

PISTON: Predicting drug indications and side effects using topic modeling and natural language processing

The process of discovering novel drugs to treat diseases requires a long time and high cost. It is important to understand side effects of drugs as well as their therapeutic effects, because these can seriously damage the patients due to unexpected actions of the derived candidate drugs. In order to overcome these limitations, computational methods for predicting the therapeutic effects and side effects have been proposed. In particular, text mining is a widely used technique in the field of systems biology, because it can discover hidden relationships between drugs, genes and diseases from a large amount of literature data. Compared with in vivo/in vitro experiments, text mining derives meaningful results with less time and cost. In this study, we propose an algorithm for predicting novel drug-phenotype associations and drug-side effect associations using topic modeling and natural language processing (NLP). We extract sentences in which drugs and genes co-occur from the abstracts of the literature and identify words that describe the relationship between them using NLP. Considering the characteristics of the identified words, we determine if the drug has an up-regulation effect or a down-regulation effect on the gene. Based on genes that affect drugs and their regulatory relationships, we group the frequently occurring genes and regulatory relationships into topics, and build a drug-topic probability matrix by calculating the score that the drug will have a topic using topic modeling. Using the matrix, a classifier is constructed for predicting the novel indications and side effects of drugs considering the characteristics of known drug-phenotype associations or drug-side effect associations. The proposed method predicts both indications and side effects with a single algorithm, and it can exclude drugs with serious side effects or side effects that patients do not want to experience from among the candidate drugs provided for the treatment of the phenotype. Furthermore, lists of novel candidate drugs for phenotypes and side effects can be continuously updated with our algorithm every time a document is added. More than a thousand documents are produced per day, and it is possible for our algorithm to efficiently derive candidate drugs because it requires less cost than the existing drug repositioning methods. The resource of PISTON is available at databio.gachon.ac.kr/tools/PISTON.

Extractive summarization using semigraph (ESSg)

Summary is the meaningful concise version of a text document. Generally existing statistical, knowledge based and discourse based extractive summarization methods use sentence similarity to extract informative sentences. This paper presents an innovative application of semigraph which includes the processes of semigraph construction and sentence extraction. Multilevel association among significant features of the text document can be represented using semigraph. Multi vertices property of semigraph helps in finding linear and nonlinear relationship between features. Some variation in semigraph in context of text document is proposed in this paper. The threshold for sentence length is calculated dynamically based on the sentence score. Challenge of measuring and analyzing performance is countered using proposed HIT ratio and ROUGE measures. Substantial experiments on benchmark dataset demonstrate that the proposed solution achieves encouraging performance. Multi directed mapping among summaries generated, using existing method is used to calculate effective index.

Exploiting User Posts for Web Document Summarization

Relevant user posts such as comments or tweets of a Web document provide additional valuable information to enrich the content of this document. When creating user posts, readers tend to borrow salient words or phrases in sentences. This can be considered as word variation. This article proposes a framework that models the word variation aspect to enhance the quality of Web document summarization. Technically, the framework consists of two steps: scoring and selection. In the first step, the social information of a Web document such as user posts is exploited to model intra-relations and inter-relations in lexical and semantic levels. These relations are denoted by a mutual reinforcement similarity graph used to score each sentence and user post. After scoring, summaries are extracted by using a ranking approach or concept-based method formulated in the form of Integer Linear Programming. To confirm the efficiency of our framework, sentence and story highlight extraction tasks were taken as a case study on three datasets in two languages, English and Vietnamese. Experimental results show that: (i) the framework can improve ROUGE-scores compared to state-of-the-art baselines of social context summarization and (ii) the combination of the two relations benefits the sentence extraction of single Web documents.

Movie Question Answering: Remembering the Textual Cues for Layered Visual Contents

Movies provide us with a mass of visual content as well as attracting stories. Existing methods have illustrated that understanding movie stories through only visual content is still a hard problem. In this paper, for answering questions about movies, we put forward a Layered Memory Network (LMN) that represents frame-level and clip-level movie content by the Static Word Memory module and the Dynamic Subtitle Memory module, respectively. Particularly, we firstly extract words and sentences from the training movie subtitles. Then the hierarchically formed movie representations, which are learned from LMN, not only encode the correspondence between words and visual content inside frames, but also encode the temporal alignment between sentences and frames inside movie clips. We also extend our LMN model into three variant frameworks to illustrate the good extendable capabilities. We conduct extensive experiments on the MovieQA dataset. With only visual content as inputs, LMN with frame-level representation obtains a large performance improvement. When incorporating subtitles into LMN to form the clip-level representation, we achieve the state-of-the-art performance on the online evaluation task of 'Video+Subtitles'. The good performance successfully demonstrates that the proposed framework of LMN is effective and the hierarchically formed movie representations have good potential for the applications of movie question answering.

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Automatic multi-document summarization needs to find representative sentences not only by sentence distribution to select the most important sentence but also by how informative a term is in a sentence. Sentence distribution is suitable for obtaining important sentences by determining frequent and well-spread words in the corpus but ignores the grammatical information that indicates instructive content. The presence or absence of informative content in a sentence can be indicated by grammatical information which is carried by part of speech (POS) labels. In this paper, we propose a new sentence weighting method by incorporating sentence distribution and POS tagging for multi-document summarization. Similarity-based Histogram Clustering (SHC) is used to cluster sentences in the data set. Cluster ordering is based on cluster importance to determine the important clusters. Sentence extraction based on sentence distribution and POS tagging is introduced to extract the representative sentences from the ordered clusters. The results of the experiment on the Document Understanding Conferences (DUC) 2004 are compared with those of the Sentence Distribution Method. Our proposed method achieved better results with an increasing rate of 5.41% on ROUGE-1 and 0.62% on ROUGE-2.

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Automatic sentence extraction for the detection of scientific paper relations

The relations between scientific papers are very useful for researchers to see the interconnection between scientific papers quickly. By observing the inter-article relationships, researchers can identify, among others, the weaknesses of existing research, performance improvements achieved to date, and tools or data typically used in research in specific fields. So far, methods that have been developed to detect paper relations include machine learning and rule-based methods. However, a problem still arises in the process of sentence extraction from scientific paper documents, which is still done manually. This manual process causes the detection of scientific paper relations longer and inefficient. To overcome this problem, this study performs an automatic sentences extraction while the paper relations are identified based on the citation sentence. The performance of the built system is then compared with that of the manual extraction system. The analysis results suggested that the automatic sentence extraction indicates a very high level of performance in the detection of paper relations, which is close to that of manual sentence extraction.

ASHuR: Evaluation of the Relation Summary-Content Without Human Reference Using ROUGE

In written documents, the summary is a brief description of important aspects of a text. The degree of similarity between the summary and the content of a document provides reliability about the summary. Some efforts have been done in order to automate the evaluation of a summary. ROUGE metrics can automatically evaluate a summary, but it needs a model summary built by humans. The goal of this study is to find a quantitative relation between an article content and its summary using ROUGE tests without a model summary built by humans. This work proposes a method for automatic text summarization to evaluate a summary (ASHuR) based on extraction of sentences. ASHuR extracts the best sentences of an article based on the frequency of concepts, cue-words, title words, and sentence length. Extracted sentences constitute the essence of the article; these sentences construct the model summary. We performed two experiments to assess the reliability of ASHuR. The first experiment compared ASHuR against similar approaches based on sentences extraction; the experiment placed ASHuR in the first place in each applied test. The second experiment compared ASHuR against human-made summaries, which yielded a Pearson correlation value of 0.86. Assessments made to ASHuR show reliability to evaluate summaries written by users in collaborative sites (e.g. Wikipedia) or to review texts generated by students in online learning systems (e.g. Moodle).

A resource-light method for cross-lingual semantic textual similarity

Recognizing semantically similar sentences or paragraphs across languages is beneficial for many tasks, ranging from cross-lingual information retrieval and plagiarism detection to machine translation. Recently proposed methods for predicting cross-lingual semantic similarity of short texts, however, make use of tools and resources (e.g., machine translation systems, syntactic parsers or named entity recognition) that for many languages (or language pairs) do not exist. In contrast, we propose an unsupervised and a very resource-light approach for measuring semantic similarity between texts in different languages. To operate in the bilingual (or multilingual) space, we project continuous word vectors (i.e., word embeddings) from one language to the vector space of the other language via the linear translation model. We then align words according to the similarity of their vectors in the bilingual embedding space and investigate different unsupervised measures of semantic similarity exploiting bilingual embeddings and word alignments. Requiring only a limited-size set of word translation pairs between the languages, the proposed approach is applicable to virtually any pair of languages for which there exists a sufficiently large corpus, required to learn monolingual word embeddings. Experimental results on three different datasets for measuring semantic textual similarity show that our simple resource-light approach reaches performance close to that of supervised and resource-intensive methods, displaying stability across different language pairs. Furthermore, we evaluate the proposed method on two extrinsic tasks, namely extraction of parallel sentences from comparable corpora and cross-lingual plagiarism detection, and show that it yields performance comparable to those of complex resource-intensive state-of-the-art models for the respective tasks.

Ontology-Based Sentence Extraction for Answering Why-Question

Ontology-Based Sentence Extraction for Answering Why-Question

Linguistic challenges in automatic summarization technology

Automatic summarization is a field of Natural Language Processing that is increasingly used in industry today. The goal of the summarization process is to create a summary of one document or a multiplicity of documents that will retain the sense and the most important aspects while reducing the length considerably, to a size that may be user-defined. One differentiates between extraction-based and abstraction-based summarization. In an extraction-based system, the words and sentences are copied out of the original source without any modification. An abstraction-based summary can compress, fuse or paraphrase sections of the source document. As of today, most summarization systems are extractive. Automatic document summarization technology presents interesting challenges for Natural Language Processing. It works on the basis of coreference resolution, discourse analysis, named entity recognition (NER), information extraction (IE), natural language understanding, topic segmentation and recognition, word segmentation and part-of-speech tagging. This study will overview some current approaches to the implementation of auto summarization technology and discuss the state of the art of the most important NLP tasks involved in them. We will pay particular attention to current methods of sentence extraction and compression for single and multi-document summarization, as these applications are based on theories of syntax and discourse and their implementation therefore requires a solid background in linguistics. Summarization technologies are also used for image collection summarization and video summarization, but the scope of this paper will be limited to document summarization.