Sentence Selection Research Articles

DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension

Multi-choice reading comprehension is a challenging task to select an answer from a set of candidate options when given passage and question. Previous approaches usually only calculate question-aware passage representation and ignore passage-aware question representation when modeling the relationship between passage and question, which cannot effectively capture the relationship between passage and question. In this work, we propose dual co-matching network (DCMN) which models the relationship among passage, question and answer options bidirectionally. Besides, inspired by how humans solve multi-choice questions, we integrate two reading strategies into our model: (i) passage sentence selection that finds the most salient supporting sentences to answer the question, (ii) answer option interaction that encodes the comparison information between answer options. DCMN equipped with the two strategies (DCMN+) obtains state-of-the-art results on five multi-choice reading comprehension datasets from different domains: RACE, SemEval-2018 Task 11, ROCStories, COIN, MCTest.

TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection

We propose TandA, an effective technique for fine-tuning pre-trained Transformer models for natural language tasks. Specifically, we first transfer a pre-trained model into a model for a general task by fine-tuning it with a large and high-quality dataset. We then perform a second fine-tuning step to adapt the transferred model to the target domain. We demonstrate the benefits of our approach for answer sentence selection, which is a well-known inference task in Question Answering. We built a large scale dataset to enable the transfer step, exploiting the Natural Questions dataset. Our approach establishes the state of the art on two well-known benchmarks, WikiQA and TREC-QA, achieving the impressive MAP scores of 92% and 94.3%, respectively, which largely outperform the the highest scores of 83.4% and 87.5% of previous work. We empirically show that TandA generates more stable and robust models reducing the effort required for selecting optimal hyper-parameters. Additionally, we show that the transfer step of TandA makes the adaptation step more robust to noise. This enables a more effective use of noisy datasets for fine-tuning. Finally, we also confirm the positive impact of TandA in an industrial setting, using domain specific datasets subject to different types of noise.

Capturing Sentence Relations for Answer Sentence Selection with Multi-Perspective Graph Encoding

This paper focuses on the answer sentence selection task. Unlike previous work, which only models the relation between the question and each candidate sentence, we propose Multi-Perspective Graph Encoder (MPGE) to take the relations among the candidate sentences into account and capture the relations from multiple perspectives. By utilizing MPGE as a module, we construct two answer sentence selection models which are based on traditional representation and pre-trained representation, respectively. We conduct extensive experiments on two datasets, WikiQA and SQuAD. The results show that the proposed MPGE is effective for both types of representation. Moreover, the overall performance of our proposed model surpasses the state-of-the-art on both datasets. Additionally, we further validate the robustness of our method by the adversarial examples of AddSent and AddOneSent.

Rule Based Extractive Summarization a nd Title Generation

Text summarying is a process by which the most important information from the source document is precisely found. It stands for the information condensed to a longer text. Text summary is broken down into two approaches: extractive summary and abstractive summary. The proposed method creates an extractive summary of a given text and generate an appropriate title for the generated summary. Extractive summary is generated through sentence selection by using Rule-based concept. Eight different features are considered to rank each sentence according to its importance. Ranking assigns a numerical measure to each sentence. After ranking, sentences that has high rank compared to others will be selected to form the summary. The frequently occurring bi-gram is selected as the title for the summary. The system performs better than existing extractive summarization techniques like Graph-based system and achieved a precision of 0.7

End-to-end response selection based on multi-level context response matching

This paper presents our work on the Dialog System Technology Challenges 7 (DSTC7). We took part in Track 1 on sentence selection which evaluates response retrieving in dialog systems on more realistic test scenarios compared to the state-of-the-art evaluations. Our proposed dialog system matches the context with the best response by computing their semantic similarity on word and sequence levels. Evaluation results on the datasets provided show the effectiveness of our system by achieving higher performance compared to the provided baseline system. Our system enjoys the advantages of its simple and end-to-end architecture making its training and adaptation to other domains easier.

Enhanced Language Modeling with Proximity and Sentence Relatedness Information for Extractive Broadcast News Summarization

The primary task of extractive summarization is to automatically select a set of representative sentences from a text or spoken document that can concisely express the most important theme of the original document. Recently, language modeling (LM) has been proven to be a promising modeling framework for performing this task in an unsupervised manner. However, there still remain three fundamental challenges facing the existing LM-based methods, which we set out to tackle in this article. The first one is how to construct a more accurate sentence model in this framework without resorting to external sources of information. The second is how to take into account sentence-level structural relationships, in addition to word-level information within a document, for important sentence selection. The last one is how to exploit the proximity cues inherent in sentences to obtain a more accurate estimation of respective sentence models. Specifically, for the first and second challenges, we explore a novel, principled approach that generates overlapped clusters to extract sentence relatedness information from the document to be summarized, which can be used not only to enhance the estimation of various sentence models but also to render sentence-level structural relationships within the document, leading to better summarization effectiveness. For the third challenge, we investigate several formulations of proximity cues for use in sentence modeling involved in the LM-based summarization framework, free of the strict bag-of-words assumption. Furthermore, we also present various ensemble methods that seamlessly integrate proximity and sentence relatedness information into sentence modeling. Extensive experiments conducted on a Mandarin broadcast news summarization task show that such integration of proximity and sentence relatedness information is indeed beneficial for speech summarization. Our proposed summarization methods can significantly boost the performance of an LM-based strong baseline (e.g., with a maximum ROUGE-2 improvement of 26.7% relative) and also outperform several state-of-the-art unsupervised methods compared in the article.

Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer

With increasing research interests in dialogue modeling, there is an emerging branch that formulates this task as next sentence selection, where given the partial dialogue context, the goal is to determine the most probable next sentence. To model natural language information, recurrent models have been applied to sequence modeling and shown promising results in various NLP tasks (Sutskever et al., 2014). Recently, the Transformer (Vaswani et al., 2017) has advanced modeling semantics for natural language sentences via attention, achieving improvement for sequence modeling. However, the Transformer focuses on modeling the intra-sentence attention but ignores inter-sentence information. In terms of dialogue modeling, the cross-sentence information is salient to understand dialogue content, so that the response selection can be better determined. Therefore, this paper proposes a novel attention mechanism based on multi-head attention, called highway attention, in order to allow the model to pass information through multiple sentences, and then builds a recurrent model based on the Transformer and the proposed highway attention. We call this model Highway Recurrent Transformer. This model focuses on not only intra-sentence dependency, but also inter-sentence dependency in the structure of dialogues. Experiments on the response selection task of the seventh Dialog System Technology Challenge (DSTC7) demonstrate that the proposed Highway Recurrent Transformer is capable of modeling both utterance-level and dialogue-level information for achieving better performance than the original Transformer in the single positive response scenario.

WEST GERMANY AND EAST GERMANY IN ACCORDANCE TO THE INTERPRETATION OF THE SIGNS IN THE SONG LYRICS OF OVER DE MUUR (1984)

This study discusses the lyrics of the song Over De Muur by Klein Orkest which was released in 1984. The purpose of this study is to discuss the situation of West Germany and East Germany in 1984 through the interpretation of the meaning of the signs contained in the Over De Muur song lyrics. The theory used in this research is the theory of myth by Roland Barthes. First, a selection of words, phrases or sentences is needed as a sign that reflected the state of West Germany and East Germany at that time. Then the analysis continued using the two steps semiosis process; The first is a primary analysis consisting of an analysis of denotative meanings and then in the secondary analysis consists of connotative meanings analysis to identify the myths that contained in the lyrics of Over De Muur . The results show that the song Over De Muur portrays that West Germany and East Germany have different ideologies, and although they have different ideologies, people in the two countries share a common goals that they both want to unite and are shown through the semantic aspects of the song's lyrics.

Overview of the seventh Dialog System Technology Challenge: DSTC7

This paper provides detailed information about the seventh Dialog System Technology Challenge (DSTC7) and its three tracks aimed to explore the problem of building robust and accurate end-to-end dialog systems. In more detail, DSTC7 focuses on developing and exploring end-to-end technologies for the following three pragmatic challenges: (1) sentence selection for multiple domains, (2) generation of informational responses grounded in external knowledge, and (3) audio visual scene-aware dialog to allow conversations with users about objects and events around them.This paper summarizes the overall setup and results of DSTC7, including detailed descriptions of the different tracks, provided datasets and annotations, overview of the submitted systems and their final results. For Track 1, LSTM-based models performed best across both datasets, allowing teams to effectively handle task variants where no correct answer was present or when multiple paraphrases were included. For Track 2, RNN-based architectures augmented to incorporate facts by using two types of encoders: a dialog encoder and a fact encoder plus using attention mechanisms and a pointer-generator approach provided the best results. Finally, for Track 3, the best model used Hierarchical Attention mechanisms to combine the text and vision information obtaining a 22% better result than the baseline LSTM system for the human rating score.More than 220 participants were registered and about 40 teams participated in the final challenge. 32 scientific papers reporting the systems submitted to DSTC7, and 3 general technical papers for dialog technologies, were presented during the one-day wrap-up workshop at AAAI-19. During the workshop, we reviewed the state-of-the-art systems, shared novel approaches to the DSTC7 tasks, and discussed the future directions for the challenge (DSTC8).

Sparse two level topic model for extraction of general summary words

Extractive multi-document summarization methods based on topic models find relevant general concepts or topics that are most representative of the documents. These topics are used for sentence ranking and selection. In this paper, a two level topic model using spike and slab prior is proposed that identify better general topics for summarization. Spike and slab prior is used earlier for finding aspect specific topics. Proposed two level model uses spike and slab prior to achieve better general topics at high level of topic hierarchy. Experiments conducted on DUC2007 dataset show that proposed model is able to identify more summary oriented general words and improve ROUGE score.

A Joint Sentence Scoring and Selection Framework for Neural Extractive Document Summarization

Extractive document summarization methods aim to extract important sentences to form a summary. Previous works perform this task by first scoring all sentences in the document then selecting most informative ones; while we propose to jointly learn the two steps with a novel end-to-end neural network framework. Specifically, the sentences in the input document are represented as real-valued vectors through a neural document encoder. Then the method builds the output summary by extracting important sentences one by one. Different from previous works, the proposed joint sentence scoring and selection framework directly predicts the relative sentence importance score according to both sentence content and previously selected sentences. We evaluate the proposed framework with two realizations: a hierarchical recurrent neural network based model; and a pre-training based model that uses BERT as the document encoder. Experiments on two datasets show that the proposed joint framework outperforms the state-of-the-art extractive summarization models which treat sentence scoring and selection as two subtasks.

A Near-Real-Time Answer Discovery for Open-Domain With Unanswerable Questions From the Web

With the proliferation of question and answering (Q&A) services, studies on building a knowledge base (KB) using various information extraction (IE) methodologies from unstructured data on the Web have received significant attention. Existing IE approaches, including machine reading comprehension (MRC), can find the correct answer to a question if the correct answer exists in the document. However, most are prone to extracting incorrect answers rather than producing no answers when the correct answer does not exist in the given documents. This problem is likely to cause serious real-world problems when we apply such technologies to practical services such as AI speakers. We propose a novel open-domain IE system to alleviate the weaknesses of previous approaches. The proposed system integrates an elaborated document selection, sentence selection, and knowledge extraction ensemble method to obtain high specificity while maintaining a realistically achievable level of precision. Based on this framework, we extract answers on Korean open-domain user queries from unstructured documents collected from multiple Web sources. For evaluating our system, we build a benchmark dataset with the SKTelecom AI Speaker log. The baseline models KYLIN infobox generator and BiDAF were used to evaluate the performance of the proposed approach. The experimental results demonstrate that the proposed method outperforms the baseline models and is practically applicable to real-world services.

Automatic Keyword and Sentence-Based Text Summarization for Software Bug Reports

Text Summarization is a process which efficiently retrieves the relevant information from documents. The objective of the proposed, unsupervised approach is to summarize bug reports (software artefacts) with complete content and diversified information. The proposed approach utilizes Rapid Automatic Keyword Extraction and term frequency-inverse document frequency method to extract meaningful keywords and key-phrases with a relevant score. For sentence extraction, fuzzy C-means clustering is used to extracts sentences having high degree of membership from each cluster above a set threshold value. A rule-engine is used for sentence selection. The rules are generated with the domain knowledge and based on the extracted information by the keywords and sentences selected by the clustering method. Cohesive and coherent summary is generated by the proposed method on apache bug reports. For redundancy removal and to re-rank generated summary, hierarchical clustering is presented to enrich the extracted summary. The proposed approach is evaluated on newly constructed Apache project Bug Report Corpus (APBRC) and existing Bug Report Corpus (BRC). The results are compared on the basis of performance metrics such as precision, recall, pyramid precision and F-score. The experimental results depict that our proposed approach attains significant improvement over other baseline approaches such as BRC and LRCA. It also attains significant improvement over existing state-of-art unsupervised approaches such as Hurried, centroid and others. It extracts significant keyword phrases and sentences from each cluster to achieve full coverage and coherent summary. The results evaluated on APBRC corpus attains an average value of 78.22%, 82.18%, 80.10% and 81.66% for precision, recall, f-score and pyramid precision respectively.

Topic-based automatic summarization algorithm for Chinese short text.

Most current automatic summarization methods are for English texts. The distinction between words in Chinese text is large, the types of parts of speech are many and complex, and polysemy or ambiguous words appear frequently. Therefore, compared with English text, Chinese text is more difficult to extract useful feature words. Due to the complex syntax of Chinese, there are currently relatively few automatic summarization methods for Chinese text. In the past, only the important sentences in the original text can be selected and simply arranged to obtain a summary with chaotic sentences and insufficient coherence. Meanwhile, because Chinese short text usually contains more redundant information and the sentence structure is not neat, we propose a topic-based automatic summary method for Chinese short text. Firstly, a key sentence selection method is proposed combining topic words and TF-IDF to obtain the score of each text corresponding to the topic in the original text data. Then the sentence with the highest score as the topic sentence of the topic is selected. Considering that the short text of Weibo may contain a lot of irrelevant information and sometimes even lack some important components of topic, three retouching mechanisms are proposed to improve the conciseness, richness and readability of topic sentence extraction results. We validate our approach on natural disaster and social hot event datasets from Sina Weibo. The experimental results show that the polished topic summary not only reflects the exact relationship between topic sentences and natural disasters or social hot events, but also has rich semantic information. More importantly, we can almost grasp the basic elements of natural disaster or social hot event from the topic sentence, so as to help the government guide disaster relief or meet the needs of users for quickly obtaining information of social hot events.

Extracting Deep Personae Social Relations in Microblog Posts

Numerous studies have been conducted to extract relationships from different documents. However, extracting relationships from microblog posts is rarely studied. In this paper, we improve a novel kernel-based learning algorithm to mine the personae social relationships from microblog posts by combining the syntax and semantic meanings of the dependency trigram kernels (DTK). To deeply extract the personal social relationships of microblog posts, we define the relation feature words, provide seven rules for extracting these feature words, and propose a rule-based approach that mines these relation feature words from microblog posts. We construct relation feature word dictionaries for different relation types because of the lack of prominent relation features in microblog posts. We propose an algorithm to classify relation feature words by considering two features of the relation feature words, namely, syntax and semantic similarities between relation feature words in microblog posts and by using relation feature word dictionaries. Experimental results show that the average recall, precision, and F-measure of our proposed approach outperforms the original DTK in sentence selection, personae social relation extraction, and personae social relation classification. Finally, the relation graphs of five topics clarify that our proposed approach is effective for extracting personae social relations from microblog posts.

An evaluation of sentence selection methods on the different phone-sized units for constructing Indonesian speech corpus

Collecting phonetically balanced text corpus is an important step to develop automatic speech recognition and text-to-speech systems. A corpus should have a small number of sentences but contains all phonetic units, such as monophone, triphone, and pentaphone units. There are exist least-to-most greedy algorithm (LTM + Greedy) and its variant to select the minimum sentence set. The variant is on the sentence scoring method, which affect the number of selected sentences. In this paper, we evaluate the sentence scoring methods by Zhang and Suyanto on LTM + Greedy algorithm. The sentence scoring methods are conducted on triphone and pentaphone units on the collection of sentence set. Triphone and pentaphone units have offered higher quality synthesized speech than monophone unit. The dataset of this paper is Indonesian sentences that collected from holy book translation, news, novel, dialog, monologue, and question sentences. Totally 115,489 sentences are used for the experiments. Based on the experiments, LTM + Greedy by Suyanto produces a smaller number of sentences that contain large number of phone units.

A neural graph embedding approach for selecting review sentences

Abstract Product reviews written by the crowd on e-commerce shopping websites have become a critical information source for making purchasing decisions. An important challenge, however, is that the vast majority of products (e.g., 90 % of products on amazon.com) do not receive enough attention and lack sufficient reviews by the users; hence, they constitute the so-called cold products. One solution to address cold products, which has already been studied in the literature, is to generate reviews for these products by sampling review sentences from closely related warm products. Our method proposed in this paper is specifically focused on such a solution. While a majority of the works in the literature rely on product specification similarity to identify relevant reviews that can be used for review sentence selection, our work differs in that it not only employs product specification similarity but also employs product-review, product-user, and user-review interactions when determining the suitability of a review sentence to be selected. More specifically, the contributions of our work can be enumerated as follows: (1) We propose that the selection of review sentences from other products should not only consider product-product similarity but also consider product-review, user-review, and user-user relationships. As such, we show how neural graph embeddings can be used to encode product, user, and review information into an attributed heterogeneous graph representation based on which similarities can be calculated. (2) We further propose how review relevance and importance can be considered using graph traversal to select appropriate review sentences for a given cold product. (3) Finally, we systematically compare the performance of our work with those of several state-of-the-art baselines on five datasets collected from CNET.com and rottentomatoes.com with different characteristics from both quantitative (e.g., the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics) and qualitative aspects and show how our proposed approach was able to provide statistically significantly improved performance over various strong baselines.

Children’s Story Classification in Indian Languages Using Linguistic and Keyword-based Features

The primary objective of this work is to classify Hindi and Telugu stories into three genres: fable, folk-tale, and legend . In this work, we are proposing a framework for story classification (SC) using keyword and part-of-speech (POS) features. For improving the performance of SC system, feature reduction techniques and combinations of various POS tags are explored. Further, we investigated the performance of SC by dividing the story into parts depending on its semantic structure. In this work, stories are (i) manually divided into parts based on their semantics as introduction, main, and climax ; and (ii) automatically divided into equal parts based on number of sentences in a story as initial, middle, and end . We have also examined sentence increment model, which aims at determining an optimum number of sentences required to identify story genre by incremental selection of sentences in a story. Experiments are conducted on Hindi and Telugu story corpora consisting of 300 and 150 short stories, respectively. The performance of SC system is evaluated using different combinations of keyword and POS-based features, with three well-established machine learning classifiers: (i) Naive Bayes (NB), (ii) k-Nearest Neighbour (KNN), and (iii) Support Vector Machine (SVM). Performance of the classifier is evaluated using 10-fold cross-validation and effectiveness of classifier is measured using precision, recall, and F-measure. From the classification results, it is observed that adding linguistic information boosts the performance of story classification. In view of the structure of the story, main, and initial parts of the story have shown comparatively better performance. The results from the sentence incremental model have indicated that the first nine and seven sentences in Hindi and Telugu stories, respectively, are sufficient for better classification of stories. In most of the studies, SVM models outperformed the other models in classification accuracy.

Crosslinguistic Differences in the Encoding of Causality: Transitivity Preferences in English and Japanese Children and Adults

ABSTRACTLanguages differ in how they encode causal events, placing greater or lesser emphasis on the agent or patient of the action. Little is known about how these preferences emerge and the relative influence of cognitive biases and language-specific input at different stages in development. In these studies, we investigated the emergence of sentence preferences to describe causal events in English- and Japanese-speaking children (aged three and five years) and compared this to preferences displayed by adults. We studied two factors suggested to influence this choice: Language (Corpus study & Experiment 1) and Intentionality (Experiment 2). Participants watched videos depicting familiar and novel causal actions, and made a best-match choice between a transitive and intransitive description. We found a stronger preference for intransitive sentences with causal verbs and more frequent argument omissions in Japanese child-directed speech than in English child-directed speech. The trajectory of acquisition in the selection of transitive sentences for causal events differed between languages. For intentionality, with familiar verbs both Japanese and English speakers selected fewer transitives for accidental than intentional scenes, but this pattern was more pronounced in Japanese speakers. However, with novel verbs, only five-year-olds and adults showed this preference. These data provide important new information to constrain theories about the process of learning to map event structure to language, and its interdependence with intentionality and the distributional properties of linguistic input to children.

Predicting and Integrating Expected Answer Types into a Simple Recurrent Neural Network Model for Answer Sentence Selection

Since end-to-end deep learning models have started to replace traditional pipeline architectures of question answering systems, features such as expected answer types which are based on the question semantics are seldom used explicitly in the models. In this paper, we propose a convolution neural network model to predict these answer types based on question words and a recurrent neural network model to find sentence similarity scores between question and answer sentences. The proposed model outperforms the current state of the art results on an answer sentence selection task in open domain question answering by 1.88% on MAP and 2.96% on MRR scores.