Answer Sentence Research Articles

Select, Answer and Explain: Interpretable Multi-Hop Reading Comprehension over Multiple Documents

Interpretable multi-hop reading comprehension (RC) over multiple documents is a challenging problem because it demands reasoning over multiple information sources and explaining the answer prediction by providing supporting evidences. In this paper, we propose an effective and interpretable Select, Answer and Explain (SAE) system to solve the multi-document RC problem. Our system first filters out answer-unrelated documents and thus reduce the amount of distraction information. This is achieved by a document classifier trained with a novel pairwise learning-to-rank loss. The selected answer-related documents are then input to a model to jointly predict the answer and supporting sentences. The model is optimized with a multi-task learning objective on both token level for answer prediction and sentence level for supporting sentences prediction, together with an attention-based interaction between these two tasks. Evaluated on HotpotQA, a challenging multi-hop RC data set, the proposed SAE system achieves top competitive performance in distractor setting compared to other existing systems on the leaderboard.

TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection

We propose TandA, an effective technique for fine-tuning pre-trained Transformer models for natural language tasks. Specifically, we first transfer a pre-trained model into a model for a general task by fine-tuning it with a large and high-quality dataset. We then perform a second fine-tuning step to adapt the transferred model to the target domain. We demonstrate the benefits of our approach for answer sentence selection, which is a well-known inference task in Question Answering. We built a large scale dataset to enable the transfer step, exploiting the Natural Questions dataset. Our approach establishes the state of the art on two well-known benchmarks, WikiQA and TREC-QA, achieving the impressive MAP scores of 92% and 94.3%, respectively, which largely outperform the the highest scores of 83.4% and 87.5% of previous work. We empirically show that TandA generates more stable and robust models reducing the effort required for selecting optimal hyper-parameters. Additionally, we show that the transfer step of TandA makes the adaptation step more robust to noise. This enables a more effective use of noisy datasets for fine-tuning. Finally, we also confirm the positive impact of TandA in an industrial setting, using domain specific datasets subject to different types of noise.

Capturing Sentence Relations for Answer Sentence Selection with Multi-Perspective Graph Encoding

This paper focuses on the answer sentence selection task. Unlike previous work, which only models the relation between the question and each candidate sentence, we propose Multi-Perspective Graph Encoder (MPGE) to take the relations among the candidate sentences into account and capture the relations from multiple perspectives. By utilizing MPGE as a module, we construct two answer sentence selection models which are based on traditional representation and pre-trained representation, respectively. We conduct extensive experiments on two datasets, WikiQA and SQuAD. The results show that the proposed MPGE is effective for both types of representation. Moreover, the overall performance of our proposed model surpasses the state-of-the-art on both datasets. Additionally, we further validate the robustness of our method by the adversarial examples of AddSent and AddOneSent.

Building interactive sentence-aware representation based on generative language model for community question answering

Semantic matching between question and answer sentences involves recognizing whether a candidate answer is relevant to a particular input question. Given the fact that such matching does not examine a question or an answer individually, context information outside the sentence should be considered equally important to the within-sentence syntactic context. This motivates us to design a new question-answer matching model, built upon a cross-sentence, context-aware, bi-directional long short-term memory architecture. The interactive attention mechanisms are proposed which automatically select salient positional sentence representations, that contribute more significantly towards the relevance between two question and answer. A new quantity called context information jump is proposed to facilitate the formulation of the attention weights, and is computed via the joint states of adjacent words. An interactive-aware sentence representation is constructed by connecting a combination of multiple sentence positional representations to each hidden representation state. In the experiments, the proposed method is compared with existed models, using four public community datasets, and the evaluations show that it is very competitive. In particular, it offers 0.32%-1.8% improvement over the best performing model for three out of four datasets, while for the remaining one performance is around 0.2% of the best performer.

Bio-AnswerFinder: a system to find answers to questions from biomedical texts.

The ever accelerating pace of biomedical research results in corresponding acceleration in the volume of biomedical literature created. Since new research builds upon existing knowledge, the rate of increase in the available knowledge encoded in biomedical literature makes the easy access to that implicit knowledge more vital over time. Toward the goal of making implicit knowledge in the biomedical literature easily accessible to biomedical researchers, we introduce a question answering system called Bio-AnswerFinder. Bio-AnswerFinder uses a weighted-relaxed word mover's distance based similarity on word/phrase embeddings learned from PubMed abstracts to rank answers after question focus entity type filtering. Our approach retrieves relevant documents iteratively via enhanced keyword queries from a traditional search engine. To improve document retrieval performance, we introduced a supervised long short term memory neural network to select keywords from the question to facilitate iterative keyword search. Our unsupervised baseline system achieves a mean reciprocal rank score of 0.46 and Precision@1 of 0.32 on 936 questions from BioASQ. The answer sentences are further ranked by a fine-tuned bidirectional encoder representation from transformers (BERT) classifier trained using 100 answer candidate sentences per question for 492 BioASQ questions. To test ranking performance, we report a blind test on 100 questions that three independent annotators scored. These experts preferred BERT based reranking with 7% improvement on MRR and 13% improvement on Precision@1 scores on average.

Predicting and Integrating Expected Answer Types into a Simple Recurrent Neural Network Model for Answer Sentence Selection

Since end-to-end deep learning models have started to replace traditional pipeline architectures of question answering systems, features such as expected answer types which are based on the question semantics are seldom used explicitly in the models. In this paper, we propose a convolution neural network model to predict these answer types based on question words and a recurrent neural network model to find sentence similarity scores between question and answer sentences. The proposed model outperforms the current state of the art results on an answer sentence selection task in open domain question answering by 1.88% on MAP and 2.96% on MRR scores.

Alias Identification from Narratives for Question Answering

Alias is the term used to give a person’s name after giving his or her real name. The name can be Pronouns or Generic (Noun Phrases) NPs. For identifying pronouns co-reference resolution is being used. Identifying Generic NPs from a narrative is a crucial task, because we observed that standard Natural Language Processing (NLP) tool-kits work poorly for generic NPs. NLP applications using the co-reference resolution technique are Timeline creation, Question-answering, Summarization and Information extraction. In this work, we propose an approach using linguistic knowledge for Alias Identification and Question Answering system to answer the given question. For encoding the linguistic knowledge Markov Logic Network (MLN) is used. The proposed system is a combination of Alias Identification and a Question Answering. Public narratives/paragraph of varying linguistic complexity is selected as input to the alias identification system. Input to the question answering system is output alias identification system and question. From the narratives/paragraph answers to the corresponding questions will be extracted. The system generates the answer sentence by finding the first pronoun or generic NP which is replaced with canonical form.

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering.

Deep learning is the crucial technology in intelligent question answering research tasks. Nowadays, extensive studies on question answering have been conducted by adopting the methods of deep learning. The challenge is that it not only requires an effective semantic understanding model to generate a textual representation but also needs the consideration of semantic interaction between questions and answers simultaneously. In this paper, we propose a stacked Bidirectional Long Short-Term Memory (BiLSTM) neural network based on the coattention mechanism to extract the interaction between questions and answers, combining cosine similarity and Euclidean distance to score the question and answer sentences. Experiments are tested and evaluated on publicly available Text REtrieval Conference (TREC) 8-13 dataset and Wiki-QA dataset. Experimental results confirm that the proposed model is efficient and particularly it achieves a higher mean average precision (MAR) of 0.7613 and mean reciprocal rank (MRR) of 0.8401 on the TREC dataset.

Transformer-Based Neural Network for Answer Selection in Question Answering

Answer selection is a crucial subtask in the question answering (QA) system. Conventional avenues for this task mainly concentrate on developing linguistic tools that are limited in both performance and practicability. Answer selection approaches based on deep learning have been well investigated with the tremendous success of deep learning in natural language processing. However, the traditional neural networks employed in existing answer selection models, i.e., recursive neural network or convolutional neural network, typically suffer from obtaining the global text information due to their operating mechanisms. The recent Transformer neural network is considered to be good at extracting the global information by employing only self-attention mechanism. Thus, in this paper, we design a Transformer-based neural network for answer selection, where we deploy a bidirectional long short-term memory (BiLSTM) behind the Transformer to acquire both global information and sequential features in the question or answer sentence. Different from the original Transformer, our Transformer-based network focuses on sentence embedding rather than the seq2seq task. In addition, we employ a BiLSTM rather than utilizing the position encoding to incorporate sequential features as the universal Transformer does. Furthermore, we apply three aggregated strategies to generate sentence embeddings for question and answer, i.e., the weighted mean pooling, the max pooling, and the attentive pooling, leading to three corresponding Transformer-based models, i.e., QA-TF $_{{W\!P}}$ , QA-TF $_{{M\!P}}$ , and QA-TF $_{{A\!P}}$ , respectively. Finally, we evaluate our proposals on a popular QA dataset WikiQA. The experimental results demonstrate that our proposed Transformer-based answer selection models can produce a better performance compared with several competitive baselines. In detail, our best model outperforms the state-of-the-art baseline by up to 2.37%, 2.83%, and 3.79% in terms of MAP, MRR, and accuracy, respectively.

A Pipeline Approach to Free‐Description Question Answering in Chinese Gaokao Reading Comprehension

This study attempted to answer complicated free-description questions in Chinese Gaokao Reading comprehension (RC) tasks. We found that quite a few questions can be answered by extracting sentences from the document and combining them, so we used a pipeline approach with two components: Answer sentence extraction (ASE) and Answer sentence fusion (ASF). Semantic vector similarity and topical distribution similarity were explored for ASE. Integer linear programming strategy was used for ASF, which combined dependencies with the language model, based on word importance. As a first step towards the new challenge, we obtained some encouraging results on actual exam questions in Chinese subject's RC tasks of Beijing Gaokao, which helped us obtain insights into techniques needed to solve real-word complex questions.

Document Retrieval for Biomedical Question Answering with Neural Sentence Matching.

Document retrieval (DR) forms an important component in end-to-end question-answering (QA) systems where particular answers are sought for well-formed questions. DR in the QA scenario is also useful by itself even without a more involved natural language processing component to extract exact answers from the retrieved documents. This latter step may simply be done by humans like in traditional search engines granted the retrieved documents contain the answer. In this paper, we take advantage of datasets made available through the BioASQ end-to-end QA shared task series and build an effective biomedical DR system that relies on relevant answer snippets in the BioASQ training datasets. At the core of our approach is a question-answer sentence matching neural network that learns a measure of relevance of a sentence to an input question in the form of a matching score. In addition to this matching score feature, we also exploit two auxiliary features for scoring document relevance: the name of the journal in which a document is published and the presence/absence of semantic relations (subject-predicate-object triples) in a candidate answer sentence connecting entities mentioned in the question. We rerank our baseline sequential dependence model scores using these three additional features weighted via adaptive random research and other learning-to-rank methods. Our full system placed 2nd in the final batch of Phase A (DR) of task B (QA) in BioASQ 2018. Our ablation experiments highlight the significance of the neural matching network component in the full system.

Context-Aware Answer Sentence Selection With Hierarchical Gated Recurrent Neural Networks

In this paper, we study the task of reading comprehension style answer sentence selection that aims to select the best sentence from a given passage to answer a question. Unlike most previous works that match the question and each candidate sentence separately, we observe that the context information among sentences in the same passage plays a vital role in this task. We propose modeling context information with hierarchical gated recurrent neural networks. Specifically, we first apply a word level recurrent neural network to model the context independent matching between the question and each candidate sentence. We then employ a sentence level recurrent neural network to incorporate the context information among all candidate sentences. Moreover, we introduce the gate mechanism to select matching information before feeding into recurrent neural networks at both word and sentence level. Experiments on the WikiQA and SQuAD datasets show that our model outperforms state-of-the-art methods.

Distributed storage and recall of sentences

Abstract The human brain is able to learn language by processing written or spoken language. Recently, several deep neural networks have been successfully used for natural language generation. Although it is possible to train such networks, it remains unknown how these networks (or the brain) actually process language. A scalable method for distributed storage and recall of sentences within a neural network is presented. A corpus of 59 million words was used for training. A system using this method can efficiently identify sentences that can be considered reasonable replies to an input sentence. The system first selects a small number of seeds words which occur with low frequency in the corpus. These seed words are then used to generate answer sentences. Possible answers are scored using statistical data also obtained from the corpus. A number of sample answers generated by the system are shown to illustrate how the method works.

Utilizing typed dependency subtree patterns for answer sentence generation in question answering systems

Question Answering over Linked Data (QALD) refer to the use of Linked Data by question answering systems, and in recent times this has become increasingly popular as it opens up a massive Linked Data cloud which is a rich source of encoded knowledge. However, a major shortfall of current QALD systems is that they focus on presenting a single fact or factoid answer which is derived using SPARQL (SPARQL Protocol and RDF Query Language) queries. There is now an increased interest in development of human-like systems which would be able to answer questions and even hold conversations by constructing sentences akin to humans. In this paper, we introduce a new answer construction and presentation system, which utilizes the linguistic structure of the source question and the factoid answer to construct an answer sentence which closely emanates a human-generated answer. We employ both semantic Web technology and the linguistic structure to construct the answer sentences. The core of the research resides on extracting dependency subtree patterns from the questions and utilizing them in conjunction with the factoid answer to generate the answer sentence with a natural feel akin to an answer from a human when asked the question. We evaluated the system for both linguistic accuracy and naturalness using human evaluation. These evaluation processes showed that the proposed approach is able to generate answer sentences which have linguistic accuracy and natural readability quotients of more than 70%. In addition, we also carried out a feasibility analysis on using automatic metrics for answer sentence evaluation. The results from this phase showed that the there is not a strong correlation between the results from automatic metric evaluation and the human ratings of the machine-generated answers.

A Joint Model for Answer Sentence Ranking and Answer Extraction

Answer sentence ranking and answer extraction are two key challenges in question answering that have traditionally been treated in isolation, i.e., as independent tasks. In this article, we (1) explain how both tasks are related at their core by a common quantity, and (2) propose a simple and intuitive joint probabilistic model that addresses both via joint computation but task-specific application of that quantity. In our experiments with two TREC datasets, our joint model substantially outperforms state-of-the-art systems in both tasks.

Odor-induced mood state modulates language comprehension by affecting processing strategies

It is controversial whether mood affects cognition by triggering specific processing strategies or by limiting processing resources. The current event-related potential (ERP) study pursued this issue by examining how mood modulates the processing of task relevant/irrelevant information. In question-answer pairs, a question context marked a critical word in the answer sentence as focus (and thus relevant) or non-focus (thereby irrelevant). At the same time, participants were exposed to either a pleasant or unpleasant odor to elicit different mood states. Overall, we observed larger N400s when the critical words in the answer sentences were semantically incongruent (rather than congruent) with the question context. However, such N400 effect was only found for focused words accompanied by a pleasant odor and for both focused and non-focused words accompanied by an unpleasant odor, but not for non-focused words accompanied by a pleasant odor. These results indicate top-down attentional shift to the focused information in a positive mood state and non-selective attention allocated to the focused and non-focused information in a less positive mood state, lending support to the “processing strategy” hypothesis. By using a novel approach to induce mood states, our study provides fresh insights into the mechanisms underlying mood modulation of language comprehension.

Demonstrative Nouns of INI and AINI in Muna Language

The article titled “Demonstrative Nouns of ini and aini in Muna Language.” The purpose of this article is to analyze the form, meaning, and function of ini and aini in Muna language. The method used is descriptive qualitative with distributional technique. This method is used to describe and analyze the form, meaning, and function of ini and aini. The results of the analysis found that the form of ini is a base or monomorpheme, while the form of aini is a derivative or polymorpheme which consists of morpheme a- and morpheme aini. In construction of phrase, clause, or sentence generally have the same meaning, namely ‘this’. For instance, O lambu ini damasoe and lambu aini damasoe the meaning is ‘The house will be sold.’ However, when it’s examined more closely, both forms of demonstrative ini and aini in distributional are not interchangeable. For instance, *O lambu aini damasoe dan *Lambu ini damasoe. Both of constructions are ungrammatical caused by form ini has a meaning ‘this’, while form aini means ‘which this’. The bound morpheme a- in aini has a function as a noun marker which pointed in article ini called relative noun marker. Thus, demonstrative aini can stand alone as a minor sentence in the answer sentence, e.g., Question: Hamai bokuku kabasa? ‘Where is my reading book?’ Answer: Aini! ‘This’ < (Here is it!). Demonstrative aini when substituted with ini, the construction became ungrammatical, e.g., Question: Hamai bokuku kabasa? ‘Where is my reading book?’ Answer: *ini. Hereafter, the form ini cannot be formed in more complexes, whereas the form aini can be formed again with another bound morpheme, such as: ainihakanau ‘This is me’, ainihako ‘This is you’, ainihakoomu ‘These are you’, ainihae ‘This is he’, ainihada ‘These are they’, and ainihakasami ‘These are we’.

How information structure influences the processing of rhythmic irregularities: ERP evidence from German phrases

This study explores the influence of focus and givenness on the cognitive processing of rhythmic irregularities occurring in natural speech. Previous ERP studies showed that even subtle rhythmic deviations are detected by the brain if attention is directed towards the rhythmic structure. By using question–answer pairs, it was investigated whether subtle rhythmic irregularities in form of stress clashes (two adjacent stressed syllables) and stress lapses (two adjacent unstressed syllables) are still perceived when presented in post-focus position in an answer sentence and attention is directed away from them, towards the meaning of the element in narrow focus position by the preceding wh-question. Moreover, by visually presenting the lexical-semantic input of the deviating structure in the question, the influence of rhythmical and lexical properties in these two forms of rhythmic deviations are disentangled. While words in the present stress clash condition do not deviate from lexical stress, stress lapses contain deviations from metrical and lexical stress. The data reveal an early negativity effect for stress clashes but not for stress lapses, supporting the assumption that they are processed differently. The absence of a negative component for stress lapses indicates that the metrical deviation alone is not salient enough to be registered in non-focus position. Moreover, the lack of a late positive component suggests that subtle rhythmic deviations are less perceivable and hence more acceptable when presented in non-focus position. Thus, these results show that attentional shift induced by information structure influences the degree of the processing of rhythm.

On the Syntax of Multiple Fragments in Korean

In a multiple question and answer pair, the answer sentence in Korean (as in other languages) is composed of the two or more fragment(ary) answer phrases. Following Choe (1987), when an answer phrase is embedded and trapped within an island-forming structure, the unmarked option is to utilize the pied-piping strategy to move the larger constituent of island structure containing the answer phrase. Since multiple fragment (MF) answers to multiple questions are assumed to be calculated anyway via post-QR absorption at the interpretive component (May and Higginbotham (1981)), the syntactic formation of MFs involving (TP) ellipsis is achieved by movement of the answer phrase via an overt instantiation of post-QR absorption (cf. Sohn (1994)). After the syntactic formation of MFs via pied-piping and absorption-accompanied movement, MF answer phrase remnants undergo two additional processes. First, the final larger remnant constituent has its right periphery ‘stripped’ away. We propose that Max-Elide (cf. Merchant (2008)) comes into play in this ‘stripping’ process, eliding both TP and an additional right periphery of the final larger remnant constituent. Second, the answer phrase surviving from the first larger remnant constituent may take the hanging topic strategy, being base-generated at the left edge without a Case-marker/postposition. To boot, we show that in addition to word-order preserving MF formation via absorption-accompanied movement, the potential second or third remnant answer phrase from a non-island structure can scramble over the potential first or second one, feeding into word-order changing MF formation.

SISTEM PENILAIAN ESAI OTOMATIS PADA E-LEARNING DENGAN ALGORITMA WINNOWING

Exam essay is an evaluation of learning in the form of essay questions that have answers more variable than multiple choice questions. Variations of these answers provide its own difficulties for teachers in assessing response. Essay grading system is built to be one solution that can speed up and simplify the process of grading. Essay grading system in this study was done by measuring the similarity of responses of the students and teacher answer key. This research use winnowing algorithm. Winnowing algorithm is an algorithm for text similarity measure. Winnowing algorithm produces fingerprint of text that will represent the answer to the calculation of similarity with jaccard coeficient equation. Testing was conducted to determine the ability of the algorithm winnowing to provide an essay grading using n-gram and window value changes of the winnowing algorithm. The test results showed the use of the value of n-grams and window on the method of winnowing effect on fingerprint similarities were found. The more similarities fingerprint found, the value of the resulting system also be higher. The accuracy of the grading system showed better results on text answers that have the answer sentence structure same with key answer.