Bi-Directional Attention Flow Research Articles

Arabic Span Extraction-based Reading Comprehension Benchmark (ASER) and Neural Baseline Models

Machine reading comprehension (MRC) requires machines to read and answer questions about a given text. This can be achieved through either predicting answers or extracting them. Extracting answers from text involves predicting the first and last index of the answer span within the paragraph. Training machines to answer questions requires datasets that are created for such a purpose. The lack of availability of benchmarking datasets for the Arabic language has hindered research into machine reading comprehension from Arabic text. The aim of this article is to propose an Arabic Span-Extraction-based Reading Comprehension Benchmark (ASER) and complement it with neural baseline models for performance evaluations. Detailed steps are depicted for building and evaluating ASER, which is an Arabic dataset created manually for the task of machine reading comprehension. It contains 10,000 records from different domains and is divided into training and testing sets. The results of ASER evaluation led to the conclusion that it is a challenging benchmark since the answers have varying lengths and human performance resulted in an exact match of 42%. On the other hand, two main baseline models were the focus of ASER experimentation: the sequence-to-sequence (Seq2Seq) model with different neural networks and the bidirectional attention flow (BIDAF) model. These experiments were implemented using different embeddings, and the results showed an exact match with lower values than human performance.

A Spatiotemporal Bidirectional Attention-Based Ride-Hailing Demand Prediction Model: A Case Study in Beijing During COVID-19

The COVID-19 pandemic has severely affected urban transport patterns, including the way residents travel. It is of great significance to predict the demand of urban ride-hailing for residents’ healthy travel, rational platform operation, and traffic control during the epidemic period. In this paper, we propose a deep learning model, called MOS-BiAtten, based on multi-head spatial attention mechanism and bidirectional attention mechanism for ride-hailing demand prediction. The model follows the encoder-decoder framework with a multi-output strategy for multi-steps prediction. The pre-predicted result and the historical demand data are extracted as two aspects of bidirectional attention flow, so as to further explore the complicated spatiotemporal correlations between the historical, present and future information. The proposed model is evaluated on the real-world dataset during COVID-19 in Beijing, and the experimental results demonstrate that MOS-BiAtten achieves a better performance compared with the state-of-art methods. Meanwhile, another dataset is used to verify the generalization performance of the model.

ScienceQA: a novel resource for question answering on scholarly articles

Machine Reading Comprehension (MRC) of a document is a challenging problem that requires discourse-level understanding. Information extraction from scholarly articles nowadays is a critical use case for researchers to understand the underlying research quickly and move forward, especially in this age of infodemic. MRC on research articles can also provide helpful information to the reviewers and editors. However, the main bottleneck in building such models is the availability of human-annotated data. In this paper, firstly, we introduce a dataset to facilitate question answering (QA) on scientific articles. We prepare the dataset in a semi-automated fashion having more than 100k human-annotated context–question–answer triples. Secondly, we implement one baseline QA model based on Bidirectional Encoder Representations from Transformers (BERT). Additionally, we implement two models: the first one is based on Science BERT (SciBERT), and the second is the combination of SciBERT and Bi-Directional Attention Flow (Bi-DAF). The best model (i.e., SciBERT) obtains an F1 score of 75.46%. Our dataset is novel, and our work opens up a new avenue for scholarly document processing research by providing a benchmark QA dataset and standard baseline. We make our dataset and codes available here at https://github.com/TanikSaikh/Scientific-Question-Answering.

Nested block self-attention multiple resolution residual network for multiorgan segmentation from CT.

Fast and accurate multiorgans segmentation from computed tomography (CT) scans is essential for radiation treatment planning. Self-attention(SA)-based deep learning methodologies provide higher accuracies than standard methods but require memory and computationally intensive calculations, which restricts their use to relatively shallownetworks. Our goal was to develop and test a new computationally fast and memory-efficient bidirectional SA method called nested block self-attention (NBSA), which is applicable to shallow and deep multiorgan segmentationnetworks. A new multiorgan segmentation method combining a deep multiple resolution residual network with computationally efficient SA called nested block SA (MRRN-NBSA) was developed and evaluated to segment 18 different organs from head and neck (HN) and abdomen organs. MRRN-NBSA combines features from multiple image resolutions and feature levels with SA to extract organ-specific contextual features. Computational efficiency is achieved by using memory blocks of fixed spatial extent for SA calculation combined with bidirectional attention flow. Separate models were trained for HN (n = 238) and abdomen (n = 30) and tested on set aside open-source grand challenge data sets for HN (n = 10) using a public domain database of computational anatomy and blinded testing on 20 cases from Beyond the Cranial Vault data set with overall accuracy provided by the grand challenge website for abdominal organs. Robustness to two-rater segmentations was also evaluated for HN cases using the open-source data set. Statistical comparison of MRRN-NBSA against Unet, convolutional network-based SA using criss-cross attention (CCA), dual SA, and transformer-based (UNETR) methods was done by measuring the differences in the average Dice similarity coefficient (DSC) accuracy for all HN organs using the Kruskall-Wallis test, followed by individual method comparisons using paired, two-sided Wilcoxon-signed rank tests at 95% confidence level with Bonferroni correction used for multiplecomparisons. MRRN-NBSA produced an average high DSC of 0.88 for HN and 0.86 for the abdomen that exceeded current methods. MRRN-NBSA was more accurate than the computationally most efficient CCA (average DSC of 0.845 for HN, 0.727 for abdomen). Kruskal-Wallis test showed significant difference between evaluated methods (p=0.00025). Pair-wise comparisons showed significant differences between MRRN-NBSA than Unet (p=0.0003), CCA (p=0.030), dual (p=0.038), and UNETR methods (p=0.012) after Bonferroni correction. MRRN-NBSA produced less variable segmentations for submandibular glands (0.82 ± 0.06) compared to two raters (0.75 ± 0.31). MRRN-NBSA produced more accurate multiorgan segmentations than current methods on two different public data sets. Testing on larger institutional cohorts is required to establish feasibility for clinicaluse.

Coarse-grained decomposition and fine-grained interaction for multi-hop question answering

In recent years, question answering (QA) and reading comprehension (RC) has attracted much attention, and most research on QA has focused on multi-hop QA task which requires connecting multiple pieces of evidence scattered in a long context to answer the question. The key to the multi-hop QA task is semantic feature interaction between documents and questions, which is widely processed by Bi-directional Attention Flow (Bi-DAF), but Bi-DAF generally captures only the surface semantics of words in complex questions, and fails to capture implied semantic feature of intermediate answers, as well as ignoring parts of contexts related to the question and failing to extract the most important parts of multiple documents. In this paper, we propose a new model architecture for multi-hop question answering by applying two completion strategies:(1) Coarse-Grained complex question Decomposition (CGDe) strategy is introduced to decompose complex questions into simple ones without any additional annotations; (2) Fine-Grained Interaction (FGIn) strategy is introduced to explicitly represent each word in documents and extract more comprehensive and accurate sentences related to the inference path. The above two strategies are combined and tested on the SQuAD and HotpotQA datasets, and the experimental results show that our method outperforms state-of-the-art baselines.

Research and Implementation of Open Domain Question Answering System Based on DuReader Dataset and BIDAF Model

At present, major search engines emerge in endlessly, and the pop-up result list of search keywords makes people dizzy. In order to improve this problem and make the question query more concise and convenient, this article is based on the DuReader data set and reproduces BiDAF (Bi-Directional Attention Flow) The model builds an intelligent question answering system in the open field. The final model can extract answers to questions more accurately after training. This article embeds deep learning technology into the system and uses intelligent chat to show them.

Deep learning-based question answering system for intelligent humanoid robot

BackgroundThe development of Intelligent Humanoid Robot focuses on question answering systems that can interact with people is very limited. In this research, we would like to propose an Intelligent Humanoid Robot with the self-learning capability for accepting and giving responses from people based on Deep Learning and Big Data knowledge base. This kind of robot can be used widely in hotels, universities, and public services. The Humanoid Robot should consider the style of questions and conclude the answer through conversation between robot and user. In our scenario, the robot will detect the user’s face and accept commands from the user to do an action.FindingsThe question from the user will be processed using deep learning, and the result will be compared to the knowledge base on the system. We proposed our Deep Learning approach, based on Recurrent Neural Network (RNN) encoder, Convolution Neural Network (CNN) encoder, with Bidirectional Attention Flow (BiDAF).ConclusionsOur evaluation indicates that using RNN based encoder with BiDAF gives a higher score, than CNN encoder with the BiDAF. Based on our experiment, our model get 82.43% F1 score and the RNN based encoder will give a higher EM/F1 score than using the CNN encoder.

Correlation analysis of law-related news combining bidirectional attention flow of news title and body

Correlation analysis of law-related news is a task to of dividing news into law-related or law-unrelated news, which is the basis of public opinion analysis. Public opinion news consists of the title and the body. The title describes the theme of the news, and the body describes the content of the news. They are equally important and interdependent in the analysis of lawrelated news. Therefore, we make full use of the dependence between the title and the body and propose a learning method that combines the bidirectional attention flow of the title and the body. This method encodes the title and the body respectively by using a bidirectional gated recurrent unit (BiGRU) to obtain the word-level feature matrix of the title and the word-level feature matrix of the body. Then it further extracts the law relevant key features from the body feature matrix, to obtain the word-level feature representation of the body. Finally, we combine the word-level feature representation of the title and the body to build bidirectional attention flow. In this way, the information of the two is fully integrated and interacted to improve the accuracy of the legal correlation analysis of news. To verify the validity of the method in this paper, we conducted experiments on the analysis of law-related news. The results show that our method has achieved good results. Compared with the baseline method, the F1 values of our method is increased by 2.2%, which strongly proves that the interaction between title and body has a good supporting effect on news text classification.

Video Alignment Using Bi-Directional Attention Flow in a Multi-Stage Learning Model

Recently, deep learning techniques have contributed to solving a multitude of computer vision tasks. In this paper, we propose a deep-learning approach for video alignment, which involves finding the best correspondences between two overlapping videos. We formulate the video alignment task as a variant of the well-known machine comprehension (MC) task in natural language processing. While MC answers a question about a given paragraph, our technique determines the most relevant frame sequence in the context video to the query video. This is done by representing the individual frames of the two videos by highly discriminative and compact descriptors. Next, the descriptors are fed into a multi-stage network that is able, with the help of the bidirectional attention flow mechanism, to represent the context video at various granularity levels besides estimating the query-aware context part. The proposed model was trained on 10k video-pairs collected from “YouTube”. The obtained results show that our model outperforms all known state of the art techniques by a considerable margin, confirming its efficacy.