Open-ended Answering Research Articles

ASMR: Aggregated Semantic Matching Retrieval Unleashing Commonsense Ability of LLM through Open-Ended Question Answering

Commonsense reasoning refers to the ability to make inferences, draw conclusions, and understand the world based on general knowledge and commonsense. Whether Large Language Models (LLMs) have commonsense reasoning ability remains a topic of debate among researchers and experts. When confronted with multiple-choice commonsense reasoning tasks, humans typically rely on their prior knowledge and commonsense to formulate a preliminary answer in mind. Subsequently, they compare this preliminary answer to the provided choices, and select the most likely choice as the final answer. We introduce Aggregated Semantic Matching Retrieval (ASMR) as a solution for multiple-choice commonsense reasoning tasks. To mimic the process of humans solving commonsense reasoning tasks with multiple choices, we leverage the capabilities of LLMs to first generate the preliminary possible answers through open-ended question which aids in enhancing the process of retrieving relevant answers to the question from the given choices. Our experiments demonstrate the effectiveness of ASMR on popular commonsense reasoning benchmark datasets, including CSQA, SIQA, and ARC (Easy and Challenge). ASMR achieves state-of-the-art (SOTA) performance with a peak of +15.3% accuracy improvement over the previous SOTA on SIQA dataset.

Knowledge-Constrained Answer Generation for Open-Ended Video Question Answering

Open-ended Video question answering (open-ended VideoQA) aims to understand video content and question semantics to generate the correct answers. Most of the best performing models define the problem as a discriminative task of multi-label classification. In real-world scenarios, however, it is difficult to define a candidate set that includes all possible answers. In this paper, we propose a Knowledge-constrained Generative VideoQA Algorithm (KcGA) with an encoder-decoder pipeline, which enables out-of-domain answer generation through an adaptive external knowledge module and a multi-stream information control mechanism. We use ClipBERT to extract the video-question features, extract framewise object-level external knowledge from a commonsense knowledge base and compute the contextual-aware episode memory units via an attention based GRU to form the external knowledge features, and exploit multi-stream information control mechanism to fuse video-question and external knowledge features such that the semantic complementation and alignment are well achieved. We evaluate our model on two open-ended benchmark datasets to demonstrate that we can effectively and robustly generate high-quality answers without restrictions of training data.

EKTVQA: Generalized Use of External Knowledge to Empower Scene Text in Text-VQA

The open-ended question answering task of Text-VQA often requires reading and reasoning about rarely seen or completely unseen scene-text content of an image. We address this zero-shot nature of the problem by proposing the generalized use of external knowledge to augment our understanding of the scene text. We design a framework to extract, validate, and reason with knowledge using a standard multimodal transformer for vision language understanding tasks. Through empirical evidence and qualitative results, we demonstrate how external knowledge can highlight instance-only cues and thus help deal with training data bias, improve answer entity type correctness, and detect multiword named entities. We generate results comparable to the state-of-the-art on three publicly available datasets, under the constraints of similar upstream OCR systems and training data.

Problems Affecting Physics Learning and its Effect on the Academic Performance of Students in Integrated Teacher Training Programme

Physics is one of the most common problematic subjects for the students in the degree and post graduate level. Students want to spend more time on the subject to get mastery over it. Similarly, to maintain the quality education, the teacher education institutes have a prominent role; they want to focus more on the education part of the subject. Students who are enrolled in the teacher training colleges should play a crucial role in developing and maintaining future science education. This study focussed on the problems affecting the Physics learning of the prospective teachers who are enrolled in the integrated teacher training programs and the different types of problems that they are faced to achieve better results. The study was conducted among prospective teachers of integrated teacher training college who are pursuing the MSc.Ed program. Data was collected by using a questionnaire prepared by the researchers, including provisions for open-ended answering. The sample of the study includes the students from various academic years. Results show that the majority of the students having a medium level of problems with Physics learning and male students are facing more problems compared to female students. The academic year also an influencing factor of problems in Physics learning. Other factors such as previous school education, current grade point average have no relationship with the problems affecting Physics learning.

Attention Based Multi-Modal Fusion Architecture for Open-Ended Video Question Answering Systems

Abstract Open-ended Video Question Answering systems is a very challenging problem with widespread applications in real life. Existing systems tend to focus on single word video question answering system, which cannot be easily extended to develop. In this paper, we propose using an architecture, popularly used for video captioning systems to solve the problem of open-ended video based question answering systems. For generating good answers, the model is required to focus on each frame separately as well as understand how to link information from different frames to generate the answer. The model also needs to keep in mind the different modalities and adapt itself accordingly while processing the videos as well as the questions. We propose an attention based multimodal fusion architecture for Video Question Answering (AMF-VQA) that uses attention mechanism at every time to output a word. Such kind of mechanism allows the model to focus on different frames as well as focus on different modalities while outputting every single word. The proposed model is very flexible were we can just add other modalities such as audio features, captions, etc. to the existing model and fine-tune the model to get improve results if these new features are available.

Long-Form Video Question Answering via Dynamic Hierarchical Reinforced Networks.

Open-ended long-form video question answering is a challenging task in visual information retrieval, which automatically generates a natural language answer from the referenced long-form video contents according to a given question. However, the existing works mainly focus on short-form video question answering, due to the lack of modeling semantic representations from long-form video contents. In this paper, we introduce a dynamic hierarchical reinforced network for open-ended long-form video question answering, which employs an encoder-decoder architecture with a dynamic hierarchical encoder and a reinforced decoder. Concretely, we first propose a frame-level dynamic long-short term memory (LSTM) network with binary segmentation gate to learn frame-level semantic representations according to the given question. We then develop a segment-level highway LSTM network with a question-aware highway gate for segment-level semantic modeling. Furthermore, we devise the reinforced decoder with a hierarchical attention mechanism to generate natural language answers. We construct a large-scale long-form video question answering dataset. The extensive experiments on the long-form dataset and another public short-form dataset show the effectiveness of our method.

Unifying the Video and Question Attentions for Open-Ended Video Question Answering.

Video question answering is an important task toward scene understanding and visual data retrieval. However, current visual question answering works mainly focus on a single static image, which is distinct from the dynamic and sequential visual data in the real world. Their approaches cannot utilize the temporal information in videos. In this paper, we introduce the task of free-form open-ended video question answering. The open-ended answers enable wider applications compared with the common multiple-choice tasks in Visual-QA. We first propose a data set for open-ended Video-QA with the automatic question generation approaches. Then, we propose our sequential video attention and temporal question attention models. These two models apply the attention mechanism on videos and questions, while preserving the sequential and temporal structures of the guides. The two models are integrated into the model of unified attention. After the video and the question are encoded, the answers are generated wordwisely from our models by a decoder. In the end, we evaluate our models on the proposed data set. The experimental results demonstrate the effectiveness of our proposed model.

VQA: Visual Question Answering

We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and answers are open-ended. Visual questions selectively target different areas of an image, including background details and underlying context. As a result, a system that succeeds at VQA typically needs a more detailed understanding of the image and complex reasoning than a system producing generic image captions. Moreover, VQA is amenable to automatic evaluation, since many open-ended answers contain only a few words or a closed set of answers that can be provided in a multiple-choice format. We provide a dataset containing $$\sim $$~0.25 M images, $$\sim $$~0.76 M questions, and $$\sim $$~10 M answers (www.visualqa.org), and discuss the information it provides. Numerous baselines and methods for VQA are provided and compared with human performance. Our VQA demo is available on CloudCV (http://cloudcv.org/vqa).

Cognitive, Emotional, And Verbal Responses In Unfair Everyday Discourse

Research about argumentational unfairness has focused primarily on the cognitive evaluation of argumentational rule violations, applying written argumentational scenarios, and structured answering format. This study investigates cognitive, emotional, and verbal responses to unfair contributions (nonrational vs. noncooperative speech acts) using an open-ended answering format in various presentation modalities (written, auditory, and role-play). No differences appeared between reactions to nonrational versus noncooperative contributions or between the different presentation modalities. The results replicate previous findings and provide support for the validity of attributional models in unfair everyday discourse, as well as for the position that argumentative behavior is rule governed by reciprocal expectancies.