Answer To Question Research Articles

OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience

Existing studies in conversational AI mostly treat task-oriented dialog (TOD) and question answering (QA) as separate tasks. Towards the goal of constructing a conversational agent that can complete user tasks and support information seeking, it is important to develop a system that can handle both TOD and QA with access to various external knowledge sources. In this work, we propose a new task, Open-Book TOD (OB-TOD), which combines TOD with QA and expands the external knowledge sources to include both explicit sources (e.g., the web) and implicit sources (e.g., pre-trained language models). We create a new dataset OB-MultiWOZ, where we enrich TOD sessions with QA-like information-seeking experience grounded on external knowledge. We propose a unified model OPERA ( Op en-book E nd-to-end Task-o r iented Di a log) which can appropriately access explicit and implicit external knowledge to tackle the OB-TOD task. Experimental results show that OPERA outperforms closed-book baselines, highlighting the value of both types of knowledge. 1

Modular Co-attention Networks in Nepali Visual Question Answering Systems

Visual question answering (VQA) has been regarded as a challenging task requiring a perfect blend of computer vision and natural language processing. As no dataset was available to train such a model for the Nepali language, a new dataset was developed during the research by translating the VQAv2 dataset. Then the dataset consisting of 202,577 images and 886,560 questions was used to train an attention-based VQA model. The dataset consists of yes/no, counting, and other questions with primarily one-word answers. Modular Co-attention Network (MCAN) was applied to the visual features extracted using the Faster RCNN framework and question embeddings extracted using the Nepali GloVe model. After co-attending the visual and language features for a few cascaded MCAN layers, the features are fused to train the whole network. During evaluation, an overall accuracy of 69.87% was obtained with 81.09% accuracy in yes/no type questions. The results surpassed the performance of models developed for Hindi and Bengali languages. Overall, novel research has been done in the Nepali Language VQA domain paving the way for further advancements.

Multi-stage reasoning on introspecting and revising bias for visual question answering

Visual Question Answering (VQA) is a task that involves predicting an answer to a question depending on the content of an image. However, recent VQA methods have relied more on language priors between the question and answer rather than the image content. To address this issue, many debiasing methods have been proposed to reduce language bias in model reasoning. However, the bias can be divided into two categories: good bias and bad bias. Good bias can benefit to the answer prediction, while the bad bias may associate the models with the unrelated information. Therefore, instead of excluding good and bad bias indiscriminately in existing debiasing methods, we proposed a bias discrimination module to distinguish them. Additionally, bad bias may reduce the model’s reliance on image content during answer reasoning and thus attend little on image features updating. To tackle this, we leverage Markov theory to construct a Markov field with image regions and question words as nodes. This helps with feature updating for both image regions and question words, thereby facilitating more accurate and comprehensive reasoning about both the image content and question. To verify the effectiveness of our network, we evaluate our network on VQA v2 and VQA cp v2 datasets and conduct extensive quantity and quality studies to verify the effectiveness of our proposed network. Experimental resu- lts show that our network achieves significant performance against the previous state-of-the-art methods.

A Survey of Robot Intelligence with Large Language Models

Since the emergence of ChatGPT, research on large language models (LLMs) has actively progressed across various fields. LLMs, pre-trained on vast text datasets, have exhibited exceptional abilities in understanding natural language and planning tasks. These abilities of LLMs are promising in robotics. In general, traditional supervised learning-based robot intelligence systems have a significant lack of adaptability to dynamically changing environments. However, LLMs help a robot intelligence system to improve its generalization ability in dynamic and complex real-world environments. Indeed, findings from ongoing robotics studies indicate that LLMs can significantly improve robots’ behavior planning and execution capabilities. Additionally, vision-language models (VLMs), trained on extensive visual and linguistic data for the vision question answering (VQA) problem, excel at integrating computer vision with natural language processing. VLMs can comprehend visual contexts and execute actions through natural language. They also provide descriptions of scenes in natural language. Several studies have explored the enhancement of robot intelligence using multimodal data, including object recognition and description by VLMs, along with the execution of language-driven commands integrated with visual information. This review paper thoroughly investigates how foundation models such as LLMs and VLMs have been employed to boost robot intelligence. For clarity, the research areas are categorized into five topics: reward design in reinforcement learning, low-level control, high-level planning, manipulation, and scene understanding. This review also summarizes studies that show how foundation models, such as the Eureka model for automating reward function design in reinforcement learning, RT-2 for integrating visual data, language, and robot actions in vision-language-action models, and AutoRT for generating feasible tasks and executing robot behavior policies via LLMs, have improved robot intelligence.

Driving sustainable energy transitions with a multi-source RAG-LLM system

By 2035, the UK aims to upgrade all homes to achieve a net-zero economy by 2050, thereby reducing energy consumption, household costs, and improving living conditions. Small and Medium-sized Enterprises (SMEs) play a crucial role in this transition. However, many SME contractors lack essential information on Sustainable Energy Initiatives (SEIs) and the relevant Energy landscape necessary for driving Sustainable Energy Transitions (SETs). This knowledge gap poses risks to SME interventions, potentially leading to increased costs and inefficiencies. Accessing timely information on SEIs including government policies, funding, technologies, and environmental impacts from various media sources is essential for guiding effective SETs and understanding the relevant Energy landscape, thereby facilitating informed decision-making. Currently, SMEs lack an integrated system that consolidates data from diverse media sources into a centralized Information System (IS), limiting their ability to effectively navigate SEIs. To address this gap, this research introduces an Energy Chatbot, a sustainable IS that utilizes Large Language Models (LLMs) integrated with multi-source Retrieval Augmented Generation (RAG). This system encompasses diverse media sources, including news articles, government reports, industry publications, academic research, and social media. The Energy Chatbot is designed to enhance decision-making for SMEs by providing comprehensive Energy sector insights through a Question Answering (QA) system. Key findings emphasize that this approach reduces costs by utilizing open-source models. Moreover, the Energy Chatbot provides SMEs with access to up-to-date information, enabling them to identify long-term sustainability strategies and maintain a competitive edge in the evolving Energy landscape.

Embracing AI in Change Management

Abstract: This article discusses how organizations can leverage artificial intelligence (AI) to support rather than hinder change management efforts amid technological transformation. It first establishes a research foundation on common sources of resistance to change, such as job insecurity, lack of understanding, and lack of involvement. The article then presents examples of how AI can augment traditional change management approaches to systematically address each resistance factor. Specifically, AI enhances two-way communication through personalized outreach and question answering. It also enables immersive, dynamic training solutions for ubiquitous, experiential skill development. AI fosters wider participation and feedback while monitoring employee well-being and performance. By addressing causes of resistance at each stage, AI strengthens critical aspects of communication, education, participation, and support to build understanding of changes rather than apathy. The article argues leaders who view AI as an ally in change management will empower their workforce through disruption.

Language-based reasoning graph neural network for commonsense question answering

Language model (LM) has played an increasingly important role in the common-sense understanding and reasoning in the CSQA task (Common Sense Question Answering). However, due to the amount of model parameters, increasing training data helps little in further improving model performance. Introducing external knowledge through graph neural networks (GNNs) proves positive in boosting performance, but exploiting different knowledge sources and capturing contextual information between text and knowledge inside remains a challenge. In this paper, we propose LBR-GNN, a Language-Based Reasoning Graph Neural Network method to address these problems, by representing the question with each answer and external knowledge using a language model and predicting the reasoning score with a designed language-based GNN. Our LBR-GNN will first regulate external knowledge into a consistent textual form and encode it using a standard LM to capture the contextual information. Then, we build a graph neural network using the encoded information, especially the language-level edge representation. Finally, we design a novel edge aggregation method to select the edge information for GNN update and the language-guided GNN reasoning. We assess the performance of LBR-GNN across the CommonsenseQA, CommonsenseQA-IH, and OpenBookQA datasets. Our evaluation reveals a performance boost of more than 5% compared to the state-of-the-art methods on the CSQA dataset, achieved with a similar number of additional parameters.

Natural language processing in finance: A survey

This survey presents an in-depth review of the transformative role of Natural Language Processing (NLP) in finance, highlighting its impact on ten major financial applications: (1) financial sentiment analysis, (2) financial narrative processing, (3) financial forecasting, (4) portfolio management, (5) question answering, virtual assistant and chatbot, (6) risk management, (7) regulatory compliance monitoring, (8) ESG (Environmental, Social, Governance) and sustainable finance, (9) explainable artificial intelligence (XAI) in finance and (10) NLP for digital assets. With the integration of vast amounts of unstructured financial data and advanced NLP techniques, the study explores how NLP enables data-driven decision-making and innovation in the financial sector, alongside the limitations and challenges. By providing a comprehensive analysis of NLP applications combining both academic and industrial perspectives, this study postulates the future trends and evolution of financial services. It introduces a unique review framework to understand the interaction of financial data and NLP technologies systematically and outlines the key drivers, transformations, and emerging areas in this field. This survey targets researchers, practitioners, and professionals, aiming to close their knowledge gap by highlighting the significance and future direction of NLP in enhancing financial services.

Application of a Question Answering System in the Ontology-Based Health Domain and Question Templates to Improve the Quality of Answers

Developing an effective Question Answering System (QAS) to address a wide range of health-related inquiries presents a significant challenge. This study introduces an innovative approach using an ontology-based QAS within the health domain, leveraging the Resource Description Framework (RDF) and SPARQL query language. By utilizing ontology, the system can more accurately map health concepts, improving the relevance and precision of its responses. Additionally, the incorporation of structured Question Templates enhances the system’s ability to understand and respond to diverse user queries. The system's performance was evaluated using Black Box Testing, which demonstrated that it consistently provides accurate and relevant answers. The system achieved a Mean Average Precision (MAP) of 79.6%, indicating its potential to effectively address a broad spectrum of health issues.

Zero-Shot Prompting Strategies for Table Question Answering with a Low-Resource Language

This work explores the application of zero-shot prompting strategies for table question answering (TQA) in Portuguese, focusing specifically on the Text2SQL task. This task involves translating questions posed in natural language into Structured Query Language (SQL) queries, which can be executed against a database to answer the original question. Given the popularity of relational databases across various domains, advancements in this field can substantially impact the accessibility and democratization of data as simpler and more intuitive interfaces for database interaction are developed. Despite this significant potential, progress in developing Portuguese TQA solutions remains limited. The proposed approach leverages Large Language Models (LLMs)—specifically the GPT-3.5 and GPT-4 models—through zero-shot prompting. The primary objectives are to assess the effectiveness of such LLMs in this task and to identify the most suitable prompt styles. These are evaluated using a Portuguese translation of the popular Spider Text2SQL benchmark. Results reveal that the proposed approach can generate adequate SQL queries to answer Portuguese language questions about various databases, mainly when using GPT-4. The findings suggest that including schema information and database content in the prompts is critical for satisfactory outcomes. Doi: 10.28991/ESJ-2024-08-05-020 Full Text: PDF

VG-CALF: A vision-guided cross-attention and late-fusion network for radiology images in Medical Visual Question Answering

Image and question matching is essential in Medical Visual Question Answering (MVQA) in order to accurately assess the visual-semantic correspondence between an image and a question. However, the recent state-of-the-art methods focus solely on the contrastive learning between an entire image and a question. Though contrastive learning successfully model the global relationship between an image and a question, it is less effective to capture the fine-grained alignments conveyed between image regions and question words. In contrast, large-scale pre-training poses significant drawbacks, including extended training times, handling substantial data volumes, and necessitating high computational power. To address these challenges, we propose the Vision-Guided Cross-Attention based Late Fusion (VG-CALF) network, which integrates image and question features into a unified deep model without relying on pre-training for MVQA tasks. In our proposed approach, we use self-attention to effectively leverage intra-modal relationships within each modality and implement vision-guided cross-attention to emphasize the inter-modal relationships between image regions and question words. By simultaneously considering intra-modal and inter-modal relationships, our proposed method significantly improves the overall performance of MVQA without the need for pre-training on extensive image-question pairs. Experimental results on benchmark datasets, such as, SLAKE and VQA-RAD demonstrate that our proposed approach performs competitively with existing state-of-the-art methods.

Automated construction safety reporting system integrating deep learning-based real-time advanced detection and visual question answering

The construction sector is globally acknowledged as one of the most hazardous industries, owing to the vulnerability of its workers to accidents, injuries, and even loss of life. Effective precautionary measures are necessary and ensuring the use of personal protective equipment (PPE) by workers is crucial for protecting them from accidents. Existing deep learning-based PPE detection systems mainly use simple vision-based target detection methods for tasks such as the identification of helmets or vests, and they tend to be task-specific. However, the identification of specific PPE based on respective job types and maintaining detailed safety records, requires further innovative approaches. In this paper, we propose an innovative intelligent system that not only accurately recognizes specific PPE according to the needs of different work types but also automatically generates safety inspection reports and establishes complete safety records, thus providing critical data to support accident investigations. The proposed system integrates a target detection model, visual question answering model, and text-based analysis of the relevant regulations to realize real-time detection of PPE and automatic generation of safety inspection reports. The experimental results show that the proposed YOLOv8n-DCA network strikes a good balance between performance and computational cost—, with a mAP value of 86%. Compared to the original YOLOv8n network, the mAP value is improved by 5.1%, while the model parameters and size are significantly reduced. Further, the visual question answering model exhibited a precision is 95.9. Finally, the automatic generation of safety inspection reports was successfully realized, verifying the feasibility of the developed system. This innovative system promises a comprehensive and efficient PPE management solution for the construction industry to ensure worker safety and provide strong data support.

Intelligent Recommendation Algorithm for Product Information on E-commerce Platforms Based on Robot Customer Service Automatic Q&A

In order to enhance the shopping experience of customers and retain them, thereby increasing sales volume, the author proposes the research topic of an intelligent recommendation system for product information on e-commerce platforms based on robot customer service automatic question answering. Firstly, starting directly from the question itself, the system can provide feedback to customers by simply segmenting the questions submitted by customers and matching them with semantic templates; Secondly, the system automatically builds and updates the user’s personalized knowledge base, using this to predict user purchasing tendencies and achieve the function of recommending products to customers. The implementation of the intelligent shopping robot system has passed 365 question and answer tests on 5G mobile phone sales terms, and is feasible in the professional field. The experimental results indicate that, when the number of training corpora increased to 300, the accuracy of the system was 0.85, 0.90, and 0.98 using 100, 2003, and 300 tests respectively. Such a system is perfect for natural language processing, so we can improve the system by expanding and improving the knowledge base. The intelligent shopping robot recommendation system studied by the author is still in the analysis and demonstration stage, sincerely hope that the processing method used in this project can have reference significance for similar recommendation systems in the near future.

OphGLM: An ophthalmology large language-and-vision assistant

Vision computer-aided diagnostic methods have been used in early ophthalmic disease screening and diagnosis. However, the limited output formats of these methods lead to poor human–computer interaction and low clinical applicability value. Thus, ophthalmic visual question answering is worth studying. Unfortunately, no practical solutions exist before Large Language Models(LLMs). In this paper, we investigate the ophthalmic visual diagnostic interaction problem. We construct an ophthalmology large language-and-vision assistant, OphGLM, consisting of an image encoder, a text encoder, a fusion module, and an LLM module. We establish a new Chinese ophthalmic fine-tuning dataset, FundusTuning-CN, including the fundus instruction and conversation sets. Based on FundusTuning-CN, we establish a novel LLM-tuning strategy to introduce visual model understanding and ophthalmic knowledge into LLMs at a low cost and high efficiency. Leveraging the pre-training of the image encoder, OphGLM demonstrates strong visual understanding and surpasses open-source visual language models in common fundus disease classification tasks. The FundusTuning-CN enables OphGLM to surpass open-source medical LLMs in both ophthalmic knowledge and interactive capabilities. Our proposed OphGLM has the potential to revolutionize clinical applications in ophthalmology. The dataset, code, and models will be publicly available at https://github.com/ML-AILab/OphGLM.

Deep Fuzzy Multiteacher Distillation Network for Medical Visual Question Answering

Deep Fuzzy Multiteacher Distillation Network for Medical Visual Question Answering

Retrieval In Decoder benefits generative models for explainable complex question answering

Large-scale Language Models (LLMs) utilizing the Chain-of-Thought prompting demonstrate exceptional performance in a variety of tasks. However, the persistence of factual hallucinations remains a significant challenge in practical applications. Prevailing retrieval-augmented methods treat the retriever and generator as separate components, which inadvertently restricts the generator’s capabilities to those of the retriever through intensive supervised training. In this work, we propose an unsupervised Retrieval In Decoder framework for multi-granularity decoding called RID, which integrates retrieval directly into the decoding process of generative models. It dynamically adjusts decoding granularity based on retrieval outcomes, and duly corrects the decoding direction through its direct impact on the next token. Moreover, we introduce a reinforcement learning-driven knowledge distillation method for adaptive explanation generation to better apply to Small-scale Language Models (SLMs). The experimental results across six public benchmarks surpass popular LLMs and existing retrieval-augmented methods, which demonstrates the effectiveness of RID in models of different scales and verifies its applicability and scalability.

COMPARATIVE ANALYSIS OF MULTILINGUAL QA MODELS AND THEIR ADAPTATION TO THE KAZAKH LANGUAGE

This paper presents a comparative analysis of large pretrained multilingual models for question-answering (QA) systems, with a specific focus on their adaptation to the Kazakh language. The study evaluates models including mBERT, XLM-R, mT5, AYA, and GPT, which were tested on QA tasks using the Kazakh sKQuAD dataset. To enhance model performance, fine-tuning strategies such as adapter modules, data augmentation techniques (back-translation, paraphrasing), and hyperparameter optimization were applied. Specific adjustments to learning rates, batch sizes, and training epochs were made to boost accuracy and stability. Among the models tested, mT5 achieved the highest F1 score of 75.72%, showcasing robust generalization across diverse QA tasks. GPT-4-turbo closely followed with an F1 score of 73.33%, effectively managing complex Kazakh QA scenarios. In contrast, native Kazakh models like Kaz-RoBERTa showed improvements through fine-tuning but continued to lag behind larger multilingual models, underlining the need for additional Kazakh-specific training data and further architectural enhancements. Kazakh’s agglutinative morphology and the scarcity of high-quality training data present significant challenges for model adaptation. Adapter modules helped mitigate computational costs, allowing efficient fine-tuning in resource-constrained environments without significant performance loss. Data augmentation techniques, such as back-translation and paraphrasing, were instrumental in enriching the dataset, thereby enhancing model adaptability and robustness. This study underscores the importance of advanced fine-tuning and data augmentation strategies for QA systems tailored to low-resource languages like Kazakh. By addressing these challenges, this research aims to make AI technologies more inclusive and accessible, offering practical insights for improving natural language processing (NLP) capabilities in underrepresented languages. Ultimately, these findings contribute to bridging the gap between high-resource and low-resource language models, fostering a more equitable distribution of AI solutions across diverse linguistic contexts.

Explanatory argument extraction of correct answers in resident medical exams

Developing technology to assist medical experts in their everyday decision-making is currently a hot topic in the field of Artificial Intelligence (AI). This is specially true within the framework of Evidence-Based Medicine (EBM), where the aim is to facilitate the extraction of relevant information using natural language as a tool for mediating in human–AI interaction. In this context, AI techniques can be beneficial in finding arguments for past decisions in evolution notes or patient journeys, especially when different doctors are involved in a patient’s care. In those documents the decision-making process towards treating the patient is reported. Thus, applying Natural Language Processing (NLP) techniques has the potential to assist doctors in extracting arguments for a more comprehensive understanding of the decisions made. This work focuses on the explanatory argument identification step by setting up the task in a Question Answering (QA) scenario in which clinicians ask questions to the AI model to assist them in identifying those arguments. In order to explore the capabilities of current AI-based language models, we present a new dataset which, unlike previous work: (i) includes not only explanatory arguments for the correct hypothesis, but also arguments to reason on the incorrectness of other hypotheses; (ii) the explanations are written originally in Spanish by doctors to reason over cases from the Spanish Residency Medical Exams. Furthermore, this new benchmark allows us to set up a novel extractive task by identifying the explanation written by medical doctors that supports the correct answer within an argumentative text. An additional benefit of our approach lies in its ability to evaluate the extractive performance of language models using automatic metrics, which in the Antidote CasiMedicos dataset corresponds to a 74.47 F1 score. Comprehensive experimentation shows that our novel dataset and approach is an effective technique to help practitioners in identifying relevant evidence-based explanations for medical questions.

Histopathology in focus: a review on explainable multi-modal approaches for breast cancer diagnosis.

Precision and timeliness in breast cancer detection are paramount for improving patient outcomes. Traditional diagnostic methods have predominantly relied on unimodal approaches, but recent advancements in medical data analytics have enabled the integration of diverse data sources beyond conventional imaging techniques. This review critically examines the transformative potential of integrating histopathology images with genomic data, clinical records, and patient histories to enhance diagnostic accuracy and comprehensiveness in multi-modal diagnostic techniques. It explores early, intermediate, and late fusion methods, as well as advanced deep multimodal fusion techniques, including encoder-decoder architectures, attention-based mechanisms, and graph neural networks. An overview of recent advancements in multimodal tasks such as Visual Question Answering (VQA), report generation, semantic segmentation, and cross-modal retrieval is provided, highlighting the utilization of generative AI and visual language models. Additionally, the review delves into the role of Explainable Artificial Intelligence (XAI) in elucidating the decision-making processes of sophisticated diagnostic algorithms, emphasizing the critical need for transparency and interpretability. By showcasing the importance of explainability, we demonstrate how XAI methods, including Grad-CAM, SHAP, LIME, trainable attention, and image captioning, enhance diagnostic precision, strengthen clinician confidence, and foster patient engagement. The review also discusses the latest XAI developments, such as X-VARs, LeGrad, LangXAI, LVLM-Interpret, and ex-ILP, to demonstrate their potential utility in multimodal breast cancer detection, while identifying key research gaps and proposing future directions for advancing the field.

The Effect of Question Answer Relationship Strategy on Iraqi EFL Intermediate Pupils’ Reading Comprehension

The Question Answer Relationship strategy is useful for teaching pupils how to understand and reply of several forms of the questions. Understanding that some questions have “Right There” solutions in the text, that others require the reader to “Think and Search,” and that others can only be answered “On My Own” helps pupils recognize the importance of thinking about the topic before finding the answer. The aims of the current study are finding out the effect of Question Answer Relationship strategy on Iraqi EFL intermediate pupil’s achievement and finding out the effect of Question Answer Relationship strategy on developing Iraqi EFL intermediate pupil’s reading comprehension. The aims of the study are supposed to be achieved through verifying the following hypothesis: There are no statistically significant differences between the mean scores of the experimental group which is taught by question answer relationship strategy and the control group who is taught by prescribed method in pupil’s post achievement test . The second hypothesis states that there is no statistical significant differences between the mean scores of the experimental group in pre and post reading comprehension test. The sample of the current study consists of (60) pupils from the second grade at Karam AL Rahman Intermediate School for Boys is chosen at random to be the sample of the study. The sample is split into two groups: one for the experiment and the other is for the control. Both groups are similar in terms of their age, parent’s educational level and previous year’s English language scores. The experiment lasted for ten weeks ,the researcher taught the two groups during the academic year 2022-2023. To analyze the obtained data, different statistical means have been used T-test, Weighted mean to measure the pupils’ achievement. After the statistically treatment of the data.