Natural Language Database Query Research Articles

Insights into Natural Language Database Query Errors: From Attention Misalignment to User Handling Strategies

Querying structured databases with natural language (NL2SQL) has remained a difficult problem for years. Recently, the advancement of machine learning (ML), natural language processing (NLP), and large language models (LLM) have led to significant improvements in performance, with the best model achieving ∼ 85% percent accuracy on the benchmark Spider dataset. However, there is a lack of a systematic understanding of the types, causes, and effectiveness of error-handling mechanisms of errors for erroneous queries nowadays. To bridge the gap, a taxonomy of errors made by four representative NL2SQL models was built in this work, along with an in-depth analysis of the errors. Second, the causes of model errors were explored by analyzing the model-human attention alignment to the natural language query. Last, a within-subjects user study with 26 participants was conducted to investigate the effectiveness of three interactive error-handling mechanisms in NL2SQL. Findings from this paper shed light on the design of model structure and error discovery and repair strategies for natural language data query interfaces in the future.

A NEW FEASIBLE NATURAL LANGUAGE DATABASE QUERY METHOD

A metadata search approach is proposed to provide practical solutions to the natural language database query problem; where the metadata grow in largely linear manner and the search is linguistics-free. A new class of reference dictionary integrates four types of enterprise metadata: enterprise information models, database values, user-words, and query cases. The layered and scalable information models allow user-words to stay in original forms as users articulated them, as opposed to relying on permutations of individual words contained in the original query. A graphical representation method turns the dictionary into searchable graphs representing all possible interpretations of the input. A branch-and-bound algorithm then identifies optimal interpretations, which lead to SQL implementation of the original queries. Query cases enhance both the metadata and the search of metadata, as well as providing case-based reasoning to directly answer the queries. This design assures feasible solutions at the termination of the search, even when the search is incomplete (i.e., the results contain the correct answer to the original query). The necessary condition is that the text input contains at least one entry in the reference dictionary. The sufficient condition is that the text input contains a set of entries corresponding to a complete, correct single SQL query. Laboratory testing shows that the system obtained accurate results for most cases that satisfied only the necessary condition.

Chapter 10 - WITHDRAWN: Natural Language

This book has been removed at request of the author.

Generating and evaluating domain-oriented multi-word terms from texts

It is generally agreed that natural language database query systems need to be tailored for each application in which they are used. This process, often called customization, involves, among other things, specification of the domain vocabulary and the grammatical properties of that vocabulary. In a previous paper, techniques for automatically generating domain vocabularies from large text collections were discussed. This paper is a continuation of that line of research, focusing on the problem of generating multi-word vocabulary terms (specifically pairs). It also discusses some of the statistical issues associated with word co-occurrences likely to be of use in a natural language interface for the application in question. Most importantly, an attempt is made to provide a more objective evaluation of the selection procedures used. Absent substantial experimentation with subjects using a working query system, all evaluation is necessarily subjective. This paper uses a surrogate for experimentation by relying on pre-existing dictionaries as indicators of domain relevance.

Transportable natural language database update

Natural language database access requires support of both query and update capabilities. Although a great deal of research effort has gone to support natural language database query, little effort has gone to support update. We describe a model of action that supports natural language database update, as well as query, and the implementation of a system that supports the model. A major goal of this research is to design a system that is easily transportable both to different database domains and different database management systems.

Interpretation of Natural Language Database Queries Using Optimization Methods

The automatic interpretation of natural language (English) database questions formulated by a user untrained in the technical aspects of database querying is an established problem in the field of artificial intelligence. State-of-the-art approaches involve the analysis of queries with syntactic and semantic grammars expressed in phrase structure grammer or transition network formalisms. With such methods difficulties exist with the detection and resolution of ambiguity, with the misinterpretation possibilities inherent with finite length look-ahead, and with the modification and extension of a mechanism for other sources of semantic knowledge. The potential of optimization techniques to solve these problems and interpret natural language database queries is examined.

Designing a Portable Natural Language Database Query System

One barrier to the acceptance of natural language database query systems is the substantial installation effort required for each new database. Much of this effort involves the encoding of semantic knowledge for the domain of discourse, necessary to correctly interpret and respond to natural language questions. For such systems to be practical, techniques must be developed to increase their portability to new domains. This paper discusses several issues involving the portability of natural language interfaces to database systems, and presents the approach taken in CO-OP — a natural language database query system that provides cooperative responses to English questions and operates with a typical CODA-SYL database system. CO-OP derives its domain-specific knowledge from a lexicon (the list of words known to the system) and the information already present in the structure and content of the underlying database. Experience with the implementation suggests that strategies that are not directly derivative of cognitive or linguistic models may nonetheless play an important role in the development of practical natural language systems.

Cooperative responses from a portable natural language query system

This paper describes the design and implementation of CO-OP, a natural language database query system that provides conversationally cooperative responses to natural language requests for data retrieval and operates with a typical CODASYL database management system. A conversationally cooperative response to a question is one which addresses the goals and intentions of the questioner while observing the conventions normally associated with discourse, to demonstrate that the chosen approach captures some fundamental properties of conversational cooperation, CO-OP was designed to be portable to new database domains with a minimum of effort—the lexicon and database itself are the only sources of domain specific knowledge. A limited theory of cooperative behavior in question-answering is informally presented, and a practical technique for applying this theory in a natural language query system is described. Sample transcripts of the system that illustrate cooperative responses in two contrasting applications are given.

A mechanism for natural language access to database systems

This reports on an investigation focused on a transform mechanism for a natural language database query system being undertaken at the University of Cincinnati. The mechanism studies the questions involved in a natural language interface between the human environment and the database query languages used to access machine stored data. A major product of the research would be a methodology for the transformation of database queries in a natural language into database queries stated in the data manipulation language of a DBMS.

Research at the Artificial Intelligence corp.

The Artificial Intelligence Corporation was formed in 1975 to develop and market a commercial natural language database query product. The company currently employs 30 people to enhance, support and market its product known as INTELLECT, which is the productization of the earlier prototype system known as ROBOT.

USING THE DATA BASE AS A SEMANTIC COMPONENT TO AID IN THE PARSING OF NATURAL LANGUAGE DATA BASE QUERIES

Many of the recent advances in artificial intelligence (Al) have been brought about through the use of domain specific knowledge. In this same spirit, this paper presents an approach to understanding natural language data base queries that employs the use of domain specific knowledge to aid the parsing process. The unique aspect of 1he data base query environment relative to other AI problem domains is that the required body of knowledge already exists in the form of the data base being queried. Thus, there is an important benefit of making use of this knowledge in the form in which it is maintained by the data base management system, since the very difficult problems of gathering and representing this information can be circumvented. The paper describes the problems encountered in the parsing of data base queries that can be solved by the semantic use of the data base, as well as a precise description of how these problems can be solved by existing data base systems. The proposed use of the data base as ...