SQL Generation Research Articles

Domain-Specific Few-Shot Table Prompt Question Answering via Contrastive Exemplar Selection

As a crucial task in natural language processing, table question answering has garnered significant attention from both the academic and industrial communities. It enables intelligent querying and question answering over structured data by translating natural language into corresponding SQL statements. Recently, there have been notable advancements in the general domain table question answering task, achieved through prompt learning with large language models. However, in specific domains, where tables often have a higher number of columns and questions tend to be more complex, large language models are prone to generating invalid SQL or NoSQL statements. To address the above issue, this paper proposes a novel few-shot table prompt question answering approach. Specifically, we design a prompt template construction strategy for structured SQL generation. It utilizes prompt templates to restructure the input for each test data and standardizes the model output, which can enhance the integrity and validity of generated SQL. Furthermore, this paper introduces a contrastive exemplar selection approach based on the question patterns and formats in domain-specific contexts. This enables the model to quickly retrieve the relevant exemplars and learn characteristics about given question. Experimental results on the two datasets in the domains of electric energy and structural inspection show that the proposed approach outperforms the baseline models across all comparison settings.

CodeS: Towards Building Open-source Language Models for Text-to-SQL

Language models have shown promising performance on the task of translating natural language questions into SQL queries (Text-to-SQL). However, most of the state-of-the-art (SOTA) approaches rely on powerful yet closed-source large language models (LLMs), such as ChatGPT and GPT-4, which may have the limitations of unclear model architectures, data privacy risks, and expensive inference overheads. To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, specifically designed for the text-to-SQL task. CodeS is a fully open-source language model, which achieves superior accuracy with much smaller parameter sizes. This paper studies the research challenges in building CodeS. To enhance the SQL generation abilities of CodeS, we adopt an incremental pre-training approach using a specifically curated SQL-centric corpus. Based on this, we address the challenges of schema linking and rapid domain adaptation through strategic prompt construction and a bi-directional data augmentation technique. We conduct comprehensive evaluations on multiple datasets, including the widely used Spider benchmark, the newly released BIRD benchmark, robustness-diagnostic benchmarks such as Spider-DK, Spider-Syn, Spider-Realistic, and Dr.Spider, as well as two real-world datasets created for financial and academic applications. The experimental results show that our CodeS achieves new SOTA accuracy and robustness on nearly all challenging text-to-SQL benchmarks.

Criteria2Query 3.0: Leveraging generative large language models for clinical trial eligibility query generation

ObjectiveAutomated identification of eligible patients is a bottleneck of clinical research. We propose Criteria2Query (C2Q) 3.0, a system that leverages GPT-4 for the semi-automatic transformation of clinical trial eligibility criteria text into executable clinical database queries. Materials and MethodsC2Q 3.0 integrated three GPT-4 prompts for concept extraction, SQL query generation, and reasoning. Each prompt was designed and evaluated separately. The concept extraction prompt was benchmarked against manual annotations from 20 clinical trials by two evaluators, who later also measured SQL generation accuracy and identified errors in GPT-generated SQL queries from 5 clinical trials. The reasoning prompt was assessed by three evaluators on four metrics: readability, correctness, coherence, and usefulness, using corrected SQL queries and an open-ended feedback questionnaire. ResultsOut of 518 concepts from 20 clinical trials, GPT-4 achieved an F1-score of 0.891 in concept extraction. For SQL generation, 29 errors spanning seven categories were detected, with logic errors being the most common (n = 10; 34.48 %). Reasoning evaluations yielded a high coherence rating, with the mean score being 4.70 but relatively lower readability, with a mean of 3.95. Mean scores of correctness and usefulness were identified as 3.97 and 4.37, respectively. ConclusionGPT-4 significantly improves the accuracy of extracting clinical trial eligibility criteria concepts in C2Q 3.0. Continued research is warranted to ensure the reliability of large language models.

Improving the Capabilities of Large Language Model based Marketing Analytics Copilots with Semantic Search and Fine-Tuning

Artificial intelligence (AI) is widely deployed to solve problems related to marketing attribution and budget optimization. However, AI models can be quite complex, and it can be difficult to understand model workings and insights without extensive implementation teams. In principle, recently developed large language models (LLMs), like GPT-4, can be deployed to provide marketing insights, reducing the time and effort required to make critical decisions. In practice, there are substantial challenges that need to be overcome to reliably usesuch models. We focus on domain-specific question-answering, SQL generation needed for data retrieval, and tabular analysis and show how a combination of semantic search, prompt engineering, and fine-tuning can be applied to dramatically improve the ability of LLMs to execute these tasks accurately. We compare both proprietary models, like GPT-4, and open-source models, like Llama-2-70b, as well as various embedding methods. These models are tested on sample use cases specific to marketing mix modeling and attribution.

HFD: Hierarchical feature decoupling for SQL generation from text

Text-to-SQL, a computational linguistics task, seeks to facilitate the conversion of natural language queries into SQL queries. Recent methodologies have leveraged the concept of slot-filling in conjunction with predetermined SQL templates to effectively bridge the semantic gap between natural language questions and structured database queries, achieving commendable performance by harnessing the power of multi-task learning. However, employing identical features across diverse tasks is an ill-suited practice, fraught with inherent drawbacks. Firstly, based on our observation, there are clear boundaries in the natural language corresponding to SELECT and WHERE clauses. Secondly, the exclusive features integral to each subtask are inadequately emphasized and underutilized, thereby hampering the acquisition of discriminative features for each specific subtask. In an endeavor to rectify these issues, the present work introduces an innovative approach: the hierarchical feature decoupling model for SQL query generation from natural language. This novel approach involves the deliberate separation of features pertaining to subtasks within both SELECT and WHERE clauses, further dissociating these features at the subtask level to foster better model performance. Empirical results derived from experiments conducted on the WikiSQL benchmark dataset reveal the superiority of the proposed approach over several state-of-the-art baseline methods in the context of text-to-SQL query generation.

GTR: An SQL Generator With Transition Representation in Cross-Domain Database Systems.

Recent studies have focused on using natural language (NL) to automatically retrieve useful data from database (DB) systems. As an important component of autonomous DB systems, the NL-to-SQL technique can assist DB administrators in writing high-quality SQL statements and make persons with no SQL background knowledge learn complex SQL languages. However, existing studies cannot deal with the issue that the expression of NL inevitably mismatches the implementation details of SQLs, and the large number of out-of-domain (OOD) words makes it difficult to predict table columns. In particular, it is difficult to accurately convert NL into SQL in an end-to-end fashion. Intuitively, it facilitates the model to understand the relations if a "bridge" transition representation (TR) is employed to make it compatible with both NL and SQL in the phase of conversion. In this article, we propose an automatic SQL generator with TR called GTR in cross-domain DB systems. Specifically, GTR contains three SQL generation steps: 1) GTR learns the relation between questions and DB schemas; 2) GTR uses a grammar-based model to synthesize a TR; and 3) GTR predicts SQL from TR based on the rules. We conduct extensive experiments on two commonly used datasets, that is, WikiSQL and Spider. On the testing set of the Spider and WikiSQL datasets, the results show that GTR achieves 58.32% and 71.29% exact matching accuracy which outperforms the state-of-the-art methods, respectively.

A survey of deep learning based text-to-SQL generation

Text-to-SQL is an important application of automation software engineering. It is also a research hotspot in the field of semantic parsing. The text-to-SQL task aims to automatically generate the SQL statement according to the natural language description. It allows nonprofessionals to access the database without understanding SQL syntax. With the development of large-scale text-to-SQL datasets and artificial intelligence technologies, the text-to-SQL task is also making great progress. Compared with the traditional text-to-SQL generation, the deep learning-based text-to-SQL has the advantages of high accuracy, flexibility, and iterative learning. In recent years, several studies have focused on SQL generation based on deep learning. This research summarizes existing works from the aspects of text-to-SQL scenarios, datasets, model structures, and evaluation methods.

NL2SQL Generation with Noise Labels based on Multi-task Learning

With the rapid development of artificial intelligence technology, semantic recognition technology is becoming more and more mature, providing the preconditions for the development of natural language to SQL (NL2SQL) technology. In the latest research on NL2SQL, the use of pre-trained models as feature extractors for natural language and table schema has led to a very significant improvement in the effectiveness of the models. However, the current models do not take into account the degradation of the noisy labels on the overall SQL statement generation. It is crucial to reduce the impact of noisy labels on the overall SQL generation task and to maximize the return of accurate answers. To address this issue, we propose a restrictive constraint-based approach to mitigate the impact of noise-labeled labels on other tasks. In addition, parameter sharing approach is used in noiseless-labeled labels to capture each part’s correlations and improve the robustness of the model. In addition, we propose to use Kullback-Leibler divergence to constrain the discrepancy between hard and soft constrained coding of noisy labels. Our model is compared with some recent state-of-the-art methods, and experimental results show a significant improvement over the approach in this paper.

다국어 BERT를 활용한 한국어 자연어 질의의 SQL 변환

Text-to-SQL is one of semantic parsing methods that converts natural language questions into SQL queries, and it aims to extract data from any relational database without knowledge of SQL query configuration. Although development of large amounts of datasets (WikiSQL, SPIDER) and development of pre-trained language models (BERT) contributed to the improvement of Text-to-SQL performance in English, language-specific dataset construction and model research have not been much progressed. Therefore, this study proposes a multilingual BERT-based Text-to-SQL methodology that converts the natural language question in Korean into SQL query for an English database. To this end, four strategies for translating Korean queries into English were explored, and their effectiveness was verified by applying each strategy to three text-to-SQL model structures. As a result of the experiment, it was confirmed that it showed a significant SQL generation performance even for Korean questions. The proposed methodology is meaningful in that it shows semantic inferences between database tables, column information, and questions composed of different languages are possible, and it is expected to support efficient database access by Korean users who lack proficiency in writing SQL queries.

SQL Generation from Natural Language: A Sequence-to-Sequence Model Powered by the Transformers Architecture and Association Rules

Using Natural Language (NL) to interacting with relational databases allows users from any background to easily query and analyze large amounts of data. This requires a system that understands user questions and automatically converts them into structured query language such as SQL. The best performing Text-to-SQL systems use supervised learning (usually formulated as a classification problem) by approaching this task as a sketch-based slot-filling problem, or by first converting questions into an Intermediate Logical Form (ILF) then convert it to the corresponding SQL query. However, non-supervised modeling that directly converts questions to SQL queries has proven more difficult. In this sense, we propose an approach to directly translate NL questions into SQL statements. In this study, we present a Sequence-to-Sequence (Seq2Seq) parsing model for the NL to SQL task, powered by the Transformers Architecture exploring the two Language Models (LM): Text-To-Text Transfer Transformer (T5) and the Multilingual pre-trained Text-To-Text Transformer (mT5). Besides, we adopt the transformation-based learning algorithm to update the aggregation predictions based on association rules. The resulting model achieves a new state-of-the-art on the WikiSQL DataSet, for the weakly supervised SQL generation.

Pattern Mining and Detection of Malicious SQL Queries on Anonymization Mechanism

With the striking development of big data, individual privacy and data security obtain unprecedented importance. Database anonymization mechanism is created for protecting individual privacy by adding noise to the result of a query, which finds a tradeoff between the privacy and utility of personal data. However, corresponding attacks are emerging continuously resulting in a high risk of individual identification. In this paper, we learn patterns of malicious SQL queries and propose a novel detection method. Association rules are used to mine patterns and features of noise-exploitation attacks, and parse trees are applied to the feature extraction of SQL, thereby we construct feature vectors and input them into the classifiers. At the same time, we also propose a SQL generation method to generate query samples based on a real database for model training and testing. Experiments show that our detection method can significantly prevent noise-exploitation attacks including almost all differential attacks and 91% cloning attacks based on the synthetic dataset, which ensures a strong degree of data utility.

Natural Language to SQL Generation for Semantic Knowledge Extraction in Social Web Sources

Enormous evolution of web data creates a peculiar myth in the field of computer and information technology for extracting the meaningful content from the web. Many organizations and social networks use databases for storing information and the data will be fetched from the specified data store. Data can be retrieved or accessed by SQL queries whereas the query is in the form of natural lingual statement which has to be processed. So, the primary objective of this research article is to find the suitable way to convert natural language query to SQL and make the data apt for semantic extraction. This Research paper also aims to derive an automatic query translator for Natural Language based questions into their associated SQL queries and provides an user friendly interface between end user and the database for easy access of social web data from different web sources such as facebook, twitter and linkedIn etc.,. This paper is implemented using java as the front end, SQL server as the back end and R-tool is used to collect the data from social web sources. This research article provides an optimized SQL query generation for the Natural Language question provided by the end user.