Intent Classification Research Articles

Few‐Shot Contrastive Learning‐Based Multi‐Round Dialogue Intent Classification Method

ABSTRACTTraditional text classification models face challenges in handling long texts and understanding topic transitions in dialogue scenarios, leading to suboptimal performance in automatic speech recognition (ASR)‐based multi‐round dialogue intent classification. In this article, we propose a few‐shot contrastive learning‐based multi‐round dialogue intent classification method. First, the ASR texts are partitioned, and role‐based features are extracted using a Transformer encoder. Second, refined sample pairs are forward‐propagated, adversarial samples are generated by perturbing word embedding matrices and contrastive loss is applied to positive sample pairs. Then, positive sample pairs are input into a multi‐round reasoning module to learn semantic clues from the entire scenario through multiple dialogues, obtain reasoning features, input them into a classifier to obtain classification results, and calculate multi‐task loss. Finally, a prototype update module (PUM) is introduced to rectify the biased prototypes by using gated recurrent unit (GRU) to update the prototypes stored in the memory bank and few‐shot learning (FSL) task. Experimental evaluations demonstrate that the proposed method outperforms state‐of‐the‐art methods on two public datasets (DailyDialog and CM) and a private real‐world dataset.

Learning and semiautomatic intention labeling for classification models: a COVID-19 dialog attendance study for chatbots

Abstract It is increasingly common to use chatbots as an interface to services. One of the main components of a chatbot is the Natural Language Understanding (NLU) model, which is responsible for interpreting the text and extracting the intent and entities present in that text. It’s possible to focus only on one of these tasks of NLU, such as intent classification. To train an NLU intent classification model, it’s generally necessary to use a considerable amount of annotated data, where each sentence of the dataset receives a label indicating an intent. Performing manually labeling data is arduous and impracticable, depending on the data volume. Thus, an unsupervised machine learning technique, such as data clustering, could be applied to find and label patterns in the data. For this task, it is essential to have an effective vector embedding representation of texts that depicts the semantic information and helps the machine understand the context, intent, and other nuances of the entire text. This paper extensively evaluates different text embedding models for clustering and labeling. We also apply some operations to improve the dataset’s quality, such as removing sentences and establishing various strategies for distance thresholds (cosine similarity) for the clusters’ centroids. Then, we trained some intent classification Models with two different architectures, one built with the Rasa framework and the other with a neural network (NN) using the attendance text from the Coronavirus Platform Service of Ceará, Brazil. We also manually annotated a dataset to be used as validation data. We conducted a study on semiautomatic labeling, implemented through clustering and visual inspection, which introduced some labeling errors to the intent classification models. However, it would be unfeasible to annotate the entire dataset manually. Nevertheless, results of competitive accuracy were still achieved with the trained models.

Enhancing Intent Classifier Training with Large Language Model-generated Data

ABSTRACT Intent classification is essential in Natural Language Processing, serving applications like virtual assistants and customer service by categorizing user inputs into predefined classes. Despite its importance, the effectiveness of intent classifiers is often constrained by the scarcity of labeled data, as acquiring substantial, annotated datasets is costly and impractical. Data augmentation addresses this by expanding datasets with modified or synthetic examples, a common practice in computer vision but more complex in NLP due to the discrete nature of language. Traditional NLP data augmentation methods have been explored but exhibit limitations. This paper investigates the use of Large Language Models for generating labeled data to enhance intent classification. We explore whether LLM-generated data can effectively augment training sets, comparing its impact on intent classifier performance against traditional augmentation methods. Our study reveals that LLMs can generate diverse and realistic data, potentially improving classifier accuracy in low-data scenarios, thereby providing valuable insights into leveraging generative AI for NLP tasks in real-world applications.

Empowering geoportals HCI with task-oriented chatbots through NLP and deep transfer learning

ABSTRACT In the past ten years, chatbot development has matured to become one of the most well-distinguished outcomes of artificial intelligence. Despite some criticism, Bing AI, ChatGPT and other natural language processing (NLP) products of similar nature are becoming more popular. The creation of chatbots can close several gaps in geographic information retrieval as well. This research introduces and successfully implements, for the first time, a model for integrating task-oriented chatbots into geoportals, with the goal of easing user requests, improving access to geospatial services, and fostering human-computer interactions (HCI). Additionally, it presents a novel recommendation solution for matching the most appropriate volunteer to the user’s geospatial needs based on expertise similarity, semantic similarity, and community feedback. The three categories of finding map services, discovering geoprocessing services, and volunteer expert recommendations were shown to be the most significant geoportal bot intents. Depending on the requirement, each intent additionally includes various entities such as time, place, description, skill, etc. The notion of deep transfer learning (DTL) was then put into practice by customizing a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model for our particular aim and creating a task-oriented conversational agent. According to the results, effective intent classification and entity recognition in the geospatial domain could arise from this approach. We performed the training process with 200 sample data, 20% of which were utilized in a stratified manner for testing, and we obtained f1-scores of at least 0.75. Finally, a pilot Geoportal Chabot that combines crowdsourcing and conversational agents’ approaches was put into use and tested with success. In keeping with SDI technical purposes, the system might direct users to common geospatial web services, namely WMS and WPS, in addition to including natural language understanding (NLU) and natural language generation (NLG) capabilities. Result of user-centered evaluation indicated that the integration of a chatbot significantly reduces the average time required to access geospatial data and processing services by more than 50%. Notably, this effect is even more pronounced when locating an expert, with a fivefold decrease in the time required. Finally, overall user satisfaction rose from 86% to 94%.

Analysis of Dataset Limitations in Semantic Knowledge-Driven Multi-Variant Machine Translation

In this study, we explore the implications of dataset limitations in semantic knowledge-driven machine translation (MT) for intelligent virtual assistants (IVA). Our approach diverges from traditional single-best translation techniques, utilizing a multi-variant MT method that generates multiple valid translations per input sentence through a constrained beam search. This method extends beyond the typical constraints of specific verb ontologies, embedding within a broader semantic knowledge framework. We evaluate the performance of multi-variant MT models in translating training sets for Natural Language Understanding (NLU) models. These models are applied to semantically diverse datasets, including a detailed evaluation using the standard MultiATIS++ dataset. The results from this evaluation indicate that while multi-variant MT method is promising, its impact on improving intent classification (IC) accuracy is limited when applied to conventional datasets such as MultiATIS++. However, our findings underscore that the effectiveness of multi-variant translation is closely associated with the diversity and suitability of the datasets utilized. Finally, we provide an in-depth analysis focused on generating variant-aware NLU datasets. This analysis aims to offer guidance on enhancing NLU models through semantically rich and variant-sensitive datasets, maximizing the advantages of multi-variant MT.

Enhancing Chatbot Intent Classification using Active Learning Pipeline for Optimized Data Preparation

This study presents a novel approach to enhancing chatbot intent classification through an optimized data preparation combined with Active Learning. We applied the clustering mechanism using a state-of-the-art sentence-transformers model with cosine similarity for cluster detection in order to categorize messages. This process was further refined through a dedicated Active Learning Pipeline, which focused on the most essential observations for labeling. Incorporating externally sourced labeled data from Scale AI, the labeling process was fine-tuned iteratively, until the model's performance stabilized. This approach shows promise for various datasets and tasks, suggesting a scalable solution for preparing data for supervised modeling and achieving optimal model performance in real-world commercial chatbot scenarios.

Integrating Chatbot Functionality in a Patient Summary Based Healthcare System.

The integration of chatbots in healthcare has gained attention due to their potential to enhance patient engagement and satisfaction. This paper presents a healthcare chatbot providing comprehensive access to patient summaries, aligned with the European Patient Summary. Leveraging Natural Language Processing (NLP) capabilities, our chatbot employs intent classification using the fine-tuned bioBERT model to categorize user queries effectively and extract relevant information from the patient summary stored in a database. We detail our methodology, which involves dataset creation, hyperparameter tuning, and model evaluation. Results demonstrate the effectiveness of our approach, with the trained model achieving high precision, recall, and F1 score across intent classes. Our study underscores the potential of emerging NLP techniques in patient interaction and healthcare delivery, covering the way for smarter, user-friendly companions.

Three-way open intent classification with nearest centroid-based representation

Open intent classification aims to identify the unknown (open) intents and simultaneously classify the known ones under the open-world assumption. However, the existing studies still face two challenges, i.e., coarse-grained representation learning and uncertain decision boundary. On the one hand, previous methods viewed each class as a unified entity during representation learning, which fails to capture the fine-grained intra-class data structure. On the other hand, traditional two-way decision for open classification struggle to classify the uncertain samples distributed at the edge of the decision boundary, increasing the risk of misclassification. To overcome these limitations, we present a three-way open intent classification method that utilizes the nearest centroid to learn representations, named 3WNC-Open. Specifically, we learn a structured representation by extracting fine-grained information from the sub-classes within each class. Then, we design a three-way open classification strategy to handle uncertainty, initially identifying uncertain samples and then processing them using an effective alternative approach. Experiments on challenging datasets demonstrate that 3WNC-Open outperforms strong baselines.

Few-shot multi-domain text intent classification with Dynamic Balance Domain Adaptation Meta-learning

User intents are ever-changing, which requires deep learning models to have the ability to classify unknown intents. Meta-learning aims to solve this problem by improving the model’s generalization ability to unknown intent. However, learning on a small amount of text can easily lead to overfitting of the model. Domain adaptation can help us train a more robust model. However, most existing methods only focus on global feature alignment and ignore alignment in subdomains. Therefore, in this study, we first consider the case where the model can maintain robustness with a small amount of data and then explore and mine the higher quality transferable features. Based on these ideas, we propose Dynamic Balance Domain Adaptation Meta-learning (DBDAML), which adaptively learns higher quality transferable features in both the global domain and subdomains.11in this paper, each subdomain represents the domain under a separate class. At the same time, we define a dynamic balance factor to enable DBDAML to dynamically focus on the global domain and subdomains. This allows the model to give different attention to different domain adaptations and prevents it from overfitting of a domain feature alignment. The dynamic balance factor is estimated by the contribution of different domain discriminators to the loss, which also makes it easy to calculate and accurate. Finally, we use the meta-learning framework to model the entire theoretical idea. Extensive experiments demonstrate that our approach achieves better performance than state-of-the-art baseline methods.

FarmTechBot (FTB): Empowering farmers for better harvest

In an evolving world, the FarmTechBot (FTB) emerges as a pioneering digital assistant, empowering farmers with essential insights and real-time information crucial for optimizing agricultural practices. The presence of chatbot functionality capable of addressing queries related to soil, pest, and market linkage associates to overcome hurdles for users seeking real-time information. Furthermore, the lack of voice recognition capabilities limits accessibility, especially for users who prefer or require voice-based interaction. Additionally, the manual process of pest identification from uploaded crop images hampers efficiency, prolonging the time required for pest management. The FTB represents a significant advancement in agricultural technology, addressing the existing system's limitations and empowering farmers with real-time information, enhanced accessibility, and streamlined pest management capabilities. The FTB utilizes a combination of techniques including Histogram of Oriented Gradients (HOG) for image feature extraction and classification using Support Vector Classifier (SVC). In Natural Language Processing (NLP), it employs tokenization and TF-IDF vectorization for text preprocessing and representation, alongside Linear Support Vector Classifier (LinearSVC) for intent classification.

Towards Building a Chatbot-Based First Aid Service in Arabic Language

In this paper, we introduce a modern standard Arabic (MSA) conversational AI in the field of first aid to assist anyone facing an emergency case. The proposed chatbot -based model not only provide adequate medical assistance, but also engage in empathetic interactions with users. Due to the lack of a reliable dataset that fits our purpose, we collect and construct a dataset of 374 records covering 55 first aid topic. Our chatbot architecture relies fine tuning pretrained language models in its intent classification and employs textual similarity to match user queries to records from the dataset. We evaluate the proposed model based using both a quantitative approach, through performance metrics of different modules, and a qualitative approach using a questionnaire.

Enhancing Customer Service in Banking with AI: Intent Classification Using Distilbert

With the increasing demand for efficient and responsive customer service in the banking sector, artificial intelligence offers a promising solution. This paper presents a comparative analysis of artificial intelligence methodologies applied to intent classification within the banking sector customer service domain. Utilizing a comprehensive dataset of banking service inquiries, we evaluate several machine learning approaches, including Naive Bayes, Logistic Regression, Support Vector Machine with Linear Kernel, Random Forest, XGBoost, and the transformer-based DistilBERT model. The models are assessed based on their accuracy, precision, recall, and F1 score metrics. Our findings indicate that DistilBERT, with its distilled architecture, not only outstrips traditional models but also demonstrates exceptional performance with an accuracy and F1 score exceeding 92%. The paper delves into the advantages of employing such an efficient and powerful model in real-time customer service settings, suggesting that DistilBERT offers a substantial enhancement over conventional methods. By providing detailed insights into the model’s capabilities, we underscore the transformative impact of employing advanced AI in the financial industry to elevate customer service standards, streamline operational efficiency, and harness the power of state-of-the-art technology for improved client interactions. The results showcased in this study are indicative of the strides being made in AI applications for financial services and set a benchmark for future exploratory and practical endeavors in the field.

EMS injury cause codes more accurate than emergency department visit ICD-10-CM codes for firearm injury intent in North Carolina.

The timeliness, accuracy, and completeness of data for firearm injury surveillance is crucial for public health surveillance efforts and informing injury prevention measures. While emergency department (ED) visit data can provide near real-time information on firearms injuries, there are concerns surrounding the accuracy of intent coding in these data. We examined whether emergency medical service (EMS) data provide more accurate firearm injury intent coding in comparison to ED data. We applied a firearm injury definition to EMS encounter data in NC's statewide syndromic surveillance system (NC DETECT), from January 1, 2021, through December 31, 2022. We manually reviewed each record to determine intent, and the corresponding manual classifications were compared to the injury cause codes entered in the EMS data and to ED visit records where EMS-ED record linkage was possible. We then calculated the sensitivity, specificity, positive and negative predictive values for each intent classification in SAS 9.4 using the manually reviewed intent classifications as the gold standard. We identified 9557 EMS encounters from January 1, 2021, through December 31, 2022 meeting our firearm injury definition. After removing false positives and duplicates, 8584 records were available for manual injury classification. Overall, our analysis demonstrated that manual and EMS injury cause code classifications were comparable. However, for the 3401 EMS encounters that could be linked to an ED visit record, sensitivity of the ED ICD-10-CM codes was low for assault and intentional self-harm encounters at 18.2% (CI 16.5-19.9%) and 22.2% (CI 16-28.5%), respectively. This demonstrates a marked difference in the reliability of the intent coding in the two data sources. This study illustrates both the value of examining EMS encounters for firearm injury intent, and the challenges of accurate intent coding in the ED setting. EMS coding has the potential for more accurate intent coding than ED coding within the context of existing hospital-based coding guidance. This may have implications for future firearm injury research, especially for nonfatal firearm injuries.

ESIE-BERT: Enriching sub-words information explicitly with BERT for intent classification and slot filling

Natural language understanding (NLU) has two core tasks: intent classification and slot filling. The success of pre-training language models resulted in a significant breakthrough in the two tasks. The architecture based on autoencoding (BERT-based model) can optimize the two tasks jointly. However, we note that BERT-based models convert each complex token into multiple sub-tokens by the Wordpiece algorithm, which generates an out-of-alignment between the length of the tokens and the labels. This leads to BERT-based models not performing well in label prediction, limiting model performance improvement. Many existing models can address this issue, but some hidden semantic information is discarded during fine-tuning. We addressed the problem by introducing a novel joint method on top of BERT. This method explicitly models multiple sub-token features after the Wordpiece tokenization, contributing to both tasks. Our proposed method effectively extracts contextual features from complex tokens using the Sub-words Attention Adapter (SAA), preserving overall utterance information. Additionally, we propose an Intent Attention Adapter (IAA) to acquire comprehensive sentence features, assisting users in predicting intent. Experimental results confirm that our proposed model exhibits significant improvements on two public benchmark datasets. Specifically, the slot-filling F1 score improves from 96.5 to 98.2 (an absolute increase of 1.7%) on the Airline Travel Information Systems (ATIS) dataset.

Natural Language Understanding for Navigation of Service Robots in Low-Resource Domains and Languages: Scenarios in Spanish and Nahuatl

Human–robot interaction is becoming increasingly common to perform useful tasks in everyday life. From the human–machine communication perspective, achieving effective interaction in natural language is one challenge. To address it, natural language processing strategies have recently been used, commonly following a supervised machine learning framework. In this context, most approaches rely on the use of linguistic resources (e.g., taggers or embeddings), including training corpora. Unfortunately, such resources are scarce for some languages in specific domains, increasing the complexity of solution approaches. Motivated by these challenges, this paper explores deep learning methods for understanding natural language commands emitted to service robots that guide their movements in low-resource scenarios, defined by the use of Spanish and Nahuatl languages, for which linguistic resources are scarcely unavailable for this specific task. Particularly, we applied natural language understanding (NLU) techniques using deep neural networks and transformers-based models. As part of the research methodology, we introduced a labeled dataset of movement commands in the mentioned languages. The results show that models based on transformers work well to recognize commands (intent classification task) and their parameters (e.g., quantities and movement units) in Spanish, achieving a performance of 98.70% (accuracy) and 96.96% (F1) for the intent classification and slot-filling tasks, respectively). In Nahuatl, the best performance obtained was 93.5% (accuracy) and 88.57% (F1) in these tasks, respectively. In general, this study shows that robot movements can be guided in natural language through machine learning models using neural models and cross-lingual transfer strategies, even in low-resource scenarios.

Campus Companion : Creating a Supportive Chat – Assistant for Students

This article explores the creation and usage of chatbots, or intelligent conversational agents, for online communication. Python and other machine learning methods are employed in the chatbot's design. To be more explicit, TensorFlow/Keras is used for natural language processing, MySQL is utilized for database administration, and Flask is used for web hosting. The initiative attempts to accomplish its goals by enhancing the user experience and fostering more effective communication on the website. Using massive datasets, the chatbot will be educated to comprehend users' objectives and offer replies that fit with them. Natural language processing technologies, including voice recognition, natural language interpretation, and natural language production, will be used to foster natural discussions. A number of machine learning approaches, such as transformers, attention processes, and recurrent neural networks, will be investigated in order to classify intent and deliver replies. The objective is to construct a powerful and versatile chatbot that can grasp arguments in their context, preserve the debate's status, and answer in a manner that sounds human. Assessment criteria such as accuracy, recall, and precision will be utilized to develop and adapt the chatbot. Numerous businesses, including education, entertainment, customer service, and other disciplines, may profit from the employment of this chatbot. The implementation of intelligent conversational interfaces on websites has the potential to greatly enhance user experience. Keywords— Chatbot, Flask, MySQL, TensorFlow, Keras, Website Interaction, Natural Language Processing (NLP), Conversational AI, Intent Classification, Response Generation, Machine Learning Models, Recurrent Neural Networks (RNNs), Transformer Models, Attention Mechanisms, User Experience (UX), Contextual Conversations, Dialogue State Management, Human-like Responses, Customer Service Automation, Educational Chatbot, Entertainment Applications.

Conversational AI: An Explication of Few-Shot Learning Problem in Transformers-Based Chatbot Systems

With the recent advancements in conversational artificial intelligence (AI), the practical applications of chatbots have risen significantly in diverse domains such as healthcare, education, e-government, customer support systems, social platforms, and entertainment. The chatbot converses with humans in natural language and responds to their queries with precise and relevant answers. The relevant literature presents several methods for intent classification and slot mapping for chatbot question answering. However, the existing chatbot architectures still suffer from few-shot learning problem (imbalance ratio of class labels over samples) and ineffective dialog management to retain the context and slots mapping. This study aims to introduce the architecture of chatbot by focusing few-shot learning problem and context management in dialog-based conversations. First, to mitigate the few-shot learning problem with chatbots, we propose a novel hybrid intent and slots transformers (HIST) model. The HIST chatbot architecture utilizes transformers and self-attention mechanisms along with bigated recurrent unit and combines conditional random field algorithms for intent classification and slot extraction. Second, to address dialog management, we introduce a hybrid interaction strategy for slots mapping and effective conversational context management. To validate the proposed model’s effectiveness, comprehensive empirical analysis is carried out using three benchmark datasets including airline travel information system (ATIS), banking77, and conversational language interface for natural conversation 150 (CLINC150). The results show that HIST outperforms against the state-of-the-art existing methods with a clear margin and obtained an accuracy of 94.89% and 96.17% for intent classification and slots extraction, respectively. The empirical results confirm the effectiveness of the HIST chatbot for resolving the few-shot learning problem with effective dialog management in chatbot systems.

Commit-Level Software Change Intent Classification Using a Pre-Trained Transformer-Based Code Model

Software evolution is driven by changes made during software development and maintenance. While source control systems effectively manage these changes at the commit level, the intent behind them are often inadequately documented, making understanding their rationale challenging. Existing commit intent classification approaches, largely reliant on commit messages, only partially capture the underlying intent, predominantly due to the messages’ inadequate content and neglect of the semantic nuances in code changes. This paper presents a novel method for extracting semantic features from commits based on modifications in the source code, where each commit is represented by one or more fine-grained conjoint code changes, e.g., file-level or hunk-level changes. To address the unstructured nature of code, the method leverages a pre-trained transformer-based code model, further trained through task-adaptive pre-training and fine-tuning on the downstream task of intent classification. This fine-tuned task-adapted pre-trained code model is then utilized to embed fine-grained conjoint changes in a commit, which are aggregated into a unified commit-level vector representation. The proposed method was evaluated using two BERT-based code models, i.e., CodeBERT and GraphCodeBERT, and various aggregation techniques on data from open-source Java software projects. The results show that the proposed method can be used to effectively extract commit embeddings as features for commit intent classification and outperform current state-of-the-art methods of code commit representation for intent categorization in terms of software maintenance activities undertaken by commits.

Decoupling Representation and Knowledge for Few-Shot Intent Classification and Slot Filling

Few-shot intent classification and slot filling are important but challenging tasks due to the scarcity of finely labeled data. Therefore, current works first train a model on source domains with sufficiently labeled data, and then transfer the model to target domains where only rarely labeled data is available. However, experience transferring as a whole usually suffers from gaps that exist among source domains and target domains. For instance, transferring domain-specific-knowledge-related experience is difficult. To tackle this problem, we propose a new method that explicitly decouples the transferring of general-semantic-representation-related experience and the domain-specific-knowledge-related experience. Specifically, for domain-specific-knowledge-related experience, we design two modules to capture intent-slot relation and slot-slot relation respectively. Extensive experiments on Snips and FewJoint datasets show that our method achieves state-of-the-art performance. The method improves the joint accuracy metric from 27.72% to 42.20% in the 1-shot setting, and from 46.54% to 60.79% in the 5-shot setting.

Tayseer: A Novel AI-Powered Arabic Chatbot Framework for Technical and Vocational Student Helpdesk Services and Enhancing Student Interactions

The rise of conversational agents (CAs) like chatbots in education has increased the demand for advisory services. However, student–college admission interactions remain manual and burdensome for staff. Leveraging CAs could streamline the admission process, providing efficient advisory support. Moreover, limited research has explored the role of Arabic chatbots in education. This study introduces Tayseer, an Arabic AI-powered web chatbot that enables instant access to college information and communication between students and colleges. This study aims to improve the abilities of chatbots by integrating features into one model, including responding with audiovisuals, various interaction modes (menu, text, or both), and collecting survey responses. Tayseer uses deep learning models within the RASA framework, incorporating a customized Arabic natural language processing pipeline for intent classification, entity extraction, and response retrieval. Tayseer was deployed at the Technical College for Girls in Najran (TCGN). Over 200 students used Tayseer during the first semester, demonstrating its efficiency in streamlining the advisory process. It identified over 50 question types from inputs with a 90% precision in intent and entity predictions. A comprehensive evaluation illuminated Tayseer’s proficiency as well as areas requiring improvement. This study developed an advanced CA to enhance student experiences and satisfaction while establishing best practices for education chatbot interfaces by outlining steps to build an AI-powered chatbot from scratch using techniques adaptable to any language.