Slot Tagging Research Articles

CEPTNER: Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) systems aim to classify unseen named entity types with limited labeled examples. Significant progress has been made in the use of large-scale pre-trained language models. However, boundary information and hidden relationships between entities beyond the sequence, which provide additional information and play crucial roles in few-shot NER, have received little attention in recent methods that achieve the state of the art. In this paper, we propose CEPTNER, a Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition, which leverages meta-learning and prototypical network to identify unseen entity types with limited labeled data. Concretely, we first detect candidate boundaries of entities in the stage of boundary detector, then we employ a prototypical network to filter false boundaries and type for the remaining spans in the stage of entity classifier. Additionally, we conduct entity-level contrastive learning to explore the internal relationship between entities that provides extra information while optimizing the prototypical network. CEPTNER is evaluated on two widely-used few-shot NER datasets and a few-shot slot tagging dataset: Few-NERD, CrossNER and SNIPS. Extensive experiments on various benchmarks show the superiority of CEPTNER over previous methods for few-shot NER.

A comparison of few-shot and traditional named entity recognition models for medical text.

Many research problems involving medical texts have limited amounts of annotated data available (e.g., expressions of rare diseases). Traditional supervised machine learning algorithms, particularly those based on deep neural networks, require large volumes of annotated data, and they underperform when only small amounts of labeled data are available. Few-shot learning (FSL) is a category of machine learning models that are designed with the intent of solving problems that have small annotated datasets available. However, there is no current study that compares the performances of FSL models with traditional models (e.g., conditional random fields) for medical text at different training set sizes. In this paper, we attempted to fill this gap in research by comparing multiple FSL models with traditional models for the task of named entity recognition (NER) from medical texts. Using five health-related annotated NER datasets, we benchmarked three traditional NER models based on BERT-BERT-Linear Classifier (BLC), BERT-CRF (BC) and SANER; and three FSL NER models-StructShot & NNShot, Few-Shot Slot Tagging (FS-ST) and ProtoNER. Our benchmarking results show that almost all models, whether traditional or FSL, achieve significantly lower performances compared to the state-of-the-art with small amounts of training data. For the NER experiments we executed, the F1-scores were very low with small training sets, typically below 30%. FSL models that were reported to perform well on non-medical texts significantly underperformed, compared to their reported best, on medical texts. Our experiments also suggest that FSL methods tend to perform worse on data sets from noisy sources of medical texts, such as social media (which includes misspellings and colloquial expressions), compared to less noisy sources such as medical literature. Our experiments demonstrate that the current state-of-the-art FSL systems are not yet suitable for effective NER in medical natural language processing tasks, and further research needs to be carried out to improve their performances. Creation of specialized, standardized datasets replicating real-world scenarios may help to move this category of methods forward.

A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario

This study evaluates the impacts of slot tagging and training data length on joint natural language understanding (NLU) models for medication management scenarios using chatbots in Spanish. In this study, we define the intents (purposes of the sentences) for medication management scenarios and two types of slot tags. For training the model, we generated four datasets, combining long/short sentences with long/short slots, while for testing, we collect the data from real interactions of users with a chatbot. For the comparative analysis, we chose six joint NLU models (SlotRefine, stack-propagation framework, SF-ID network, capsule-NLU, slot-gated modeling, and a joint SLU-LM model) from the literature. The results show that the best performance (with a sentence-level semantic accuracy of 68.6%, an F-score of 76.4% for slot filling, and an accuracy of 79.3% for intent detection) is achieved using short sentences and short slots. Our results suggest that joint NLU models trained with short slots yield better results than those trained with long slots for the slot filling task. The results also indicate that short slots could be a better choice for the dialog system because of their simplicity. Importantly, the work demonstrates that the performance of the joint NLU models can be improved by selecting the correct slot configuration according to the usage scenario.

Natural language understanding of map navigation queries in Roman Urdu by joint entity and intent determination.

Navigation based task-oriented dialogue systems provide users with a natural way of communicating with maps and navigation software. Natural language understanding (NLU) is the first step for a task-oriented dialogue system. It extracts the important entities (slot tagging) from the user’s utterance and determines the user’s objective (intent determination). Word embeddings are the distributed representations of the input sentence, and encompass the sentence’s semantic and syntactic representations. We created the word embeddings using different methods like FastText, ELMO, BERT and XLNET; and studied their effect on the natural language understanding output. Experiments are performed on the Roman Urdu navigation utterances dataset. The results show that for the intent determination task XLNET based word embeddings outperform other methods; while for the task of slot tagging FastText and XLNET based word embeddings have much better accuracy in comparison to other approaches.

Graph Based Training Resource Allocation Scheme for CoMP Transmission in CRAN: A Low Complexity Solution

Coordinated Multi Point (CoMP) transmission in Cloud Radio Area Network (CRAN) strives to provide the required Quality of Service (QoS), irrespective of user's location. Specifically, CoMP enables intra-cluster Access Points (APs) to maximize user's performance utilizing channel information, which is obtained by exchanging prior-known pilot signals among the intended users and intra-cluster APs. These signals are often referred as training signals, and are shared over Training Resources (TRs). However, CoMP is pillared on user-centric network, thereby the resulting clusters are inevitably overlapped. As a result, the TR allocation schemes for these networks require considerably large computational complexity. In this paper, we introduce the notion of Time Slot Tagging (TST), in which each AP is tagged with an appropriate training slot. Further, clusters are formed on the basis of this tagged information. Specifically, TST eliminates the requirement of resource allocation for individual users which saves a significant time in each clustering cycle, and thus enhances the link utilization factor. Also, TST is designed in a manner that it assures the presence of preferred APs in each intended cluster. Further, it is shown that the proposed approach performs significantly close to the most optimistic scenario.

From context-aware to knowledge-aware: Boosting OOV tokens recognition in slot tagging with background knowledge

Neural-based context-aware models for slot tagging tasks in language understanding have achieved state-of-the-art performance, especially deep contextualized models, such as ELMo, BERT. However, the presence of out-of-vocab (OOV) words significantly degrades the performance of neural-based models, especially in a few-shot scenario. In this paper, we propose a novel knowledge-aware slot tagging model to integrate contextual representation of input text and the large-scale lexical background knowledge. Besides, we use multi-level graph attention to explicitly reason via lexical relations. We aim to leverage both linguistic regularities covered by deep language models (LM) and high-quality background knowledge derived from curated knowledge bases (KB). Consequently, our model could infer rare and unseen words in the test dataset by incorporating contextual semantics learned from the training dataset and lexical relations from ontology. The experiments show that our proposed knowledge integration mechanism achieves consistent improvements across settings with different sizes of training data on two public benchmark datasets. We also show through detailed analysis that incorporating background knowledge effectively alleviates issues of data scarcity.

Tsarray: An Intelligent Access Control System Using Backscatter Communication

With the help of computer vision-based technology, current face recognition access control systems can effectively reduce the hassle of swiping cards. However, it is not suitable for special industry areas where privacy is concerned, and it cannot be used in environments where light conditions are not satisfied. In this paper, we propose Tsarray, an RFID-based method that intelligent senses access control events by vertically deploying large-scale passive tags to the attachment plane. By fully studying the correlation between access control events, and the received RF signals, we choose a tag array deployment strategy of 4 rows, and 15 columns to eliminate environmental interference, and extract distinctive time slot tag array maps for direction tracking. Besides, by setting the same direction conversion module, the signal feature differences caused by the direction are eliminated. Moreover, we built a model to extract the spatiotemporal characteristics, and explore the model structure that is most suitable for the access control event. We evaluate the performance of Tsarray in a real environment, and used a reserved unseen test sample as a system test. Results show that our system can achieve the optimal performance in direction tracking, the accuracy of the 10 volunteers' recognition is 97.5%, the accuracy of the height recognition is 95%, and the accuracy of the weight recognition is 92.5%.

Multi-Level Cross-Lingual Transfer Learning With Language Shared and Specific Knowledge for Spoken Language Understanding

Recently conversational agents effectively improve their understanding capabilities by neural networks. Such deep neural models, however, do not apply to most human languages due to the lack of annotated training data for various NLP tasks. In this paper, we propose a multi-level cross-lingual transfer model with language shared and specific knowledge to improve the spoken language understanding of low-resource languages. Our method explicitly separates the model into the language-shared part and language-specific part to transfer cross-lingual knowledge and improve the monolingual slot tagging, especially for low-resource languages. To refine the shared knowledge, we add a language discriminator and employ adversarial training to reinforce information separation. Besides, we adopt novel multi-level knowledge transfer in an incremental and progressive way to acquire multi-granularity shared knowledge rather than a single layer. To mitigate the discrepancies between the feature distributions of language specific and shared knowledge, we propose the neural adapters to fuse knowledge automatically. Experiments show that our proposed model consistently outperforms monolingual baseline with a statistically significant margin up to 2.09%, even higher improvement of 12.21% in the zero-shot setting.

Zero-Shot Adaptive Transfer for Conversational Language Understanding

Conversational agents such as Alexa and Google Assistant constantly need to increase their language understanding capabilities by adding new domains. A massive amount of labeled data is required for training each new domain. While domain adaptation approaches alleviate the annotation cost, prior approaches suffer from increased training time and suboptimal concept alignments. To tackle this, we introduce a novel Zero-Shot Adaptive Transfer method for slot tagging that utilizes the slot description for transferring reusable concepts across domains, and enjoys efficient training without any explicit concept alignments. Extensive experimentation over a dataset of 10 domains relevant to our commercial personal digital assistant shows that our model outperforms previous state-of-the-art systems by a large margin, and achieves an even higher improvement in the low data regime.

Design of a planar slot tag antennas with circular polarization for ultra‐high frequency radio frequency identification applications

Slot radio frequency identification (RFID) Tag antennas designed for ultra‐high frequency (UHF) band are presented. Two orthogonal slots are used to achieve circular polarization (CP). By selecting an appropriate length of feed‐lines, the input impedance of the tag antenna can be controlled. Experimental results show a good reading range between 9 m and 10 m. A wide band CP band was measured. The design procedure for the proposed antennas are described and the performances are presented and discussed.

Grouped Dynamic Frame Slotted ALOHA Tag Anti-collision Algorithm Based on Parallelizable Identification

In order to solve the problem of low throughput rate and efficiency of the current dynamic frame slot ALOHA collision algorithms, a grouped dynamic frame slotted ALOHA tag anti-collision algorithm based on Parallelizable identification (PIGDFSA) is proposed. Based on the experiments, the method and strategy of increasing the system throughput rate and lowering the tag collision rate are presented by exploring effects of the number of the tags and its groups, the frame length on the system throughout and tag collision rate. Combining the multi-antenna of the RFID system and FastICA technology, the collision slot can be redefined, and the number of the unrecognized tags can be used to set the number of groups and frame length adaptively. The simulation results show that the PIGDFSA algorithm can stabilize the throughput rate more than 92% when the number of tags reaches 2000, and it has higher throughput rate, lesser idle slot and higher algorithm efficiency compared with the FSA-256, GDFSA, and BSDBG algorithm.

Miniaturized decoupled slotted patch RFID tag antennas for wearable health care

In this article, a couple of two-layered RFID tag antenna designs exhibiting improved performance descriptors for on-body applications are presented. The antennas are designed to operate in the microwave band (2.4–2.48 GHz) ensuring high data transmission rates ideal for real-time subject monitoring applications. The radiating element of both the antennas is a slotted patch structure provisioned with a pair of T-shaped slots realized on a commercial FR4 substrate. The augmentation of a systematic sequence of narrow comb-like etchings into the design enhances the impedance bandwidth considerably. A high permittivity silicon layer embedded with the radiating patch provides resilience from the human body dielectric losses. A modified antenna design utilizing patch miniaturization technique, resulting in an overall footprint reduction by 32%, is also proposed. The designed tag antennas offer a gain of more than 1.8 dBi and an attractive read range greater than 6.8 m in the operating band.

Alumina ceramic slot antenna for UHF RFID tag embedded in vehicles

ABSTRACTTo address the metallic interference problem in vehicular RFID (radio‐frequency identification) applications, this letter proposes a novel alumina ceramic slot tag antenna (ACSTA). The proposed antenna consists of a grounded thin alumina ceramic substrate and a slotted top layer rectangular patch, and the all four sides of the substrate are shorting walls. The slot can be equivalent to a horizontal magnetic current flow, so an excellent radiation performance is achieved according to the electromagnetic image theory. Meanwhile, due to the existence of the ground plane, the influences of metallic objects behind the ground to the tag antenna impedance can be maximally reduced. Combined with a commercial chip, this ACSTA is fabricated and measured, and the thickness of the novel tag is only 1mm. The tag antenna offers a half power coefficient bandwidth of 6.9% (0.87–0.93 GHz) with a stable gain about 3.6 dBi, and a similar half power coefficient bandwidth of 6.9% (0.89–0.95 GHz) with a gain about 4.8 dBi when it is attached to an 100 × 100 mm2 metallic plate, respectively. Moreover, the experimental results show that the maximum read range is about 27 or 30 m without or with a same attached 100 × 100 mm2 metallic plate, respectively. Simulation and measurement results show that the operating frequency band of the ACSTA covers US legal UHF RFID (902—928 MHz) band, and so it is a quite suitable candidate for vehicular RFID applications. © 2016 Wiley Periodicals, Inc. Microwave Opt Technol Lett 58:1150–1154, 2016

An Empirical Investigation of Word Class-Based Features for Natural Language Understanding

There are many studies that show using class-based features improves the performance of natural language processing (NLP) tasks such as syntactic part-of-speech tagging, dependency parsing, sentiment analysis, and slot filling in natural language understanding (NLU), but not much has been reported on the underlying reasons for the performance improvements. In this paper, we investigate the effects of the word class-based features for the exponential family of models specifically focusing on NLU tasks, and demonstrate that the performance improvements could be attributed to the regularization effect of the class-based features on the underlying model. Our hypothesis is based on empirical observation that shrinking the sum of parameter magnitudes in an exponential model tends to improve performance. We show on several semantic tagging tasks that there is a positive correlation between the model size reduction by the addition of the class-based features and the model performance on a held-out dataset. We also demonstrate that class-based features extracted from different data sources using alternate word clustering methods can individually contribute to the performance gain. Since the proposed features are generated in an unsupervised manner without significant computational overhead, the improvements in performance largely come for free and we show that such features provide gains for a wide range of tasks from semantic classification and slot tagging in NLU to named entity recognition (NER).

Card-type slot antenna for UHF RFID tag close to the chest of a human body

This article presents a UHF radio frequency identification tag antenna used in proximity to the chest of a human body for student ID card identification.The proposed tag antenna is a rectangular slot-type patch structure printed on a PVC card. For near-body applications, stratified square plates with physical parameters are used as a human model. The radiation properties of the slotted tag antenna placed near the model and the absorption effects due to human body are investigated. The research used the HFSS simulator to optimally design the antenna and employed the FEKO simulator to verify the simulated results. The designed tag was constructed for testing the reading range. The measured reading range achieved is around 4.2 m as the tag is put close to the chest. © 2013 Wiley Periodicals, Inc. Microwave Opt Technol Lett 55:1626–1631, 2013; View this article online at wileyonlinelibrary.com. DOI 10.1002/mop.27610

An effective impedance matching method for RFID slot tag antennas

This article presents an impedance matching method suitable for various radio frequency identification (RFID) slot tag antennas.By loading a matching slot onto a radiating slot, the impedance of the proposed slot tag antenna can be easily tuned to match the various impedance values of chips by simply changing the length of the matching slot. Two examples of the slot tag antennas are studied. Results show that the designed tags exhibit offer good reading range performances and are excellent candidates for RFID applications. © 2011 Wiley Periodicals, Inc. Microwave Opt Technol Lett, 2011; View this article online at wileyonlinelibrary.com. DOI 10.1002/mop.26130

Compact slotted planar inverted-F RFID tag mountable on metallic objects

A compact slotted PIFA-type RFID tag (900 MHz) mountable on metallic objects is presented. The proposed structure can be easily matched to most microchips and facilitates their attachment on its slot. The performance of the tag is evaluated by monitoring its radar cross-sections (RCSs) for various incident angles and chip impedances. The measured detection distance is 4 m.