Low-resource Data Research Articles

Cross-Domain NER under a Divide-and-Transfer Paradigm

Cross-domain Named Entity Recognition (NER) transfers knowledge learned from a rich-resource source domain to improve the learning in a low-resource target domain. Most existing works are designed based on the sequence labeling framework, defining entity detection and type prediction as a monolithic process. However, they typically ignore the discrepant transferability of these two sub-tasks: the former locating spans corresponding to entities is largely domain-robust, whereas the latter owns distinct entity types across domains. Combining them into an entangled learning problem may contribute to the complexity of domain transfer. In this work, we propose the novel divide-and-transfer paradigm in which different sub-tasks are learned using separate functional modules for respective cross-domain transfer. To demonstrate the effectiveness of divide-and-transfer, we concretely implement two NER frameworks by applying this paradigm with different cross-domain transfer strategies. Experimental results on 10 different domain pairs show the notable superiority of our proposed frameworks. Experimental analyses indicate that significant advantages of the divide-and-transfer paradigm over prior monolithic ones originate from its better performance on low-resource data and a much greater transferability. It gives us a new insight into cross-domain NER. Our code is available on GitHub. 1

A Prompt-Based Topic-Modeling Method for Depression Detection on Low-Resource Data

Depression has a large impact on one’s personal life, especially during the COVID-19 pandemic. People have been trying to develop reliable methods for the depression detection task. Recently, methods based on deep learning have attracted much attention from the research community. However, they still face the challenge that data collection and annotation are difficult and expensive. In many real-world applications, only a small number of or even no training data are available. In this context, we propose a Prompt-based Topic-modeling method for Depression Detection (PTDD) on low-resource data, aiming to establish an effective way of depression detection under the above challenging situation. Instead of learning discriminating features from a small amount of labeled data, the proposed framework turns to leverage the generalization power of pretrained language models. Specifically, based on the question-and-answer routine during the interview, we first reorganize the text data according to the predefined topics for each interviewee. Via the prompt-based framework, we then predict whether the next-sentence prompt is emotionally positive or not. Finally, the depression detection task can be achieved based on the obtained topicwise predictions through a simple voting process. In the experiments, we validate the effectiveness of our model under several low-resource data settings. The results and analysis demonstrate that our PTDD achieves acceptable performance when only a few training samples or even no training samples are available.

A Systematic Review on Semantic Role Labeling for Information Extraction in Low-Resource Data

A Systematic Review on Semantic Role Labeling for Information Extraction in Low-Resource Data

Bacterial Classification Using Deep Structured Convolutional Neural Network for Low Resource Data

Bacterial identification is an essential task in medical disciplines and food hygiene. The characteristics of bacteria can be examined under a microscope using culture techniques. However, traditional clinical laboratory culture methods require considerable work, primarily physical and manual effort. An automated process using deep learning technology has been widely used for increasing accuracy and decreasing working costs. In this paper, our research evaluates different types of existing deep CNN models for bacterial contamination classification when low-resource data are used. They are baseline CNN, GCNN, ResNet, and VGGNet. The performance of CNN models was also compared with the traditional machine learning method, including SIFT+SVM. The performance of the DIBaS dataset and our own collected dataset have been evaluated. The results show that VGGNet achieves the highest accuracy. In addition, data augmentation was performed to inflate the dataset. After fitting the model with augmented data, the results show that the accuracy increases significantly. This improvement is consistent in all models and both datasets.

Improving Tibetan End-To-End Speech Recognition with Transfer Learning

End-to-end architecture has shown outstanding performance in the field of speech recognition, but achieving such performance typically requires a large amount of annotated data. In languages with sufficient corpus and resources, satisfactory recognition results have been achieved. However, for some low-resource data with insufficient training data, the lack of training data remains a bottleneck in building speech recognition systems. This paper presents an approach that utilizes self-supervised feature extraction and transfer learning to enhance the performance of acoustic models in low-resource languages. The proposed strategy involves retraining the fundamental acoustic model, originally trained on resource-rich languages, using a limited amount of low-resource speech data within an end-to-end architecture. By doing so, it aims to construct an improved acoustic model tailored for low-resource languages. The results of the model in the Tibetan language dataset show significant improvement, with a relative decrease of 26% in word error rate from 13.8% to 10.2% on the test set.

A Research Toward Chinese Named Entity Recognition Based on Transfer Learning

To improve the performance of named entity recognition in the lack of well-annotated entity data, a transfer learning-based Chinese named entity recognition model is proposed in this paper. The specific tasks are as follows: (1) first/, a data transfer method based on entity features is proposed. By calculating the similarity of feature distribution between low resource data and high resource data, the most representative entity features are selected for feature transfer mapping, and the distance of entity distribution between the two domains is calculated to make up the gap between the data of the two domains then model is trained by high resource data. (2) Then, an entity boundary detection method is proposed. This method utilizes the BiLSTM+CRF as the main structure and integrates character boundary information to assist the attention network to improve the model’s ability to recognize entity boundaries. (3) Finally, multiple named entity recognition methods are selected as baseline methods for comparison, and experiments are conducted on several datasets. The results show that the model proposed in this paper improves the accuracy of named entity recognition by 1%, the recall rate by 2%, and the F1 value by 2% on average in the field with low-resource.

Exploiting All Samples in Low-Resource Sentence Classification: Early Stopping and Initialization Parameters

To improve deep-learning performance in low-resource settings, many researchers have redesigned model architectures or applied additional data (e.g., external resources, unlabeled samples). However, there have been relatively few discussions on how to make good use of small amounts of labeled samples, although it is potentially beneficial and should be done before applying additional data or redesigning models. In this study, we assume a low-resource setting in which only a few labeled samples (i.e., 30–100 per class) are available, and we discuss how to exploit them without additional data or model redesigns. We explore possible approaches in the following three aspects: training validation splitting, early stopping, and weight initialization. Extensive experiments are conducted on six public sentence classification datasets. Performance on various evaluation metrics (e.g., accuracy, loss, and calibration error) significantly varied depending on the approaches that were combined in the three aspects. Based on the results, we propose an integrated method, which is to initialize the model with a weight averaging method and use a non-validation stop method to train all samples. This simple integrated method consistently outperforms the competitive methods; e.g., the average accuracy of six datasets of this method was 1.8% higher than those of conventional validation-based methods. In addition, the integrated method further improves the performance when adapted to several state-of-the-art models that use additional data or redesign the network architecture (e.g., self-training and enhanced structural models). Our results highlight the importance of the training strategy and suggest that the integrated method can be the first step in the low-resource setting. This study provides empirical knowledge that will be helpful when dealing with low-resource data in future efforts. Our code is publicly available at https://github.com/DMCB-GIST/exploit_all_samples.

METHOD OF LOW-RESOURCE HASHING TYPE "DATA – GENERATOR"

Creating a secure and efficient structure of a cryptographic algorithm is one of the key cryptographic tasks. Recently, cryptography for low-resource devices has attracted considerable attention of world scientists. A significant portion of the research is dedicated to the examination of block encryption methods, and conversely, there are notably fewer publicly disclosed proposals for low-resource hashing methods. Many hash functions recommended for use in low-resource devices by well-known standardization organizations are based on block encryption, which offers a sufficient level of security but demands substantial computational resources—something critical for their application in such devices. The urgency of investigating low-resource data hashing methods stems from the need to guarantee an adequate level of hash function security while minimizing computational resource usage through adjustments to the hashing process. This article reviews established approaches to constructing hash functions of varying complexities and examines the latest research and publications focused on low-resource hashing. Based on this, the structure and approach for developing a low-resource data hashing method were chosen. A novel low-resource hashing method, founded on the Merkle-Damgård construction and utilizing an iterative byte-oriented approach, is introduced. The process of low-resource hashing, according to the new method, is formally described. Statistical testing of the proposed method was conducted in accordance with NIST SP 800-22. An overview of the hardware implementation of the proposed low-resource hashing method is presented in the form of a generalized structural diagram. The complexity of the proposed hardware implementation is quantified in conventional units [GE] for hash value calculations of 128, 192, and 256 bits. A comparison of the proposed "data-generator" type hashing method with established low-resource hash functions, in terms of hardware costs, is conducted.

Transformer-based multitask learning for reaction prediction under low-resource circumstances.

Recently, effective and rapid deep-learning methods for predicting chemical reactions have significantly aided the research and development of organic chemistry and drug discovery. Owing to the insufficiency of related chemical reaction data, computer-assisted predictions based on low-resource chemical datasets generally have low accuracy despite the exceptional ability of deep learning in retrosynthesis and synthesis. To address this issue, we introduce two types of multitask models: retro-forward reaction prediction transformer (RFRPT) and multiforward reaction prediction transformer (MFRPT). These models integrate multitask learning with the transformer model to predict low-resource reactions in forward reaction prediction and retrosynthesis. Our results demonstrate that introducing multitask learning significantly improves the average top-1 accuracy, and the RFRPT (76.9%) and MFRPT (79.8%) outperform the transformer baseline model (69.9%). These results also demonstrate that a multitask framework can capture sufficient chemical knowledge and effectively mitigate the impact of the deficiency of low-resource data in processing reaction prediction tasks. Both RFRPT and MFRPT methods significantly improve the predictive performance of transformer models, which are powerful methods for eliminating the restriction of limited training data.

Voice Activation for Low-Resource Languages

Voice activation systems are used to find a pre-defined word or phrase in the audio stream. Industry solutions, such as “OK, Google” for Android devices, are trained with millions of samples. In this work, we propose and investigate several ways to train a voice activation system when the in-domain data set is small. We compare self-training exemplar pre-training, fine-tuning a model pre-trained on another domain, joint training on both an out-of-domain high-resource and a target low-resource data set, and unsupervised pre-training. In our experiments, the unsupervised pre-training and the joint-training with a high-resource data set from another domain significantly outperform a strong baseline of fine-tuning a model trained on another data set. We obtain 7–25% relative improvement depending on the model architecture. Additionally, we improve the best test accuracy on the Lithuanian data set from 90.77% to 93.85%.

Data augmentation-based conditional Wasserstein generative adversarial network-gradient penalty for XSS attack detection system.

The rapid growth of the worldwide web and accompanied opportunities of web applications in various aspects of life have attracted the attention of organizations, governments, and individuals. Consequently, web applications have increasingly become the target of cyberattacks. Notably, cross-site scripting (XSS) attacks on web applications are increasing and have become the critical focus of information security experts’ reports. Machine learning (ML) technique has significantly advanced and shown impressive results in the area of cybersecurity. However, XSS training datasets are often limited and significantly unbalanced, which does not meet well-developed ML algorithms’ requirements and potentially limits the detection system efficiency. Furthermore, XSS attacks have multiple payload vectors that execute in different ways, resulting in many real threats passing through the detection system undetected. In this study, we propose a conditional Wasserstein generative adversarial network with a gradient penalty to enhance the XSS detection system in a low-resource data environment. The proposed method integrates a conditional generative adversarial network and Wasserstein generative adversarial network with a gradient penalty to obtain necessary data from directivity, which improves the strength of the security system over unbalance data. The proposed method generates synthetic samples of minority class that have identical distribution as real XSS attack scenarios. The augmented data were used to train a new boosting model and subsequently evaluated the model using a real test dataset. Experiments on two unbalanced XSS attack datasets demonstrate that the proposed model generates valid and reliable samples. Furthermore, the samples were indistinguishable from real XSS data and significantly enhanced the detection of XSS attacks compared with state-of-the-art methods.

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

To improve the performance of deep learning methods in case of a lack of labeled data for entity annotation in entity recognition tasks, this study proposes transfer learning schemes that combine the character to be the word to convert low-resource data symmetry into high-resource data. We combine character embedding, word embedding, and the embedding of the label features using high- and low-resource data based on the BiLSTM-CRF model, and perform the feature-transfer and parameter-sharing tasks in two domains of the BiLSTM network to annotate with zero resources. Before transfer learning, we must first calculate the label similarity between two different domains and select the label features with large similarity for feature transfer mapping. All training parameters of the source domain in the model are shared during the BiLSTM network processing and CRF layer. In addition, we also use the method of combining characters and words to reduce the problem of word segmentation across domains and reduce the error rate in label mapping. The results of experiments show that in terms of the overall F1 score, the proposed model without supervision was superior by 9.76 percentage points to the general parametric shared transfer learning method, and by 9.08 and 12.38 percentage points, respectively, to two recent high–low resource learning methods. The proposed scheme improves performance in terms of transfer learning between the high- and low-resource data and can identify the predicted data in the target domain.

Dual Adversarial Transfer for Sequence Labeling.

We propose a new architecture for addressing sequence labeling, termed Dual Adversarial Transfer Network (DATNet). Specifically, the proposed DATNet includes two variants, i.e., DATNet-F and DATNet-P, which are proposed to explore effective feature fusion between high and low resource. To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD) and adopt adversarial training to boost model generalization. We investigate the effects of different components of DATNet across different domains and languages, and show that significant improvement can be obtained especially for low-resource data. Without augmenting any additional hand-crafted features, we achieve state-of-the-art performances on CoNLL, Twitter, PTB-WSJ, OntoNotes and Universal Dependencies with three popular sequence labeling tasks, i.e., Named entity recognition (NER), Part-of-Speech (POS) Tagging and Chunking.

Improved Wasserstein conditional generative adversarial network speech enhancement

The speech enhancement based on the generative adversarial network has achieved excellent results with large quantities of data, but performance in the low-data regime and tasks like unseen data learning still lag behind. In this work, we model Wasserstein Conditional Generative Adversarial Network-Gradient Penalty speech enhancement system and introduce the elastic network into the objective function to simplify and improve the performance of the model in low-resource data environment. We argue that the regularization is significant in learning with small amounts of data and the available information of the input data is key in speech enhancement performance and generalization ability of the model, which means that network parameters and network structure can be set up and designed according to the characteristics of actual input data. Experiments on the noisy speech corpus show that the improved algorithm outperforms previous generative adversarial network speech enhancement approach.

The mortuary as a source of injury data: Progress towards a mortuary data guideline for fatal injury surveillance

To help bridge the obstacle of inadequate injury fatality data in low and medium income countries (LMICs) a simple cost effective system for mortuary surveillance of fatal injuries is being developed in consultation with the WHO. This will inform, direct and monitor injury prevention (IP) interventions and policies in LMICs. This article uses CDC's ‘attributes of a successful surveillance system’ to describe the process, the barriers and solutions in development of this mortuary data guideline. The consultative process utilised generated feedback from key stakeholders including forensic pathologists, Ministry of Health officials and injury prevention experts. An International Advisory Group was also convened to guide the guideline development. These assisted the adjustment of the proposed guideline to maximise flexibility, acceptability and stability; whilst minimising resource implications. Representativeness and the securing of government support perhaps remain the most significant challenges. Consultation with the advisory group and the wider stakeholders has been effective in developing a widely acceptable, user-friendly, low resource data form to gather useful data. Further strategies to overcome barriers need to be developed over the course of the pilot study and this should be done in consultation with the advisory group and stakeholders.

Protocol Development for Ovarian Cancer Treatment in Kenya: A Brief Report

Ovarian cancer is a leading cause of cancer death for Kenyan women. Most women are diagnosed with an advanced stage of disease. The current North American standard of care includes surgery followed by carboplatin and paclitaxel. Neither drug is available for Kenyan women. We performed a literature search investigating chemotherapy in low resource countries with the aim to write an evidence-based chemotherapy protocol for women diagnosed with ovarian cancer in Eldoret, Kenya, at the Moi Teaching and Referral Hospital. We systematically searched PubMed and EMBASE for articles describing chemotherapy treatment outcomes of ovarian epithelial cancer in low-resource settings. After data analysis, a secondary review was undertaken on randomized controlled trials(RCTs) aligning with chemotherapy availability in Kenya. We identified 1184 articles. Fourteen met our criteria: ovarian epithelial cancer,low resource, chemotherapy use, and survival or response data. No publications were RCTs or had a cohort larger than 100 patients. There was no consistency in drug choice between studies. After this search, we reviewed commonly quoted and relevant RCTs and meta-analyses conducted on ovarian cancer since the 1980s. Although RCTs in the developed world suggest carboplatin and taxol provide optimal survival benefit, these drugs are unavailable in Kenya. Cyclophosphamide and cisplatin provide the next most optimal survival benefit, with acceptable and manageable toxicity. Because these drugs are more available and affordable in Kenya, we have developed a protocol recommending their use, which has been accepted by the Moi Teaching and Referral Hospital. Currently, there is a paucity of published RCTs that may guide treatment in low-resource settings. One considerable barrier to establishing and evaluating chemotherapy protocols in low-resource settings may be the cost of chemotherapy drugs. There needs to be an international movement to make cancer chemotherapeutics available at lower prices in low-resource settings.

The Mortuary as a source of injury data: progress towards a mortuary data guideline for fatal injury surveillance

BackgroundInjury is a substantial and preventable public health problem accounting for 10% of deaths worldwide, of which 90% are estimated to occur in low and middle income countries (LMICs). To...