Unigram Word Research Articles

Machine translation of classical Chinese based on unigram segmentation transformer framework

In the translation work of Chinese ancient books, traditional manual translation is difficult and inefficient. As an important field of natural language processing, machine translation is expected to solve this problem. Due to the rapid development of NLP technology, prior works mainly follow the pipeline of Transformer when dealing with the machine translation task, which can extract the high-quality feature representation with its self-attention mechanism. The great success of Transformer has inspired the direction of our ancient text translation work. In this paper, we screen the Unigram word division by exploring and comparing, and propose a solution for the translation of ancient literary texts. Specifically, we adopt the evaluation of BLEU value and achieve the BLEU values of 43.4 and 40.03 for short and long sentences respectively. When compared with the results of Baidu Translation, our BLEU values increase by 8.12 and 5.18. Additionally, our translation results are more in line with the original text than Baidu Translation, demonstrating the potential and advantage of the model in bridging the ancient and modern Chinese era rift.

Semantic-Preserving Adversarial Text Attacks

Deep learning models are known immensely brittle to adversarial text examples. Existing text adversarial attack strategies can be roughly divided into character-level, word-level, and sentence-level attacks. Despite the success brought by recent text attack methods, how to induce misclassification with minimal text modifications while keeping the lexical correctness, syntactic soundness, and semantic consistency is still a challenge. In this paper, we devise a Bigram and Unigram-based adaptive Semantic Preservation Optimization (BU-SPO) approach which attacks text documents not only at a unigram word level but also at a bigram level to avoid generating meaningless sentences. We also present a hybrid attack strategy that collects substitution words from both synonyms and sememe candidates, to enrich the potential candidate set. Besides, a Semantic Preservation Optimization (SPO) method is devised to determine the word substitution priority and reduce the perturbation cost. Furthermore, we constrain the SPO with a semantic Filter (dubbed SPOF) to improve the semantic similarity. To estimate the effectiveness of our proposed methods, BU-SPO and BU-SPOF, we attack four victim deep learning models trained on three text datasets. Experimental results demonstrate that our approaches accomplish the highest semantics consistency and attack success rates by making minimal word modifications compared with competitive methods.

Bigram and Unigram Based Text Attack via Adaptive Monotonic Heuristic Search

Deep neural networks (DNNs) are known to be vulnerable to adversarial images, while their robustness in text classification are rarely studied. Several lines of text attack methods have been proposed in the literature, such as character-level, word-level, and sentence-level attacks. However, it is still a challenge to minimize the number of word distortions necessary to induce misclassification, while simultaneously ensuring the lexical correctness, syntactic correctness, and semantic similarity. In this paper, we propose the Bigram and Unigram based Monotonic Heuristic Search (BU-MHS) method to examine the vulnerability of deep models. Our method has three major merits. Firstly, we propose to attack text documents not only at the unigram word level but also at the bigram level to avoid producing meaningless outputs. Secondly, we propose a hybrid method to replace the input words with both their synonyms and sememe candidates, which greatly enriches potential substitutions compared to only using synonyms. Lastly, we design a search algorithm, i.e., Monotonic Heuristic Search (MHS), to determine the priority of word replacements, aiming to reduce the modification cost in an adversarial attack. We evaluate the effectiveness of BU-MHS on IMDB, AG's News, and Yahoo! Answers text datasets by attacking four state-of-the-art DNNs models. Experimental results show that our BU-MHS achieves the highest attack success rate by changing the smallest number of words compared with other existing models.

APLIKASI SENTIMENT MONITORING UNTUK TWITTER DENGAN ALGORITMA NAIVE-BAYES CLASSIFIER

Every day there are millions of opinion spread across social networks. This is often utilized by various parties to determine the opinion and sentiment of the public towards the product, brand or figures that they hold. Given the abundance of data and opinions, it is not possible to do sentiment analysis manually. In this research, author performs design and implementation of sentiment monitoring application, that could monitor people’s sentiment about a particular keyword, so it is known how the people response to those keywords, whether positive, negative or neutral. From various existing social networks, Twitter is chosen as the source of data that will be monitored. Classification algorithm used here is Naive-Bayes Classifier with Boolean Multinomial model, and feature extraction using unigram word. The training data used is 400,000 data for each type of sentiment, so the total is 1.200.000 data. In the process of classification and training, application will perform stemming to take the root words contained within the tweet. Stemming algorithm used here is Confix Stripping. The methodology of application development that used here is staged delivery. Implementation of application is done using PHP programming language. The result of this research is a sentiment monitoring application that can monitor public sentiment about a particular keyword in a particular time frame. From testing using k-fold cross validation, obtained accuracy rate for sentiment classification amounted to 85%.

PMCVec: Distributed phrase representation for biomedical text processing

Distributed semantic representation of biomedical text can be beneficial for text classification, named entity recognition, query expansion, human comprehension, and information retrieval. Despite the success of high-quality vector space models such as Word2Vec and GloVe, they only provide unigram word representations and the semantics for multi-word phrases can only be approximated by composition. This is problematic in biomedical text processing where technical phrases for diseases, symptoms, and drugs should be represented as single entities to capture the correct meaning. In this paper, we introduce PMCVec, an unsupervised technique that generates important phrases from PubMed abstracts and learns embeddings for single words and multi-word phrases simultaneously. Evaluations performed on benchmark datasets produce significant performance gains both qualitatively and quantitatively.

An improved Urdu stemming algorithm for text mining based on multi-step hybrid approach

ABSTRACTStemming is the basic operation in Natural language processing (NLP) to remove derivational and inflectional affixes without performing a morphological analysis. This practice is essential to extract the root or stem. In NLP domains, the stemmer is used to improve the process of information retrieval (IR), text classifications (TC), text mining (TM) and related applications. In particular, Urdu stemmers utilize only uni-gram words from the input text by ignoring bigrams, trigrams, and n-gram words. To improve the process and efficiency of stemming, bigrams and trigram words must be included. Despite this fact, there are a few developed methods for Urdu stemmers in the past studies. Therefore, in this paper, we proposed an improved Urdu stemmer, using hybrid approach divided into multi-step operation, to deal with unigram, bigram, and trigram features as well. To evaluate the proposed Urdu stemming method, we have used two corpora; word corpus and text corpus. Moreover, two different evaluation metrics have been applied to measure the performance of the proposed algorithm. The proposed algorithm achieved an accuracy of 92.97% and compression rate of 55%. These experimental results indicate that the proposed system can be used to increase the effectiveness and efficiency of the Urdu stemmer for better information retrieval and text mining applications.

Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards

Abstract Typing on tiny QWERTY keyboards on smartwatches is considered challenging or even impractical due to the limited screen space. In this paper, we describe three user studies undertaken to investigate users’ typing abilities and preferences on tiny QWERTY keyboards. The first two studies, using a smartphone as a substitute for a smartwatch, tested five different keyboard sizes (2, 2.5, 3, 3.5 and 4 cm). Study 1 collected typing data from participants using keyboards and given asterisk feedback. We analyzed both the distribution of touch points (e.g., the systematic offset and shape of the distribution) and the effect of keyboard size. Study 2 adopted a Bayesian algorithm based on a touch model derived from Study 1 and a unigram word language model to perform input prediction. We found that on the smart keyboard, participants could type between 26.8 and 33.6 words per minute (WPM) across the five keyboard sizes with an uncorrected character error rate ranging from 0.4% to 1.9%. Participants’ subjective feedback indicated that they felt most comfortable with keyboards larger than 2.5 cm. Study 3 replicated the 3.0 and 3.5 cm keyboard tests on a real smartwatch and verified that in terms of text entry speed, error rate and user preference, there was no significant difference between the results measured on a smartphone and that on a smartwatch with same sized keys. This study result indicated that the results of Study 1 and 2 are applicable to smartwatch devices. Finally, we conducted a simulation to investigate the performance of different touch/language models based on our collected data. The results showed that using either a bigram language model or a detailed touch model can effectively correct imprecision in users’ input. Our results suggest that achieving satisfactory levels of text input on tiny QWERTY keyboards is possible.